Added logic to create table from schema. by subkanthi · Pull Request #219 · Altinity/ice

subkanthi · 2026-09-15T19:52:50Z

closes: #218

Add support for creating table by passing schema.

subkanthi · 2026-09-19T10:35:32Z

create-table ns1.t1 --format-version=3   --schema '[{"name":"id","type":"int","required":true},{"name":"payload","type":"unknown"}]'

describe -s ns1.t1
2026-09-19 12:44:16 [main/3332010] INFO o.a.i.r.auth.AuthManagers > Loading AuthManager implementation: org.apache.iceberg.rest.auth.NoopAuthManager
---
- kind: Table
  metadata:
    id: ns1.t1
  data:
    schemaRaw: |-
      table {
        1: id: required int
        2: payload: optional unknown
      }
    partitionSpecRaw: "[]"
    sortOrderRaw: "[]"
    formatVersion: 3
    location: s3://bucket1/ns1/t1

subkanthi · 2026-09-21T15:47:13Z

+      }
+
+      if (schemaJson != null && !schemaJson.isEmpty()) {
+        CreateTable.run(


wanted to not to touch the current flow of CreateTable

xieandrew

There's an issue with field id ordering with nested schemas plus some other comments.

Also, not exactly from this PR but I noticed that nested fields don't support the required option, in IcebergTypeParser it always uses optional. Not sure if that belongs in this PR or could be added later.

xieandrew · 2026-09-21T18:10:32Z

+      int fieldId = nextId.incrementAndGet();
+      Type type = IcebergTypeParser.parseType(field.type(), nextId);
+      boolean required = field.required() != null && field.required();
+      columns.add(
+          required
+              ? Types.NestedField.required(fieldId, field.name(), type, field.doc())
+              : Types.NestedField.optional(fieldId, field.name(), type, field.doc()));
+    }
+    return new Schema(columns);


The field id ordering from this method doesn't match what the metadata actually creates for nested schemas with struct. For example ice create-table schema.nested --schema '[{"name":"a","type":"struct<x:string,y:long>"},{"name":"b","type":"long"}]' results in "a": 1, "x": 3, "y": 4, "b": 2 in the real table metadata but this method produces "a": 1, "x": 2, "y": 3, "b": 4 which ends up in the table property "schema.name-mapping.default" (mismatched from the real schema). I think the fix is to add TypeUtil.assignIncreasingFreshIds(schema) before the return.

xieandrew · 2026-09-21T18:13:36Z

  }

-  private static Type parseType(String typeString, AtomicInteger nextId) {
+  public static Type parseType(String typeString, AtomicInteger nextId) {


Create table with variant type does not work: ice create-table schema.variant --schema '[{"name":"v","type":"variant"}]'

Error:

java.lang.IllegalArgumentException: Cannot parse type string: variant is not a primitive type at org.apache.iceberg.types.Types.fromPrimitiveString(Types.java:116) at com.altinity.ice.cli.internal.util.IcebergTypeParser.parseType(IcebergTypeParser.java:52) at com.altinity.ice.cli.internal.util.IceSchemaParser.toSchema(IceSchemaParser.java:77) at com.altinity.ice.cli.internal.util.IceSchemaParser.parse(IceSchemaParser.java:56) at com.altinity.ice.cli.Main.createTable(Main.java:403) at com.altinity.ice.cli.Main.lambda$main$4(Main.java:1258) [9 skipped] at com.altinity.ice.cli.Main.main(Main.java:1265) [1 skipped]

created #222

xieandrew · 2026-09-21T18:15:18Z

-          formatVersion,
-          partitions,
-          sortOrders);
+      if ((schemaFile == null) == (schemaJson == null || schemaJson.isEmpty())) {


I think schemaFile should check isEmpty too since --schema-from-parquet "" with empty string would currently be allowed.

xieandrew · 2026-09-21T18:18:37Z

+    Map<Integer, Types.NestedField> byId = TypeUtil.indexById(schema.asStruct());
+    // a, x, y, b, list element
+    assertThat(byId).hasSize(5);
+    assertThat(byId.keySet()).containsExactlyInAnyOrder(1, 2, 3, 4, 5);


This should check the exact order not just a set to verify the ordering issue from the comment in IceSchemaParser.

…ID order in IceSchemaParserTest

… critical)

…ds to match catalog

…moved from quay.io/dockerhub)

xieandrew

Looks good to me

Added logic to create table from schema.

50ae288

subkanthi linked an issue Sep 15, 2026 that may be closed by this pull request

Add support for creating table with schema #218

Closed

Add pyiceberg test resources and iceberg-v3-unknown-type scenario

6066f02

subkanthi force-pushed the 218-add-support-for-creating-table-with-schema branch from c8b7049 to 6b6e467 Compare September 15, 2026 21:02

Add pyarrow extra to pyiceberg test image

0144dd4

subkanthi force-pushed the 218-add-support-for-creating-table-with-schema branch from e238b47 to 0144dd4 Compare September 15, 2026 21:55

Use noop auth in pyiceberg validators (catalog test config is anonymous)

7d50554

subkanthi force-pushed the 218-add-support-for-creating-table-with-schema branch from 4a93449 to 7d50554 Compare September 16, 2026 16:30

ADded new createTable to reflect config json

c258763

subkanthi closed this Sep 16, 2026

subkanthi reopened this Sep 16, 2026

subkanthi mentioned this pull request Sep 19, 2026

Antalya 26.6: Added support for iceberg v3 unknown data type Altinity/ClickHouse#2363

Open

30 tasks

subkanthi commented Sep 21, 2026

View reviewed changes

subkanthi requested a review from xieandrew September 21, 2026 15:47

xieandrew reviewed Sep 21, 2026

View reviewed changes

subkanthi added 4 commits September 24, 2026 21:08

create-table: reject empty --schema-from-parquet; assert exact field …

3ae2986

…ID order in IceSchemaParserTest

Bump anyio 4.9.0 -> 4.15.1 in pyiceberg test lock (GHSA-82r6-8w77-94w6,…

67c9970

… critical)

IceSchemaParser: assign field IDs via TypeUtil.assignIncreasingFreshI…

fe1f02a

…ds to match catalog

Replace MinIO with rustfs:1.0.0 in integration tests (minio images re…

1cb5961

…moved from quay.io/dockerhub)

subkanthi requested a review from xieandrew September 25, 2026 03:41

xieandrew approved these changes Sep 25, 2026

View reviewed changes

subkanthi added this to the v0.23.0 milestone Sep 25, 2026

subkanthi added the ice Relates to ice label Sep 25, 2026

subkanthi merged commit ca476fa into master Sep 25, 2026
10 checks passed

Navigation Menu

Added logic to create table from schema. - #219

Added logic to create table from schema.#219
subkanthi merged 9 commits into
masterfrom
218-add-support-for-creating-table-with-schema

subkanthi commented Sep 15, 2026

Uh oh!

subkanthi commented Sep 19, 2026 •

edited

Loading

Uh oh!

subkanthi Sep 21, 2026

Uh oh!

xieandrew left a comment

Uh oh!

xieandrew Sep 21, 2026

Uh oh!

xieandrew Sep 21, 2026

Uh oh!

subkanthi Sep 25, 2026

Uh oh!

xieandrew Sep 21, 2026

Uh oh!

subkanthi Sep 25, 2026

Uh oh!

xieandrew Sep 21, 2026

Uh oh!

subkanthi Sep 25, 2026

Uh oh!

xieandrew left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

subkanthi commented Sep 15, 2026

Uh oh!

subkanthi commented Sep 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

xieandrew left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

xieandrew left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

subkanthi commented Sep 19, 2026 •

edited

Loading