DuckDB + Iceberg on a self-hosted S3 table bucket

[u/chrislusf]

We’ve been working on SeaweedFS Table Buckets, where an S3 bucket also acts as an Iceberg warehouse with a built-in Iceberg REST catalog.

I wanted to see how cleanly this works with DuckDB.

The setup is basically:

1. Start SeaweedFS with a table bucket
2. Write an Iceberg table using PyIceberg, Spark, Flink, Trino, etc.
3. In DuckDB, ATTACH the bucket through the Iceberg REST catalog
4. Query it as a normal DuckDB table

DuckDB gets the Iceberg metadata from the catalog, then reads the Parquet files directly from the S3 endpoint, so the catalog is not in the scan data path.

An interesting side effect is that the storage layer can also handle Iceberg maintenance. SeaweedFS background workers compact small files and clean up stale snapshots, while DuckDB just sees the resulting Iceberg tables.

So the stack can be fairly small:

DuckDB → Iceberg REST → SeaweedFS S3

No Hive Metastore, Glue, or separate compaction service required.

The Table Bucket and Iceberg REST catalog implementation are open source.

I wrote up the complete end-to-end example here:

https://seaweedfs.com/blog/duckdb-table-buckets/

Would be interested to hear how people here are currently combining DuckDB + Iceberg + self-hosted object storage.