Experimental Iceberg Extractor and Writer Components Released

We’re excited to announce the release of two new experimental components: the Iceberg Extractor and Iceberg Writer. These component enable seamless integration with Apache Iceberg - a high-performance table format for large-scale analytics bringing modern data lake capabilities into the Keboola platform.

Both components currently support REST API catalogs and are ideal for advanced data lakehouse use cases.

Iceberg Extractor Highlights

• Flexible data selection: Choose between full extraction or selecting specific columns.
• Multiple load modes: Supports full, incremental, or append loads.
• Efficient output: Option to store results as Parquet files for optimized downstream processing.

Iceberg Writer Highlights

• Write modes: Overwrite, append, or upsert support.
• Partitioning: Enable efficient queries with table partitioning.
• Storage flexibility: Write to external tables in blob storage or native Databricks tables via DBX Warehouse.
• Performance tuning: Configure batch size and insertion order for external tables.

As experimental components, we welcome your feedback and use cases to help shape their future development. Let us know what you think!