Faster Workspace Loads with Simplified Output Mapping (Snowflake Only)
We’ve simplified the logic for loading data from SQL workspaces to Storage, significantly improving performance and reducing load times - especially for large datasets.
Introducing a performance-focused optimization in how data is loaded from SQL workspaces to Storage in Snowflake transformations. This update simplifies the output mapping process and significantly speeds up large table imports.
What’s new?
The new logic replaces complex branching with a simplified execution path. When enabled, you can configure the Output Mapping in Transformations to use Insert instead of the default Upsert. This significantly speeds up data loads by skipping deduplication and type casting, making it ideal for large datasets where performance is critical.
Limitation: Only supported for Snowflake SQL workspaces.
NOTE: Feature Flag Required
This feature is currently available only via support-enabled feature flag. It is intended for advanced users and high data maturity teams who fully understand the implications. When you disable deduplication, you take full responsibility for data integrity - incorrect use may lead to broken or inconsistent tables in Storage. Reach out via our Support if you would like to try it in some of your projects.
We’re also pleased to share that we’re actively working on additional improvements to Output Mapping and data import to further accelerate transformations and provide greater control and flexibility to advanced users. If you would like to participate and provide any inputs in this area please reach out to some of our Keboola's colleagues you are in touch with or via our Support, thank you!