Keboola logoCHANGELOG
Platform

2024 Release 1 - Data Management

Project DevOps Manager (Private Beta)

Manage the entire development process with a code-first approach across your environments.

What

Manage Keboola environments via new CLI capabilities. Integrate with GIT for full lifecycle management across development, production, and testing environments, ensuring robust project governance and compliance.

We are working on a real-life model scenario and a set of supporting scripts that showcase how to leverage our CLI to manage the DEV/PROD environment lifecycle.

Who

  • Data Analysts: More options for code-first approach. Tighter relationship between CLI definition and GUI branches.
  • Data Engineers: More granular control over deployment process. Separate DWH style development environments. Ability to implement CI/CD principles.
  • Business users: More options to define and comply with specific enterprise security standards.

Example Use Cases

  • Multi-project environment: Manage entire DEV/PROD(/TEST) multi-project architecture environments via GIT and git flow principles with complete freedom on setting up the branch security and deployment process principles (e.g. advanced SOX).
  • Project deployment automation: Project representation into a different project using the command line interface (CLI). E.g. one project (e.g. “template”) can be deployed from a single repository to multiple projects.
  • Git-based Keboola GUI branch management: Merge or rebase Keboola GUI Branches via git rather than UI. Even merge Keboola GUI branches between each other via git.

Change Data Capture (CDC) (Public Beta)

Load every change fast and save on costs

What

Log-based Change Data Capture (CDC) connectors enable incremental data sync without overloading the source system. This efficient yet robust method supports a broad range of database systems. Micro-batch approach ensures data consistency and cost control.

Who

  • Data Engineers: Reduced latency, efficient data replication, automated schema change handling, simple setup.
  • Business users, Data Analysts and Data Scientists: Near-real-time access to updated data, Improved data quality and accuracy

Example Use Cases

  • Robust data transformation: Ensures precise and dependable data transformations, tailored to fit the requirements of your data warehouse and downstream analytics, ensuring high-quality data readiness.
  • Complete slowly-changing dimension: Captures database events with high accuracy, ensuring dimension tables are consistently updated and reflect all data changes, supporting better decision-making processes.
  • Streamlined data extraction: Efficiently extract data from various sources for seamless integration into the data warehouse, reducing processing time and enhancing data flow.
  • Save costs: Micro Batch Syncs allow you to set frequency to your business needs and save resources.

Learn more about CDC in the announcement or in documentation.

Universal Extractor (Public Beta)

Easily connect to (almost) any data source.

What

Our Generic Extractor now offers a user-friendly visual builder for ingesting data from any REST-like JSON APIs without requiring the need for code. Additional features include importing from cURL commands and mapping inference analyzer, which builds the ‌proper tabular structure from all objects.

Who

  • Data Analysts, business users: Individuals with any level of technical expertise can confidently create a Generic Extractor with ease.
  • Data Scientists, Data Engineers: Ease of use. The UX is rated highly among new users. It reduces the time needed to create a new generic Ex Configuration. & Fewer errors.

Example Use Cases

  • Gathering financial data: Connect with financial APIs, such as stock market data providers or banking APIs. Extract real-time stock prices, historical data, account balances, and more.
  • Collecting IoT sensor data: Interact with IoT device APIs, enabling you to extract sensor data from various sources for further analysis or integration with other systems.
  • Extracting social media data: Extract data from various social media APIs, such as LinkedIn, Facebook, or Instagram. Easily retrieve specific data points and metrics for further analysis.
  • Integrating E-commerce platforms: Connect with different e-commerce APIs, such as Shopify, WooCommerce, or Magento. Extract product information, sales data, customer details, and order history to consolidate and analyze your e-commerce operations in Keboola.

Read the full announcement and learn more.

Google Cloud Services Adoption (General Availability)

Get the most out of Google Cloud

What

Keboola now offers Google Cloud private and shared environments, with BigQuery — allowing for increased optionality. Additionally, Google Cloud Marketplace enables ‌the deployment of Keboola, so you can start working instantly.

Who

  • CTO & CDO: Burn down your GCP Commit and drive real value.
  • Data teams: The best of the simplicity with Keboola with all the perks of Google Cloud.
  • Partners: Enrich your existing Google Cloud Services by accessing Keboola through the Marketplace.

Example Use Cases

  • Seamless integration with BigQuery: Avoid architectural complexity by seamlessly deploying Keboola on top of your existing BigQuery infrastructure.
  • Rapid single-tenant deployment: Quickly deploy a dedicated, single-tenant environment on the Google Cloud platform while ensuring data security and isolation.
  • Streamlined data acquisition: Eliminate for manual intervention by seamlessly signing up for Keboola in the cloud through Google Marketplace.
  • ‌Data gravity: Whether you are migrating or consolidating data sets to your main DWH, Keboola can help.

External Datasets (General Availability)

Plug your existing Snowflake or BigQuery warehouse into Keboola in seconds.

What

External Data Access is particularly beneficial for clients who already utilize their own Snowflake or Big Query for data storage and wish to leverage Keboola's powerful data transformation capabilities without the time-consuming setup processes traditionally involved.

Who

  • Data Analysts, Scientists and business users: All data available quickly in Keboola for analyzing, prototyping, experimentation.
  • Data Engineers: No need to manually set up extractors to transfer the data.

Example Use Cases

  • Enhanced flexibility: A prime example of the feature’s utility is demonstrated when a client with a Snowflake or Big Query containing hundreds of gigabytes needs a swiftly connect data to Keboola and play with them. What used to be a potentially lengthy and complicated process can now be achieved in mere seconds.
  • Ease of use: Connect any Snowflake or Big Query DB scheme to Keboola Storage within seconds, bypassing the need to set up and run an extractor.

Bring your own database (General Availability)

Keep your existing data warehouse and natively integrate it with Keboola

What

Connect your Snowflake or Big Query data warehouse seamlessly. No time-consuming setup or migration needed. You can immediately work with your data in Keboola thanks to native integration.

Who

  • Data Analysts, Scientists and business users: All data available quickly in Keboola for analyzing, rapid prototyping, experimentation.
  • Data Engineers: No need to manually set up extractors to transfer the data.

Example Use Cases

  • Enhanced flexibility: You might have hundreds of gigabytes in Big Query or Snowflake and you can connect them to Keboola in seconds (!), ready to apply transformations you need.
  • Data Warehouse agnostic: Keep your data warehouse, whichever it is, and keep working with it. No need to change or migrate.
Share on: Twitter Linkedin Facebook