Databricks releases, every cloud
- Dedicated group clusters will soon be available by default for workspaces with the compliance security profile enabled Coming soon
Enables shared compute for groups with compliance security profile, supporting Databricks Runtime for ML and RDD APIs.
AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(not on this cloud) - Object metadata column is now in Public Preview
Exposes object properties like MIME type and tags, available in Databricks Runtime 18.1.
AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(not on this cloud) - MySQL integrated CDC pipeline is now in Beta
Replaces separate ingestion gateway, requires single pipeline update.
AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(not on this cloud) - AUTO CDC flows for streaming tables in Databricks SQL are generally available
Automatically handles out-of-order records as SCD type 1 or 2, replacing complex MERGE INTO logic.
AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(not on this cloud) - Lakehouse Replay (Beta)
Automatically replays serverless workloads against upcoming runtime releases. Requires no setup.
AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(not on this cloud) - Schedule deferred policy enforcement for all-purpose compute
Applies on next restart or immediately, and admins can cancel scheduled enforcements.
AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(not on this cloud) - Run the change, do not just read it. Hands-on labs in your own Databricks workspace, graded when you submit. Browse labs
- Instance events and instance pools system tables are now available (Public Preview)
Track instance state transitions and instance pool configurations via system.compute.instance_events and system.compute.instance_pools tables.
AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(not on this cloud) - Databricks Container Services for standard compute is now available (Beta)
Requires Databricks Runtime 18.3 or later, allows custom Docker images in shared compute.
AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(not on this cloud) - Spark Declarative Pipelines on Lakeflow updates are retained for 60 days
Retention period increased from 30 to 60 days, applying to UI and REST API updates.
AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(not on this cloud) - Compute log delivery to volumes is now GA
Delivers Spark driver, worker, and event logs to Unity Catalog volumes, recommended for log storage.
AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(not on this cloud) - SQL alert task for jobs (Beta)
Evaluates SQL alerts as part of workflows, enabling automated condition checks and notifications.
AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(not on this cloud) - Serverless compute is enabled by default
Eligible Google Cloud workspaces are affected, account admins no longer need to manually enable it.
Headlines, dates and product areas are Databricks' own, from the release notes published for each cloud, and every item links to the note it came from. The one-line summaries are ours. Not a Databricks product and not affiliated with Databricks, Inc.