Databricks releases, every cloud
- Incremental ingestion for Salesforce formula fields in Lakeflow Connect will soon be generally available Coming soon
Replaces full snapshot default, improving performance and reducing cost.
- Apache Arrow support for Zerobus Ingest is now generally available
Replaces JSON and Avro formats, requires Lakeflow Connect.
- HITRUST and IRAP compliance controls are now generally available
Available through compliance security profile, required for HIPAA, HITRUST, and IRAP.
- Zerobus Ingest into streaming tables is now generally available
Works like writing to a managed Delta table with same limits and quotas.
- Consumer access to query Unity Gateway services will become generally available Coming soon
Requires account admins to set budgets and rate limits beforehand to control model traffic.
- Salesforce connector mTLS authentication is now generally available
Replaces OAuth client credentials flow for API-only users, requires client certificates.
- Run the change, do not just read it. Hands-on labs in your own Databricks workspace, graded when you submit. Browse labs
- Role-based access control (RBAC) is generally available
Requires account and workspace admin setup. Enables exclusive access to sensitive data.
- JAR tasks on serverless compute are now generally available
Requires matching Scala, Java Development Kit, and Databricks Connect versions.
- Serverless compute access control is now generally available
Governs access to notebooks, jobs, and pipelines through Default Interactive and Automated Compute objects. Existing workloads run unchanged.
- Email notifications for expiring service principal tokens are now generally available
Sends emails to workspace admins 7 days before token expiration for used tokens with lifetime over 7 days. No configuration is required.
- Context-based ingress control for workspaces public access is now generally available
Uses allow and deny rules to control access based on identities, network sources, and request types.
- REPLACE WHERE flows are now generally available
Replaces full table reprocessing with row-level updates based on a predicate.
AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(not on this cloud) - Lakeflow Connect row filtering is now generally available
Works like a SQL WHERE clause, applies to initial and incremental updates.
AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(not on this cloud) - SQL alert task in Lakeflow Jobs is now generally available
Evaluates a Databricks SQL alert as part of a Lakeflow Job, returning its state for downstream tasks.
AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(not on this cloud) - AUTO CDC flows for streaming tables in Databricks SQL are generally available
Automatically handles out-of-order records as SCD type 1 or 2, replacing complex MERGE INTO logic.
AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(not on this cloud) - The update_flow API for Spark Declarative Pipelines on Lakeflow is generally available
Writes to a sink in update output mode, emitting only changed rows. Supports stateful aggregations without a watermark.
AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(not on this cloud) - Inbound Private Link for performance-intensive services is now generally available
Supports Lakebase Autoscaling and Zerobus Ingest.
- Query-based connectors for Lakeflow Connect are now generally available
Ingests data from databases like Oracle and PostgreSQL without CDC configuration.
AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(not on this cloud) - Get this table by email. One Monday mail covering the week, filtered to the clouds and products you run. Weekly digest
- Refresh policies and EXPLAIN CREATE MATERIALIZED VIEW for materialized views are now generally available
Requires Databricks Runtime 17.3 and above. Available in Databricks SQL.
AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(not on this cloud) - Automatic identity management with Microsoft Entra ID is now GA
Replaces SCIM provisioning, syncs users and groups from Microsoft Entra ID.
AWS, read it on their docsAzure(not on this cloud)GCP, read it on their docsSAP, read it on their docs - Unified runs list is generally available
Replaces separate job and pipeline run lists, adds filtering and error code visualization.
AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(not on this cloud) - Spark Declarative Pipelines on Lakeflow sinks are now generally available
Supports writing to Delta tables, Kafka topics, and Azure Event Hubs.
AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(not on this cloud) - Databricks SQL alerts are now generally available
Monitors data and KPIs by running scheduled queries, notifying recipients when conditions are met.
- Lakeflow Pipelines Editor is now generally available
Offers agent-first experience with Genie Code for creating production pipelines.
AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(not on this cloud) - Disabling tasks in Lakeflow Jobs is now GA
Disabled tasks retain config and run history, can be re-enabled later.
AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(not on this cloud) - Workspace base environments are now generally available
Lets workspace admins create pre-built environments for serverless notebooks, enabled by default in compliance security profile workspaces.
AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(not on this cloud) - C5, TISAX, and K-FSI compliance controls are now generally available
Available on Azure, providing enhancements for workspace compliance requirements.
- Scoped personal access tokens are now generally available
Restrict tokens to specific API operations with specified permissions. Coming soon to compliance security profile workspaces.
AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(not on this cloud) - Customer-managed keys for Unity Catalog are generally available
Encrypts data in Unity Catalog catalogs using cloud KMS keys. Requires default storage.
- Update workspace network configuration to VNet injection is now GA
Migrates from Databricks-managed VNet to customer-owned VNet.
- Azure virtual network service endpoint policies are now generally available
Apply to classic compute for outbound storage access filtering. Requires configuration.
- COMMENT ON streaming tables is now GA
Adds comments to streaming tables, previously only available for non-streaming tables.
- Enhanced Security and Compliance add-on is now generally available
includes compliance security profile and enhanced security monitoring.
- Serverless egress control is now generally available
Manages outbound connections with restricted access and FQDN filtering.
- Applying filters, masks, tags, and comments to pipeline-created datasets is now GA
Modifies ETL and ingestion pipelines using CREATE, ALTER, or Lakeflow UI. Applies to Spark Declarative Pipelines on Lakeflow and Lakeflow Connect.
- Serverless workspaces are now generally available
Come pre-configured with serverless compute and default storage.
- Updated Lakeflow Jobs UI is generally available
Simplifies job creation and editing, see Lakeflow Jobs for details.
- Spark Declarative Pipelines on Lakeflow stream progress metrics are generally available
Supports querying event log for stream progress metrics.
- GCP Private Service Connect generally available
No account team request needed to enable.
- Migrate Spark Declarative Pipelines on Lakeflow pipelines from legacy publishing mode is generally available
Replaces legacy mode that only published to a single catalog and schema. Now supports multiple catalogs and schemas.
Headlines, dates and product areas are Databricks' own, from the release notes published for each cloud, and every item links to the note it came from. The one-line summaries are ours. Not a Databricks product and not affiliated with Databricks, Inc.