Databricks releases, every cloud
- Databricks Documentation site table of contents tabs
Replaces static sidebar with tabbed navigation, includes 6 context tabs.
- Databricks Connector for Google Sheets updates
Longer query execution timeout and UI improvements added.
AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(not on this cloud) - Databricks JDBC driver 2.8.0 is available
Replaces Log4j with SLF4J 2.0.13, enables telemetry by default.
AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(not on this cloud) - Databricks SQL pipelines support notifications and performance mode (Beta)
Supports failure notifications and serverless performance mode for materialized views and streaming tables.
AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(not on this cloud) - Workday HCM connector (Beta)
Ingests Workday Human Capital Management data into Databricks.
AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(not on this cloud) - Change to use_case field in VPC endpoint API responses Coming soon
Changes from WORKSPACE_ACCESS to GENERAL_ACCESS, see API docs for details.
- Run the change, do not just read it. Hands-on labs in your own Databricks workspace, graded when you submit. Browse labs
- TikTok Ads connector (Beta)
Ingests data from TikTok Ads into Databricks.
- HubSpot connector (Beta)
Ingests HubSpot Marketing Hub data into Databricks via Lakeflow Connect.
- Google Ads connector (Beta)
Ingests data from Google Ads into Databricks via Lakeflow Connect.
- Select tables and create pivot tables in Google Sheets
Imports data via Databricks Connector, selecting from Catalog Explorer.
- Zendesk Support connector (Beta)
Ingests ticket, help center, and community forum data from Zendesk Support.
- Trigger on update for pipelines (GA)
Refreshes pipeline when source table changes, usable in Databricks SQL pipeline schedules.
- View and restore your recent workspace tab sessions
Restores up to 10 recent sessions.
- Google Drive connector (Beta)
Supports read_files, spark.read, COPY INTO, and Auto Loader for ingesting files.
- Ingest Salesforce formula fields incrementally (Beta)
Replaces snapshotting with incremental ingestion, improving performance and reducing costs.
- Row filtering for managed ingestion connectors (Beta)
Applies conditions like a SQL WHERE clause, available for Google Analytics, Salesforce, and ServiceNow connectors.
- Meta Ads connector (Beta)
Ingests data from Meta Ads, requires setup as a data source.
- Login required to download ODBC driver
Requires login and license acceptance, except for AWS GovCloud which needs account team access.
- Get this table by email. One Monday mail covering the week, filtered to the clouds and products you run. Weekly digest
- Confluence connector (Beta)
Ingests Confluence spaces and pages into Databricks, requires OAuth U2M configuration.
- Change owner for materialized views or streaming tables defined in Databricks SQL
Done through Catalog Explorer, replacing previous methods.
- Customizable SharePoint connector (Beta)
Ingests structured, semi-structured, and unstructured files into Delta tables with customizable schema inference and parsing options.
- Discover files in Auto Loader efficiently using file events
Requires file events enabled on external locations. Replaces directory listing for efficiency.
- Jira connector (Beta)
Ingests Jira issues, comments, and attachments metadata into Databricks.
- Configure full refresh behavior for managed database connectors
Applies to Lakeflow Connect managed database connectors, allowing scheduled full refresh snapshots and automatic recovery from unsupported schema changes.
- Jobs can now be triggered on source view update
Supports views referencing supported table types, see documentation for specifics.
- Improved full refresh flow for managed database connectors
Reduces downtime by delaying table refresh until snapshot completion, minimizing PENDING_RESET errors.
- Lakeflow Declarative Pipelines has been renamed to Lakeflow Spark Declarative Pipelines
Replaces Lakeflow Declarative Pipelines, see Spark Declarative Pipelines for details.
- JAR tasks are now supported on serverless compute
Run JAR jobs without a cluster.
- Salesforce ingestion connector supports history tracking (SCD type 2)
Enables history tracking, overwriting default SCD type 1 behavior.
- Jobs can now be triggered on source table update
Triggers require Databricks Runtime 7.0 or later.
- Create backfill job runs
Triggers job runs to load past data, useful for repairing processing failures.
- New requirement to create connections for Salesforce ingestion
Requires Databricks connected app installed. Applies to new connections only.
- Google Analytics Raw Data connector GA
Enables ingestion of raw Google Analytics data into Databricks, requires Google Analytics 4 and Google BigQuery setup.
- Python custom data sources can be used with Spark Declarative Pipelines on Lakeflow
Supports Python custom data sources and sinks in pipeline definitions.
- SQL Server connector supports SCD type 2
Maintains a history of changes, rather than overwriting outdated records.
- Microsoft SQL Server connector GA
Replaces manual ingestion scripts, requires Lakeflow Connect setup.
- Set the run-as user for Spark Declarative Pipelines on Lakeflow
Replaces user accounts with a service principal for automated workloads.
- Jobs in continuous mode can now have task-level retries for failed tasks
Task-level retries replace job-level retries, require continuous mode.
- ServiceNow connector GA
Replaces manual API configuration, requires Lakeflow Connect setup.
- Jobs & Pipelines list now includes Databricks SQL pipelines
Includes materialized views and streaming tables created with Databricks SQL.
Headlines, dates and product areas are Databricks' own, and every item links to the note it came from. The one-line summaries are ours.