Skip to content

Databricks releases, every cloud

All GAPreviewBehavior changeDeprecationUpdate

157 releases of 1321

Last updated

  1. Change owner for materialized views or streaming tables defined in Databricks SQL

    Done through Catalog Explorer, replacing previous methods.

    #Lakeflow
    AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(unknown)
  2. Customizable SharePoint connector (Beta)

    Ingests structured, semi-structured, and unstructured files into Delta tables with customizable schema inference and parsing options.

    #Data ingestion#Lakeflow
    AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(unknown)
  3. Discover files in Auto Loader efficiently using file events

    Requires file events enabled on external locations. Replaces directory listing for efficiency.

    #Auto Loader#Lakeflow
    AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(unknown)
  4. Jira connector (Beta)

    Ingests Jira issues, comments, and attachments metadata into Databricks.

    #Data Ingestion#Lakeflow
    AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(unknown)
  5. Microsoft Dynamics 365 connector (Public Preview)

    Ingests data from Dynamics 365 apps into Databricks. Requires Lakeflow Connect.

    #Data ingestion#Lakeflow
    AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(unknown)
  6. NetSuite connector (Public Preview)

    Ingests data from NetSuite2.com via API, CLI, or notebook.

    #Lakeflow
    AWS, read it on their docsAzure, read it on their docsGCP(not on this cloud)SAP(unknown)
  7. Run the change, do not just read it. Hands-on labs in your own Databricks workspace, graded when you submit. Browse labs
  8. ForEachBatch for Spark Declarative Pipelines on Lakeflow is available (Public Preview)

    Allows processing streams as micro-batches in Python. Requires Lakeflow setup.

    #Lakeflow
    AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(unknown)
  9. Spark Declarative Pipelines on Lakeflow stream progress metrics are generally available

    Supports querying event log for stream progress metrics.

    #Lakeflow
    AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(unknown)
  10. Configure full refresh behavior for managed database connectors

    Applies to Lakeflow Connect managed database connectors, allowing scheduled full refresh snapshots and automatic recovery from unsupported schema changes.

    #Lakeflow
    AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(unknown)
  11. Jobs can now be triggered on source view update

    Supports views referencing supported table types, see documentation for specifics.

    #Lakeflow
    AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(unknown)
  12. Improved full refresh flow for managed database connectors

    Reduces downtime by delaying table refresh until snapshot completion, minimizing PENDING_RESET errors.

    #Lakeflow
    AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(unknown)
  13. SFTP connector (Public Preview)

    Extends Auto Loader to ingest files from SFTP servers.

    #Auto Loader#Lakeflow
    AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(unknown)
  14. Lakeflow Declarative Pipelines has been renamed to Lakeflow Spark Declarative Pipelines

    Replaces Lakeflow Declarative Pipelines, see Spark Declarative Pipelines for details.

    #Lakeflow
    AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(unknown)
  15. JAR tasks are now supported on serverless compute

    Run JAR jobs without a cluster.

    #Lakeflow
    AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(unknown)
  16. Salesforce ingestion connector supports history tracking (SCD type 2)

    Enables history tracking, overwriting default SCD type 1 behavior.

    #Lakeflow
    AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(unknown)
  17. Zerobus Ingest connector in Lakeflow Connect (Public Preview)

    Enables record-by-record ingestion into Delta tables via gRPC API.

    #Lakeflow
    AWS, read it on their docsAzure, read it on their docsGCP(not on this cloud)SAP(unknown)
  18. Unified runs list (Public Preview)

    Combines job and pipeline runs into one list.

    #Lakeflow
    AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(unknown)
  19. Jobs can now be triggered on source table update

    Triggers require Databricks Runtime 7.0 or later.

    #Lakeflow
    AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(unknown)
  20. Get this table by email. One Monday mail covering the week, filtered to the clouds and products you run. Weekly digest
  21. Create backfill job runs

    Triggers job runs to load past data, useful for repairing processing failures.

    #Lakeflow
    AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(unknown)
  22. Lakeflow Pipelines Editor is now in public preview

    Replaces multi-file editor, edits pipelines as files in asset browser, defaults to Python and SQL code.

    #Lakeflow
    AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(unknown)
  23. New requirement to create connections for Salesforce ingestion

    Requires Databricks connected app installed. Applies to new connections only.

    #Lakeflow
    AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(unknown)
  24. Migrate Spark Declarative Pipelines on Lakeflow pipelines from legacy publishing mode is generally available

    Replaces legacy mode that only published to a single catalog and schema. Now supports multiple catalogs and schemas.

    #Lakeflow
    AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(unknown)
  25. Discover files in Auto Loader efficiently using file events without enrollment (Public Preview)

    Requires file events enabled on external locations.

    #Auto Loader#Lakeflow
    AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(unknown)
  26. Google Analytics Raw Data connector GA

    Enables ingestion of raw Google Analytics data into Databricks, requires Google Analytics 4 and Google BigQuery setup.

    #Lakeflow
    AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(unknown)
  27. Python custom data sources can be used with Spark Declarative Pipelines on Lakeflow

    Supports Python custom data sources and sinks in pipeline definitions.

    #Lakeflow
    AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(unknown)
  28. Spark Declarative Pipelines on Lakeflow now supports stream progress metrics in Public Preview

    Supports querying event log for stream progress metrics.

    #Lakeflow
    AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(unknown)
  29. Migrate Spark Declarative Pipelines on Lakeflow from legacy publishing mode is rolled back to Public Preview

    Reverts migration feature to Public Preview due to issues. Requires re-enabling default publishing mode in pipelines.

    #Lakeflow
    AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(unknown)
  30. SQL Server connector supports SCD type 2

    Maintains a history of changes, rather than overwriting outdated records.

    #Lakeflow
    AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(unknown)
  31. Migrate Spark Declarative Pipelines on Lakeflow from legacy publishing mode is now GA

    Replaces legacy single-catalog publishing with multi-catalog support.

    #Lakeflow
    AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(unknown)
  32. Microsoft SQL Server connector GA

    Replaces manual ingestion scripts, requires Lakeflow Connect setup.

    #Lakeflow
    AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(unknown)
  33. Set the run-as user for Spark Declarative Pipelines on Lakeflow

    Replaces user accounts with a service principal for automated workloads.

    #Lakeflow
    AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(unknown)
  34. Spark Declarative Pipelines on Lakeflow template in bundles in the workspace (Public Preview)

    Creates ETL pipelines in workspace bundles using Lakeflow template. Requires New ETL pipeline setup.

    #Bundles#Lakeflow
    AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(unknown)
  35. Jobs in continuous mode can now have task-level retries for failed tasks

    Task-level retries replace job-level retries, require continuous mode.

    #Lakeflow
    AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(unknown)
  36. ServiceNow connector GA

    Replaces manual API configuration, requires Lakeflow Connect setup.

    #Lakeflow
    AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(unknown)
  37. Jobs & Pipelines list now includes Databricks SQL pipelines

    Includes materialized views and streaming tables created with Databricks SQL.

    #Databricks SQL#Lakeflow
    AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(unknown)
  38. Moving tables between Spark Declarative Pipelines on Lakeflow pipelines is now GA

    Supports moving tables in Unity Catalog ETL pipelines.

    #Lakeflow
    AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(unknown)
  39. Parent tasks (Run job and For each) now have a separate limit

    Separate limit doesn't count against overall limit, see Resource limits.

    #Lakeflow
    AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(unknown)

Headlines, dates and product areas are Databricks' own, and every item links to the note it came from. The one-line summaries are ours.

We measure how the site is used. PostHog and Google Analytics, set to measurement only with advertising features off. We do not sell your data or share it for advertising, and you can turn this off. What this stores.