Databricks releases, every cloud
- Data Classification now scans Unity Catalog views (Beta)
Scans Unity Catalog views in addition to tables, using the same detection engine, and is currently in Beta.
- Data Classification is now available by default for additional compliance standards
Requires compliance security profile with C5, TISAX, or IRAP controls.
- Data Classification is now available by default for some workspaces with the compliance security profile enabled
Requires compliance security profile and specific controls like HIPAA or HITRUST.
- Reorganized Spark Declarative Pipelines on Lakeflow documentation
Conceptual topics are now under Concepts.
AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(not on this cloud) - Databricks Data Classification is now GA
Automatically classifies and tags sensitive data in Unity Catalog using AI.
AWS, read it on their docsAzure, read it on their docsGCP, read it on their docsSAP(not on this cloud) - Databricks Documentation site table of contents tabs
Replaces static sidebar with tabbed navigation, includes 6 context tabs.
- Run the change, do not just read it. Hands-on labs in your own Databricks workspace, graded when you submit. Browse labs
- File events enabled by default on new external locations
Enabled on new locations, detects file changes for storage jobs and pipelines, can be disabled after creation.
- Discover files in Auto Loader efficiently using file events
Requires file events enabled on external locations. Replaces directory listing for efficiency.
- SFTP connector (Public Preview)
Extends Auto Loader to ingest files from SFTP servers.
- Data Classification (Public Preview)
Supports all catalog types, consolidates results into a system table.
- Data classification system table Beta
Captures sensitive data detections at column level across enabled catalogs.
- Discover files in Auto Loader efficiently using file events without enrollment (Public Preview)
Requires file events enabled on external locations.
Headlines, dates and product areas are Databricks' own, and every item links to the note it came from. The one-line summaries are ours.