What’s new in Databricks for September 2023




  • You can use Structured Streaming to perform streaming reads from views registered with Unity Catalog. For more information : https://docs.databricks.com/en/structured-streaming/views.html (DBR 14.1 required)
  • You can filter sensitive data with row filters and column masks. For more information https://www.youtube.com/watch?v=Ph32H5HTWlA&t=3s
  • System tables now include Marketplace schema and pricing information.
  • Data explorer is now Catalog Explorer
  • You can now delegate privileges for managing items in the allowlist for Unity Catalog. For more information https://docs.databricks.com/en/data-governance/unity-catalog/manage-privileges/allowlist.html

Databricks SQL

  • Lakeview Dashboards allow you to share visualizations and datasets using a simplified and unified data governance model. For more information https://docs.databricks.com/en/dashboards/lakeview.html
  • New charts are now available, featuring faster render performance, beautiful colors, and improved interactivity.
  • In the graph view of Query Profile, you can now view the Join type on any node containing a join in the query plan.

Machine learning

  • Inference tables for model serving endpoints is public preview. You can capture incoming and outgoing responses to those endpoints and log them as a Unity Catalog delta table. For more information : https://docs.databricks.com/en/machine-learning/model-serving/inference-tables.html
  • GPU Model serving in public preview.
  • On demand Feature computation is now in Unity Catalog is now in public preview. For more information https://docs.databricks.com/en/machine-learning/feature-store/on-demand-features.html

Delta Lake

  • Row-level concurrency reduces conflicts between concurrent write operations by detecting changes at the row-level and automatically resolving competing changes in concurrent writes that update or delete different rows in the same data file. For more information https://docs.databricks.com/en/optimizations/isolation-level.html#row-level-concurrency

Partner connect

  • You can use Partner connect to connect your databricks workspace to Snowplow.

Laisser un commentaire

Votre adresse e-mail ne sera pas publiée. Les champs obligatoires sont indiqués avec *