What’s new in Databricks for April 2023


Delta Lake

  • For Delta Live tables The number of allowed notebooks per pipeline is increased to 100. The previous limit was 25 notebooks.
  • This release allows use of additional commands in a spark.sql(<command>) function, including DESC,DESCRIBE, SHOW TBLPROPERTIES, SHOW TABLES, SHOW NAMESPACES, SHOW COLUMNS IN, SHOW FUNCTIONS, SHOW CATALOGS, SHOW SCHEMAS, SHOW DATABASES, and SHOW CREATE TABLE.
  • Unity Catalog and Delta Live Tables Integration targeting ungated Public Preview for both SQL/Python. For more information : https://docs.databricks.com/delta-live-tables/unity-catalog.html


  • You can use the settings kafka.ssl.truststore.location and kafka.ssl.keystore.location to store Kafka certificates on external locations managed by Unity Catalog when you use structured Streaming on shared access clusters. ( Require DBR 13)
  • The new privileges USE SHARE, USE RECIPIENT, USE PROVIDER, and SET SHARE PERMISSION enable delegation of share, recipient, and provider management tasks that would otherwise require the metastore admin role or object ownership. These privileges enable you to limit the number of users who have the powerful metastore admin role. For more information: https://docs.databricks.com/data-governance/unity-catalog/manage-privileges/privileges.html#delta-sharing


Databricks SQL

  • Return text generated by a selected large language model (LLM) given the prompt with ai_generate_text. This function is only available as a public preview on Databricks SQL Pro and Serverless. 
  • The TIMESTAMP_NTZ type represents values comprising of fields year, month, day, hour, minute, and second. All operations are performed regardless of time zone. 

Laisser un commentaire

Votre adresse e-mail ne sera pas publiée. Les champs obligatoires sont indiqués avec *