At it's Data+AI Summit in San Francisco today, Databricks announced LakehouseIQ, a cutting-edge knowledge engine that provides natural language access to a company's unique business data. With LakehouseIQ, the company aims to democratize data to every employee.
As companies adopt generative AI technology, they are recognizing the need for models that are tailored to their own data, specific jargon, and org structure.
LakehouseIQ stands out by harnessing signals from across the Databricks Lakehouse platform, enabling it to understand a business's unique data environment. By reflecting how data is practically used within an organization, it fosters the development of accurate and relevant AI-powered tools that can enhance the performance of numerous natural language interfaces throughout the Databricks ecosystem.
The first feature utilizing LakehouseIQ will be an AI assistant within Databricks’ SQL Editor and Notebooks that can generate and explain queries, and answer questions about data. Databricks claims this assistant is already saving their users hundreds of hours.
LakehouseIQ will also enhance Databricks’ search functionality. Rather than just returning raw data, it interprets, aligns and presents data to help users start working faster.
Across various management and troubleshooting workflows, LakehouseIQ can suggest meaningful comments on data sets, and understand and debug jobs, data pipelines, SQL and Spark queries.
Importantly, Databricks is exposing LakehouseIQ's main capabilities through an API to allow integration into third party apps.
This approach simplifies the development of business-specific AI applications, allowing organizations to deploy custom solutions with a deep understanding of their unique business language and data practices.
Finally, in a world where data security and governance is vital, LakehouseIQ incorporates Databricks' Unity Catalog, the company's leading solution for security and governance. This integration ensures users have access only to datasets they're authorized to view, providing businesses with the confidence to democratize data synthesis and analysis without compromising security.
By enabling intuitive, contextual, natural language interactions with data at scale, LakehouseIQ makes data not just accessible, but intelligible, valuable, and primed for action.
Databricks plans to roll out LakehouseIQ features over the coming year, and is inviting customers to provide feedback as they strive to innovate and enhance the data experience for all.