Databricks Expands Mosaic AI with New Tools for Enterprise LLMs

A year after acquiring MosaicML for $1.3 billion, Databricks has rebranded the platform as Mosaic AI, making it central to its AI solutions. At the company's Data + AI Summit, Databricks announced several new features for Mosaic AI, aimed at enhancing enterprise capabilities with large language models (LLMs).

New Mosaic AI Tools Unveiled

Databricks introduced five new tools under the Mosaic AI umbrella:

Mosaic AI Agent Framework
Mosaic AI Agent Evaluation
Mosaic AI Tools Catalog
Mosaic AI Model Training
Mosaic AI Gateway

These tools aim to address key concerns such as improving model quality, cost efficiency, and data privacy.

Key Insights from Databricks Leaders

During the summit, Databricks co-founders CEO Ali Ghodsi and CTO Matei Zaharia discussed the significance of these tools. Ghodsi emphasized the focus on enhancing model reliability, cost efficiency, and data privacy. Zaharia highlighted the need for modular systems in deploying LLMs, allowing enterprises to integrate multiple components and tools for optimized performance.

AI Agent Framework and Tools Catalog

The Mosaic AI Agent Framework allows developers to build Retrieval-Augmented Generation (RAG) applications using Databricks' serverless vector search functionality. The AI Tools Catalog provides a comprehensive set of tools for developing AI applications, integrating classic keyword-based search with embedding search.

Enhanced Governance with Unity Catalog

Databricks has extended its Unity Catalog to help enterprises govern AI tools and functions that LLMs can access. This ensures compliance and enhances discoverability of services across an organization. The Unity Catalog is also being gradually open-sourced.

Building Custom AI Solutions

Developers can use Mosaic AI tools to build custom agents by chaining models and functions using tools like Langchain or LlamaIndex. Zaharia noted that many Databricks customers are already leveraging these tools for their AI applications.

Evaluating AI Performance

To ensure the effectiveness of new AI applications, Databricks introduced the Mosaic AI Agent Evaluation tool. This combines LLM-based judges with user feedback to assess AI performance. The evaluation tool includes a UI component from Databricks' acquisition of Lilac, enabling users to visualize and search large text datasets.

Fine-Tuning AI Models

Databricks now offers the Mosaic AI Model Training service, allowing users to fine-tune models with their organization's private data. This customization improves model performance for specific tasks.

Unified Interface with AI Gateway

The Mosaic AI Gateway provides a unified interface for querying, managing, and deploying both open-source and proprietary models. This tool includes a centralized credentials store, rate limits for cost management, and usage tracking for debugging.

Evolving Market Trends

Ghodsi noted a shift in the market, with customers becoming more sophisticated in their use of open models. This has necessitated new tools to address the challenges and opportunities presented by advanced AI technologies.

Conclusion

Databricks' expansion of Mosaic AI with new tools underscores its commitment to empowering enterprises with advanced AI capabilities. The integration of these tools aims to enhance model reliability, cost efficiency, and data privacy, paving the way for the future of mission-critical AI applications.