Knowledge marts are now supported on Databricks

Knowledge mart tasks automate the transformation, vectorization and loading of data into a vector store to support custom semantic search and retrieval-augmented-generative (RAG) AI applications. Structured data can be curated into domain oriented documents that can be vectorized alongside unstructured documents for a variety of use cases. Previously knowledge marts only supported Snowflake as the data platform

Knowledge Mart and File-based Knowledge Mart tasks now support:

Delivery to vector stores:

• Databricks Vector Store — NEW!

• Snowflake Cortex

• ElasticSearch

• OpenSearch

• PineCone Vector Search

Embedding via customer-provided LLMs:

• Databricks LLMs –NEW!

• Snowflake Cortex. Required when Snowflake is your vector store.

• Amazon Bedrock

• Azure OpenAI

• OpenAI

Creating knowledge marts