Databricks Summit, Model Context Layer, Salesforce Gate Slack API #88 w/e 13 June 2025
Join the 5,400-strong data herd getting all you need to know about Data for your Friday roundup
As predicted there were some massive announcements at DATBRICKS Summit this time. Oosht
🧩 Lakebase – Operational Database for AI-Native Apps
Databricks introduced Lakebase, a Postgres-compatible transactional database natively integrated with the lakehouse.
Why it matters: It collapses OLTP + OLAP into a single system. No more fragile ETL. You can now build AI-first, data-intensive apps (like chatbots and AI dashboards) on unified infrastructure. It’s Postgres—but with lakehouse superpowers.
🤖 Mosaic AI Agents + MLflow 3.0
Databricks launched Agent Bricks (Beta) and MLflow 3.0, forming the backbone of its end-to-end GenAI stack.
Why it matters: You can now build, evaluate, deploy, and monitor AI agents like a real software system. MLflow 3.0 brings prompt versioning, cross-platform model tracking, and deep observability—even for external deployments.
✨ AI/BI Genie – Natural Language Analytics (GA)
AI/BI Genie is now GA: a natural-language assistant for exploring and visualizing enterprise data.
Why it matters: This is ChatGPT for your BI—complete with traceable answers, visualizations, and upcoming features like Deep Research to generate hypotheses and cite sources. It’s self-service analytics done right.
🔄 Lakeflow – No-Code, Reliable Data Pipelines
Lakeflow is now generally available, offering visual ETL workflows and robust scheduling for modern data teams.
Why it matters: Anyone can build reliable, declarative pipelines without writing code. This unlocks data engineering workflows for analysts and ops teams who live in Excel, not Spark.
🧠Vector Search + Serverless GPU + AI Functions in SQL
Three major infrastructure upgrades for production AI:
Vector Search (Preview): Billion-scale RAG with 7× lower cost.
Serverless GPU Compute (Beta): Instant, infra-free access to A10G (and soon, H100s).
AI SQL Functions: Run AI tasks like
ai_parse_documentinside SQL—now 3× faster and 4× cheaper.
Why it matters: This trio powers real-time AI apps, search, and document intelligence, all from the lakehouse.
📦 Unity Catalog Upgrades + Databricks Apps (GA)
Unity Catalog just got enterprise-grade:
Metrics layer, Iceberg support, attribute-based access, and an internal data/AI marketplace.
Databricks Apps (GA): Now anyone can build governed, interactive apps right inside Databricks.
Why it matters: It’s not just about data or models anymore. It’s about building production apps—secure, governed, and powered by your lakehouse.
Orchestra Supports Dataform
You can now run Dataform Jobs in Orchestra.
Pretty neat. This is a great option for anyone looking to build a scalable architecture on GCP without having to use dbt or the hassle of Airflow Cloud Composer.
Check out our GCP Architecture here.
Medium ðŸ§
🧠Can AI Truly Develop a Memory That Adapts Like Ours? (link)
🧠Postgres Support in Orchestra (link)
🧠User Authorisation in Streamlit With OIDC and Google (link)
🧠Migrating from Teradata to the Databricks Lakehouse: How Data Modeling Evolves (link)
🧠Data Modeling on Databricks Lakehouse: A Real-World Scenario with Billion-Row Facts and Changing Dimensions (link)
🧠The Future of Compute: How Snowflake is Making Compute Faster, Easier, and More Price/Performant (link)
🧠The Future of Data Science: How AI Agents Are Revolutionizing ML Development (link)
LinkedIn🕴
🕴 Tracking Attribute Changes Over Time in BigQuery (link)
🕴 Shrink Your Data, Save Your Budget: BigQuery Tips (link)
🕴 Talking Autonomous Data Products with Zhamak Dehghani (link)
🕴 How Attackers Are Outsmarting Even Smart Devs (link)
🕴 Snowflake vs. Databricks: 2X Faster Data Masking (link)
🕴 How OM1 Simplified Healthcare Data with Snowflake (link)
Editor’s Pick
🕴Databricks runs about 3x more compute than Snowflake for similar performance (link)
News 📰
Editor’s Pick
📰 Databricks CEO says AI still in "explosion" phase (link)
📰 Vast Data Aiming for $25 Billion Valuation in New Funding Round (link)
📰 AI storage platform Vast Data sought funding at $25bn valuation earlier this year - report (link)
📰 Databricks continues M&A spree, will buy Neon for $1 billion in AI-agent push (link)
📰 Blend Announces $300M M&A Push to Power Next Era of Scalable, Specialized AI Solutions (link)
YouTube and Podcast 🎥
Editor’s Pick
🎥 Common Data Engineering Patterns: S3 Sensors in Orchestra (link)
🎥 Dremio MCP Server using a Langchain Based MCP Client (link)
🎥 A Tour of Dremio's Enterprise Features for Building Apache Iceberg Lakehouses (link)
🎥 Hybrid deployment for python and dbt using Orchestra on AWS ECS (link)
Special 💫
💫 Build Better Data Pipelines: Constructing and Orchestrating with SQL and Python in Snowflake (link)
💫 Databricks research report (link)
Jobs 💼
💼 Data Scientist - West Bend, Wisconsin at Delta Defense (link)
💼 Director of Data and Technology at Areté Rising (link)
💼 Senior Data Engineer at Billee Technologies (link)
💼 Data & Analytics Engineer at Backstage (link)
💼 Senior Data Analyst FTC at Genio (link)
Want to save on your ingestion bills? You’ll love this
You can leverage Python for lightweight ELT integrations. Here you’re only paying for compute and not being penalised by row-based pricing models. Pretty neat right? Check it out below / head to Orchestra and start today.
The best place to run dbt?
Don’t believe us? Watch the video below.
dbt, dbt core and dbt labs are all trademarks of dbt labs inc