DAILY BRIEFING · WEDNESDAY, JUNE 17, 2026
Databricks' Data + AI Summit dominates the week — Lakehouse//RT, LTAP, and a wave of Unity Catalog AI-governance launches push the lakehouse toward a single governed substrate for operational, analytical, and agentic workloads, while Collibra, Pinecone, dbt, and NewCore race to wrap that substrate in context, retrieval, and agent-grade trust.
⇣ Jump To
CDC · ELT/ETL Ingestion · Transformation Frameworks
Lakehouses · Table Formats · Architectural Patterns
Semantic Layers & Retrieval · Enterprise RAG & Retrieval · BI & Analytics
Catalogs & Metadata · Governance, Security & Compliance · FinOps for Data
⚡ QUICK TAKES
| Story | Signal |
|---|---|
| ↗ Lakebase Change Data Feed lands | CDC moves inside the lakehouse — fewer hops between OLTP writes and analytical reads. |
| ↗ Lakeflow Connect adds SharePoint connector | Managed ingestion expands to unstructured sources — document corpora become first-class inputs. |
| ↗ dbt Core v2.0 ships the Fusion engine | State-aware transforms make incremental the default — compute scales with change, not DAG size. |
| ↗ Databricks unveils Lakehouse//RT | The serving tier folds into the lakehouse — one governed copy for analytics and low-latency reads. |
| ↗ Iceberg v3 hits public preview on Databricks | Iceberg v3 erases the Delta-vs-Iceberg feature gap — format choice stops dictating performance. |
| ↗ Databricks launches LTAP architecture | Operational and analytical planes merge — the ETL seam between OLTP and OLAP is designed away. |
| ↗ Unity Catalog Business Semantics goes GA, open source | The semantic layer moves into the catalog — one governed metric definition for BI and agents. |
| ↗ Pinecone Nexus compiles context for agents | Retrieval shifts left to a compile step — agents read prebuilt context, not raw vectors. |
| ↗ Sigma becomes Lakehouse//RT launch partner | BI plugs straight into the real-time lakehouse — the extract/cube layer starts disappearing. |
| ↗ Unity Catalog now governs models, agents, MCP | The catalog absorbs AI assets — agents and MCP services become governed objects beside tables. |
| ↗ Collibra and Databricks deepen governance tie | Federated governance persists — UC owns the platform, Collibra brokers cross-platform context. |
| ↗ Unity AI Gateway governs inference calls | Inference joins the governed perimeter — model/agent calls get query-grade policy and audit. |
| ↗ NewCore raises $66M for agent identities | Agent identity becomes its own security primitive — credentials scoped per agent, not per service. |
| ↗ Unravel ships autonomous cost optimization | Data FinOps goes closed-loop — optimization acts on spend automatically rather than just flagging it. |
Databricks Blog · June 2026
Databricks added a native Change Data Feed to Lakebase, its serverless Postgres-on-object-storage layer, so row-level inserts, updates, and deletes are emitted as a consumable stream without bolting on Debezium or a standalone CDC connector. Paired with the new LTAP architecture, those change events land in the same governed lakehouse storage that analytics and agents already read. For platform teams it collapses the operational-to-analytical replication path that historically demanded Kafka plus a separate CDC tool.
✍️ Databricks · Read article →
Microsoft Community Hub · June 2026
Among the Summit updates, Lakeflow Connect gained a managed SharePoint connector (Beta), extending point-and-click ingestion from enterprise apps and unstructured document stores directly into Delta tables. Lakeflow Connect now spans 100+ managed connectors alongside the open Airbyte and Debezium libraries. The clear intent is to make unstructured-content ingestion a first-class fuel for agentic retrieval, not just structured application data.
✍️ Microsoft / Azure Databricks · Read article →
Data Engineer Things · June 2026
dbt Core v2.0 ships the open-source, Rust-based Fusion engine, which fingerprints model state and skips unchanged nodes — turning a 90-minute DAG into roughly five minutes when only a leaf node has new data, with a reported 10%+ cut in enterprise cloud-compute spend. Fusion also performs column-level lineage during clone and preview. With Fivetran's dbt Labs acquisition now closed as of June 1, Fusion is the transformation centerpiece of the merged ingestion-plus-transformation stack.
✍️ Ana Escobar, Data Engineer Things · Read article →
BigDATAwire · June 2026
Lakehouse//RT, unveiled at Summit and powered by a new compute engine called Reyden, delivers sub-100ms latency at 12,000 queries/sec directly on governed Delta and Iceberg tables — up to 16x faster than bolting a separate real-time serving store onto the lakehouse. Every query runs inside Unity Catalog with no proprietary format, no CDC sync, and no second permissions layer. It targets the agentic-app concurrency wall where analytical stores have traditionally handed off to Redis- or Pinot-style serving tiers.
✍️ BigDATAwire · Read article →
Databricks Blog · June 2026
Databricks put Apache Iceberg v3 into public preview, bringing deletion vectors, row lineage, and the VARIANT type natively to Iceberg tables — so teams get Delta-class performance features without giving up Iceberg engine compatibility. The bet is a single physical copy of data that every engine reads, killing replication pipelines between format camps. It lands the same fortnight Snowflake's Horizon Catalog reached Iceberg v3 GA, meaning convergence is now happening across both major platforms at once.
✍️ Databricks · Read article →
Databricks · June 2026
LTAP unifies transactions, analytics, streaming, and operational data on a single copy of lake storage by combining Lakebase — serverless Postgres on open object storage — with the Lakehouse under one Unity Catalog governance model. The pitch is a single governed foundation to "read, reason, and act on" without pipelines, replicas, or ETL overhead. It's an explicit shot at the HTAP and operational-versus-analytical split that has defined data architecture for a decade.
✍️ Databricks · Read article →
Databricks Blog · June 2026
Databricks made Unity Catalog Business Semantics generally available and open-sourced the spec, putting governed metric and dimension definitions in the catalog itself so BI tools and AI agents query the same certified semantics rather than re-deriving them per tool. It's a direct bid to be the semantic layer for both dashboards and agentic consumption. Open-sourcing the format is aimed squarely at the Cube, dbt Semantic Layer, and MetricFlow fragmentation problem.
✍️ Databricks · Read article →
Pinecone · June 2026
Pinecone's Nexus replaces the classic embed-and-retrieve RAG pipeline with a context compiler that pre-builds task-specific knowledge artifacts per agent, plus a composable retriever (KnowQL) that formats context for how each agent needs it. Pinecone's own benchmarks claim task completion above 90%, completion up to 30x faster, and token spend cut by as much as 90% versus standard RAG. Early access is open for regulated, knowledge-dense domains such as financial services, healthcare, and legal.
✍️ Pinecone · Read article →
Business Wire · June 2026
Sigma was named Databricks' BI Partner of the Year and signed on as a launch partner for Lakehouse//RT, wiring its spreadsheet-style interface directly onto the new millisecond real-time lakehouse. The architectural takeaway for platform teams: BI can now sit on the governed lakehouse at interactive latency without a separate extract or serving cube. It follows Sigma's recent $80M raise and pivot toward "agentic analytics."
✍️ Sigma Computing (Business Wire) · Read article →
Databricks Blog · June 2026
Unity Catalog now registers and governs models, MCP services, agents, and skills alongside tables — with the same access controls, lineage, discovery, and audit applied to data assets. The move positions the catalog as the control plane for agentic infrastructure, not just datasets. For governance engineers it means agent permissions and data permissions finally share a single policy model rather than living in parallel systems.
✍️ Databricks · Read article →
BigDATAwire · June 2026
Databricks named Collibra its Governance Partner of the Year and launched a bi-directional Unity Catalog integration that pushes Collibra's business context — sensitivity, ownership, policies — into Databricks across Genie and Agent Bricks. The integration builds on Collibra's recently launched AI Command Center for governing every agent across the enterprise. The signal: even as Unity Catalog expands, large enterprises still want a vendor-neutral governance plane sitting above the platform.
✍️ BigDATAwire · Read article →
Databricks Blog · June 2026
The Unity AI Gateway adds centralized policy, guardrails, and traceability for model and agent calls routed through the platform, extending data access controls to inference. Combined with Lakehouse//RT and LTAP, the theme is one governance boundary spanning data, models, and agent actions. It's Databricks' answer to the agent-observability and AI-policy gap practitioners have been stitching together by hand.
✍️ Databricks · Read article →
SiliconANGLE · June 2026
NewCore emerged with a $66M seed (Cyberstarts, Index Ventures, Evolution Equity) and a GA platform that issues verifiable, security-first identities to AI agents, so agent actions can be authenticated, scoped, and audited like human or service identities. It targets the gap where agents inherit broad credentials and leave no traceable identity trail. The raise underscores how quickly agent identity and access is becoming its own governance category for data teams enabling agentic access.
✍️ SiliconANGLE · Read article →
SiliconANGLE · May 2026
Unravel shipped an autonomous optimization engine that profiles workloads across Databricks, Snowflake, and BigQuery and applies cost and performance fixes — tuning, sizing, and query rewrites — without waiting for an engineer to act on a recommendation. It pushes data FinOps from advisory dashboards toward closed-loop remediation. The cross-platform scope is the differentiator as warehouse and AI-workload spend keeps climbing.
✍️ SiliconANGLE · Read article →