Daily Briefing — Wednesday, November 18, 2026

Data & AI Platforms Briefing

Ignite week pulls the platforms toward agent-ready plumbing: CDC and ingestion fold into the lakehouse, retrieval and reranking land in warehouse SQL, and catalogs start governing agent skills and MCP access alongside the data itself.

⇣ Jump To

Click any section or topic below to jump to it.

Move & Transform

Streaming & Messaging ·  CDC ·  ELT/ETL Ingestion ·  Transformation Frameworks

Store & Architect

Cloud Data Warehouses ·  Table Formats ·  Architectural Patterns ·  Vector & Specialty Stores

Consume & Activate

AI-Driven Consumption ·  Semantic Layers & Retrieval ·  Enterprise RAG & Retrieval

Govern & Operate

Catalogs & Metadata ·  Governance, Security & Compliance

⚡ Quick Takes

Story Signal
↗  Confluent Cloud adds full-record backup and restore to its Azure Blob Storage connectors Topic backup to object storage is becoming a first-class connector mode, not a DIY job
↗  Client-side field-level encryption with Azure Key Vault now works over Private Link Field-level encryption on streams is being fitted to private-network-only enterprise estates
↗  Databricks puts an integrated PostgreSQL CDC pipeline into Beta Lakehouse vendors keep absorbing CDC into the platform, squeezing standalone replication tools
↗  Snowflake Openflow deployments go GA in Azure Government regions Native ingestion is following warehouses into sovereign and government clouds
↗  Snowflake ships a GA Spark Monitoring UI for Snowpark Connect workloads Snowflake is building the operator tooling needed to win Spark workloads, not just run them
↗  BigQuery adds a GA REST API for cross-region dataset replication and failover reservations Warehouse disaster recovery is moving into infrastructure-as-code
↗  Cortex AI Function Studio and AI Function Optimization reach general availability In-warehouse AI is gaining the eval and cost-tuning tooling that production pipelines need
↗  BigQuery previews Apache Iceberg views through its Lakehouse runtime catalog REST endpoint Iceberg interoperability is moving up from tables to views
↗  Microsoft launches Azure HorizonDB GA and previews a serverless Fabric HorizonDB at Ignite Translytical Postgres inside the analytics platform is now table stakes for the big three
↗  LanceDB raises $57.5M for its multimodal data lakehouse Multimodal AI training data is creating demand for a table format beyond Parquet-era assumptions
↗  Dell puts a single gateway between enterprise AI agents and the data they need MCP gateways are becoming the default control point for agent-to-data access
↗  Fabric IQ goes GA in Copilot, with Excel access in preview Semantic models are becoming the agent context layer, not just a BI artifact
↗  BigQuery makes AI.RANK and AI.SEARCH reranking GA, plus an optimized mode for AI.IF and AI.CLASSIFY Retrieval and reranking are collapsing into warehouse SQL
↗  Unity Catalog agent skills can now be shared through OpenSharing and Databricks Marketplace Catalogs are starting to govern agent capabilities as shareable assets
↗  Governed tag propagation extends to classic compute Tag-based access control only works if propagation covers every compute path
↗  Ataccama named sole Visionary in the 2026 Gartner Magic Quadrant for D&A Governance Platforms The new governance MQ is out; agentic stewardship is the headline pitch
↗  Snowflake makes SHOW GRANTS ... INCLUDE HIERARCHY generally available Effective-privilege visibility is becoming a native query, not a script
↗  Snowflake previews SCIM provisioning at the organization level Identity governance is moving up to the organization tier in multi-account platforms
↗  Microsoft Ignite Day 1: MCP Server for Enterprise GA and Purview network DLP Security and compliance metadata is being exposed to agents through MCP by default

Move & Transform

› Streaming & Messaging

Confluent Cloud Release Notes — November 2026

Confluent Cloud adds full-record backup and restore to its Azure Blob Storage connectors

The Azure Blob Storage Sink and Source connectors gain a BACKUP_FULL_RECORD / RESTORE_BACKUP_FULL_RECORD mode that preserves the whole Kafka record rather than just the value payload. That turns object storage into a credible topic-level backup and replay target on Azure, which matters for DR runbooks and for rehydrating topics into new clusters without custom tooling.

✍️ Confluent · Read article →

Confluent Cloud Release Notes — November 2026

Client-side field-level encryption with Azure Key Vault now works over Private Link

Confluent Cloud now supports CSFLE using Azure Key Vault reached over Azure Private Link, in Limited Availability and opt-in through the account team. For regulated Azure shops, it closes the gap where field-level encryption required a public KMS endpoint, keeping key traffic on the private network end to end.

✍️ Confluent · Read article →

› CDC

Databricks Release Notes — November 2026

Databricks puts an integrated PostgreSQL CDC pipeline into Beta

Lakeflow Connect can now ingest PostgreSQL change data capture without deploying a separate ingestion gateway, per the November 15 release note. Removing the gateway hop simplifies the operational footprint for Postgres-to-lakehouse replication and puts Databricks in more direct competition with Debezium-based and Fivetran HVR-style pipelines.

✍️ Databricks · Read article →

› ELT/ETL Ingestion

Snowflake Release Notes — November 2026

Snowflake Openflow deployments go GA in Azure Government regions

Openflow, Snowflake's managed NiFi-based ingestion service, is now generally available for Snowflake-hosted deployments in Azure Government. Public-sector teams that were blocked on FedRAMP-style region requirements can now run ingestion inside the platform rather than staging data through third-party ELT tools.

✍️ Snowflake · Read article →

› Transformation Frameworks

Snowflake Release Notes — November 2026

Snowflake ships a GA Spark Monitoring UI for Snowpark Connect workloads

Snowsight now offers a generally available monitoring UI for Spark batch jobs and sessions running through Snowpark Connect for Spark. Teams migrating PySpark code onto Snowflake compute get Spark-style job visibility in place, which removes one of the practical objections to lifting Spark transforms off dedicated clusters.

✍️ Snowflake · Read article →

↑ Top

 

Store & Architect

› Cloud Data Warehouses

BigQuery Release Notes — November 2026

BigQuery adds a GA REST API for cross-region dataset replication and failover reservations

Cross-region dataset replication and failover reservations can now be managed through datasets.insert, patch, update and get in the BigQuery REST API, now generally available. That makes warehouse DR configurable as code through Terraform or internal platform tooling instead of console-driven setup.

✍️ Google Cloud · Read article →

Snowflake Release Notes — November 2026

Cortex AI Function Studio and AI Function Optimization reach general availability

Snowflake made Cortex AI Function Studio GA for creating, evaluating and optimizing AI functions, alongside a separate GA AI Function Optimization feature aimed at more efficient execution. Together they give platform teams an eval-and-tune loop for SQL-callable LLM functions, which is where token cost and output quality are actually controlled once AI runs inside pipelines.

✍️ Snowflake · Read article →

› Table Formats

BigQuery Release Notes — November 2026

BigQuery previews Apache Iceberg views through its Lakehouse runtime catalog REST endpoint

Iceberg views can now be created in Google's Lakehouse runtime catalog via the Iceberg REST endpoint and queried from Spark and Trino, in Preview. Views are the piece of the Iceberg spec that most catalogs still handle poorly, so engine-portable view definitions are a meaningful step toward a genuinely shared semantic surface across engines.

✍️ Google Cloud · Read article →

› Architectural Patterns

Microsoft Tech Community — November 2026

Microsoft launches Azure HorizonDB GA and previews a serverless Fabric HorizonDB at Ignite

Azure HorizonDB, Microsoft's PostgreSQL-compatible database with built-in vector search, is now GA, while Fabric HorizonDB enters preview as a serverless variant that provisions in seconds, pauses after five minutes idle, and pushes analytical queries down to Fabric Data Warehouse. Microsoft also made the Cosmos DB semantic reranker GA and previewed automatic embedding generation, reinforcing the push to keep operational, analytical and vector workloads on one data estate.

✍️ Shireesh Thota, Microsoft · Read article →

› Vector & Specialty Stores

SiliconANGLE — November 2026

LanceDB raises $57.5M for its multimodal data lakehouse

LanceDB raised $57.5 million led by Bessemer Venture Partners and Innovation Endeavors, bringing total funding to $98.5 million. Built on the open-source Lance format and querying data in place in customer cloud storage, it reports tables of 10 to 20 PB in large training workflows, with Runway, Netflix and Microsoft named as users.

✍️ Kyt Dotson, SiliconANGLE · Read article →

↑ Top

 

Consume & Activate

› AI-Driven Consumption

SiliconANGLE — November 2026

Dell puts a single gateway between enterprise AI agents and the data they need

Dell introduced an MCP-backed Agentic Data Gateway as part of its AI Data Platform, claiming agent answer accuracy rose from roughly 60% to about 94% in its own testing. The figure is vendor-reported from a Dell-sponsored theCUBE segment, but the pattern of a governed MCP layer brokering agent access to enterprise data is spreading beyond the cloud data platforms.

✍️ Ryan Stevens, SiliconANGLE (Dell-sponsored segment) · Read article →

› Semantic Layers & Retrieval

Microsoft Ignite Book of News — November 2026

Fabric IQ goes GA in Copilot, with Excel access in preview

Microsoft made Fabric IQ, its business-semantics layer over OneLake, generally available inside Copilot, with Excel integration in preview; the Book of News also lists Fabric apps and the Fabric Database Hub as GA. For architects, the significance is that the ontology layer defined in Fabric now becomes the context agents consume across the Microsoft surface, raising the stakes on getting that model right.

✍️ Microsoft · Read article →

› Enterprise RAG & Retrieval

BigQuery Release Notes — November 2026

BigQuery makes AI.RANK and AI.SEARCH reranking GA, plus an optimized mode for AI.IF and AI.CLASSIFY

A new GA AI.RANK function reorders rows by how well a text column answers a query, and AI.SEARCH gains a GA rerank option, putting a full retrieve-then-rerank pattern into SQL. An optimized mode for AI.IF and AI.CLASSIFY, also GA, cuts LLM token use and latency and adds multi-label classification.

✍️ Google Cloud · Read article →

↑ Top

 

Govern & Operate

› Catalogs & Metadata

Databricks Release Notes — November 2026

Unity Catalog agent skills can now be shared through OpenSharing and Databricks Marketplace

In Beta, data providers can add Unity Catalog agent skills to shares and Marketplace listings alongside tables and models. That extends the catalog's governed-sharing model from data assets to agent capabilities, meaning lineage and access policy questions now apply to the skills an agent loads, not only the data it reads.

✍️ Databricks · Read article →

Databricks Release Notes — November 2026

Governed tag propagation extends to classic compute

Unity Catalog governed tag propagation now works on classic compute running Runtime 19 and above, not just serverless, in Beta. Since tag-driven ABAC policies only hold if tags follow data through derived tables, closing the classic-compute gap removes a common hole in classification coverage for mixed estates.

✍️ Databricks · Read article →

› Governance, Security & Compliance

GlobeNewswire — November 2026

Ataccama named sole Visionary in the 2026 Gartner Magic Quadrant for D&A Governance Platforms

Ataccama announced its placement in Gartner's 2026 Magic Quadrant for Data and Analytics Governance Platforms, published November 12, highlighting its ONE AI steward agent, Data Trust Index and MCP Server. The vendor release is a signal that the governance MQ is out, and that agentic stewardship and MCP exposure are now part of how vendors pitch against the catalog incumbents.

✍️ Ataccama · Read article →

Snowflake Release Notes — November 2026

Snowflake makes SHOW GRANTS ... INCLUDE HIERARCHY generally available

SHOW GRANTS TO can now return the full inherited role hierarchy in one call, GA as of November 16. Effective-privilege resolution has long required recursive scripts against ACCOUNT_USAGE, so this makes access reviews and entitlement audits considerably easier to automate.

✍️ Snowflake · Read article →

Snowflake Release Notes — November 2026

Snowflake previews SCIM provisioning at the organization level

Organization users and organization user groups can now be provisioned via SCIM in private preview. For multi-account Snowflake estates, it moves identity lifecycle from per-account integrations to a single org-level feed from the IdP, reducing orphaned-user drift across accounts.

✍️ Snowflake · Read article →

Gethyn Ellis (GRE Solutions) — November 2026

Microsoft Ignite Day 1: MCP Server for Enterprise GA and Purview network DLP

A practitioner roundup of Ignite Day 1 reports that Microsoft's MCP Server for Enterprise, which exposes Entra, Purview and Intune data to AI tools, is GA, along with network DLP through Global Secure Access. It also flags Fabric IQ previews in ChatGPT and Claude and the retirement of standalone Power BI Copilot by end of January 2027.

✍️ Gethyn Ellis · Read article →

↑ Top

Compiled by Rainvil Labs — Wednesday, November 18, 2026
Sources verified via live web research on November 18, 2026, drawing on Microsoft (Ignite Book of News, Tech Community), SiliconANGLE, GlobeNewswire, Gethyn Ellis, and vendor release notes from Databricks, Snowflake, Google Cloud BigQuery and Confluent. Vendor-reported figures are noted as such. This briefing is for informational purposes only and does not constitute legal, regulatory, or investment advice.