Daily Briefing — Friday, September 25, 2026

Data & AI Platforms Briefing

Agents are pulling the data stack toward context and control: Kafka absorbs replication, CDC starts feeding vector stores directly, warehouses add evaluation loops to in-platform AI, and catalogs and security vendors race to enforce rules on agents they didn't build.

⇣ Jump To

Click any section or topic below to jump to it.

Move & Transform

Streaming & Messaging ·  CDC ·  ELT/ETL Ingestion ·  Stream Processing ·  Transformation Frameworks ·  In-Process Compute

Store & Architect

Cloud Data Warehouses ·  Lakehouses ·  Table Formats ·  Query Engines ·  Vector & Specialty Stores

Consume & Activate

AI-Driven Consumption ·  Semantic Layers & Retrieval ·  Enterprise RAG & Retrieval

Govern & Operate

Data Observability ·  Catalogs & Metadata ·  Data Contracts & Lineage ·  Governance, Security & Compliance ·  FinOps for Data

⚡ Quick Takes

Story Signal
↗  Data liberation: Apache Kafka's native cluster mirroring DR and migration become broker features; MirrorMaker 2's days look numbered.
↗  How Knock Saved $87K a Year by Migrating Off Kinesis to WarpStream Diskless Kafka wins cost-driven exits from Kinesis where sub-second latency isn't required.
↗  Debezium 3.7.0.CR1 Released CDC is becoming the default way to keep RAG indexes fresh.
↗  Snowpipe Streaming: Partitioned Apache Iceberg tables (GA) Warehouse-native streaming into Iceberg erodes the case for a separate Kafka-to-Iceberg hop.
↗  Announcing Apache Fluss 1.0: Real-Time Data Foundation for AI Streaming storage repositions as a real-time context layer agents query directly.
↗  DuckDB Now Ships inside dbt v2 Local-first transformation on embedded engines becomes a first-class dbt path.
↗  DuckDB and Hugging Face: Querying Datasets Directly Embedded engines are becoming the default prep layer for ML training data.
↗  Snowflake Code Bundles add warehouse execution and Spark batch jobs (Preview) Snowflake keeps chipping at Databricks' Spark moat: bring your jobs, keep your code.
↗  Cortex AI Function Evaluation and Optimization (Public Preview) In-warehouse AI gets an eval-and-optimize loop, not just inference endpoints.
↗  Databricks September release notes: shared metric views, ABAC-aware time travel, UC Python UDFs GA Semantics and ABAC now survive sharing and time-travel queries.
↗  Apache Data Lakehouse Weekly: September 15 to 23, 2026 How embeddings land in Parquet is being decided now, for years.
↗  MotherDuck introduces prompt_jev() Table-scale classification becomes an analytics-speed SQL function.
↗  PhoenixAI joins Cloudera as a foundational database partner Real-time OLAP vendors rebrand for agents and ride hybrid estates.
↗  Pinecone vector database managed service expands BYOC to AWS, Azure and GCP Pinecone concedes data residency; SaaS-only vector stores are over.
↗  Databricks acquires spreadsheet startup Row Zero to enhance its AI capabilities The governed spreadsheet becomes an agent interface over the lakehouse.
↗  Neo4j makes the case for knowledge graphs as shared context for AI agents Shared context layers pitched as the fix for per-agent semantic drift.
↗  Tabular Editor 3: September 2026 Release Semantic model engineering gets agent write access with revertible guardrails.
↗  Query decomposition doesn't fix context starvation — it just moves it RAG failures shift from retrieval to context packing, where few teams measure.
↗  Buyers are pricing model risk; their AI breaches are operational failures Agent observability and AI security converge into one budget line.
↗  Collibra Launches New Capabilities to Reduce the Hallucination Tax on Enterprise AI The catalog becomes the runtime rulebook for third-party agents.
↗  Liquibase Secure Now Available on Snowflake Marketplace Database change governance expands from schema to access objects.
↗  Proofpoint Breaks Down the Divide Between Data Security and AI Security with Unified Agentic System DSPM and AI runtime security merge; agent intent becomes an access-control input.
↗  Dataiku Unveils Agent Management to Track Enterprise AI Agents Across Platforms Cross-platform agent inventory emerges as its own governance category.
↗  CloudZero launches AI Signals to put finance leaders back in control of AI spend AI spend becomes attributable per inference, not a month-end surprise.

Move & Transform

› Streaming & Messaging

Red Hat Developer — September 2026

Data liberation: Apache Kafka's native cluster mirroring

A walkthrough of KIP-1279, which builds cross-cluster replication into the Kafka broker and retires MirrorMaker 2's Connect workers. Mirroring is byte-for-byte, so offsets stay identical, nothing is recompressed, consumer-group offsets and ACLs are synced, and failover is one kafka-cluster-mirrors.sh --stop command. Sources as old as Kafka 2.1 are supported, which opens a direct ZooKeeper-to-KRaft migration path; synchronous zero-loss mirroring is proposed in KIP-1360.

✍️ Federico Valeri · Read article →

WarpStream — September 2026

How Knock Saved $87K a Year by Migrating Off Kinesis to WarpStream

Knock replaced a Kinesis fan-out design, which copied events into child streams per consumer, with Kafka topics and consumer groups on WarpStream BYOC. Expected annual cost falls from about $112K to $25K; one engineer moved 5 topics and ~20 consumers in a month, and the main cluster runs on three m6in.xlarge nodes at ~20% CPU with sub-2-second end-to-end latency. Data stays in S3 in Knock's own account, which simplified the compliance review.

✍️ Andrea Leopardi (Knock) · Read article →

› CDC

Debezium — September 2026

Debezium 3.7.0.CR1 Released

The first 3.7 release candidate introduces PyDebeziumAI, which turns change events into LangChain-compatible documents so vector stores such as Chroma, PGVector and Milvus stay in sync with source tables, with proper upserts, deletes and a dead-letter queue. Oracle LogMiner gains an abandoned-transaction metric, a signal to dump pending transactions, and automatic retry on ORA-02002, and MongoDB now reconnects to the preferred replica after failover.

✍️ Chris Cranford · Read article →

› ELT/ETL Ingestion

Snowflake Release Notes — September 2026

Snowpipe Streaming: Partitioned Apache Iceberg tables (GA)

Snowpipe Streaming can now write into partitioned Snowflake-managed Iceberg v2 and v3 tables using each table's current partition spec, via both Elastic and Named Channels; classic Snowpipe Streaming and externally managed Iceberg tables are excluded. A same-day companion release adds event-table monitoring with SQL-queryable row counts, server-side latency and channel errors.

✍️ Snowflake · Read article →

› Stream Processing

Apache Fluss — September 2026

Announcing Apache Fluss 1.0: Real-Time Data Foundation for AI

Fluss's first major release since graduating to a top-level Apache project lands over 1,000 commits: a stateless Rust REST gateway so agents can reach tables over HTTP (MCP and Postgres-protocol access planned), Python/Rust/C++ clients on a shared Rust core, server-side filter pushdown, row-level TTL on primary-key tables and Hudi tiering. It also adds Flink 2.3 support, Spark union reads across live and historical data, and coordinator HA.

✍️ Keith Lee and Jark Wu · Read article →

› Transformation Frameworks

DuckDB — September 2026

DuckDB Now Ships inside dbt v2

dbt 2.0, running on the Rust Fusion engine, is the first dbt release with a built-in DuckDB adapter, connected via ADBC and pinned to a specific DuckDB build. The v2-only catalogs.yml configures DuckLake and Iceberg REST catalogs, and project metadata plus locally generated column-level lineage are written as Parquet that can be queried with plain DuckDB SQL in CI. Migration guidance: validate with dbt parse --use-v2-parser on 1.12, then run dbt-autofix.

✍️ Geertjan Wielenga · Read article →

› In-Process Compute

DuckDB — September 2026

DuckDB and Hugging Face: Querying Datasets Directly

A practitioner deep dive on DuckDB's hf:// protocol (available since 2024): querying Hub datasets in place with column pruning, globbing across files, commit pinning with @ for reproducible runs, and reading the auto-converted ~parquet branch. Private datasets authenticate through the secrets manager, and the recommended pattern is filtering remotely in SQL, then COPY-ing the training subset to local Parquet.

✍️ DuckDB team · Read article →

↑ Top

 

Store & Architect

› Cloud Data Warehouses

Snowflake Release Notes — September 2026

Snowflake Code Bundles add warehouse execution and Spark batch jobs (Preview)

Code Bundles, which run non-SQL code on Snowflake compute without stored-procedure wrappers, can now target virtual warehouses (compute_type: warehouse) in addition to compute pools. They also accept Scala, Java or Python Spark applications as batch jobs (type: spark) built on Snowpark Connect, so existing Spark code runs without a rewrite.

✍️ Snowflake · Read article →

Snowflake Release Notes — September 2026

Cortex AI Function Evaluation and Optimization (Public Preview)

AI_FUNCTION_EVALUATION scores custom AI Functions and Cortex AI calls against labeled datasets for quality, cost and token usage, and AI_FUNCTION_OPTIMIZATION searches across prompts and models for a better-performing implementation. Both run from SQL or Cortex AI Function Studio, putting in-warehouse inference quality and cost under the same governed workflow as any other workload.

✍️ Snowflake · Read article →

› Lakehouses

Databricks Release Notes — September 2026

Databricks September release notes: shared metric views, ABAC-aware time travel, UC Python UDFs GA

Metric views can now be shared through OpenSharing and queried with MEASURE() on serverless (GA, Sept 24). Time travel on Unity Catalog managed tables now enforces ABAC row filters and column masks (Beta, DBR 19), and scalar and batch Unity Catalog Python UDFs are GA with the five-UDF-per-query limit removed. Together they close a real gap: governed semantics and policy now carry through sharing and historical reads.

✍️ Databricks · Read article →

› Table Formats

DEV Community — September 2026

Apache Data Lakehouse Weekly: September 15 to 23, 2026

Parquet opened review on a Modular Footer so wide-table readers no longer parse the full Thrift footer, and a PR for a LIST-based VECTOR logical type for embeddings is drawing pushback. Iceberg C++ 0.4.0 went to vote with v3 types, deletion vectors and row lineage, and Arrow Rust 60.0.0 is the first to ship ALP float encoding.

✍️ Alex Merced · Read article →

› Query Engines

MotherDuck — September 2026

MotherDuck introduces prompt_jev()

prompt_jev() is a SQL scalar function running TypeSafe's Jev model that returns typed labels, scores or booleans with calibrated probabilities rather than prose. On a 100K-row AG News benchmark MotherDuck reports 89% accuracy in 40 seconds for $0.50, versus about 32 minutes and $37.58 for the closest-accuracy LLM. Preview on paid plans in us-east-1 and us-west-2.

✍️ Hamilton Ulmer, Till Döhmen and Jacob Matson · Read article →

BigDATAwire — September 2026

PhoenixAI joins Cloudera as a foundational database partner

PhoenixAI, the renamed CelerData behind StarRocks, will list its engine on the Cloudera Anywhere Cloud Marketplace for deployment inside private and sovereign clouds. It claims sub-second SQL over streaming and lakehouse data in place, without denormalization or pre-aggregation, plus RBAC and per-query auditing, giving StarRocks a route into Cloudera's hybrid installed base.

✍️ PhoenixAI · Read article →

› Vector & Specialty Stores

Blocks & Files — September 2026

Pinecone vector database managed service expands BYOC to AWS, Azure and GCP

Pinecone's bring-your-own-cloud option now keeps vectors, metadata and payloads in the customer's own account on all three hyperscalers, managed by Pinecone's control plane via outbound-only calls with no SSH, VPN or standing cross-account IAM role. Toyota Motor North America runs it in production, and an air-gapped self-managed option is in development, bringing Pinecone level with Zilliz, Weaviate and Qdrant on deployment choice.

✍️ Chris Mellor · Read article →

↑ Top

 

Consume & Activate

› AI-Driven Consumption

SiliconANGLE — September 2026

Databricks acquires spreadsheet startup Row Zero to enhance its AI capabilities

Row Zero is a cloud spreadsheet that handles up to a billion rows, runs each workbook on its own EC2 instance and syncs bidirectionally with cloud data platforms, including write-back. Databricks will make it a spreadsheet surface for Genie, sitting on Genie Ontology for business terms and Unity Catalog for governance; it follows a string of recent tuck-ins.

✍️ Maria Deutscher · Read article →

› Semantic Layers & Retrieval

SiliconANGLE — September 2026

Neo4j makes the case for knowledge graphs as shared context for AI agents

Neo4j's Jesús Barrasa argues teams are embedding context in each agent's prompts and skills separately, recreating the inconsistent-metrics problem of the BI era. His alternative is a governed, provenance-tracked knowledge layer shared by all agents, built use case by use case with LLMs accelerating ontology construction, measured by the marginal cost of each new agent and the cost of reconciling conflicting answers.

✍️ Victoria Gayton · Read article →

Tabular Editor — September 2026

Tabular Editor 3: September 2026 Release

Tabular Editor added an MCP server so external agents such as Claude Code or Copilot can work on an open Power BI or Fabric semantic model, with permissions that separate model edits from data reads. Per-property change markers allow selective revert, file-writing scripts are held for review, and every AI and MCP action is logged locally; the release also improves Databricks Metric View YAML editing.

✍️ Ruben Van de Voorde · Read article →

› Enterprise RAG & Retrieval

The New Stack — September 2026

Query decomposition doesn't fix context starvation — it just moves it

Testing on 10,000 chunks of GitLab docs with multi-intent queries, the author finds sub-query decomposition raises recall while the context-packing step silently drops answers. In one five-part example all five correct passages were retrieved but only one survived a 2,000-token budget, versus three without decomposition. The takeaway: measure per-sub-question coverage in the packed context, not just recall@k.

✍️ Kayode Ojo · Read article →

↑ Top

 

Govern & Operate

› Data Observability

Monte Carlo — September 2026

Buyers are pricing model risk; their AI breaches are operational failures

Moses uses IBM's 2026 breach data to argue AI breaches are operational failures: 92% of organizations whose models were attacked lacked AI access controls, with similar breach rates for open-source and third-party models. Monte Carlo's own survey of 260 practitioners found 71% cite security and governance as the top barrier to deploying more agents.

✍️ Barr Moses · Read article →

› Catalogs & Metadata

PR Newswire — September 2026

Collibra Launches New Capabilities to Reduce the Hallucination Tax on Enterprise AI

Collibra introduced Maestro, a no-code builder for governance agents on its knowledge graph (Studio and Assistant in public preview); Live Map, a context graph over unstructured documents from its Deasy Labs acquisition; and Agent Contracts with Guardian Agents, machine-readable rules enforced at runtime on agents built outside Collibra, due in October. The pitch leans on Collibra's Harris Poll finding that 76% of firms hit roadblocks moving agents to production.

✍️ Collibra · Read article →

› Data Contracts & Lineage

BigDATAwire — September 2026

Liquibase Secure Now Available on Snowflake Marketplace

Liquibase Secure is now purchasable against committed Snowflake spend and governs change beyond schemas, covering roles, shares, stages, warehouses and tasks. It adds pre-production policy checks, drift detection, audit evidence and rollback, treating changes to access and data movement as governed deployments.

✍️ Liquibase · Read article →

› Governance, Security & Compliance

GlobeNewswire — September 2026

Proofpoint Breaks Down the Divide Between Data Security and AI Security with Unified Agentic System

Proofpoint's Agentic Data and AI Security system runs on a knowledge graph linking each agent's intent to identity, access and data sensitivity, with three agents for detection, investigation and remediation (access fixes, DLP tuning) under human oversight. Semantic Business Policies turn plain-language rules into runtime controls; availability is expected by year-end 2026.

✍️ Proofpoint · Read article →

BigDATAwire — September 2026

Dataiku Unveils Agent Management to Track Enterprise AI Agents Across Platforms

A standalone, vendor-neutral product that discovers and inventories agents across Bedrock, Databricks, Vertex, Copilot Studio, Agentforce and Snowflake Cortex, with OpenTelemetry for custom stacks, and keeps certification status, risks and scheduled re-tests for high-risk agents; GA in October. A companion Dataiku/Harris Poll survey of 685 CIOs found 81% say they have lost oversight of their agents and only 21% see AI costs in near real time by team.

✍️ Dataiku · Read article →

› FinOps for Data

BigDATAwire — September 2026

CloudZero launches AI Signals to put finance leaders back in control of AI spend

AI Signals captures AI usage events as they happen and attributes spend by person, team and unit of work across models and providers, adding an Overview page and Monitors to the existing per-call Livestream and pre-invoice AI Explorer views. Usage data is processed inside the customer's own cloud network so prompts never leave it; available now.

✍️ CloudZero · Read article →

↑ Top

Compiled by Rainvil Labs — Friday, September 25, 2026
Sources verified via live web research on September 25, 2026, drawing on Red Hat Developer, WarpStream, Debezium, Apache Fluss, DuckDB, Snowflake and Databricks release notes, DEV Community, MotherDuck, BigDATAwire, Blocks & Files, SiliconANGLE, The New Stack, Tabular Editor, Monte Carlo, PR Newswire and GlobeNewswire. Vendor-reported benchmarks and survey figures are attributed as such. This briefing is for informational purposes only and does not constitute legal, regulatory, or investment advice.