Daily Briefing — Monday, September 28, 2026

Data & AI Platforms Briefing

The agent era is being operationalized from both ends: platforms are adding log semantics, CDC hardening and incremental refresh underneath, while MCP credentials, runtime kill switches and decision-level observability become the governance problems on top.

⇣ Jump To

Click any section or topic below to jump to it.

Move & Transform

Streaming & Messaging ·  CDC ·  Transformation Frameworks

Store & Architect

Cloud Data Warehouses ·  Architectural Patterns ·  Query Engines ·  Vector & Specialty Stores

Consume & Activate

AI-Driven Consumption ·  Semantic Layers & Retrieval ·  Enterprise RAG & Retrieval

Govern & Operate

Data Observability ·  Catalogs & Metadata ·  Governance, Security & Compliance

⚡ Quick Takes

Story Signal
↗  Amazon EventBridge relaunches event buses for enterprise scale AWS’s serverless router is acquiring log semantics — ordering, retention, replay — and blurring the bus-vs-log line.
↗  Amazon Kinesis Data Streams adds service-managed partition keys Managed streams are absorbing partitioning decisions for workloads that don’t need per-key ordering.
↗  Snowflake Openflow runtime updates harden Oracle, PostgreSQL and SQL Server CDC Built-in warehouse CDC is maturing through unglamorous correctness fixes rather than headline launches.
↗  Amazon EMR on EKS adds interactive sessions via Spark Connect Spark Connect is becoming the default interface to Spark in managed services.
↗  Amazon EMR introduces Long Term Support, starting with Spark 4.1 Hyperscalers are pairing long support windows with AI upgrade agents to cut the cost of Spark version moves.
↗  CHECK constraints on hybrid tables (GA) Snowflake keeps adding the OLTP integrity guarantees operational workloads expect, one feature at a time.
↗  Snowpipe Streaming: event table monitoring Streaming-ingest observability is moving into the platform itself.
↗  Dynamic tables: incremental refresh now supports the VALUES clause Declarative pipelines get cheaper as the incremental-refresh envelope widens.
↗  Data Dialogue: How S&P Global Energy made its structured data estate conversational with Genie Agents and MCP A reference pattern for serving a governed data estate to agents: domain-owned, mesh-style, MCP-composed.
↗  ClickHouse Release 26.9 Real-time OLAP engines keep absorbing warehouse-style incremental processing.
↗  Perplexity replaces DynamoDB with home-built CobbleDB Retrieval at AI scale is breaking per-byte-priced managed KV stores, pushing builders to split serving from durable storage.
↗  Dataiku Expands Cobuild to Turn Business Questions Into Governed AI Work Data platforms are becoming governed backends that external coding agents drive, not destinations users log into.
↗  Ekai raises $1.7M to give enterprise AI agents verified business context A counter-position to auto-mining semantics from query logs: define first, validate, then generate.
↗  Persistent AI memory adds another layer to enterprise data Retrieved context now includes derived, stateful memory that needs lineage, authority rules, and a lifecycle.
↗  AWS CloudWatch Omni goes after the hardest question in agentic AI: Why did the agent do that? Observability is shifting from “is it running?” to “was the decision right?” — and hyperscalers want the evaluation layer.
↗  Collibra brings runtime governance to enterprise AI agents Catalog vendors are moving from design-time policy to runtime enforcement in the agent path.
↗  Hardcoded credentials in public MCP files open door to cyberattacks MCP config files are the new leaked .env; agent credentials need joiner-mover-leaver discipline.
↗  Pulling the plug: Why the AI kill switch might be a dead end Containment design — credential TTLs, reversibility tiers, dependency maps — is becoming a platform-engineering requirement.
↗  New Okta for AI Agents innovations increase visibility, secure connections at runtime, and enforce continuous agent governance Identity providers are positioning as the agent control plane between agents and data.

Move & Transform

› Streaming & Messaging

AWS What’s New — September 2026

Amazon EventBridge relaunches event buses for enterprise scale

A new “enhanced” custom event bus adds strict ordering, content-based deduplication, CloudEvents publishing with schema preserved, 24 hours of built-in retention (extendable to a year for replay), and cross-account sharing via AWS RAM; pricing shifts from per-event to per-data-transferred, and the old bus becomes “classic.” Ordering plus replay moves EventBridge into territory teams previously reserved for Kafka, Kinesis, or SNS+SQS fan-out, which reopens the “do we need a log here?” question for event-driven integration.

✍️ Amazon Web Services · Read article →

AWS What’s New — September 2026

Amazon Kinesis Data Streams adds service-managed partition keys

On-demand streams (Standard and Advantage) can now accept records without a partition key; the service distributes them across shards based on warm capacity, eliminating hot-key throttling. The usual workaround — random UUID keys — still produced uneven shard load. Requires the latest SDK or KPL, available in all commercial regions at no extra cost.

✍️ Amazon Web Services · Read article →

› CDC

Snowflake Documentation — September 2026

Snowflake Openflow runtime updates harden Oracle, PostgreSQL and SQL Server CDC

The September 24 runtime (2026.9.24.2) adds client-side ROWID chunking for Oracle CDC on read-only databases and very large tablespaces, removes a scheduling lag on PostgreSQL CDC merges, and fixes schema evolution when a removed column is renamed; the September 22 runtime added cursor-based SQL Server capture for sources without Change Tracking or CDC and fixed a Change Tracking bug that could permanently skip changes. These edge cases decide whether Openflow can credibly displace GoldenGate, Qlik Replicate, or self-run Debezium estates.

✍️ Snowflake · Read article →

› Transformation Frameworks

AWS What’s New — September 2026

Amazon EMR on EKS adds interactive sessions via Spark Connect

Interactive Spark sessions can now be launched from SageMaker Unified Studio notebooks or local IDEs (Jupyter, VS Code) against existing EKS clusters, running as pods under an IAM execution role and tagged by project and user. Supported on EMR 7.14 (Spark 3.5) and emr-spark-8.1.0 (Spark 4.1). It turns Spark Connect’s thin-client split into a production pattern on customer-run Kubernetes.

✍️ Amazon Web Services · Read article →

AWS What’s New — September 2026

Amazon EMR introduces Long Term Support, starting with Spark 4.1

emr-spark-8.1.0 is EMR’s first LTS release, with 36 months of security, bug, and data-corruption fixes, full Iceberg v3 support, catalog-by-name references (including cross-account and S3 Tables) with automatic Iceberg/Delta/Hudi detection, and broader fine-grained access control. AWS pairs it with its Apache Spark Upgrade Agent to migrate older jobs — a three-year Spark baseline plus AI-assisted migration materially changes upgrade planning for platform teams.

✍️ Amazon Web Services · Read article →

↑ Top

 

Store & Architect

› Cloud Data Warehouses

Snowflake Release Notes — September 2026

CHECK constraints on hybrid tables (GA)

Hybrid (Unistore) tables now enforce CHECK constraints — inline or multi-column — on every INSERT, UPDATE, MERGE, and CTAS, moving integrity rules out of application code. The limits matter: constraints can only be declared at CREATE time (no ALTER ADD), and COPY INTO fails against a constrained hybrid table, so bulk-load paths need rework.

✍️ Snowflake · Read article →

Snowflake Release Notes — September 2026

Snowpipe Streaming: event table monitoring

Snowpipe Streaming now emits ingestion telemetry — row counts, server-side latency, row and channel errors, channel activity — to the default event table, enabled per schema with LOG_EVENT_LEVEL=INFO and queryable in SQL alongside SDK-side Prometheus metrics. Combined with last week’s partitioned-Iceberg streaming GA, streaming into Snowflake-managed Iceberg is becoming something you can operate and alert on without bespoke tooling.

✍️ Snowflake · Read article →

Snowflake Release Notes — September 2026

Dynamic tables: incremental refresh now supports the VALUES clause

Incremental refresh (INCREMENTAL, AUTO, and ADAPTIVE modes) now handles VALUES clauses such as inline lookup tables; a companion Sept 23 note extends incremental support to approximate aggregates (APPROX_COUNT_DISTINCT, APPROX_PERCENTILE, APPROX_TOP_K) in the SELECT list. Fewer dynamic tables will silently fall back to full refresh — a direct compute saving on declarative pipelines.

✍️ Snowflake · Read article →

› Architectural Patterns

Databricks Blog — September 2026

Data Dialogue: How S&P Global Energy made its structured data estate conversational with Genie Agents and MCP

S&P Global Energy has subject-matter experts curate one narrow Genie Agent per dataset group (LNG cargo, outages, netbacks) over native and federated tables; each becomes a Unity Catalog-governed MCP server, and a FastMCP proxy composes them into namespaced per-commodity endpoints — also exposed to external customers’ agents. Lessons: keep agents narrow and compose at the MCP layer, federate before building pipelines, and benchmark accuracy continuously.

✍️ Debaprasad Satapathy & Eric Henderson, Databricks · Read article →

› Query Engines

ClickHouse Blog — September 2026

ClickHouse Release 26.9

The release ships 56 features, 135 performance optimizations, and 464 bug fixes, headlined by incremental refreshes for append-only materialized views, disk spilling for high-cardinality DISTINCT, conditional LIMIT boundaries, time-limited access tokens, and broader PromQL support. Incremental refresh and spill-to-disk remove two common reasons to run a separate transform layer or oversize nodes for real-time rollups.

✍️ ClickHouse · Read article →

› Vector & Specialty Stores

InfoQ — September 2026

Perplexity replaces DynamoDB with home-built CobbleDB

Perplexity split storage into Pillar (durable store on YTsaurus), Lorry (partition-aligned batch writes to S3), and CobbleDB, a Rust/RocksDB key-value store on local NVMe that serves batched reads of passages and embeddings using async replicas, same-zone routing, and hedged reads. p50 latency fell from 31.4 ms to 5.6 ms, p99 from 123 ms to 24.2 ms, with storage cost down at least 20%; Perplexity plans to open-source it.

✍️ Olimpiu Pop, InfoQ · Read article →

↑ Top

 

Consume & Activate

› AI-Driven Consumption

BigDATAwire — September 2026

Dataiku Expands Cobuild to Turn Business Questions Into Governed AI Work

Cobuild Insights answers plain-language questions from governed data, while Dataiku Headless lets Claude Code, Codex, and Cursor build and modify pipelines, models, and agents inside a live Dataiku environment — including migrating Excel or Alteryx logic into governed projects. The AI Catalog now lists semantic models, so assembled answers and pipelines draw on vetted assets with lineage.

✍️ Dataiku (press release) · Read article →

› Semantic Layers & Retrieval

SiliconANGLE — September 2026

Ekai raises $1.7M to give enterprise AI agents verified business context

Ekai “forward-engineers” semantic models and transformation code from domain-expert definitions, validating everything against the warehouse before it ships, and explicitly rejects inferring semantics from dashboards, query history, or dbt projects. It claims semantic modeling that took 3–6 months done in about 6 hours; it runs on Snowflake with support for Databricks, BigQuery, Redshift, Postgres, ClickHouse, and DuckDB.

✍️ Duncan Riley, SiliconANGLE · Read article →

› Enterprise RAG & Retrieval

TechTarget — September 2026

Persistent AI memory adds another layer to enterprise data

Agent memory is becoming a distributed data layer — Writer’s team-level Agent Memory, AWS AgentCore Memory’s expiration and relevance-decay lifecycle, Couchbase’s AI Data Plane, Gemini Enterprise Memory Bank. The piece proposes treating memory as an overlay on the data map and checking it against the authoritative system before any consequential action, correcting or expiring stale memory on conflict.

✍️ James Alan Miller, TechTarget · Read article →

↑ Top

 

Govern & Operate

› Data Observability

SiliconANGLE — September 2026

AWS CloudWatch Omni goes after the hardest question in agentic AI: Why did the agent do that?

Built on OpenTelemetry/OpenInference, CloudWatch Omni ships 17 built-in evaluators (faithfulness, routing correctness, and others) that run continuously on live traffic, storing agent traces alongside app and infrastructure telemetry. Kerravala warns that chatty agent spans can push ingestion costs past the AI budget itself, and argues investigation history should be treated as governance evidence.

✍️ Zeus Kerravala, ZK Research · Read article →

› Catalogs & Metadata

SiliconANGLE — September 2026

Collibra brings runtime governance to enterprise AI agents

CEO Felix Van de Maele details how Agent Contracts are compiled and deployed into existing AI gateways, with Guardian Agents blocking non-compliant actions before execution. Live Map pre-builds reusable context from unstructured documents for token efficiency, and Maestro steward agents claim to automate up to 80% of governance work — the operational detail behind last week’s launch.

✍️ Victoria Gayton, SiliconANGLE · Read article →

› Governance, Security & Compliance

TechTarget — September 2026

Hardcoded credentials in public MCP files open door to cyberattacks

Hush Security analyzed roughly 82,000 public MCP config files: 12% of credential slots held a hardcoded secret, and 24% of those were both non-expiring and broad-scope, including model-provider API keys. Remediation: rotate at the provider (scrubbing Git history isn’t enough), use environment variables or secret managers, adopt an MCP gateway, and move to the 2026-07-28 MCP spec.

✍️ Alissa Irei, TechTarget · Read article →

TechTarget — September 2026

Pulling the plug: Why the AI kill switch might be a dead end

Proposed kill-switch legislation collides with technical reality: agents spawn sub-agents and queue jobs, so killing the parent can leave work running. Security leaders argue for “positive control” instead — short-lived scoped credentials treated as “a lease, not a license,” with human approval gated by how reversible an action is.

✍️ Richard Livingston, TechTarget · Read article →

Okta Newsroom — September 2026

New Okta for AI Agents innovations increase visibility, secure connections at runtime, and enforce continuous agent governance

Agent SSO (Cross App Access replacing non-expiring keys with short-lived tokens), agent-to-agent connections, and resource access certifications are GA; Agent Gateway — policy enforcement and logging on every tool call — plus shadow-agent discovery are slated next, with a gateway-level kill switch that revokes all active tokens and in-flight sessions planned for Q4. For data platform teams, this puts the IdP directly in the MCP tool-call path.

✍️ Okta · Read article →

↑ Top

Compiled by Rainvil Labs — Monday, September 28, 2026
Sources verified via live web research on September 28, 2026, drawing on AWS What’s New, Snowflake documentation and release notes, the Databricks and ClickHouse blogs, InfoQ, SiliconANGLE, BigDATAwire, TechTarget, and the Okta newsroom. This briefing is for informational purposes only and does not constitute legal, regulatory, or investment advice.