Release Notes v3.1.0

Released: May 7, 2026

Version 3.1.0 introduces first-class backpressure handling across every stage of the data plane (ingestor, dedup, join, sink, and the OTLP receiver), surfaces a configurable MaxMsgs cap on inter-stage NATS streams, chunks the NATS async publisher to keep per-request memory bounded, and ships a new one-shot data migration container that turns v2-format pipeline configs into v3 on chart upgrade. This product release is accompanied by Helm chart v0.5.17; use chart v0.5.17 when installing or upgrading to v3.1.0.

What’s New

Backpressure across the data plane

Every stage that publishes to a NATS JetStream stream now detects backpressure (the downstream stream hitting its MaxMsgs cap), retries with bounded backoff, and emits observability data so operators can see episodes as they happen.

• Kafka ingestor processes batches with backpressure awareness and re-queues failing records into a carry slice for the next batch attempt (#708 ).
• JetStream publisher is now backpressure-aware on both the sync and async paths.
• Dedup streaming component retries with backoff when its output stream applies backpressure (#715 ).
• Join component propagates backpressure on its output stream rather than silently looping (#710 ).
• OTLP receiver applies backpressure to incoming requests when the downstream stream cannot accept publishes (#711 ).
• Observability emits a gfm_ingestor_backpressure_* metric family covering active state, episode count, and per-episode duration.

Configurable NATS stream cap (MaxMsgs)

The inter-stage NATS streams now expose an explicit MaxMsgs cap on the pipeline definition, so operators can size the buffer to their throughput and latency requirements (#701 ).

• Pipeline config. resources.nats.stream.maxMsgs is the new knob. Defaults are automatically computed from the sink batch size, so no manual tuning is necessary across most pipelines.
• Backpressure trigger. When a stream reaches MaxMsgs, the producing component enters a backpressure episode (see above) rather than dropping events.

NATS publisher chunking

The NATS async batch writer now chunks payloads so per-request memory is bounded regardless of upstream batch size (#712 ).

• Each publish chunk is sized to a hard cap on memory rather than to the upstream batch.
• Prevents OOM scenarios when a single Kafka or OTLP request would otherwise produce an outsized NATS write.

Data migration container

A one-shot Kubernetes Job container migrates any persisted v2-format pipeline configs in your Postgres database to the v3 format.

• What it does. Runs once on chart install or upgrade, picks up v2 configs, applies the same transformation as the migrate-preview API, and writes the v3 result back.
• What it does not do. It does not invert filter expression semantics (that remains a human-judgment migration; see the v3.0.0 release notes).
• Image. glassflow/glassflow-etl-data-migration, bundled in chart v0.5.17.

Improvements

Security

• Patched CVE-2026-33815 in a transitive dependency (#703 ).

Bug Fixes

• Backpressure regression revert. A backpressure-related change introduced a regression and was reverted, then re-landed correctly later in the cycle (#706 ).
• ClickHouse password encoding. Removed unnecessary base64 encoding of the ClickHouse password in the UI’s generated pipeline JSON.
• v3 wizard alignment. Pipeline-config generation in the UI wizard now aligns consistently with the v3 schema.

Migration Notes

There are no breaking changes in v3.1.0 if you are already on v3.0.0. The data migration container handles any v2 pipeline configs still in Postgres on upgrade.

• Helm chart. Use chart v0.5.17 for product v3.1.0. Charts v0.5.18 through v0.5.20 are chart-only patches that continue to bundle the v3.1.0 binary (operational fixes, no product changes).
• MaxMsgs default. The newly computed default cap on inter-stage streams is sized from the sink’s max_batch_size. If you previously relied on unbounded streams, set resources.nats.stream.maxMsgs explicitly on your pipelines.
• Backpressure visibility. If your dashboards alert on “consumer lag growing while no errors are emitted”, you can now alert directly on gfm_ingestor_backpressure_active == 1 for 5m instead of inferring from rate-based queries.

Try It Out

1. Upgrade via Helm. Deploy v3.1.0 using the Kubernetes Helm charts at chart version v0.5.17. Ensure your existing cluster is on v3.0.0 first.
2. Verify the data migration. Watch for the one-shot glassflow-etl-data-migration Job to complete; check Postgres for any pipeline rows where persisted config is still in the v2 shape.
3. Size your inter-stage streams. Review resources.nats.stream.maxMsgs on your pipelines. The new default is computed from sink max_batch_size; override it if you need a different cap.
4. Wire up backpressure alerting. Add a Prometheus alert on gfm_ingestor_backpressure_active == 1 for 5m per pipeline so sustained episodes surface in your on-call channels.

Full Changelog

For the complete list of changes in v3.1.0, see the GitHub release v3.1.0 .

GlassFlow v3.1.0 closes the visibility gap on inter-stage flow control and lays the foundation for the sink-side error-handling work coming in the next release.