Skip to Content
SourcesOpenTelemetry (OTLP)

OpenTelemetry (OTLP)

OpenTelemetry is an open standard for collecting traces, metrics, and logs from distributed systems. It provides a unified way to instrument applications and export telemetry data to observability backends.

Using OpenTelemetry with GlassFlow

OTLP source pipelines let GlassFlow ingest OpenTelemetry data directly via the OTLP protocol. GlassFlow runs its own OTLP receiver that accepts data on the standard OTLP ports, so you do not need a Kafka cluster to get observability data into ClickHouse.

This is ideal when your services are already instrumented with OpenTelemetry and you want a lightweight path from OTel SDKs or Collectors to ClickHouse.

Prerequisites

The OTLP receiver must be enabled in your GlassFlow Helm installation. If it is not already running, upgrade your release to enable it:

helm upgrade glassflow glassflow/glassflow-etl \
  --set otlpReceiver.enabled=true

Verify the receiver pod is running:

kubectl get pods -n glassflow -l app=glassflow-otlp-receiver

Configuration

Each OTLP source is an entry in the sources array. OTLP sources do not need schema_fields — the schema is predefined per signal type.

"sources": [
  {
    "type": "otlp.logs",
    "source_id": "logs"
  }
]

Signal Types

Source TypeDescriptionUse Case
otlp.logsOpenTelemetry log recordsApplication logs, structured logging
otlp.tracesDistributed trace spansRequest tracing, latency analysis
otlp.metricsMetric data points (gauge, sum, histogram, summary)Infrastructure monitoring, SLIs

Protocols

The GlassFlow OTLP receiver accepts data over two protocols:

ProtocolPortDescription
gRPC4317Standard OTLP/gRPC
HTTP/JSON4318Standard OTLP/HTTP with endpoints /v1/logs, /v1/traces, /v1/metrics

Required Header

Every OTLP request must include the x-glassflow-pipeline-id header. This header routes the incoming data to the correct pipeline.

x-glassflow-pipeline-id: <your-pipeline-id>

Features

FeatureSupportedDetails
DeduplicationYesDeduplication
Temporal JoinsNo
FilterYesFilter
Stateless TransformationYesStateless Transformation

Telemetry collectors

Any OTLP-capable client can talk to the GlassFlow OTLP receiver. The pages below cover the validated integrations for popular telemetry agents and forwarders, including tested configs, field mappings, and per-tool caveats.

  • AWS CloudWatch. Pull-based via an OpenTelemetry Collector relay (awscloudwatchreceiver then otlp_http); covers both logs and metrics with one Collector instance.
  • Fluent Bit. Direct OTLP HTTP forwarding for logs and host metrics, no Collector in between.
  • Fluentd. Routed through an OpenTelemetry Collector relay (fluent_forward then otlp_http); Fluentd itself has no OTLP output that supports custom headers.
  • Grafana Alloy. Direct OTLP forwarding for logs, metrics, and traces via Alloy’s OpenTelemetry-native exporters.
  • Logstash. Routed through an OpenTelemetry Collector relay (tcplog then otlphttp); Logstash itself has no maintained OTLP output.

For OpenTelemetry SDKs and the OpenTelemetry Collector directly, see Sending Data.

Pipeline Stop and Terminate Behaviour

Because the OTLP receiver is a shared component that stays running across all pipelines, it is important to understand what happens to in-flight data when you stop or terminate an OTLP pipeline.

  • Stop and Terminate: In both cases, pipeline components scale to zero but NATS JetStream streams are preserved. Any data already ingested but not yet written to ClickHouse remains in the streams and will be processed when the pipeline is resumed. Because the OTLP receiver is a shared component, it continues accepting incoming data for the stopped/terminated pipeline — new events are published to NATS JetStream and buffered until the pipeline is resumed. The difference between stop and terminate is that stop drains in-flight messages before scaling down, while terminate scales down immediately.
  • Delete: NATS JetStream streams are only deleted when the pipeline itself is deleted. This is the only operation that permanently removes buffered data.

While a pipeline is stopped or terminated, the OTLP receiver keeps buffering incoming data into NATS JetStream. If the pipeline stays in this state for an extended period, monitor the stream size against your max_bytes and max_age limits to avoid data eviction (see Scaling Guide).

Backpressure

When the receiver cannot keep up with incoming data it signals backpressure to the upstream OTel Collector or SDK — returning 429 Too Many Requests (HTTP) or ResourceExhausted (gRPC) — instead of silently dropping events. Standard OTel exporters retry these responses automatically.

See Backpressure for how the two backpressure gates work, how to tune them, and how to resolve sustained backpressure.

Next Steps

Last updated on
Topic ConfigurationExamples