Run the stack
One file holds the whole stack: every service and every config, inline. You download it into an empty folder, start it, and send it a session with curl. It takes about two minutes once the images are pulled.
You
curl, an app, a backendBasic demo:pt_dev_demo_…
Edge
otel-collectorghcr.io/ouo11y/collector
Storage
loki · tempo · mimirgrafana/loki 3.7.7 · tempo 3.0.3 · mimir 3.2.1
rum-apighcr.io/ouo11y/rum-api
View
grafanaghcr.io/ouo11y/grafana · :3000
Before you start
Section titled “Before you start”- Docker with Compose 2.24 or later. The file uses inline
configs.content, which older Compose versions reject. Check withdocker compose version. - Pull access to the ouo11y images.
Start it
Section titled “Start it”-
Create an empty folder and go into it. The Compose project is named after the folder, so two copies in two folders never collide.
Terminal window mkdir ouo11y-stack && cd ouo11y-stack -
Download the stack as
compose.yaml.Terminal window curl -fsSLo compose.yaml https://ouo11y.com/downloads/quickstart.yamlOr save it from the page: quickstart.yaml. The full file is at the end of this page.
-
Log in to the GitHub container registry with your token. Keep the token in your shell, never in a file.
Terminal window echo $GHCR_TOKEN | docker login ghcr.io -u <github-user> --password-stdinYou see
Login Succeeded. -
Start the stack.
Terminal window docker compose up -dThe first run pulls about 1.5 GB of images. Then you see every service as
Started, andgrafana-pluginasExited: it copies the Grafana app onto the Grafana volume and stops, on every start. -
Check that the collector asks for a credential. Without one it answers
401:Terminal window curl -s -o /dev/null -w '%{http_code}\n' -X POST localhost:4318/v1/traces -H 'Content-Type: application/json' -d '{"resourceSpans":[]}'With the demo project’s credential it answers
200:Terminal window curl -s -o /dev/null -w '%{http_code}\n' -X POST localhost:4318/v1/traces -u demo:pt_dev_demo_0000000000000000 -H 'Content-Type: application/json' -d '{"resourceSpans":[]}' -
Check that rum-api reaches Loki and Tempo:
Terminal window curl -s localhost:8080/readyzYou see
{"checks":{"loki":"ok","tempo":"ok"},"ready":true}. Right after a start Loki still reportsIngester not ready: waiting for 15s after being ready; ask again after half a minute. -
Open Grafana at http://localhost:3000, log in as
admin/testing567, and go to More apps → ouo11y RUM. The Overview is empty: nothing has sent a session yet.
Send a first session
Section titled “Send a first session”A session is a set of OpenTelemetry log records with the same session.id. This command sends the two that open and close one, the way an SDK would, to the collector’s OTLP/HTTP endpoint:
NOW=$(date +%s)000000000; SID=curl-$(date +%s); curl -s -w '%{http_code}\n' -X POST localhost:4318/v1/logs -u demo:pt_dev_demo_0000000000000000 -H 'Content-Type: application/json' -d '{"resourceLogs":[{"resource":{"attributes":[{"key":"service.name","value":{"stringValue":"curl-shop"}},{"key":"service.version","value":{"stringValue":"1.0.0"}},{"key":"ouo.platform","value":{"stringValue":"web"}}]},"scopeLogs":[{"logRecords":[{"timeUnixNano":"'$NOW'","eventName":"session.start","body":{"stringValue":"session started"},"attributes":[{"key":"session.id","value":{"stringValue":"'$SID'"}},{"key":"user.id","value":{"stringValue":"u-1"}}]},{"timeUnixNano":"'$NOW'","eventName":"exception","body":{"stringValue":"TypeError: cart is undefined"},"attributes":[{"key":"session.id","value":{"stringValue":"'$SID'"}},{"key":"user.id","value":{"stringValue":"u-1"}},{"key":"exception.type","value":{"stringValue":"TypeError"}},{"key":"exception.message","value":{"stringValue":"cart is undefined"}},{"key":"ouo.exception.fatal","value":{"boolValue":false}},{"key":"ouo.exception.handled","value":{"boolValue":true}}]}]}]}]}'; echo "session $SID"You see {"partialSuccess":{}}200 and the session id. In Grafana, ouo11y RUM → Sessions now lists a curl-shop session from user u-1, status errored: it has one exception that was not fatal. Open it to see its timeline. Sessions and health explains why it is errored and not crashed.
The collector stamped ouo.project.id = demo on it from the credential. Whatever a client sends in that attribute is overwritten.
What is listening
Section titled “What is listening”| Service | Address on your machine | Who uses it |
|---|---|---|
| Grafana | http://localhost:3000 |
you, as admin / testing567 |
| Collector, OTLP/HTTP | http://localhost:4318 (/v1/traces, /v1/logs, /v1/metrics) |
every SDK and backend |
| Collector, replay | http://localhost:4319/v1/replay/chunks |
the SDKs’ replay uploads |
| Collector, Sentry bridge | http://localhost:4320 |
Sentry SDKs, only if you use the bridge |
| Collector, OTLP/gRPC | 127.0.0.1:4317 |
backends on this machine |
| Collector, health | http://127.0.0.1:13133 |
readiness checks |
| rum-api | http://localhost:8080 |
Grafana; published for debugging only |
| Loki, Tempo, Mimir | :3100, :3200, :9009 |
Grafana and rum-api; published for debugging only |
Loki, Tempo, Mimir and rum-api are published on every interface of your laptop, so other devices on your network can reach them. That is fine on a laptop. On a server, use the VM install, which binds everything except the proxy to 127.0.0.1.
Ports already taken
Section titled “Ports already taken”Put the ones you need to move in a .env file next to compose.yaml, then run docker compose up -d again:
OUO_GRAFANA_PORT=13000OUO_OTLP_HTTP_PORT=14318OUO_REPLAY_PORT=14319The other variables are OUO_SENTRY_PORT, OUO_OTLP_GRPC_PORT, OUO_COLLECTOR_HEALTH_PORT, OUO_RUM_API_PORT, OUO_LOKI_PORT, OUO_TEMPO_PORT and OUO_MIMIR_PORT. An SDK then needs both collector URLs, for example endpoint: "http://localhost:14318" and replayEndpoint: "http://localhost:14319". The SDK only guesses the replay port when the OTLP port is 4318.
Credentials in this file
Section titled “Credentials in this file”Every value below is a development value, written in this public file. They are fine on a laptop and never acceptable on a host other people can reach. Project tokens shows how to make real ones.
| What | Value | Public or secret | Used by |
|---|---|---|---|
| Grafana login | admin / testing567 |
secret | you (OUO_GRAFANA_ADMIN_PASSWORD) |
| Project credential | id demo, token pt_dev_demo_0000000000000000 |
public by design | apps and backends, as HTTP Basic |
| Internal project | id internal, token pt_dev_internal_00000000000000 |
public | rum-api’s own telemetry |
| Ingest token | it_dev_0000000000000000 |
secret | collector → rum-api replay forwarding |
| Read token | rt_dev_0000000000000000 |
secret | Grafana → rum-api reads |
| Admin token | at_dev_0000000000000000 |
secret | rum-api’s operator API (delete by user or project) |
| Source map upload token | demo:ut_dev_demo_0000000000000000 |
secret | uploading source maps for project demo from CI |
A project token ships inside your web bundle, so anyone can read it. It only lets a client write telemetry into one project, within the collector’s CORS, size and rate limits. Project credentials explains the model.
The browser origins allowed to post are http://localhost:* and http://127.0.0.1:*. Add yours with OUO_ALLOWED_ORIGINS in .env, comma separated, no spaces.
Versions
Section titled “Versions”The ouo11y images default to :latest, which follows each release. The file’s header names the versions it was generated for and their digests. To stay on exactly those, pin them in .env:
OUO_COLLECTOR_VERSION=0.4.0OUO_RUM_API_VERSION=0.5.0OUO_GRAFANA_VERSION=0.5.0When a newer latest rejects a config from this file, docker compose logs <service> names the key; pin the versions above, and download the file again after the next release.
Stop and clean up
Section titled “Stop and clean up”Stop the stack and keep its data:
docker compose downStop it and delete the data (sessions, traces, replays, Grafana settings):
docker compose down -vWhen something does not work
Section titled “When something does not work”| You see | Cause | Fix |
|---|---|---|
denied or 403 pulling ghcr.io/ouo11y/... |
no pull access, or logged in as another user | the early-access steps above; docker logout ghcr.io and log in again |
configs.content or additional properties 'content' not allowed |
Compose older than 2.24 | update Docker Desktop or the Compose plugin |
Bind for 0.0.0.0:3000 failed: port is already allocated |
something else uses that port | move it in .env, see above; do not stop the other program |
401 with the credential |
a typo in the id or token | the token is pt_dev_demo_ followed by sixteen zeros |
/readyz says loki is not ok |
Loki is still starting | wait half a minute and ask again |
| Grafana has no ouo11y RUM app | the grafana-plugin step failed |
docker compose logs grafana-plugin should end with ouo11y-rum-app 0.9.0 installed |
| A browser app gets CORS errors | its origin is not allowed | set OUO_ALLOWED_ORIGINS and docker compose up -d |
The file
Section titled “The file”compose.yaml
# ouo quickstart: the whole local stack in one file, on the latest ouo release.# GENERATED by deploy/scripts/gen-quickstart.sh; do not edit by hand.## The ouo images default to :latest, the newest release. This file was generated for, and its# configs come from, these releases (digest = the multi-arch image index at that version):## collector 0.4.0 sha256:ca0d001dc58153671fc501cbe280a69f86dc3e681a1b4621e827dedadb7727f2 OUO_COLLECTOR_VERSION# rum-api 0.5.0 sha256:1ee9e606220df2c42f4ade4131fdd3f82085d6e814676a771754a85a6ae78336 OUO_RUM_API_VERSION# grafana 0.5.0 sha256:57607a3616c88867644be1f69abb4681ca73df05d6bf2682543418f0dfeb622f OUO_GRAFANA_VERSION (plugin ouo11y-rum-app 0.9.0)## To stay on exactly these, put e.g. OUO_GRAFANA_VERSION=0.5.0 (or 0.5.0@sha256:...) in a .env# file next to this one. Each config below comes from its image's release commit; Loki, Tempo# and Mimir from grafana-v0.5.0. A later release that needs new configs comes with a regenerated# file: fetch it again (deploy/compose/quickstart.yaml in the ouo11y/ouo11y repo).## Copy this file into an empty folder as compose.yaml, then:## echo $GHCR_TOKEN | docker login ghcr.io -u <github-user> --password-stdin # images are private for now# docker compose up -d## Grafana http://localhost:3000 admin / testing567, More apps -> ouo11y RUM# OTLP/HTTP http://localhost:4318 /v1/traces /v1/logs /v1/metrics# Replay http://localhost:4319 /v1/replay/chunks# Credential HTTP Basic, project id `demo`, project token `pt_dev_demo_0000000000000000`# (Authorization: Basic ZGVtbzpwdF9kZXZfZGVtb18wMDAwMDAwMDAwMDAwMDAw)## Ports collide? Set OUO_GRAFANA_PORT, OUO_OTLP_HTTP_PORT, OUO_REPLAY_PORT, ... in a .env# file next to this one (names and defaults: deploy/compose/.env.example in the repo). An SDK# then needs both URLs, e.g. endpoint :14318 and replayEndpoint :14319.# Stop: `docker compose down`; wipe the data too: `docker compose down -v`.## Development tokens only (they are public): fine on a laptop, never on a reachable host.# A server install is deploy/vm/install.sh. Needs Docker Compose >= 2.24 (`configs.content`).## No `name:` on purpose: the project is named after the folder, so two copies of this file in# two folders are two separate stacks (a fixed name would make `down -v` in one remove the other).
services: grafana: image: ${OUO_IMAGE_REGISTRY:-ghcr.io/ouo11y}/grafana:${OUO_GRAFANA_VERSION:-latest} ports: - "${OUO_GRAFANA_PORT:-3000}:3000" environment: GF_SECURITY_ADMIN_USER: admin GF_SECURITY_ADMIN_PASSWORD: ${OUO_GRAFANA_ADMIN_PASSWORD:-testing567} GF_AUTH_ANONYMOUS_ENABLED: "false" GF_USERS_DEFAULT_THEME: dark GF_PLUGINS_ALLOW_LOADING_UNSIGNED_PLUGINS: ouo11y-rum-app GF_SECURITY_DISABLE_FRONTEND_SANDBOX_FOR_PLUGINS: ouo11y-rum-app # Read by the plugin provisioning file; must equal rum-api's OUO_READ_TOKEN. OUO_READ_TOKEN: ${OUO_READ_TOKEN:-rt_dev_0000000000000000} # Grafana feature toggles the plugin can use (comma-separated), empty by default. OUO_FEATURE_TOGGLES_ENABLE: ${OUO_FEATURE_TOGGLES_ENABLE:-} configs: - source: grafana-datasources target: /etc/grafana/provisioning/datasources/datasources.yaml - source: grafana-plugin-app target: /etc/grafana/provisioning/plugins/ouo11y-rum-app.yaml volumes: - grafana-data:/var/lib/grafana depends_on: grafana-plugin: condition: service_completed_successfully loki: condition: service_started tempo: condition: service_started mimir: condition: service_started healthcheck: test: ["CMD-SHELL", "wget -qO- http://localhost:3000/api/health >/dev/null 2>&1 || exit 1"] interval: 5s timeout: 3s retries: 30
# The grafana-data volume hides the plugin the image ships in /var/lib/grafana/plugins; # copy it onto the volume before every start so an image upgrade also upgrades the plugin. grafana-plugin: image: ${OUO_IMAGE_REGISTRY:-ghcr.io/ouo11y}/grafana:${OUO_GRAFANA_VERSION:-latest} entrypoint: ["sh", "-ec"] command: - | mkdir -p /pvc/plugins rm -rf /pvc/plugins/ouo11y-rum-app cp -a /var/lib/grafana/plugins/ouo11y-rum-app /pvc/plugins/ouo11y-rum-app echo "ouo11y-rum-app $$(sed -n 's/.*"version": *"\([^"]*\)".*/\1/p' /pvc/plugins/ouo11y-rum-app/plugin.json | head -1) installed" volumes: - grafana-data:/pvc restart: "no"
loki: image: grafana/loki:3.7.7 command: ["-config.file=/etc/loki/config.yaml", "-config.expand-env=true"] ports: - "${OUO_LOKI_PORT:-3100}:3100" environment: OUO_RETENTION_LOGS: ${OUO_RETENTION_LOGS:-720h} OUO_LOKI_COMPACTION_INTERVAL: ${OUO_LOKI_COMPACTION_INTERVAL:-10m} OUO_LOKI_RETENTION_DELETE_DELAY: ${OUO_LOKI_RETENTION_DELETE_DELAY:-2h} OUO_LOKI_DELETE_CANCEL_PERIOD: ${OUO_LOKI_DELETE_CANCEL_PERIOD:-24h} configs: - source: loki target: /etc/loki/config.yaml - source: loki-runtime target: /etc/loki/runtime.yaml volumes: - loki-data:/loki
tempo: image: grafana/tempo:3.0.3 command: ["-config.file=/etc/tempo/config.yaml", "-config.expand-env=true"] ports: - "${OUO_TEMPO_PORT:-3200}:3200" environment: OUO_RETENTION_TRACES: ${OUO_RETENTION_TRACES:-336h} configs: - source: tempo target: /etc/tempo/config.yaml volumes: - tempo-data:/var/tempo
mimir: image: grafana/mimir:3.2.1 command: ["-config.file=/etc/mimir/config.yaml", "-config.expand-env=true"] ports: - "${OUO_MIMIR_PORT:-9009}:9009" environment: OUO_RETENTION_METRICS: ${OUO_RETENTION_METRICS:-2160h} configs: - source: mimir target: /etc/mimir/config.yaml volumes: - mimir-data:/data
# The only ingest (ADR-0007): OTLP on 4318 and replay chunks on 4319, both behind HTTP # Basic project credentials checked against the projects-htpasswd config below. otel-collector: image: ${OUO_IMAGE_REGISTRY:-ghcr.io/ouo11y}/collector:${OUO_COLLECTOR_VERSION:-latest} command: ["--config=/etc/otelcol/config.yaml"] ports: - "127.0.0.1:${OUO_OTLP_GRPC_PORT:-4317}:4317" - "${OUO_OTLP_HTTP_PORT:-4318}:4318" - "${OUO_REPLAY_PORT:-4319}:4319" - "${OUO_SENTRY_PORT:-4320}:4320" - "127.0.0.1:${OUO_COLLECTOR_HEALTH_PORT:-13133}:13133" environment: OUO_INGEST_TOKEN: ${OUO_INGEST_TOKEN:-it_dev_0000000000000000} OUO_ALLOWED_ORIGINS: ${OUO_ALLOWED_ORIGINS:-http://localhost:*,http://127.0.0.1:*} OUO_RATELIMIT_CLIENT_KEY: ${OUO_RATELIMIT_CLIENT_KEY:-address} OUO_RATELIMIT_PROJECT_RATE: ${OUO_RATELIMIT_PROJECT_RATE:-1000} OUO_RATELIMIT_PROJECT_BURST: ${OUO_RATELIMIT_PROJECT_BURST:-5000} OUO_RATELIMIT_CLIENT_RATE: ${OUO_RATELIMIT_CLIENT_RATE:-100} OUO_RATELIMIT_CLIENT_BURST: ${OUO_RATELIMIT_CLIENT_BURST:-300} OUO_SENTRY_UPSTREAM: ${OUO_SENTRY_UPSTREAM:-none} OUO_SENTRY_PROJECT_ID: ${OUO_SENTRY_PROJECT_ID:-1} OUO_SENTRY_PUBLIC_KEY: ${OUO_SENTRY_PUBLIC_KEY:-publickey0000000000000000000000} OUO_SENTRY_SERVICE_NAME: ${OUO_SENTRY_SERVICE_NAME:-sentry-demo} configs: - source: collector target: /etc/otelcol/config.yaml - source: projects-htpasswd target: /etc/otelcol/projects.htpasswd depends_on: - loki - tempo - mimir
# Internal: replay chunk sink and the plugin's API. SDKs never talk to it. rum-api: image: ${OUO_IMAGE_REGISTRY:-ghcr.io/ouo11y}/rum-api:${OUO_RUM_API_VERSION:-latest} command: ["-config.file=/etc/rum-api/config.yaml", "-config.expand-env=true"] ports: - "${OUO_RUM_API_PORT:-8080}:8080" environment: GRAFANA_URL: http://localhost:${OUO_GRAFANA_PORT:-3000} OUO_INGEST_TOKEN: ${OUO_INGEST_TOKEN:-it_dev_0000000000000000} OUO_READ_TOKEN: ${OUO_READ_TOKEN:-rt_dev_0000000000000000} # Operator API /v1/admin/* and the per-project source-map upload tokens (ADR-0011). OUO_ADMIN_TOKEN: ${OUO_ADMIN_TOKEN:-at_dev_0000000000000000} OUO_UPLOAD_TOKENS: ${OUO_UPLOAD_TOKENS:-demo:ut_dev_demo_0000000000000000} REPLAY_RETENTION: ${OUO_RETENTION_REPLAYS:-720h} LOG_RETENTION: ${OUO_RETENTION_LOGS:-720h} REPLAY_SWEEP_INTERVAL: ${OUO_REPLAY_SWEEP_INTERVAL:-1h} # The image runs as nonroot. A named volume mounted on the existing nonroot home is # seeded with that owner; one mounted on /data would be root-owned and unwritable. DATA_DIR: /home/nonroot OTEL_EXPORTER_OTLP_ENDPOINT: http://otel-collector:4318 OTEL_EXPORTER_OTLP_HEADERS: ${OUO_BACKEND_OTLP_HEADERS:-Authorization=Basic%20aW50ZXJuYWw6cHRfZGV2X2ludGVybmFsXzAwMDAwMDAwMDAwMDAw} OTEL_EXPORTER_OTLP_PROTOCOL: http/protobuf OTEL_SERVICE_NAME: rum-api OTEL_RESOURCE_ATTRIBUTES: deployment.environment.name=local configs: - source: rum-api target: /etc/rum-api/config.yaml volumes: - rum-api-data:/home/nonroot depends_on: - loki - tempo - otel-collector
volumes: grafana-data: loki-data: tempo-data: mimir-data: rum-api-data:
configs: collector: # deploy/compose/collector/config.yaml at collector-v0.4.0 content: | # ouo-collector configuration for the local stack (ADR-0007: the collector is the edge). # # Image: ouo/collector, built from collector/ (collector/builder.yaml). Compose mounts this # file and projects.htpasswd into /etc/otelcol/. Ports: # 4318 OTLP/HTTP public Basic auth (project id : project token), CORS # 4319 replay public Basic auth, CORS, POST /v1/replay/chunks # 4320 Sentry public Sentry SDK envelopes (ADR-0008), Sentry DSN key auth, forwarded to Sentry # 4317 OTLP/gRPC loopback in compose, same Basic auth (send the Authorization metadata) # 13133 health loopback # 8888 self-metrics (Prometheus pull, not published; they are also pushed to Mimir) # # Environment: # OUO_INGEST_TOKEN bearer the ouoreplay exporter sends to rum-api (must equal rum-api's) # OUO_ALLOWED_ORIGINS optional comma-separated CORS origin list, no spaces # (default http://localhost:*,http://127.0.0.1:*) # OUO_RATELIMIT_CLIENT_KEY address (default) or x_forwarded_for behind a reverse proxy # OUO_RATELIMIT_{PROJECT,CLIENT}_{RATE,BURST} bucket sizes (defaults 1000/5000 and 100/300; # deploy/scripts/qa-edge.sh lowers them to test the buckets) # OUO_SENTRY_UPSTREAM Sentry ingest base URL the bridge forwards to (https://oNNN.ingest.sentry.io), # or "none" to answer locally (default none: no data leaves the laptop) # OUO_SENTRY_PROJECT_ID, OUO_SENTRY_PUBLIC_KEY the DSN's project id and public key # # Every receiver uses include_metadata: true so the resource processor and ouoratelimit # can read the authenticated project (auth.username) and request headers from the # client context. Anything that drops that context (batch) must come after them.
extensions: health_check: endpoint: 0.0.0.0:13133 basicauth/server: htpasswd: file: /etc/otelcol/projects.htpasswd # Receivers authenticate through ouoauth, which delegates to basicauth/server and # counts every rejected request (401) as ouo_auth_failures_total{reason} in Mimir. ouoauth: authenticator: basicauth/server # Receiver middleware sharing the ouoratelimit processor's buckets: the OTLP receiver # answers an empty export (no spans / records / points) 200 without calling the # pipeline, so only a middleware can charge it. Requests it charged are not charged # again by the processor; other receivers (replay, Sentry) are charged by the processor. ouoratelimit: processor: ouoratelimit
receivers: otlp: protocols: grpc: endpoint: 0.0.0.0:4317 include_metadata: true auth: authenticator: ouoauth middlewares: - id: ouoratelimit http: # 0.0.0.0 inside the container; exposure is decided by the compose port binding. endpoint: 0.0.0.0:4318 include_metadata: true max_request_body_size: 4194304 # 4 MiB, applied to the compressed and the decompressed body auth: authenticator: ouoauth middlewares: - id: ouoratelimit cors: allowed_origins: $${env:OUO_ALLOWED_ORIGINS:-http://localhost:*,http://127.0.0.1:*} allowed_headers: [Authorization, Content-Type, Content-Encoding, traceparent, baggage] exposed_headers: [Retry-After] max_age: 7200 ouoreplay: endpoint: 0.0.0.0:4319 include_metadata: true max_request_body_size: 8388608 # 8 MiB: frames chunks (rrweb chunks stop at max_rrweb_chunk_bytes, 4 MiB) auth: authenticator: ouoauth cors: allowed_origins: $${env:OUO_ALLOWED_ORIGINS:-http://localhost:*,http://127.0.0.1:*} allowed_headers: [Authorization, Content-Type, Content-Encoding, X-Ouo-Session-Id, X-Ouo-Seq] exposed_headers: [Retry-After] max_age: 7200 # Sentry SDK bridge (ADR-0008, docs/specs/sentry-bridge.md): apps change only their DSN host. # Every envelope is forwarded byte for byte to the project's upstream Sentry and Sentry's # answer goes back unchanged; accepted envelopes are translated into sessions, errors, # traces (logs/traces pipelines) and video replay chunks (routed to logs/replay). ouosentry: endpoint: 0.0.0.0:4320 upstream_timeout: 30s projects: - sentry_project_id: "$${env:OUO_SENTRY_PROJECT_ID:-1}" public_keys: ["$${env:OUO_SENTRY_PUBLIC_KEY:-publickey0000000000000000000000}"] upstream: $${env:OUO_SENTRY_UPSTREAM:-none} ouo_project_id: demo service_name: $${env:OUO_SENTRY_SERVICE_NAME:-sentry-demo}
connectors: # The Sentry bridge emits replay chunks on its logs output; send them to the replay # pipeline (ouoreplay exporter -> rum-api) and everything else to Loki. routing/sentry: default_pipelines: [logs] table: - context: log condition: log.attributes["ouo.replay.chunk"] == true pipelines: [logs/replay]
processors: memory_limiter: check_interval: 1s limit_mib: 512 spike_limit_mib: 128 # Token buckets per project (auth.username) and per client; over the limit the OTLP # receiver answers 429 + Retry-After (gRPC RESOURCE_EXHAUSTED + RetryInfo). All # pipelines share one set of buckets. Behind a reverse proxy (deploy/vm sets # OUO_RATELIMIT_CLIENT_KEY=x_forwarded_for) the TCP peer is the proxy, so key the # client bucket on the first X-Forwarded-For hop instead of the address. ouoratelimit: project: rate: $${env:OUO_RATELIMIT_PROJECT_RATE:-1000} burst: $${env:OUO_RATELIMIT_PROJECT_BURST:-5000} client: rate: $${env:OUO_RATELIMIT_CLIENT_RATE:-100} burst: $${env:OUO_RATELIMIT_CLIENT_BURST:-300} key_from: $${env:OUO_RATELIMIT_CLIENT_KEY:-address} idle_timeout: 10m # Stamp the authenticated project on every resource, overwriting whatever the client # sent (delete first so a request without auth context can never keep a spoofed value). resource/project: attributes: - key: ouo.project.id action: delete - key: ouo.project.id from_context: auth.username action: upsert batch: timeout: 1s send_batch_size: 512 send_batch_max_size: 1024 # Copy the OTLP top-level event_name (OTel Logs API `eventName`) into the # `event.name` log attribute so Loki can index it as a label regardless of # whether the SDK set it as a field or as an attribute. Also makes sure # `ouo.platform` exists as a log attribute when it was only set on the resource. transform/events: error_mode: ignore log_statements: - context: log statements: - set(log.attributes["event.name"], log.event_name) where log.event_name != "" and log.attributes["event.name"] == nil - set(log.attributes["ouo.platform"], resource.attributes["ouo.platform"]) where log.attributes["ouo.platform"] == nil and resource.attributes["ouo.platform"] != nil
# Collector >= 0.150 names the exporters otlp_grpc / otlp_http (otlp / otlphttp are deprecated aliases). exporters: otlp_grpc/tempo: endpoint: tempo:4317 tls: insecure: true otlp_http/loki: endpoint: http://loki:3100/otlp tls: insecure: true otlp_http/mimir: endpoint: http://mimir:9009/otlp tls: insecure: true # Replay chunks -> rum-api's internal sink (POST /v1/replay/chunks with # X-Ouo-Project-Id, X-Ouo-Session-Id, X-Ouo-Seq and the ingest bearer). ouoreplay: endpoint: http://rum-api:8080 ingest_token: $${env:OUO_INGEST_TOKEN} timeout: 30s sending_queue: enabled: true queue_size: 1000 retry_on_failure: enabled: true max_elapsed_time: 5m debug: verbosity: basic
service: extensions: [health_check, basicauth/server, ouoauth, ouoratelimit] telemetry: logs: level: info # Self-metrics (otelcol_receiver_*, otelcol_exporter_*, ouo_ratelimit_rejected_total, # ouo_auth_failures_total, ouo_replay_chunks_total): pushed to Mimir over OTLP every 15 s, and pullable on :8888 # inside the network for debugging. metrics: level: normal readers: - periodic: interval: 15000 exporter: otlp: protocol: http/protobuf endpoint: http://mimir:9009/otlp/v1/metrics - pull: exporter: prometheus: host: 0.0.0.0 port: 8888 pipelines: traces: receivers: [otlp, ouosentry] processors: [memory_limiter, ouoratelimit, resource/project, batch] exporters: [otlp_grpc/tempo] logs: receivers: [otlp, routing/sentry] processors: [memory_limiter, ouoratelimit, resource/project, transform/events, batch] exporters: [otlp_http/loki] metrics: receivers: [otlp] processors: [memory_limiter, ouoratelimit, resource/project, batch] exporters: [otlp_http/mimir] # Replay chunks: one log record per chunk, project stamped by the receiver. No batch: # the receiver waits for the queue hand-off and answers 202, 429 or 503 per chunk. logs/replay: receivers: [ouoreplay, routing/sentry] processors: [memory_limiter, ouoratelimit] exporters: [ouoreplay] # Sentry bridge output before routing; the project is in the client context # (auth.username), so resource/project and ouoratelimit work downstream. logs/sentry: receivers: [ouosentry] processors: [memory_limiter] exporters: [routing/sentry] projects-htpasswd: # deploy/compose/collector/projects.htpasswd at collector-v0.4.0 content: | # Development project registry for ouo-collector (basicauth extension, ADR-0007). # LOCAL USE ONLY. The demo token is public in this repository; never deploy this file. # One line per project. username = project id, password = project token, {SHA} hash. # Generate a line with htpasswd -nbs PROJECT TOKEN # Comment lines in this file must never contain a colon character (see project/kb/collector-components.md). demo:{SHA}IUpHBAUPHvXoJmvv4inmF4cN96I= # internal = the platform's own telemetry (rum-api, demo backend), token pt_dev_internal_00000000000000, # so it has its own rate-limit buckets and never spends a client project's quota. internal:{SHA}wXBw3q7GMLhIo7ifP3a0fC2Z14M= loki: # deploy/compose/loki/config.yaml at grafana-v0.5.0 content: | # Loki 3.x single-binary, filesystem storage, native OTLP ingestion. # Started with -config.expand-env=true: the OUO_* variables below come from compose.yaml (retention, # deletion timings; docs/operations/data-lifecycle.md). auth_enabled: false
server: http_listen_port: 3100 grpc_listen_port: 9096 log_level: warn
common: instance_addr: 127.0.0.1 path_prefix: /loki storage: filesystem: chunks_directory: /loki/chunks rules_directory: /loki/rules replication_factor: 1 ring: kvstore: store: inmemory
schema_config: configs: - from: "2024-01-01" store: tsdb object_store: filesystem schema: v13 index: prefix: index_ period: 24h
storage_config: tsdb_shipper: active_index_directory: /loki/tsdb-index cache_location: /loki/tsdb-cache
# Retention and deletion (roadmap 8b): the compactor applies limits_config.retention_period # (and per-project retention_stream entries from runtime.yaml) and processes delete # requests (POST /loki/api/v1/delete, sent by rum-api's delete-by-user / -project). compactor: working_directory: /loki/compactor compaction_interval: $${OUO_LOKI_COMPACTION_INTERVAL:-10m} retention_enabled: true # Chunks marked for deletion are removed after this delay. retention_delete_delay: $${OUO_LOKI_RETENTION_DELETE_DELAY:-2h} delete_request_store: filesystem # Grace period during which a delete request can be cancelled before the compactor # acts on it. delete_request_cancel_period: $${OUO_LOKI_DELETE_CANCEL_PERIOD:-24h}
# Per-project overrides (retention_stream on the ouo_project_id label), reloaded every # 10 s without a restart. runtime_config: file: /etc/loki/runtime.yaml period: 10s
pattern_ingester: enabled: true
ruler: storage: type: local local: directory: /loki/rules
limits_config: # Structured metadata is required for OTLP ingestion: every attribute that is # not promoted to a label lands there (session.id, user.id, trace_id, ...). allow_structured_metadata: true volume_enabled: true reject_old_samples: true reject_old_samples_max_age: 168h ingestion_rate_mb: 32 ingestion_burst_size_mb: 64 max_label_names_per_series: 30 # Retention of every stream (OUO_RETENTION_LOGS, at least 24h); runtime.yaml can set a # shorter or longer one per project. Delete requests remove lines from chunks. retention_period: $${OUO_RETENTION_LOGS:-720h} deletion_mode: filter-and-delete # Label promotion for OTLP (ADR-0001): only service_name, event_name, # ouo_platform and ouo_project_id (one value per project, for per-project # retention_stream and deletes) become stream labels; session.id stays # structured metadata so cardinality stays low. Loki rewrites "." to "_". otlp_config: resource_attributes: # Loki's default list also promotes service.instance.id, k8s.*, cloud.* which # would explode cardinality for browser SDKs; replace it with a fixed set. ignore_defaults: true attributes_config: - action: index_label attributes: - service.name - service.namespace - deployment.environment.name - ouo.platform - ouo.project.id log_attributes: - action: index_label attributes: - event.name - ouo.platform
analytics: reporting_enabled: false loki-runtime: # deploy/compose/loki/runtime.yaml at grafana-v0.5.0 content: | # Loki runtime overrides, reloaded every 10 s (runtime_config in config.yaml): edit and # save, no restart. Per-project log retention goes here as retention_stream entries on # the ouo_project_id stream label; the default is limits_config.retention_period # (OUO_RETENTION_LOGS). Loki's single tenant is "fake" (auth_enabled: false). # docs/operations/data-lifecycle.md, section Per-project retention. # # overrides: # fake: # retention_stream: # - selector: '{ouo_project_id="ttltest"}' # priority: 1 # period: 24h # at least 24h overrides: {} tempo: # deploy/compose/tempo/config.yaml at grafana-v0.5.0 content: | # Tempo 3.x single binary, local storage, OTLP receiver, metrics generator -> Mimir. # Started with -config.expand-env=true (trace retention from compose.yaml). # Only keys that exist in 3.0.x are used; see project/kb/collector-lgtm.md. # Grafana >= 11 streams search/trace results over gRPC on the HTTP port (h2c). stream_over_http_enabled: true
server: http_listen_port: 3200 log_level: warn
distributor: receivers: otlp: protocols: grpc: endpoint: 0.0.0.0:4317 http: endpoint: 0.0.0.0:4318
# Tempo 3.0 removed the 2.x `ingester` and `compactor` top-level blocks # (block-builder / live-store replace them); defaults are fine locally.
# Trace retention (roadmap 8b): blocks older than block_retention are deleted by the # backend scheduler's retention job (hourly) run on the backend worker. Both carry the # compaction settings in 3.0; keep them equal. Tempo has no delete API: single traces # are removed with `tempo-cli redact` (docs/operations/data-lifecycle.md). backend_scheduler: provider: compaction: compaction: block_retention: $${OUO_RETENTION_TRACES:-336h}
backend_worker: compaction: block_retention: $${OUO_RETENTION_TRACES:-336h}
querier: frontend_worker: frontend_address: 127.0.0.1:9095
query_frontend: search: duration_slo: 5s throughput_bytes_slo: 1.073741824e+09 trace_by_id: duration_slo: 5s
metrics_generator: registry: external_labels: source: tempo storage: path: /var/tempo/generator/wal remote_write: - url: http://mimir:9009/api/v1/push send_exemplars: true
storage: trace: backend: local wal: path: /var/tempo/wal local: path: /var/tempo/blocks
overrides: defaults: metrics_generator: processors: [service-graphs, span-metrics] mimir: # deploy/compose/mimir/config.yaml at grafana-v0.5.0 content: | # Mimir monolithic mode, filesystem storage, OTLP push at /otlp/v1/metrics. # Started with -config.expand-env=true (metric retention from compose.yaml). target: all multitenancy_enabled: false
server: http_listen_port: 9009 grpc_listen_port: 9095 log_level: warn
common: storage: backend: filesystem filesystem: dir: /data/blocks
blocks_storage: storage_prefix: blocks bucket_store: sync_dir: /data/tsdb-sync tsdb: dir: /data/tsdb
compactor: data_dir: /data/compactor sharding_ring: instance_addr: 127.0.0.1 kvstore: store: memberlist
distributor: ring: instance_addr: 127.0.0.1 kvstore: store: memberlist
ingester: ring: instance_addr: 127.0.0.1 replication_factor: 1 kvstore: store: memberlist
store_gateway: sharding_ring: instance_addr: 127.0.0.1 replication_factor: 1 kvstore: store: memberlist
ruler_storage: storage_prefix: rules
alertmanager_storage: storage_prefix: alertmanager
memberlist: bind_addr: [127.0.0.1] join_members: []
limits: # Accept metrics without OTel-style unit/type suffix mangling and keep resource attrs. otel_metric_suffixes_enabled: false max_global_series_per_user: 1000000 ingestion_rate: 100000 ingestion_burst_size: 500000 out_of_order_time_window: 5m # Metric retention (roadmap 8b): the compactor deletes blocks older than this. 0 keeps # everything. compactor_blocks_retention_period: $${OUO_RETENTION_METRICS:-2160h}
usage_stats: enabled: false rum-api: # deploy/compose/rum-api/config.yaml at rum-api-v0.5.0 content: | # rum-api configuration for the local stack, mounted at /etc/rum-api/config.yaml and # started with -config.file=/etc/rum-api/config.yaml -config.expand-env=true (like Loki # and Tempo). Every key and its environment override: services/rum-api/config.example.yaml. # Environment variables set in compose.yaml win over this file (GRAFANA_URL does, so the # deep links follow OUO_GRAFANA_PORT and deploy/vm/compose.tls.yaml's public domain).
server: http_listen_address: ":8080"
# Secrets come from the container environment (compose.yaml), never from this file. auth: ingest_token: "$${OUO_INGEST_TOKEN}" read_token: "$${OUO_READ_TOKEN}" admin_token: "$${OUO_ADMIN_TOKEN}"
loki: url: http://loki:3100 datasource_uid: loki
tempo: url: http://tempo:3200 datasource_uid: tempo
session: inactivity: 30m max_duration: 4h
storage: # filesystem: deploy/data/rum-api/replays on the host (bind mount, like Loki and Tempo # keep their local filesystem storage). compose.s3.yaml switches this to s3 # (STORAGE_BACKEND=s3) against the MinIO below. backend: filesystem filesystem: dir: /data s3: endpoint: minio:9000 bucket: $${OUO_RUM_API_S3_BUCKET:-ouo-replays} region: us-east-1 access_key_id: "$${S3_ACCESS_KEY_ID}" secret_access_key: "$${S3_SECRET_ACCESS_KEY}" insecure: true # plain HTTP inside the compose network force_path_style: true # MinIO
# Replay retention and the sweeper: REPLAY_RETENTION / LOG_RETENTION / REPLAY_SWEEP_INTERVAL # from compose.yaml (OUO_RETENTION_REPLAYS, OUO_RETENTION_LOGS) override these. retention: replays: 720h logs: 720h sweep_interval: 1h
# Per-project replay retention and server-side sampling caps (restart rum-api after an # edit). Keep a project's log retention in loki/runtime.yaml in step. projects: {} # ttltest: # retention: { replays: 24h } # sampling: { session_rate: 1, replay_rate: 0.25 } grafana-datasources: # deploy/compose/grafana/provisioning/datasources/datasources.yaml at grafana-v0.5.0 content: | apiVersion: 1
deleteDatasources: - name: Loki orgId: 1 - name: Tempo orgId: 1 - name: Mimir orgId: 1
datasources: - name: Loki uid: loki type: loki access: proxy url: http://loki:3100 editable: false jsonData: maxLines: 1000 derivedFields: # OTLP log records carry the trace id as structured metadata `trace_id`. - name: TraceID datasourceUid: tempo matcherType: label matcherRegex: trace_id url: "$$$${__value.raw}" urlDisplayLabel: View trace
- name: Tempo uid: tempo type: tempo access: proxy url: http://tempo:3200 editable: false jsonData: streamingEnabled: search: true nodeGraph: enabled: true serviceMap: datasourceUid: mimir tracesToMetrics: datasourceUid: mimir spanStartTimeShift: "-5m" spanEndTimeShift: "5m" lokiSearch: datasourceUid: loki tracesToLogsV2: datasourceUid: loki spanStartTimeShift: "-30m" spanEndTimeShift: "30m" filterByTraceID: false filterBySpanID: false customQuery: true # `$${__span.tags[...]}` resolves both span attributes (session.id) and resource # attributes (service.name): Grafana merges process tags into __span.tags and the # field path goes through lodash `property`, so bracket keys with dots work. # session.id is structured metadata in Loki, so it is a `|` filter, never a # stream selector. `$$$$` escapes the provisioner's own $${ENV} expansion. tags: - key: service.name value: service_name query: '{service_name="$$$${__span.tags["service.name"]}"} | session_id="$$$${__span.tags["session.id"]}"'
- name: Mimir uid: mimir type: prometheus access: proxy url: http://mimir:9009/prometheus editable: false jsonData: httpMethod: POST prometheusType: Mimir prometheusVersion: "2.9.1" timeInterval: 15s exemplarTraceIdDestination: - name: trace_id datasourceUid: tempo grafana-plugin-app: # deploy/compose/grafana/provisioning/plugins/ouo11y-rum-app.yaml at grafana-v0.5.0 content: | # Enables the ouo11y RUM app plugin (plugins/ouo11y-rum-app, loaded unsigned from # /var/lib/grafana/plugins/ouo11y-rum-app) and points its "rum" route at the # in-network rum-api. The browser never talks to rum-api directly: the plugin # calls /api/plugin-proxy/ouo11y-rum-app/rum/v1/... and Grafana forwards to rumApiUrl, # adding `Authorization: Bearer <readToken>` from secureJsonData (ADR-0007 section 6). # # $${OUO_READ_TOKEN} is expanded by the Grafana provisioner from the container's # environment (compose passes it through; default rt_dev_0000000000000000, a # development value). It must equal rum-api's OUO_READ_TOKEN. Secure fields are # write-only: the settings API reports `secureJsonFields: {readToken: true}`. # # $${OUO_FEATURE_TOGGLES_ENABLE} turns plugin feature toggles on, comma separated, like # Grafana's GF_FEATURE_TOGGLES_ENABLE (empty: all off). The list is in # plugins/ouo11y-rum-app/README.md, e.g. OUO_FEATURE_TOGGLES_ENABLE=sessionTimelinePanel. apiVersion: 1
apps: - type: ouo11y-rum-app org_id: 1 disabled: false jsonData: rumApiUrl: http://rum-api:8080 featureToggles: "$${OUO_FEATURE_TOGGLES_ENABLE}" secureJsonData: readToken: $${OUO_READ_TOKEN}Send real telemetry: with the SDK, with plain OpenTelemetry, or by building Tiny Shop against this stack.