3. Backend traces

The API gets the plain OpenTelemetry Node SDK: nothing from ouo11y. The auto-instrumentation creates a span for every HTTP request and Express middleware; you add your own spans for the work inside a request.

Step 5. Instrument the backend

Stop the API with Ctrl+C in its terminal tab, then:

cd ~/Desktop/tiny-shop/backend

npm install @opentelemetry/api @opentelemetry/sdk-node @opentelemetry/sdk-trace @opentelemetry/exporter-trace-otlp-http @opentelemetry/instrumentation-http @opentelemetry/instrumentation-express @opentelemetry/baggage-span-processor @opentelemetry/resources @opentelemetry/semantic-conventions

What each package is for:

Package	For
@opentelemetry/api	what your code calls (trace.getTracer)
@opentelemetry/sdk-node	runs the SDK
@opentelemetry/sdk-trace	the span processors
@opentelemetry/exporter-trace-otlp-http	sends spans to the collector
@opentelemetry/instrumentation-http, instrumentation-express	create the HTTP server and Express spans for you
@opentelemetry/baggage-span-processor	copies the browser’s session id onto your spans (page 4)
@opentelemetry/resources, semantic-conventions	name the service

5a. Initialise the SDK

Create backend/instrumentation.js:

backend/instrumentation.js

backend/instrumentation.js

// OpenTelemetry SDK init. Loaded before the app with `node --require ./instrumentation.js`
// so the http and express instrumentations can patch those modules before server.js loads them.
const { NodeSDK } = require("@opentelemetry/sdk-node");
const { BatchSpanProcessor } = require("@opentelemetry/sdk-trace");
const { OTLPTraceExporter } = require("@opentelemetry/exporter-trace-otlp-http");
const { HttpInstrumentation } = require("@opentelemetry/instrumentation-http");
const { ExpressInstrumentation } = require("@opentelemetry/instrumentation-express");
const { BaggageSpanProcessor, ALLOW_ALL_BAGGAGE_KEYS } = require("@opentelemetry/baggage-span-processor");
const { resourceFromAttributes } = require("@opentelemetry/resources");
const { ATTR_SERVICE_NAME, ATTR_SERVICE_VERSION } = require("@opentelemetry/semantic-conventions");

// The ouo collector, the only ingest. Moved ports? OUO_OTLP_ENDPOINT=http://localhost:14318
const endpoint = process.env.OUO_OTLP_ENDPOINT || "http://localhost:4318";

// Every request needs the project credential: Basic base64("<project id>:<project token>").
// This is the quickstart's `demo` project: base64("demo:pt_dev_demo_0000000000000000").
const exporter = new OTLPTraceExporter({
  url: `${endpoint}/v1/traces`,
  headers: { Authorization: "Basic ZGVtbzpwdF9kZXZfZGVtb18wMDAwMDAwMDAwMDAwMDAw" },
});

const sdk = new NodeSDK({
  resource: resourceFromAttributes({
    [ATTR_SERVICE_NAME]: "tiny-shop-backend",
    [ATTR_SERVICE_VERSION]: "1.0.0",
    "deployment.environment.name": "local",
  }),
  spanProcessors: [
    // The browser SDK sends `baggage: session.id=...,user.id=...`; copy it onto every span so
    // backend spans can be searched by the RUM session.
    new BaggageSpanProcessor(ALLOW_ALL_BAGGAGE_KEYS),
    // OTel JS 3.0: options object, `{ exporter }` (2.x took the exporter itself).
    new BatchSpanProcessor({ exporter }),
  ],
  instrumentations: [new HttpInstrumentation(), new ExpressInstrumentation()],
});

sdk.start();
console.log(`[otel] tracing to ${endpoint}/v1/traces`);

const shutdown = () => sdk.shutdown().finally(() => process.exit(0));
process.on("SIGINT", shutdown);
process.on("SIGTERM", shutdown);

Three things to know about it:

• The credential. The collector accepts nothing without Authorization: Basic base64("<project id>:<project token>"). The quickstart ships the demo project; check the value with printf 'demo:pt_dev_demo_0000000000000000' | base64.
• The endpoint. http://localhost:4318 unless you moved the collector in step 4c; then start the API with OUO_OTLP_ENDPOINT=http://localhost:14318 (shown below).
• OpenTelemetry JS 3.0. BatchSpanProcessor comes from @opentelemetry/sdk-trace and takes { exporter }. Snippets written for 2.x use sdk-trace-base and new BatchSpanProcessor(exporter); that version starts fine and crashes at the first export.

The resource attributes are OpenTelemetry semantic conventions: service.name, service.version and deployment.environment.name. They are how you find this service in Tempo below.

5b. Add manual spans

Replace backend/server.js with:

backend/server.js

backend/server.js

// The API. HTTP server spans come from the auto-instrumentation; the spans below are manual.
const express = require("express");
const cors = require("cors");
const { trace, SpanStatusCode } = require("@opentelemetry/api");

const tracer = trace.getTracer("tiny-shop-backend");
const app = express();
// The browser calls this API from another origin and adds `traceparent` and `baggage`
// headers; cors() answers the preflight for them.
app.use(cors());
app.use(express.json());

const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
const products = [
  { id: 1, name: "Coffee", price: 4 },
  { id: 2, name: "Tea", price: 3 },
  { id: 3, name: "Cake", price: 6 },
];

app.get("/api/products", async (req, res) => {
  // A manual span around a "database" call.
  const result = await tracer.startActiveSpan("db.select products", async (span) => {
    span.setAttribute("db.system.name", "fake-db");
    await sleep(50 + Math.random() * 100);
    span.setAttribute("app.products.count", products.length);
    span.end();
    return products;
  });
  res.json(result);
});

app.post("/api/checkout", async (req, res) => {
  const items = req.body.items || [];
  // A parent span with two nested children.
  await tracer.startActiveSpan("checkout", async (span) => {
    span.setAttribute("app.cart.items", items.length);
    await tracer.startActiveSpan("validate cart", async (child) => {
      await sleep(30);
      child.end();
    });
    await tracer.startActiveSpan("charge card", async (child) => {
      await sleep(120);
      child.addEvent("payment accepted");
      child.end();
    });
    span.end();
  });
  res.json({ ok: true, orderId: Math.floor(Math.random() * 100000) });
});

app.get("/api/fail", (req, res) => {
  // Enrich the auto-created HTTP span with the error.
  const span = trace.getActiveSpan();
  const err = new Error("inventory service unavailable");
  span?.recordException(err);
  span?.setStatus({ code: SpanStatusCode.ERROR, message: err.message });
  res.status(500).json({ error: err.message });
});

const port = Number(process.env.PORT || 5001);
app.listen(port, () => console.log(`tiny-shop API on http://localhost:${port}`));

startActiveSpan makes the new span the current one while the callback runs, so the spans started inside it become its children, and all of them become children of the HTTP span the auto-instrumentation opened for the request. Always end() a span you started.

db.system.name is the OpenTelemetry name for the kind of database; the app. attributes are this app’s own, a prefix OpenTelemetry leaves to applications.

5c. Run it with the SDK loaded first

The SDK has to load before Express, so the instrumentation can patch it. Save that as the start script:

npm pkg set scripts.start="node --require ./instrumentation.js server.js"

npm start

It prints [otel] tracing to http://localhost:4318/v1/traces. Collector on 14318 (step 4c)? Stop it and start it this way instead:

OUO_OTLP_ENDPOINT=http://localhost:14318 npm start

Your backend/package.json now has that start script and the packages above. The finished one, for comparison (your version numbers may be newer):

backend/package.json

backend/package.json

{
  "name": "tiny-shop-backend",
  "private": true,
  "version": "1.0.0",
  "description": "ouo quickstart: Express API traced with the OpenTelemetry Node SDK (auto + manual spans)",
  "type": "commonjs",
  "scripts": {
    "start": "node --require ./instrumentation.js server.js"
  },
  "engines": {
    "node": ">=24"
  },
  "dependencies": {
    "@opentelemetry/api": "^1.10.0",
    "@opentelemetry/baggage-span-processor": "^0.6.0",
    "@opentelemetry/exporter-trace-otlp-http": "^0.300.0",
    "@opentelemetry/instrumentation-express": "^0.72.0",
    "@opentelemetry/instrumentation-http": "^0.300.0",
    "@opentelemetry/resources": "^3.0.0",
    "@opentelemetry/sdk-node": "^0.300.0",
    "@opentelemetry/sdk-trace": "^3.0.0",
    "@opentelemetry/semantic-conventions": "^1.43.0",
    "cors": "^2.8.6",
    "express": "^5.3.0"
  }
}

Send some traffic from another tab:

curl -X POST -H 'content-type: application/json' -d '{"items":[1,2]}' localhost:5001/api/checkout

curl localhost:5001/api/fail

The first answers {"ok":true,"orderId":...}, the second {"error":"inventory service unavailable"}.

5d. See the backend traces in Grafana

Wait up to a minute (Tempo’s search catches up; open a trace by id and it is there sooner), then in Grafana: Explore, data source Tempo, TraceQL tab:

TraceQL

{ resource.service.name = "tiny-shop-backend" }

What you should see

Two traces at least. Open the POST /api/checkout one:

1. tiny-shop-backendPOST /api/checkoutHTTP server span, auto-instrumentation
   158 ms
2. middleware - corsMiddleware
   1 ms
3. middleware - jsonParser
   2 ms
4. request handler - /api/checkout
   1 ms
5. checkoutmanual · app.cart.items = 2
   152 ms
6. validate cartmanual
   31 ms
7. charge cardmanual · event "payment accepted"
   121 ms

The checkout trace from curl. The root is the API's HTTP span; the tick on charge card is its payment accepted event. Durations are illustrative; the sleeps in server.js set them roughly.

POST /api/checkout, then the Express middleware spans, then checkout with validate cart and charge card (with its payment accepted event under Events). The GET /api/fail trace is red, with the exception recorded on it:

1. tiny-shop-backendGET /api/failstatus error · exception: inventory service unavailable
   3 ms

The failing call's HTTP span (its Express middleware spans left out): red, with the exception and status message that server.js sets.

Note

Nothing in Tempo after a minute? Check the API’s terminal for errors and the troubleshooting table.

The backend is traced on its own. Next, the browser starts the trace.