Observability: Node Distributed Tracing with Open Zipkin Example

Intro

Eventually apps get complicated and make many requests. When building out services such as microservices or even just multi services, debugging our apps get a bit harder. The services will usually make requests to each other, especially when we need syncronous calls, async can be handled by message queues. In these situations, we can turn to Distributed Tracing to assist. In this article, we will learn Node distributed tracing with Open Zipkin.

Starting Zipkin

To start zipkin, we will use docker. We can start the server by running the following command.

docker run -d -p 9411:9411 openzipkin/zipkin

Installing modules

We will need quite a few modules for this project. Run the command below to install all the below.

npm install zipkin zipkin-instrumentation-express zipkin-transport-http node-fetch zipkin-instrumentation-fetch

Creating the Services

For this example, we will create two services. They will pretty much be the same except in name. The first service will also call the second service so that we can view the full trace.

Service 1

Start by including all the modules we installed.

touch server.js

const express = require('express');
const fetch = require('node-fetch')

const {Tracer, ExplicitContext, BatchRecorder, jsonEncoder: {JSON_V2}} = require('zipkin');
const {HttpLogger} = require('zipkin-transport-http');
const zipkinMiddleware = require('zipkin-instrumentation-express').expressMiddleware;
const wrapFetch = require('zipkin-instrumentation-fetch');

Next, we will create the tracer using Zipkin. This looks like a lot, but there are three components:

Context
Recorder
The local service name

// Create the tracer
const ctxImpl = new ExplicitContext();
const recorder = new BatchRecorder({
    logger: new HttpLogger({
      endpoint: 'http://localhost:9411/api/v2/spans',
      jsonEncoder: JSON_V2
    })
});
const localServiceName = 'service-1'; // name of this application
const tracer = new Tracer({ctxImpl, recorder, localServiceName});

Now, we will create a fetch wrapper using Zipkin (for tracing), an express app, and attach the zipkin middleware.

// Create fetch
const zipkinFetch = wrapFetch(fetch, {tracer});

const app = express();

// Add the Zipkin middleware
app.use(zipkinMiddleware({tracer}));

Finally, we create en endpoint to call our next service.

app.get('/server1', async (req, res) => {
    const server2Res = await zipkinFetch('http://localhost:3002/server2')
    const server2Data  = await server2Res.json();

    return res.json({
        name: 'server1',
        server2Data
    })
})

app.listen(3001, () => {
    console.log("Started server 1")
})

Here is the full code.

const express = require('express');
const fetch = require('node-fetch')

const {Tracer, ExplicitContext, BatchRecorder, jsonEncoder: {JSON_V2}} = require('zipkin');
const {HttpLogger} = require('zipkin-transport-http');
const zipkinMiddleware = require('zipkin-instrumentation-express').expressMiddleware;
const wrapFetch = require('zipkin-instrumentation-fetch');

// Create the tracer
const ctxImpl = new ExplicitContext();
const recorder = new BatchRecorder({
    logger: new HttpLogger({
      endpoint: 'http://localhost:9411/api/v2/spans',
      jsonEncoder: JSON_V2
    })
});
const localServiceName = 'service-1'; // name of this application
const tracer = new Tracer({ctxImpl, recorder, localServiceName});

// Create fetch
const zipkinFetch = wrapFetch(fetch, {tracer});

const app = express();

// Add the Zipkin middleware
app.use(zipkinMiddleware({tracer}));

app.get('/server1', async (req, res) => {
    const server2Res = await zipkinFetch('http://localhost:3002/server2')
    const server2Data  = await server2Res.json();

    return res.json({
        name: 'server1',
        server2Data
    })
})

app.listen(3001, () => {
    console.log("Started server 1")
})

Service 2

The second servce will be mostly the same, but will not have the fetch module calling anything, although you can add that for more tracing.

touch server2.js

const express = require('express');
const {Tracer, ExplicitContext, BatchRecorder, jsonEncoder: {JSON_V2}} = require('zipkin');
const {HttpLogger} = require('zipkin-transport-http');
const zipkinMiddleware = require('zipkin-instrumentation-express').expressMiddleware;

const ctxImpl = new ExplicitContext();
const recorder = new BatchRecorder({
    logger: new HttpLogger({
      endpoint: 'http://localhost:9411/api/v2/spans',
      jsonEncoder: JSON_V2
    })
});
const localServiceName = 'service-2'; // name of this application
const tracer = new Tracer({ctxImpl, recorder, localServiceName});

const app = express();

// Add the Zipkin middleware
app.use(zipkinMiddleware({tracer}));

app.get('/server2', (req, res) => {
    return res.json({
        name: 'server2'
    })
})

app.listen(3002, () => {
    console.log("Started server 2")
})

Starting the services

Now, let's start both services. You will need to run them in two separate terminals.

node server.js

In another terminal.

node server2.js

Now visit the first service in a browser or curl it.

curl http://localhost:3001/server1

The Zipkin Dashboard

Now that we have some trace data, let's load up the Zipkin dashboard to view it. Start by visiting this url.

http://localhost:9411/zipkin/

Click run query and you should see something similar to the below. Click one of the traces "Show" button.

Here on the trace, we can see a timeline of the api calls.