Observability: Node js Elasticsearch Example

Intro

Logging is one of the most fundemental observability concepts needed in Node.js programming. Whether we have a cli app, REST api app, or graphql app, we use logs to make sure things are going well and to debug. Logstash, part of the ELK stack, makes collecting logs simple. When using Kibana with Logstash, we get a nice dashboard to monitor our app. In this article, we will build a node js elasticsearch example.

To read more about logging in the context of architecture, I recommend this: https://microservices.io/patterns/observability/application-logging.html. Note that many of the patterns on that site are not just for microservices.

Starting Elk

To start up ELK is quite a feat, however, docker can make this easy. I recommend using docker as in practice you will either use a service or your devops team will host this using kubernetes for you.

Start by creating a config file. I called my udp.conf. Add the following to the file

input {
  udp {
    port => 5000
    type => "json"
    codec => "json"
  }
}
output {
  elasticsearch {
    hosts => ["localhost:9200"]
    document_type => "nodelog"
    manage_template => false
    index => "nodejs-%{+YYYY.MM.dd}"
  }
}

Run the following command to start up the full ELK stack.

docker run \
    -p 5601:5601 -p 9200:9200 -p 5044:5044 -p 5000:5000/udp \
    -v $PWD/elkexample/udp.conf:/etc/logstash/conf.d/99-input-udp.conf \
    -it --name elk sebp/elk

When running the above I received a few errors on windows.

• max virtual memory areas vm.max_map_count [65530] likely too low, increase to at least [262144]
• max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]

To fix this, I had to give docker more memory.

• You can view this article to do that: https://docs.docker.com/docker-for-windows/#:~:text=Memory%3A%20By%20default%2C%20Docker%20Desktop,swap%20file%20size%20as%20needed
• if you are using WSL, use this: https://docs.microsoft.com/en-us/windows/wsl/wsl-config#configure-global-options-with-wslconfig
• You might need to open WSL console and run this command. See here sysctl -w vm.max_map_count=262144 https://stackoverflow.com/questions/51445846/elasticsearch-max-virtual-memory-areas-vm-max-map-count-65530-is-too-low-inc

We can see windows docker is fun :D

Then add run this command with the new limit option.

docker run \
    --ulimit nofile=65535:65535 \
    -p 5601:5601 -p 9200:9200 -p 5044:5044 -p 5000:5000/udp \
    -v $PWD/elkexample/udp.conf:/etc/logstash/conf.d/99-input-udp.conf \
    -it --name elk sebp/elk

Create the Node Code

Start by creating a new file

touch index.js

We will use the base node libraries to send our logs. Let's start by importing the dgram module which we will use for udp socket connections and the os module to get the server host name.

const dgram = require('dgram');
const os = require('os');

const client = dgram.createSocket('udp4');
const host = os.hostname();

Next, let's create a log function which will build a payload and send this payload via our udp client above.

function log(severity, type, fields) {
    const payload = JSON.stringify({
        '@timestamp': (new Date()).toISOString(),
        "@version": 1,
        app: 'web-api', 
        environment: NODE_ENV,
        severity,
        type,
        fields,
        host
    });
    
    client.send(payload, LS_PORT, LS_HOST, (err) => {
        console.log(err)
    });
};

And finally, we can call the log method like so.

log('verbose', 'some-tag', {details: 'anobject with details'});

You can vist the url: http://localhost:9200/_search?pretty and you will see your logs under hits.

Viewing in Kibana

If you would like to view in Kibana, first go here: http://localhost:5601/app/management/kibana/indexPatterns

Then create an index pattern node*.

Then go here: http://localhost:5601/app/logs/settings, and select your index pattern.

You should then see your logs on the stream: http://localhost:5601/app/logs/stream.

Next, you can go to the Kibana visualization to create a dashboard.