How to setup elk stack in docker - Software Engineering Musings

216 words

1 minutes

How to setup elk stack in docker

2020-10-16

Introduction#

The Elastic Stack (also known as the ELK Stack) is used across a variety of use cases — from observability to security, from enterprise search to business analytics.

ELK is the acronym for three open source projects: Elasticsearch, Logstash, and Kibana.

On this blog post, we will go through necessary steps to run elk using docker.

Using docker#

Create a Docker network to enable communication between containers via container name.

 docker network create elk

Run elasticsearch docker container.

docker run -d --name elasticsearch --net elk -p 9200:9200 -e "discovery.type=single-node" elasticsearch:7.9.2

Create logstash configuration file logstash.conf.

cat <<EOF> ~/logstash.conf
input {
  tcp {
    port => 5044
    codec => json_lines
  }
}
output {
  elasticsearch {
    hosts => ["http://elasticsearch:9200"]
    index => "example-%{appname}-%{env}"
  }
}
EOF

Run logstash docker container.

docker run -d --name logstash --net elk -p 5044:5044 -v ~/logstash.conf:/usr/share/logstash/pipeline/logstash.conf logstash:7.9.2

Run kibana docker container.

docker run -d --name kibana --net elk -e "ELASTICSEARCH_URL=http://elasticsearch:9200" -p 5601:5601 kibana:7.9.2

Using docker-compose#

Create logstash configuration file logstash.conf.

cat <<EOF> ~/logstash.conf
input {
  tcp {
    port => 5044
    codec => json_lines
  }
}
output {
  elasticsearch {
    hosts => ["http://elasticsearch:9200"]
    index => "example-%{appname}-%{env}"
  }
}
EOF

docker-compose.yaml configuration for elk stack:

version: '3.1'
services:
  elasticsearch:
        environment:
      - discovery.type=single-node
    ports:
      - 9200
  logstash:
        volumes:
      - ~/logstash.conf:/usr/share/logstash/pipeline/logstash.conf
    environment:
      ELASTICSEARCH_URL: http://elasticsearch:9200
    ports:
      - 5000:5000
    links:
      - elasticsearch
  kibana:
        environment:
      ELASTICSEARCH_URL: http://elasticsearch:9200
    ports:
      - 5601:5601
    links:
      - elasticsearch