Docker deployment of Graylog

I was working in Docker for my summer internship. I had the work to scale up the graylog cluster that the company used and do that using Kubernetes. But first I needed to learn the basics. Kubernetes uses Docker inside it, and hence I needed to learn Docker first. Well then, what is Docker? In very simple terms Docker is a Utility that helps in creating and deploying applications. The applications that are deployed in the server are taken care inside containers which are like VMs. This way the same server is able to deploy more than one app using different ports on it. Docker is a tool that enables this process to occur smoothly and lets your application to receive traffic. You can create, deploy and run applications using it. But what is exactly that makes it so renowned and special? Its the concept of containers (different than VMs) and the ability of docker to quickly create ready to run applications using the concept of microservices.

I can go on to explain docker in detail but that I left for you to read. To make this short writing interesting I will explain to deploy and run graylog in docker, and dissect the code that we run. It’s gonna be fun!

All the images (the compact binaries of the application that when run by docker run command get converted to containers) are stored in an online registry called docker-hub. In any general app deployment process, we first download the docker image from the docker-hub. The images are ready to be run on the server, the one additional process that we need to do is to port-forward the docker-host port to the container port, otherwise, it will be running but won’t take any traffic unless we execute commands directly to the container. It might scare you at first go, but think of it as the docker host runs the containers and we can contact the docker-host but not the container directly. To contact the container we must allow the ports to be exposed to the outside world.

For the following commands to work, one must have docker installed in their machine, see here for details on how to install.

The commands are as follows:

$ docker run --name some-mongo -d mongo:2
$ docker run --name some-elasticsearch -d elasticsearch:2 elasticsearch -Des.cluster.name="graylog"
$ docker run --link some-mongo:mongo --link some-elasticsearch:elasticsearch -p 9000:9000 -p 1514:1514 -e GRAYLOG_WEB_ENDPOINT_URI="http://127.0.0.1:9000/api" -d graylog2/server


Code Dissection:

  • Line:1 : The docker run command directly deployed the container mongo-db, it will first search for the image if present on the local machine, if not it will download it from docker-hub. Note we don’t port forward (don’t use -p flag) this container as we don’t want the outside world to connect to this container. The name argument gives the container a name which we can use in future to access the container. Note the -d argument. It says the docker to keep running the container as a daemon (in the background) and not making the terminal inaccessible to future commands. The name of the image that needs to be downloaded is provided as mongo:2.

  • Line:2 : There’s one extra thing after the image name elastic search, they are the commands that we provide to the container of elastic-search. This is the power of microservices. Stand clear people, the same image of elastic search which can be used anywhere for our purpose is being configured to run with graylog. The same is for mongo-db, see how nicely the image that was present as a discrete entity is configured together with other images to make our whole new application - people, Microservices for you! The command for the elastic search says the container that we are using graylog named cluster to connect to and be ready for accepting data from graylog. Note we don’t port forward it as well, and always provide the -d flag.

  • Line:3 : This is the most important one. Things happening in the command in the same sequence are: the --link for linking the already created container of mongo db, elastic search with the new container being created, the -p flag for port forwarding the docker-host port 9000,1514 to 9000,1514 of the container so that this container is able to receive traffic, the -e flag for configuring the new container env variables (one must read documentation for the image for knowing this stuff) and finally the docker daemon that will run the graylog2 image from the docker hub in the background.

Now open http://127.0.0.1:9000/ in your browser and use admin:admin as user:password pair to log in, done!
We are done with the docker part for now, for those interested in Gralog may continue. To see practically that the logs are being accepted by the graylog service, you must follow the following steps:

  • Create a tcp input in the graylog web service, give the binding address as 0.0.0.0 (it’s important as it must listen on every interface since the web already has docker host running in our machine) and use any port of your choice apart from the system-ports, here we chose 1514, this is why we included it in the expose port command.

  • Add the following lines at the end of /etc/syslog.conf file:

#at the end of the .conf file
*.* @@0.0.0.0:1514

This will send all the syslogs to the given bind adress using rsyslog using tcp, for udp use only single @.

  • Do a rsyslog service restart by:
$ service rsyslog restart

Done! You have your running graylog service on your local host using Docker. Next blog we will deploy the same image using Kubernetes, stay tuned!