Part 2: Setting up Graylog Server and Web Interface

This blog will guide you through the steps of installing and configuring Graylog server and web interface on your machine.

Table of Contents

1. Introduction

In this blog, you will learn how to set up Graylog server and web interface on your machine. Graylog is an open source log management platform that allows you to collect, store, analyze, and visualize log data from various sources. Graylog can help you monitor the performance and security of your applications, servers, and networks.

Setting up Graylog involves two main components: the Graylog server and the Graylog web interface. The Graylog server is responsible for receiving, processing, and storing the log data. The Graylog web interface is a web application that allows you to access and interact with the log data stored by the Graylog server.

In this blog, you will learn how to install and configure both components on your machine. You will also learn how to test the functionality of Graylog by sending some sample log messages and viewing them on the web interface. By the end of this blog, you will have a working Graylog setup that you can use for your own log management needs.

Before you start, you need to make sure that you have some prerequisites installed and configured on your machine. Let’s see what they are in the next section.

2. Prerequisites

Before you can install and configure Graylog server and web interface, you need to make sure that you have some prerequisites installed and configured on your machine. These prerequisites are essential for the proper functioning of Graylog and its components.

The prerequisites are:

A Linux operating system, such as Ubuntu, Debian, CentOS, or Red Hat. In this blog, we will use Ubuntu 20.04 as an example.
A Java Runtime Environment (JRE) version 8 or higher. Graylog server and web interface are written in Java and require a JRE to run.
A MongoDB database server version 3.6 or higher. Graylog server uses MongoDB to store its configuration and metadata.
An Elasticsearch cluster version 6.x or 7.x. Graylog server uses Elasticsearch to store and index the log data.
A web server, such as Apache or Nginx. Graylog web interface is a web application that needs a web server to serve its static files and proxy its requests to the Graylog server.

If you don’t have these prerequisites installed and configured on your machine, you can follow the official documentation of each component to do so. You can find the links to the documentation below:

Once you have these prerequisites installed and configured on your machine, you can proceed to the next section, where we will install Graylog server.

3. Installing Graylog Server

In this section, you will learn how to install Graylog server on your machine. Graylog server is the core component of Graylog that receives, processes, and stores the log data. You can install Graylog server using one of the following methods:

Using the official Graylog repository for your Linux distribution.
Using the official Graylog Docker image.
Using the official Graylog OVA or AMI images for virtual machines or cloud platforms.

In this blog, we will use the first method, which is using the official Graylog repository for Ubuntu 20.04. This method is recommended for production environments, as it allows you to easily update Graylog server and its dependencies. If you want to use the other methods, you can refer to the official documentation of Graylog for more details.

To install Graylog server using the official repository, you need to follow these steps:

Add the Graylog repository to your system by running the following commands:

wget https://packages.graylog2.org/repo/packages/graylog-4.1-repository_latest.deb
sudo dpkg -i graylog-4.1-repository_latest.deb
sudo apt-get update

Install Graylog server and its dependencies by running the following command:

sudo apt-get install graylog-server

Start Graylog server by running the following command:

sudo systemctl start graylog-server

Check the status of Graylog server by running the following command:

sudo systemctl status graylog-server

If everything is working fine, you should see something like this:

● graylog-server.service - Graylog server
     Loaded: loaded (/lib/systemd/system/graylog-server.service; enabled; vendor preset: enabled)
     Active: active (running) since Sun 2021-10-31 12:34:56 UTC; 5min ago
       Docs: http://docs.graylog.org/
   Main PID: 1234 (graylog-server)
      Tasks: 65 (limit: 2286)
     Memory: 1.2G
     CGroup: /system.slice/graylog-server.service
             └─1234 /bin/sh /usr/share/graylog-server/bin/graylog-server

Congratulations, you have successfully installed Graylog server on your machine. In the next section, you will learn how to configure Graylog server to make it ready for receiving and processing log data.

4. Configuring Graylog Server

After installing Graylog server, you need to configure it to make it ready for receiving and processing log data. Graylog server has a configuration file called server.conf that contains various settings and parameters that affect its behavior. You can find this file in the /etc/graylog/server/ directory on your machine.

In this section, you will learn how to edit some of the most important settings in the server.conf file. These settings are:

password_secret: A secret key that is used to encrypt sensitive data in the Graylog database. You need to generate a random string of at least 64 characters and set it as the value of this setting.
root_password_sha2: A hashed password that is used to log in to the Graylog web interface as the admin user. You need to generate a SHA-256 hash of your desired password and set it as the value of this setting.
http_bind_address: The IP address and port that Graylog server listens on for HTTP requests. You need to set it to the IP address of your machine and the port 9000.
elasticsearch_hosts: A comma-separated list of Elasticsearch nodes that Graylog server connects to for storing and indexing log data. You need to set it to the IP address and port of your Elasticsearch cluster.
mongodb_uri: The URI of the MongoDB database that Graylog server uses to store its configuration and metadata. You need to set it to the IP address and port of your MongoDB server and the name of the database (graylog by default).

To edit these settings, you need to follow these steps:

Open the server.conf file with your preferred text editor, such as nano or vim. For example, you can run the following command:

sudo nano /etc/graylog/server/server.conf

Find the setting that you want to edit and uncomment it by removing the # symbol at the beginning of the line.
Replace the default value with the value that you want to set. For example, to set the password_secret setting, you can use the following command to generate a random string and copy it to the clipboard:

pwgen -N 1 -s 96 | xclip -selection clipboard

Paste the copied string as the value of the password_secret setting. It should look something like this:

password_secret = 8fj3jf83jf8j3fj83fj83fj83fj83fj83fj83fj83fj83fj83fj83fj83fj83fj

Repeat the same process for the other settings that you want to edit. Make sure to use the correct values for your machine and environment.
Save and close the server.conf file.
Restart Graylog server by running the following command:

sudo systemctl restart graylog-server

Congratulations, you have successfully configured Graylog server on your machine. In the next section, you will learn how to install Graylog web interface, which is the graphical user interface that allows you to access and interact with the log data stored by Graylog server.

5. Installing Graylog Web Interface

In this section, you will learn how to install Graylog web interface on your machine. Graylog web interface is the graphical user interface that allows you to access and interact with the log data stored by Graylog server. You can install Graylog web interface using one of the following methods:

Using the official Graylog repository for your Linux distribution.
Using the official Graylog Docker image.
Using the official Graylog OVA or AMI images for virtual machines or cloud platforms.

In this blog, we will use the first method, which is using the official Graylog repository for Ubuntu 20.04. This method is recommended for production environments, as it allows you to easily update Graylog web interface and its dependencies. If you want to use the other methods, you can refer to the official documentation of Graylog for more details.

To install Graylog web interface using the official repository, you need to follow these steps:

Add the Graylog repository to your system by running the following commands:

wget https://packages.graylog2.org/repo/packages/graylog-4.1-repository_latest.deb
sudo dpkg -i graylog-4.1-repository_latest.deb
sudo apt-get update

Install Graylog web interface and its dependencies by running the following command:

sudo apt-get install graylog-web

Start Graylog web interface by running the following command:

sudo systemctl start graylog-web

Check the status of Graylog web interface by running the following command:

sudo systemctl status graylog-web

If everything is working fine, you should see something like this:

● graylog-web.service - Graylog web interface
     Loaded: loaded (/lib/systemd/system/graylog-web.service; enabled; vendor preset: enabled)
     Active: active (running) since Sun 2021-10-31 12:45:23 UTC; 5min ago
       Docs: http://docs.graylog.org/
   Main PID: 2345 (graylog-web)
      Tasks: 23 (limit: 2286)
     Memory: 512.0M
     CGroup: /system.slice/graylog-web.service
             └─2345 /bin/sh /usr/share/graylog-web/bin/graylog-web

Congratulations, you have successfully installed Graylog web interface on your machine. In the next section, you will learn how to configure Graylog web interface to make it ready for accessing and interacting with the log data stored by Graylog server.

6. Configuring Graylog Web Interface

After installing Graylog web interface, you need to configure it to make it ready for accessing and interacting with the log data stored by Graylog server. Graylog web interface has a configuration file called web.conf that contains various settings and parameters that affect its behavior. You can find this file in the /etc/graylog/web/ directory on your machine.

In this section, you will learn how to edit some of the most important settings in the web.conf file. These settings are:

web_listen_uri: The URI that Graylog web interface listens on for HTTP requests. You need to set it to the IP address and port of your machine and the path /web.
web_endpoint_uri: The URI that Graylog web interface uses to communicate with the Graylog server API. You need to set it to the IP address and port of your Graylog server and the path /api.
web_enable_cors: A boolean value that determines whether Graylog web interface allows cross-origin resource sharing (CORS) requests. You need to set it to true to allow Graylog web interface to communicate with Graylog server from different domains.
web_enable_gzip: A boolean value that determines whether Graylog web interface compresses its responses using gzip. You need to set it to true to reduce the bandwidth usage and improve the performance of Graylog web interface.

To edit these settings, you need to follow these steps:

Open the web.conf file with your preferred text editor, such as nano or vim. For example, you can run the following command:

sudo nano /etc/graylog/web/web.conf

Find the setting that you want to edit and uncomment it by removing the # symbol at the beginning of the line.
Replace the default value with the value that you want to set. For example, to set the web_listen_uri setting, you can use the following value:

web_listen_uri = http://192.168.1.10:9000/web

Repeat the same process for the other settings that you want to edit. Make sure to use the correct values for your machine and environment.
Save and close the web.conf file.
Restart Graylog web interface by running the following command:

sudo systemctl restart graylog-web

Congratulations, you have successfully configured Graylog web interface on your machine. In the next section, you will learn how to test the functionality of Graylog by sending some sample log messages and viewing them on the web interface.

7. Testing Graylog Functionality

In this section, you will learn how to test the functionality of Graylog by sending some sample log messages and viewing them on the web interface. Graylog can receive log data from various sources, such as files, syslog, network devices, applications, and more. You can use different methods to send log data to Graylog, such as using the Graylog Collector Sidecar, using the GELF protocol, using the HTTP input, and more. You can find more details about the different methods and sources in the official documentation of Graylog.

In this blog, we will use a simple method to send some sample log messages to Graylog, which is using the logger command. The logger command is a tool that allows you to send messages to the syslog service on your machine. You can use the -P option to specify the port number that Graylog server listens on for syslog messages. By default, Graylog server listens on port 5140 for UDP syslog messages and port 5141 for TCP syslog messages. You can change these ports in the server.conf file if you want.

To test the functionality of Graylog by sending some sample log messages using the logger command, you need to follow these steps:

Open a terminal window on your machine.
Run the following command to send a UDP syslog message to Graylog server:

logger -P 5140 "This is a test UDP syslog message"

Run the following command to send a TCP syslog message to Graylog server:

logger -T -P 5141 "This is a test TCP syslog message"

Open a web browser and go to the Graylog web interface. The default URL is http://:9000/web.
Log in to the Graylog web interface using the admin username and password that you set in the server.conf file.
Go to the Search tab and click on the All messages stream. You should see the two messages that you sent using the logger command.
Click on each message to see more details, such as the source, timestamp, level, facility, and message content.

Congratulations, you have successfully tested the functionality of Graylog by sending some sample log messages and viewing them on the web interface. You can now explore the different features and functionalities of Graylog, such as creating streams, dashboards, alerts, extractors, pipelines, and more. You can also send log data from different sources and methods to Graylog and analyze them using the powerful search and visualization tools. In the next and final section, you will learn how to conclude your blog and provide some useful resources for further learning.

8. Conclusion

In this blog, you have learned how to set up Graylog server and web interface on your machine. You have also learned how to configure both components to make them ready for receiving and processing log data. Finally, you have learned how to test the functionality of Graylog by sending some sample log messages and viewing them on the web interface.

Graylog is a powerful and versatile log management platform that can help you monitor the performance and security of your applications, servers, and networks. You can use Graylog to collect, store, analyze, and visualize log data from various sources and methods. You can also use Graylog to create streams, dashboards, alerts, extractors, pipelines, and more to enhance your log analysis and visualization capabilities.

We hope that this blog has been useful and informative for you. If you want to learn more about Graylog and its features and functionalities, you can check out the following resources:

Graylog Documentation: The official documentation of Graylog that covers everything from installation and configuration to usage and administration.
Graylog Resources: A collection of resources that include blogs, webinars, podcasts, ebooks, and more about Graylog and log management.
Graylog Community: A forum where you can ask questions, share ideas, and get help from other Graylog users and developers.
Graylog GitHub: The GitHub repository of Graylog where you can find the source code, issues, and pull requests of Graylog.

Thank you for reading this blog and following along with the tutorial. We hope that you have enjoyed learning how to set up Graylog server and web interface on your machine. If you have any feedback or suggestions, please feel free to leave a comment below. Happy logging!