Elastic Stack Getting Started

ELK is the most widely deployed open source message broker. With more than 35,000 production deployments of ELK world-wide at small startups and large enterprises, ELK is the most popular open source message broker.

If you have installed Websoft9 Elastic Stack, the following steps is for your quick start

Preparation

1. Get the Internet IP of your Server on Cloud

2. Check your Inbound of Security Group Rule of Cloud Console to ensure the TCP:80 is allowed

3. Complete Five steps for Domain if you want to use Domain for Elastic Stack

4. Get default username and password of Elastic Stack

5. Log in the cloud server, run the following command, pull the ELK-related Docker image and start the container

   cd /data/apps/elastic && docker compose pull && docker compose up -d

   The Elastic Open Source License does not allow third-party distribution, but allows users to use it for free. Therefore, if you use this solution to deploy Elastic, you should first execute the above command to pull the Elastic image yourself.

Elastic Stack Initialization

Steps for you

1. Use local Chrome or Firefox to access the URL http://DNS You will enter installation wizard of Elastic Stack.

2. Log in Elastic Stack web console. (Don't have password?)

3. Set you new password from: 【Users】>【Admin】>【Permissions】>【Update this user】

More guide about Elastic Stack, please refer to ELK Documentation.

Having trouble?

Below is for you to solve problem, and you can contact Websoft9 Support or refer to Troubleshoot + FAQ to get more.

Elastic Stack QuickStart

Elastic Stack supports a variety of data sources, here we use the common logs file as an input to Logstash as an example, the steps are as follows:

1. Set the index "mytest" in Logstash configuration file and restart the container

   input{
      file{
         path => "/var/log/yum.log"
         type => "elasticsearch"
         start_position => "beginning"
      }
   }
   
   output {
      elasticsearch {
         hosts => "elasticsearch:9200"
         user => "elastic"
         password => "xxxxx"
                  index => "mytest"
      }
   }

   cd /data/apps/elastic
   docker-compose down
   docker-compose up -d

2. Verify that Elasticsearch and Logstash are successfully connected and that the index data is valid (verified by URL: http:// server public IP: 9200/cat/indices?v)

3. Log in to Kibana, click [Manage], and then click [Index Patterns] in the right menu

4. Search for "mytest" and follow the prompts to complete the creation

5. The index is created successfully in Kibana, and you can retrieve data here with a timestamp

Elastic Stack Setup

Connecting Logstash to Elasticsearch

As the data collector, how does Logstash transfer data to Elasticsearch?

1. Edit Logstash Confiugration file

2. Add a new pipeline configuration file:

   input{
      file{
         path => "/var/log/*.log"
         type => "elasticsearch"
         start_position => "beginning"
      }
   }
   
   ## Add your filters / logstash plugins configuration here
   
   output {
      elasticsearch {
         hosts => "elasticsearch:9200"
         user => "elastic"
         password => "elastic123"
                  index => "mytest"
      }
   }

output in the configuration requires the database connection account of Elasticsearch.

Configure SMTP

1. Get SMTP related parameters in the mailbox management console

2. Log in Elastic Stack Console.

3. Enter the SMTP settings.

4. Click the Test Connection. You will get the feedback "no errors were..." if SMTP is valid.

Reset Password

There are two main measures to reset password.

Changing password

Log in to Kibana and click User Profile in the upper right corner of the user icon to change the password

Forgot Password

If you forget your password, you can reset it by rerunning the container:

cd /data/apps/elastic
docker compose down && docker compose up -d

The DB_ES_PASSWORD variable in the .env file is the password after the reset

Reference sheet

The below items and General parameter sheet is maybe useful for you manage Elastic Stack

Run docker ps command, view all Containers when Elastic Stack is running:

CONTAINER ID   IMAGE                  COMMAND                  CREATED         STATUS         PORTS                                                                                                                                                                        NAMES
4c27ee6b8e98   logstash:7.13.4        "/usr/local/bin/dock…"   4 minutes ago   Up 4 minutes   0.0.0.0:5000->5000/tcp, :::5000->5000/tcp, 0.0.0.0:5044->5044/tcp, :::5044->5044/tcp, 0.0.0.0:9600->9600/tcp, 0.0.0.0:5000->5000/udp, :::9600->9600/tcp, :::5000->5000/udp   elastic-logstash
babdf8193e8d   kibana:7.13.4          "/bin/tini -- /usr/l…"   4 minutes ago   Up 4 minutes   0.0.0.0:9001->5601/tcp, :::9001->5601/tcp                                                                                                                                    elastic-kibana
de14eb80b9f9   elasticsearch:7.13.4   "/bin/tini -- /usr/l…"   4 minutes ago   Up 4 minutes   0.0.0.0:9200->9200/tcp, :::9200->9200/tcp, 0.0.0.0:9300->9300/tcp, :::9300->9300/tcp

Path

The Elastic Stack consists of components such as Elasticsearch, Kibana, Logstash, etc

Elastic Stack installation directory： /data/apps/elastic
Elastic Stack configuration directory： /data/apps/elastic/src
Logstash configuration file： /data/apps/elastic/src/logstash/pipelinelogstash.conf
Kibana configuration file： /data/apps/elastic/src/kibana/config/kibana.yml
Elasticsearch configuration file： /data/apps/elastic/src/elasticsearch/config/elasticsearch.yml

Port

Port	Use	Necessity
9200	Elasticsearch HTTP	Required
9600	Logstash API	Optional

Version

docker exec -it elastic-elasticsearch bin/elasticsearch --version

Service

sudo docker  start | stop | restart | status elastic-elasticsearch
sudo docker  start | stop | restart | status elastic-logstash
sudo docker  start | stop | restart | status elastic-kibana

CLI

SQL CLI

API

ELK API adopts the REST API 2.0 specification.