Skip to main content

Elastic Stack Getting Started

ELK is the most widely deployed open source message broker. With more than 35,000 production deployments of ELK world-wide at small startups and large enterprises, ELK is the most popular open source message broker.

If you have installed Websoft9 Elastic Stack, the following steps is for your quick start

Preparation

  1. Get the Internet IP of your Server on Cloud
  2. Check your Inbound of Security Group Rule of Cloud Console to ensure the TCP:80 is allowed
  3. Complete Five steps for Domain if you want to use Domain for Elastic Stack
  4. Get default username and password of Elastic Stack
  5. Log in the cloud server, run the following command, pull the ELK-related Docker image and start the container
cd /data/wwwroot/elk && docker-compose pull && docker-compose up -d

The Elastic Open Source License does not allow third-party distribution, but allows users to use it for free. Therefore, if you use this solution to deploy Elastic, you should first execute the above command to pull the Elastic image yourself.

Elastic Stack Initialization

Steps for you

  1. Use local Chrome or Firefox to access the URL http://DNS You will enter installation wizard of Elastic Stack.

  2. Log in Elastic Stack web console. (Don't have password?)

  3. Set you new password from: 【Users】>【Admin】>【Permissions】>【Update this user】

More guide about Elastic Stack, please refer to ELK Documentation.

Having trouble?

Below is for you to solve problem, and you can contact Websoft9 Support or refer to Troubleshoot + FAQ to get more.

Elastic Stack QuickStart

Elastic Stack supports a variety of data sources, here we use the common logs file as an input to Logstash as an example, the steps are as follows:

  1. Set the index "mytest" in Logstash configuration file and restart the container

    input{
    file{
    path => "/var/log/yum.log"
    type => "elasticsearch"
    start_position => "beginning"
    }
    }

    output {
    elasticsearch {
    hosts => "elasticsearch:9200"
    user => "elastic"
    password => "xxxxx"
    index => "mytest"
    }
    }
    cd /data/apps/elastic
    docker-compose down
    docker-compose up -d
  2. Verify that Elasticsearch and Logstash are successfully connected and that the index data is valid (verified by URL: http:// server public IP: 9200/cat/indices?v)

    ELK 验证

  3. Log in to Kibana, click [Manage], and then click [Index Patterns] in the right menu

    ELK Index

    ELK Index

    ELK Index

  4. Search for "mytest" and follow the prompts to complete the creation

    ELK Index

    ELK Index

  5. The index is created successfully in Kibana, and you can retrieve data here with a timestamp

    ELK Index

    ELK Index

Elastic Stack Setup

Connecting Logstash to Elasticsearch

As the data collector, how does Logstash transfer data to Elasticsearch?

  1. Edit Logstash Confiugration file

  2. Add a new pipeline configuration file:

    input{
    file{
    path => "/var/log/*.log"
    type => "elasticsearch"
    start_position => "beginning"
    }
    }

    ## Add your filters / logstash plugins configuration here

    output {
    elasticsearch {
    hosts => "elasticsearch:9200"
    user => "elastic"
    password => "elastic123"
    index => "mytest"
    }
    }

    output in the configuration requires the database connection account of Elasticsearch.

Configure SMTP

  1. Get SMTP related parameters in the mailbox management console

  2. Log in Elastic Stack Console.

  3. Enter the SMTP settings. Metabase SMTP

  4. Click the Test Connection. You will get the feedback "no errors were..." if SMTP is valid.

Reset Password

There are two main measures to reset password.

Changing password

Log in to Kibana and click User Profile in the upper right corner of the user icon to change the password

Forgot Password

If you forget your password, you can reset it by rerunning the container:

cd /data/apps/elastic
docker-compose down && docker-compose up -d

The DB_ES_PASSWORD variable in the .env file is the password after the reset

Reference sheet

The below items and General parameter sheet is maybe useful for you manage Elastic Stack

Run docker ps command, view all Containers when Elastic Stack is running:

CONTAINER ID   IMAGE                  COMMAND                  CREATED         STATUS         PORTS                                                                                                                                                                        NAMES
4c27ee6b8e98 logstash:7.13.4 "/usr/local/bin/dock…" 4 minutes ago Up 4 minutes 0.0.0.0:5000->5000/tcp, :::5000->5000/tcp, 0.0.0.0:5044->5044/tcp, :::5044->5044/tcp, 0.0.0.0:9600->9600/tcp, 0.0.0.0:5000->5000/udp, :::9600->9600/tcp, :::5000->5000/udp elastic-logstash
babdf8193e8d kibana:7.13.4 "/bin/tini -- /usr/l…" 4 minutes ago Up 4 minutes 0.0.0.0:9001->5601/tcp, :::9001->5601/tcp elastic-kibana
de14eb80b9f9 elasticsearch:7.13.4 "/bin/tini -- /usr/l…" 4 minutes ago Up 4 minutes 0.0.0.0:9200->9200/tcp, :::9200->9200/tcp, 0.0.0.0:9300->9300/tcp, :::9300->9300/tcp

Path

The Elastic Stack consists of components such as Elasticsearch, Kibana, Logstash, etc

Elastic Stack installation directory: /data/apps/elastic
Elastic Stack configuration directory: /data/apps/elastic/src
Logstash configuration file: /data/apps/elastic/src/logstash/pipelinelogstash.conf
Kibana configuration file: /data/apps/elastic/src/kibana/config/kibana.yml
Elasticsearch configuration file: /data/apps/elastic/src/elasticsearch/config/elasticsearch.yml

Port

PortUseNecessity
9200Elasticsearch HTTPRequired
9600Logstash APIOptional

Version

docker exec -it elastic-elasticsearch bin/elasticsearch --version

Service

sudo docker  start | stop | restart | status elastic-elasticsearch
sudo docker start | stop | restart | status elastic-logstash
sudo docker start | stop | restart | status elastic-kibana

CLI

SQL CLI

API

ELK API adopts the REST API 2.0 specification.