ElasticSearch


ElasticSearch ElasticSearch is the central point of the ELK architecture. This is where data will be aggregated and persisted.


Requirements

To install and use ELK you need:

  • JAVA 1.7.55+ (Java 8 is recommended)
java -version

Java version must be > 1.7.0_55


  • Firewall rule

Open the TCP ports 9200 + 9300, allow multicast too.

See FW input && FW output


  • Apache2 server

See Apache2 setup


  • Python
apt-get install python3
apt-get install python-pip


ElasticSearch

Installation

Source: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/setup-repositories.html


  • Add ELK repository: see Sources#ELK
  • Install application
apt-get install elasticsearch

>> Binaries in /usr/share/elasticsearch

>> Configuration in /etc/elasticsearch

>> Logs in /var/log/elasticsearch


  • Register application as a service
cd /etc/init.d
update-rc.d elasticsearch defaults 95 10


Configuration

Edit the configuration file:

vim /etc/elasticsearch/elasticsearch.yml


Set your CLUSTER and NODE name + allow Kibana access.

cluster.name: VEHCO         # line 33      
node.name: "VEHCO_MASTER"   # line 40

                            ### [...] At the end
http.cors.enabled: true
http.cors.allow-origin: https://smartcards.vehco.com     # http://192.168.1.203

!! You need to adjust your IP || hostname according to your needs.


Extensions (plugins)

You need to install some extensions (plugins) to get the full power of ElasticSearch.

The following plugins are the ones recommended by the ElasticSearch team.

cd /usr/share/elasticsearch/bin
./plugin -install karmi/elasticsearch-paramedic
./plugin -install mobz/elasticsearch-head
./plugin -install royrusso/elasticsearch-HQ


More information about each plugin:


You can access the plugins using the /_plugin/ URL:


You can search for more plugins on Google or the official ElasticSearch web-site.

Checkout http://www.elasticsearch.org/download to get a list of plugins available per official developer.


ElasticSearch tooling

Curator

Curator allows you to remove the old indices.


Installation

pip install elasticsearch-curator


Usage

# Display VEHCO- indices
curator show --show-indices --prefix vehco-
# Remove indices that are more than 10 days old


References



Start ElasticSearch

service elasticsearch start 

## OR ##
/etc/init.d/elasticsearch start


Delete indices

Reference: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-delete-index.html


curl -XDELETE 'http://localhost:9200/smartcard-monitoring-2014*/'

replace smartcard-monitoring-2014* by your own expression.



References