ansible-elasticsearch

Ansible playbook / roles / tasks for Elasticsearch. Currently it will work on Debian and RedHat based linux systems. Tested platforms are:

• Ubuntu 1404
• Debian 7
• Debian 8
• Centos 7
• Centos 8

Usage

Create your Ansible playbook with your own tasks, and include the role elasticsearch. You will have to have this repository accessible within the context of playbook, e.g.

e.g.

cd /my/repos/
git clone git@github.com:elastic/ansible-elasticsearch.git
cd /my/ansible/playbook
mkdir -p roles
ln -s /my/repos/ansible-elasticsearch ./roles/elasticsearch

Then create your playbook yaml adding the role elasticsearch. By default, the user is only required to specify a unique es_instance_name per role application.

The simplest configuration therefore consists of:

---
- name: Simple Example
  hosts: localhost
  roles:
    - { role: elasticsearch, es_instance_name: "node1" }
  vars:

All Elasticsearch configuration parameters are supported. This is achieved using a configuration map parameter 'es_config' which is serialized into the elasticsearch.yml file.
The use of a map ensures the Ansible playbook does not need to be updated to reflect new/deprecated/plugin configuration parameters.

In addition to the es_config map, several other parameters are supported for additional functions e.g. script installation. These can be found in the role's defaults/main.yml file.

The following illustrates applying configuration parameters to an Elasticsearch instance.

- name: Elasticsearch with custom configuration
  hosts: localhost
  roles:
    #expand to all available parameters
    - { role: elasticsearch, es_instance_name: "node1", es_data_dir: "/opt/elasticsearch/data", es_log_dir: "/opt/elasticsearch/logs", es_work_dir: "/opt/elasticsearch/temp", es_config: {node.name: "node1", cluster.name: "custom-cluster", discovery.zen.ping.unicast.hosts: "localhost:9301", http.port: 9201, transport.tcp.port: 9301, node.data: false, node.master: true, bootstrap.mlockall: true, discovery.zen.ping.multicast.enabled: false } }
  vars:
    es_scripts: false
    es_templates: false
    es_version_lock: false
    es_heap_size: 1g

The playbook utilises Elasticsearch version defaults. By default, therefore, multicast is enabled for 1.x. If disabled, the user user is required to specify the following additional parameters:

1. es_config['http.port'] - the http port for the node
2. es_config['transport.tcp.port'] - the transport port for the node
3. es_config['discovery.zen.ping.unicast.hosts'] - the unicast discovery list, in the comma separated format ":,:" (typically the clusters dedicated masters)

If set to true, the ports will be auto defined and node discovery will be performed using multicast.

A more complex example:

---
- name: Elasticsearch with custom configuration
  hosts: localhost
  roles:
    #expand to all available parameters
    - { role: elasticsearch, es_instance_name: "node1", es_data_dir: "/opt/elasticsearch/data", es_log_dir: "/opt/elasticsearch/logs", es_work_dir: "/opt/elasticsearch/temp", es_config: {node.name: "node1", cluster.name: "custom-cluster", discovery.zen.ping.unicast.hosts: "localhost:9301", http.port: 9201, transport.tcp.port: 9301, node.data: false, node.master: true, bootstrap.mlockall: true, discovery.zen.ping.multicast.enabled: false } }
  vars:
    es_scripts: false
    es_templates: false
    es_version_lock: false
    es_heap_size: 1g
    es_scripts: false
    es_templates: false
    es_version_lock: false
    es_start_service: false
    es_plugins_reinstall: false
    es_plugins:
        - plugin: elasticsearch/elasticsearch-cloud-aws
          version: 2.5.0
        - plugin: elasticsearch/marvel
          version: latest
        - plugin: elasticsearch/license
          version: latest
        - plugin: elasticsearch/shield
          version: latest
        - plugin: elasticsearch/elasticsearch-support-diagnostics
          version: latest
        - plugin: lmenezes/elasticsearch-kopf
          version: master

The application of a role results in the installation of a node on a host. Multiple roles equates to multiple nodes for a single host.

In any multi node cluster configuration it is recommened the user list the master eligble roles first - especially if these are used a unicast hosts off which other nodes are 'booted'

An example of a two server deployment, each with 1 node on one server and 2 nodes on another. The first server holds the master and is thus declared first.

---

- hosts: master_nodes
  roles:
    # one master per host
    - { role: elasticsearch, es_instance_name: "node1", es_heap_size: "1g", es_config: { "discovery.zen.ping.multicast.enabled": false, discovery.zen.ping.unicast.hosts: "elastic02:9300", http.port: 9200, transport.tcp.port: 9300, node.data: false, node.master: true, bootstrap.mlockall: false, discovery.zen.ping.multicast.enabled: false } }
  vars:
    es_scripts: false
    es_templates: false
    es_version_lock: false
    es_cluster_name: test-cluster
    ansible_user: ansible
    es_plugins:
     - plugin: elasticsearch/license
       version: latest

- hosts: data_nodes
  roles:
    # two nodes per host
    - { role: elasticsearch, es_instance_name: "node1", es_data_dir: "/opt/elasticsearch", es_config: { "discovery.zen.ping.multicast.enabled": false, discovery.zen.ping.unicast.hosts: "elastic02:9300", http.port: 9200, transport.tcp.port: 9300, node.data: true, node.master: false, bootstrap.mlockall: false, discovery.zen.ping.multicast.enabled: false } }
    - { role: elasticsearch, es_instance_name: "node2", es_config: { "discovery.zen.ping.multicast.enabled": false, discovery.zen.ping.unicast.hosts: "elastic02:9300", http.port: 9201, transport.tcp.port: 9301, node.data: true, node.master: false, bootstrap.mlockall: false, discovery.zen.ping.multicast.enabled: false } }
  vars:
    es_scripts: false
    es_templates: false
    es_version_lock: false
    es_cluster_name: test-cluster
    ansible_user: ansible
    es_plugins:
     - plugin: elasticsearch/license
       version: latest

Parameters can additionally be assigned to hosts using the inventory file if desired.

Make sure your hosts are defined in your inventory file with the appropriate ansible_ssh_host, ansible_ssh_user and ansible_ssh_private_key_file values.

Then run it:

ansible-playbook -i hosts ./your-playbook.yml

Configuration

You can add the role without any customisation and it will by default install Java and Elasticsearch, without any plugins.

Following variables affect the versions installed:

• es_major_version (e.g. "1.5" )
• es_version (e.g. "1.5.2")
• es_start_service (true (default) or false)
• es_plugins_reinstall (true or false (default) )
• es_plugins (an array of plugin definitons e.g.:

    es_plugins:
      - plugin: elasticsearch-cloud-aws
        version: 2.5.0

• java_repos (an array of repositories to be added to allow java to be installed)
• java_packages (an array of packages to be installed to get Java installed)

Notes

• The role assumes the user/group exists on the server. The elasticsearch packages create the default elasticsearch user. If this needs to be changed, ensure the user exists.
• The playbook relies on the inventory_name of each host to ensure its directories are unique
• Systemd scripts are by default installed in addition to init scripts - with the exception of Debian 8. This is pending improvement and currently relies on the user to determine the preferred mechanism.
• Changing an instance_name for a role application will result in the installation of a new component. The previous component will remain.
• KitchenCI has been used for testing. This is used to confirm images reach the correct state after a play is first applied.