diff --git a/.kitchen.yml b/.kitchen.yml index 15af7cf..ac8eb77 100644 --- a/.kitchen.yml +++ b/.kitchen.yml @@ -22,7 +22,7 @@ provisioner: extra_vars: es_major_version: "<%= ENV['VERSION'] %>" <% if ENV['VERSION'] == '6.x' %> - es_version: '6.8.23' + es_version: '6.8.18' <% end %> <% end %> diff --git a/CHANGELOG.md b/CHANGELOG.md index efa5c6d..6d21bb0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,39 +1,5 @@ # Changelog -## 7.17.0 - -* 7.17.0 as default version. - - -| PR | Author | Title | -| --- | --- | --- | -| [#845](https://github.com/elastic/ansible-elasticsearch/pull/845) | [@jmlrt](https://github.com/jmlrt) | Remove CentOS 8 tests | - - -## 7.16.3 - -* 7.16.3 as default version. -* 6.8.23 as 6.x tested version - -## 7.16.2 - -* 7.16.2 as default version. -* 6.8.22 as 6.x tested version - -## 7.16.1 - -* 7.16.1 as default version. -* 6.8.21 as 6.x tested version - - -| PR | Author | Title | -| --- | --- | --- | -| [#838](https://github.com/elastic/ansible-elasticsearch/pull/838) | [@mgreau](https://github.com/mgreau) | Add warning message about 8.x versions | -| [#830](https://github.com/elastic/ansible-elasticsearch/pull/830) | [@bfontaine](https://github.com/bfontaine) | README: fix link to ES licensing change blog post | -| [#831](https://github.com/elastic/ansible-elasticsearch/pull/831) | [@gaima8](https://github.com/gaima8) | command necessarily causes a change on each invocation breaking idempotency, wait_for accomplishes the same goal without the change | -| [#833](https://github.com/elastic/ansible-elasticsearch/pull/833) | [@Pablohn26](https://github.com/Pablohn26) | Fix broken link | - - ## 7.16.0 * 7.16.0 as default version. diff --git a/README.md b/README.md index 182cdae..e68c0d2 100644 --- a/README.md +++ b/README.md @@ -1,23 +1,10 @@ -# ARCHIVED - -This project is no longer maintained. - -You are welcomed to keep using it and adapting it to work for your own needs, including with Elasticsearch [8.x](docs/8x-support.md). - -For alternative getting started experiences, you may want to try one of these options: - -- Start a [free trial on Elastic Cloud](https://www.elastic.co/cloud/elasticsearch-service/signup), our hosted service. -- Take a look at [Elastic Cloud on Kubernetes (ECK)](https://elastic.co/guide/en/cloud-on-k8s/current/k8s-quickstart.html) for launching the stack via Kubernetes. -- Read our [Running the Elastic Stack on Docker](https://www.elastic.co/guide/en/elastic-stack-get-started/current/get-started-docker.html) guide. -- Take a look at the [Elastic Stack Terraform provider.](https://github.com/elastic/terraform-provider-elasticstack) - # ansible-elasticsearch +[![Build Status](https://img.shields.io/jenkins/s/https/devops-ci.elastic.co/job/elastic+ansible-elasticsearch+main.svg)](https://devops-ci.elastic.co/job/elastic+ansible-elasticsearch+main/) [![Ansible Galaxy](https://img.shields.io/badge/ansible--galaxy-elastic.elasticsearch-blue.svg)](https://galaxy.ansible.com/elastic/elasticsearch/) +**THIS ROLE IS FOR 7.x & 6.x** -**THIS ROLE IS FOR 7.x & 6.x**, but should still work with 8.x (see [note](docs/8x-support.md)). - -Ansible role for 7.x/6.x Elasticsearch - tests used to run and pass on the below platforms: +Ansible role for 7.x/6.x Elasticsearch. Currently this works on Debian and RedHat based linux systems. Tested platforms are: * Ubuntu 16.04 * Ubuntu 18.04 @@ -26,8 +13,11 @@ Ansible role for 7.x/6.x Elasticsearch - tests used to run and pass on the below * Debian 9 * Debian 10 * CentOS 7 +* CentOS 8 * Amazon Linux 2 +The latest Elasticsearch versions of 7.x & 6.x are actively tested. + ## BREAKING CHANGES ### Notice about multi-instance support @@ -59,7 +49,7 @@ Starting from Elasticsearch 7.11.0, OSS distributions will no longer be provided This Ansible role will fail if `oss_version` is set to `true` and `es_version` is greater than `7.11.0`. -See [Doubling down on open, Part II](https://www.elastic.co/blog/licensing-change) +See [Doubling down on open, Part II](https://www.elastic.co/blog/licensing-change for more details) blog post for more details. #### How to override configuration files provided by ansible-elasticsearch? @@ -78,7 +68,7 @@ This role uses the json_query filter which [requires jmespath](https://github.co Create your Ansible playbook with your own tasks, and include the role elasticsearch. You will have to have this repository accessible within the context of playbook. ```sh -ansible-galaxy install elastic.elasticsearch,v7.17.0 +ansible-galaxy install elastic.elasticsearch,v7.16.0 ``` Then create your playbook yaml adding the role elasticsearch. @@ -92,14 +82,14 @@ The simplest configuration therefore consists of: roles: - role: elastic.elasticsearch vars: - es_version: 7.17.0 + es_version: 7.16.0 ``` -The above installs Elasticsearch 7.17.0 in a single node 'node1' on the hosts 'localhost'. +The above installs Elasticsearch 7.16.0 in a single node 'node1' on the hosts 'localhost'. **Note**: Elasticsearch default version is described in [`es_version`](https://github.com/elastic/ansible-elasticsearch/blob/main/defaults/main.yml#L2). You can override this variable in your playbook to install another version. -While we are testing this role only with one 7.x and one 6.x version (respectively [7.17.0](https://github.com/elastic/ansible-elasticsearch/blob/main/defaults/main.yml#L2) and [6.8.23](https://github.com/elastic/ansible-elasticsearch/blob/main/.kitchen.yml#L22) at the time of writing), this role should work with other versions also in most cases. +While we are testing this role only with one 7.x and one 6.x version (respectively [7.16.0](https://github.com/elastic/ansible-elasticsearch/blob/main/defaults/main.yml#L2) and [6.8.18](https://github.com/elastic/ansible-elasticsearch/blob/main/.kitchen.yml#L22) at the time of writing), this role should work with other versions also in most cases. This role also uses [Ansible tags](https://docs.ansible.com/ansible/2.9/user_guide/playbooks_tags.html). Run your playbook with the `--list-tasks` flag for more information. @@ -431,7 +421,7 @@ In addition to es_config, the following parameters allow the customization of th * ```oss_version``` Default `false`. Setting this to `true` will install the oss release of Elasticsearch (for version <7.11.0 only). * `es_xpack_trial` Default `false`. Setting this to `true` will start the 30-day trail once the cluster starts. -* ```es_version``` (e.g. "7.17.0"). +* ```es_version``` (e.g. "7.16.0"). * ```es_api_host``` The host name used for actions requiring HTTP e.g. installing templates. Defaults to "localhost". * ```es_api_port``` The port used for actions requiring HTTP e.g. installing templates. Defaults to 9200. **CHANGE IF THE HTTP PORT IS NOT 9200** * ```es_api_basic_auth_username``` The Elasticsearch username for making admin changing actions. Used if Security is enabled. Ensure this user is admin. diff --git a/defaults/main.yml b/defaults/main.yml index 47c9479..4b3d1f8 100644 --- a/defaults/main.yml +++ b/defaults/main.yml @@ -1,5 +1,5 @@ --- -es_version: "7.17.0" +es_version: "7.16.0" es_use_snapshot_release: false oss_version: false es_package_name: "elasticsearch" diff --git a/docs/8x-support.md b/docs/8x-support.md deleted file mode 100644 index 9e8d980..0000000 --- a/docs/8x-support.md +++ /dev/null @@ -1,116 +0,0 @@ -# 8.x support - -In [December 2021](https://github.com/elastic/ansible-elasticsearch/pull/838), we made the hard decision to deprecate this Ansible playbook without adding support for Elasticsearch 8.X. -We acknowledge the impact this has had on many developers and organizations, and while we are not reverting the decision, we decided to share some guidelines around how to proceed from here, for folks wanting to keep using this playbook with Elasticsearch 8.X. - ---- - -At a high level, this role is expected to work in most cases for fresh installs and upgrades from 7.17+ by only overriding the `es_version` variable as long as the security is enforced properly using the [SSL/TLS doc](ssl-tls-setup.md). - -1. install the last released version of the role from galaxy: `ansible-galaxy install elastic.elasticsearch,v7.17.0` - -2. copy the TLS PKCS12 keystore and truststore (https://www.elastic.co/guide/en/elasticsearch/reference/current/security-settings.html#security-http-pkcs12-files) - -3. write a minimal playbook to deploy 8.2.3 on localhost: -```yaml -- hosts: localhost - roles: - - elastic.elasticsearch - vars: - es_version: 8.2.3 - es_api_basic_auth_username: elastic - es_api_basic_auth_password: changeme - es_enable_http_ssl: true - es_enable_transport_ssl: true - es_ssl_keystore: "certs/keystore-password.p12" - es_ssl_truststore: "certs/truststore-password.p12" - es_ssl_keystore_password: password1 - es_ssl_truststore_password: password2 - es_validate_certs: no -``` - -4. deploy locally: `ansible-playbook es.yml` - -## Context for the below experiment - -The intent is to assess if the current playbook can still work with ES 8.X and what modifications may be needed. The testing was done on Ubuntu 20.04 and CentOS7 GCP VMs. - -The only code change done in the Ansible playbook was the override of the `es_version` variable. - -### What is working - -- ✅ Deploying a standalone Elasticsearch cluster in 8.2.3 with the security example playbook from 7.x: -- ✅ managing Elasticsearch users -- ✅ upgrading a 7.17.0 standalone cluster **with security already enabled** to 8.2.3 -- ✅ managing Elasticsearch license - -The below configuration was used in the tests - -```yaml -- hosts: localhost - roles: - - elastic.elasticsearch - vars: - es_config: - xpack.security.authc.realms.file.file1.order: 0 - es_api_basic_auth_username: elastic - es_api_basic_auth_password: changeme - es_api_sleep: 5 - es_enable_http_ssl: true - es_enable_transport_ssl: true - es_ssl_keystore: "test/integration/files/certs/keystore-password.p12" - es_ssl_truststore: "test/integration/files/certs/truststore-password.p12" - es_ssl_keystore_password: password1 - es_ssl_truststore_password: password2 - es_validate_certs: no - es_users: - file: - es_admin: - password: changeMe - roles: - - admin - testUser: - password: changeMeAlso! - roles: - - power_user - - user - es_roles: - file: - admin: - cluster: - - all - indices: - - names: '*' - privileges: - - all - power_user: - cluster: - - monitor - indices: - - names: '*' - privileges: - - all - user: - indices: - - names: '*' - privileges: - - read -``` - -### What is not working - -**Deploying an 8.X cluster with the default Ansible configuration (no security) will not work.** - -When runnin Elasticsearch 8.x outside of Ansible without any security configuration, Elasticsearch will autogenerate a security configuration and still activate security. -However, when you run Elasticsearch 8.x as part of the Ansible role without any security configuration, this will fail because the Ansible role will not be able to retrieve and use the autogenerated security configuration. - -To tackle this, you always have to specify your own security configuration based on the [SSL/TLS doc](ssl-tls-setup.md). - - -### What has not been tested - -**Deploying a 3 nodes cluster** - -When trying to deploy a 3 nodes clusters, the nodes seem to be configured successfully but they aren't able to communicate together with the test certificates (the ones used in automated standalone tests). It's highly likely that the problem lies with the tests certs themselves and not with the role. - -Should you be able to deploy a multi-node clusters, you will most likely have to change the configuration to use the new `node.roles` parameter ([example](https://github.com/elastic/ansible-elasticsearch/pull/772)) instead of the `node.master` and `node.data` (which got deprecated in 7.9, but the role never got [fixed](https://github.com/elastic/ansible-elasticsearch/issues/731). diff --git a/tasks/elasticsearch-Debian.yml b/tasks/elasticsearch-Debian.yml index d93e627..f6e4c17 100644 --- a/tasks/elasticsearch-Debian.yml +++ b/tasks/elasticsearch-Debian.yml @@ -65,7 +65,7 @@ - name: Include optional user and group creation. when: (es_user_id is defined) and (es_group_id is defined) - include_tasks: elasticsearch-optional-user.yml + include: elasticsearch-optional-user.yml - name: Debian - Get installed elasticsearch version command: dpkg-query --showformat='${Version}' --show {{ es_package_name }} diff --git a/tasks/elasticsearch-RedHat.yml b/tasks/elasticsearch-RedHat.yml index e1debfc..7d53fbf 100644 --- a/tasks/elasticsearch-RedHat.yml +++ b/tasks/elasticsearch-RedHat.yml @@ -26,7 +26,7 @@ when: es_use_repository - name: RedHat - include versionlock - include_tasks: elasticsearch-RedHat-version-lock.yml + include: elasticsearch-RedHat-version-lock.yml - name: RedHat - Remove the other elasticsearch package if switching between OSS and standard become: yes @@ -36,7 +36,7 @@ - name: Include optional user and group creation. when: (es_user_id is defined) and (es_group_id is defined) - include_tasks: elasticsearch-optional-user.yml + include: elasticsearch-optional-user.yml - name: RedHat - Install Elasticsearch become: yes diff --git a/tasks/elasticsearch.yml b/tasks/elasticsearch.yml index d6e5cb7..05fd93b 100644 --- a/tasks/elasticsearch.yml +++ b/tasks/elasticsearch.yml @@ -1,9 +1,9 @@ --- - name: Include specific Elasticsearch - include_tasks: elasticsearch-Debian.yml + include: elasticsearch-Debian.yml when: ansible_os_family == 'Debian' - name: Include specific Elasticsearch - include_tasks: elasticsearch-RedHat.yml + include: elasticsearch-RedHat.yml when: ansible_os_family == 'RedHat' diff --git a/tasks/main.yml b/tasks/main.yml index 855c5de..c0ad700 100644 --- a/tasks/main.yml +++ b/tasks/main.yml @@ -12,49 +12,49 @@ - always - name: set compatibility variables - include_tasks: compatibility-variables.yml + include: compatibility-variables.yml tags: - always - name: check-set-parameters - include_tasks: elasticsearch-parameters.yml + include: elasticsearch-parameters.yml tags: - always - name: use snapshot release - include_tasks: snapshot-release.yml + include: snapshot-release.yml when: es_use_snapshot_release - name: include java.yml - include_tasks: java.yml + include: java.yml when: es_java_install tags: - java - name: include elasticsearch.yml - include_tasks: elasticsearch.yml + include: elasticsearch.yml tags: - install - name: include elasticsearch-config.yml - include_tasks: elasticsearch-config.yml + include: elasticsearch-config.yml tags: - config - name: include elasticsearch-plugins.yml - include_tasks: elasticsearch-plugins.yml + include: elasticsearch-plugins.yml when: es_plugins is defined or es_plugins_reinstall tags: - plugins #We always execute xpack as we may need to remove features - name: include xpack/elasticsearch-xpack.yml - include_tasks: xpack/elasticsearch-xpack.yml + include: xpack/elasticsearch-xpack.yml tags: - xpack - name: include elasticsearch-ssl.yml - include_tasks: elasticsearch-ssl.yml + include: elasticsearch-ssl.yml when: es_enable_http_ssl or es_enable_transport_ssl tags: - xpack @@ -88,25 +88,25 @@ when: manage_native_realm | bool - name: activate-license - include_tasks: ./xpack/security/elasticsearch-xpack-activation.yml + include: ./xpack/security/elasticsearch-xpack-activation.yml when: es_start_service and not oss_version and es_xpack_license is defined and es_xpack_license != '' run_once: True - name: activate-trial - include_tasks: ./xpack/security/elasticsearch-xpack-trial-activation.yml + include: ./xpack/security/elasticsearch-xpack-trial-activation.yml when: es_start_service and not oss_version and es_xpack_trial run_once: True #perform security actions here now elasticsearch is started - name: include xpack/security/elasticsearch-security-native.yml - include_tasks: ./xpack/security/elasticsearch-security-native.yml + include: ./xpack/security/elasticsearch-security-native.yml when: manage_native_realm | bool run_once: True #Templates done after restart - handled by flushing the handlers. e.g. suppose user removes security on a running node and doesn't specify es_api_basic_auth_username and es_api_basic_auth_password. The templates will subsequently not be removed if we don't wait for the node to restart. #We also do after the native realm to ensure any changes are applied here first and its denf up. - name: include elasticsearch-template.yml - include_tasks: elasticsearch-template.yml + include: elasticsearch-template.yml when: es_templates | bool tags: - templates diff --git a/tasks/xpack/elasticsearch-xpack.yml b/tasks/xpack/elasticsearch-xpack.yml index 84528f4..9b49408 100644 --- a/tasks/xpack/elasticsearch-xpack.yml +++ b/tasks/xpack/elasticsearch-xpack.yml @@ -2,7 +2,7 @@ #Security configuration - name: include security/elasticsearch-security.yml - include_tasks: security/elasticsearch-security.yml + include: security/elasticsearch-security.yml when: not oss_version #Make sure elasticsearch.keystore has correct Permissions diff --git a/tasks/xpack/security/elasticsearch-security.yml b/tasks/xpack/security/elasticsearch-security.yml index 1d9aee4..184ab44 100644 --- a/tasks/xpack/security/elasticsearch-security.yml +++ b/tasks/xpack/security/elasticsearch-security.yml @@ -73,7 +73,7 @@ #-----------------------------FILE BASED REALM---------------------------------------- -- include_tasks: elasticsearch-security-file.yml +- include: elasticsearch-security-file.yml when: (es_users is defined and es_users.file is defined) or (es_roles is defined and es_roles.file is defined) #-----------------------------ROLE MAPPING ---------------------------------------- diff --git a/test/integration/custom-config.yml b/test/integration/custom-config.yml index 633f6b2..ed2dbab 100644 --- a/test/integration/custom-config.yml +++ b/test/integration/custom-config.yml @@ -2,7 +2,7 @@ - name: Setup Elasticsearch with custom config files hosts: localhost post_tasks: - - include_tasks: elasticsearch/test/integration/debug.yml + - include: elasticsearch/test/integration/debug.yml roles: - elasticsearch vars: diff --git a/test/integration/default.yml b/test/integration/default.yml index fb76ad1..7a61d17 100644 --- a/test/integration/default.yml +++ b/test/integration/default.yml @@ -2,6 +2,6 @@ - name: Setup Elasticsearch using default configuration hosts: localhost post_tasks: - - include_tasks: elasticsearch/test/integration/debug.yml + - include: elasticsearch/test/integration/debug.yml roles: - elasticsearch diff --git a/test/integration/license.yml b/test/integration/license.yml index c77e56d..cd0e234 100644 --- a/test/integration/license.yml +++ b/test/integration/license.yml @@ -2,7 +2,7 @@ - name: Elasticsearch Xpack HTTP different keystore and truststore with password hosts: localhost post_tasks: - - include_tasks: elasticsearch/test/integration/debug.yml + - include: elasticsearch/test/integration/debug.yml roles: - elasticsearch vars: diff --git a/test/integration/oss-to-default-upgrade.yml b/test/integration/oss-to-default-upgrade.yml index 8bf3112..774f725 100644 --- a/test/integration/oss-to-default-upgrade.yml +++ b/test/integration/oss-to-default-upgrade.yml @@ -2,7 +2,7 @@ - name: Deploy Elasticsearch OSS version hosts: localhost post_tasks: - - include_tasks: elasticsearch/test/integration/debug.yml + - include: elasticsearch/test/integration/debug.yml roles: - elasticsearch vars: @@ -12,7 +12,7 @@ - name: Upgrade to Elasticsearch default version hosts: localhost post_tasks: - - include_tasks: elasticsearch/test/integration/debug.yml + - include: elasticsearch/test/integration/debug.yml roles: - elasticsearch vars: diff --git a/test/integration/oss-upgrade.yml b/test/integration/oss-upgrade.yml index 1cec4d5..153b02a 100644 --- a/test/integration/oss-upgrade.yml +++ b/test/integration/oss-upgrade.yml @@ -2,7 +2,7 @@ - name: Deploy Elasticsearch OSS previous version hosts: localhost post_tasks: - - include_tasks: elasticsearch/test/integration/debug.yml + - include: elasticsearch/test/integration/debug.yml roles: - elasticsearch vars: @@ -12,7 +12,7 @@ - name: Deploy Elasticsearch OSS latest version hosts: localhost post_tasks: - - include_tasks: elasticsearch/test/integration/debug.yml + - include: elasticsearch/test/integration/debug.yml roles: - elasticsearch vars: diff --git a/test/integration/oss.yml b/test/integration/oss.yml index 95f242b..6c6ba06 100644 --- a/test/integration/oss.yml +++ b/test/integration/oss.yml @@ -2,7 +2,7 @@ - name: Setup Elasticsearch OSS 6.x hosts: localhost post_tasks: - - include_tasks: elasticsearch/test/integration/debug.yml + - include: elasticsearch/test/integration/debug.yml roles: - elasticsearch vars: diff --git a/test/integration/security.yml b/test/integration/security.yml index a00445c..bfedebf 100644 --- a/test/integration/security.yml +++ b/test/integration/security.yml @@ -2,7 +2,7 @@ - name: Elasticsearch Xpack HTTP different keystore and truststore with password hosts: localhost post_tasks: - - include_tasks: elasticsearch/test/integration/debug.yml + - include: elasticsearch/test/integration/debug.yml roles: - elasticsearch vars: diff --git a/test/integration/trial.yml b/test/integration/trial.yml index bf253f1..3221713 100644 --- a/test/integration/trial.yml +++ b/test/integration/trial.yml @@ -2,7 +2,7 @@ - name: Setup Elasticsearch with security enabled and a trial license hosts: localhost post_tasks: - - include_tasks: elasticsearch/test/integration/debug.yml + - include: elasticsearch/test/integration/debug.yml roles: - elasticsearch vars: diff --git a/test/integration/upgrade.yml b/test/integration/upgrade.yml index 4c6fa1a..2a1a27b 100644 --- a/test/integration/upgrade.yml +++ b/test/integration/upgrade.yml @@ -2,7 +2,7 @@ - name: Deploy Elasticsearch previous version hosts: localhost post_tasks: - - include_tasks: elasticsearch/test/integration/debug.yml + - include: elasticsearch/test/integration/debug.yml roles: - elasticsearch vars: @@ -11,6 +11,6 @@ - name: Deploy Elasticsearch latest version hosts: localhost post_tasks: - - include_tasks: elasticsearch/test/integration/debug.yml + - include: elasticsearch/test/integration/debug.yml roles: - elasticsearch diff --git a/test/matrix-6x.yml b/test/matrix-6x.yml index d76a5cc..c92f941 100644 --- a/test/matrix-6x.yml +++ b/test/matrix-6x.yml @@ -6,6 +6,7 @@ OS: - debian-9 - debian-10 - centos-7 + - centos-8 - amazonlinux-2 TEST_TYPE: - custom-config diff --git a/test/matrix.yml b/test/matrix.yml index 1955cf5..21538e6 100644 --- a/test/matrix.yml +++ b/test/matrix.yml @@ -6,6 +6,7 @@ OS: - debian-9 - debian-10 - centos-7 + - centos-8 - amazonlinux-2 TEST_TYPE: - custom-config