Upgrade Elasticsearch is an easy task, I always heard this sentence. Which is correct if you take care about indices and compatibility. At a customer, I had the chance to upgrade an old version of Elasticsearch, today I will share with you the Index compatibility story 😉
Index compatibility
Elastic confirm that any index you restore from a snapshot must be compatible with the current cluster’s version. If you try to restore an index created in an incompatible version, the restore attempt will fail. What about the current indices?
The following is the compatibility of index creation version with the cluster version.
In my case I need to upgrade Elasticsearch from 5.6 to 7.17, you got it, I have to upgrade from 5.6 to 6.8 and then from 6.8 to 7.17.
But what about my indices? They will stay readable throw all these versions?
I wanted to make the test myself, share it with you, even if I know the answer already 😉
Questions about your own setup? A dbi services consultant can look at your case.
Talk to an expert (opens in a new tab)Create indices with version 5.6
I installed Elasticsearch 5.6, I won’t go into installation steps this is not the aim of this blog.
My Elasticsearch 5.6 is available.
[elastic@vmelastic app]# curl -X GET "localhost:9200/"
{
"name" : "EDRyCty",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "wgLfYBK4Sq-ck9MPVhJHLA",
"version" : {
"number" : "5.6.0",
"build_hash" : "781a835",
"build_date" : "2017-09-07T03:09:58.087Z",
"build_snapshot" : false,
"lucene_version" : "6.6.0"
},
"tagline" : "You Know, for Search"
}
No index created yet.
[elastic@vmelastic app]# curl 'localhost:9200/_cat/indices?v'
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
Create some indices manually and from Logstash.
[elastic@vmelastic app]# curl -X GET "localhost:9200/_cat/indices"
yellow open ddi-2023.05.22 6mWQyvXlSuKExxEngKBjsw 5 1 4 0 25.7kb 25.7kb
yellow open ddi_index eWn5a2urRXKtP9-EFV4Uxg 5 1 0 0 1.2kb 1.2kb
yellow open .kibana wBrBaV9ZRGC41dbgk4JcaA 1 1 2 0 6.8kb 6.8kb
yellow open logstash-2023.05.22 31_uvd2KQvWLr6q30AcEZA 5 1 33 0 57.2kb 57.2kb
Now let’s upgrade Elasticsearch to 6.8
Upgrade Elasticsearch version 6.8
As for installation, I won’t go into installation steps to keep this blog readable.
My cluster has been upgraded successfully.
[elastic@vmelastic app]# curl -X GET "localhost:9200/"
{
"name" : "EDRyCty",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "wgLfYBK4Sq-ck9MPVhJHLA",
"version" : {
"number" : "6.8.23",
"build_flavor" : "default",
"build_type" : "rpm",
"build_hash" : "4f67856",
"build_date" : "2022-01-06T21:30:50.087716Z",
"build_snapshot" : false,
"lucene_version" : "7.7.3",
"minimum_wire_compatibility_version" : "5.6.0",
"minimum_index_compatibility_version" : "5.0.0"
},
"tagline" : "You Know, for Search"
}
Indices are still there and readable, which is expected according to the compatibility table shared above 😉
[elastic@vmelastic app]# curl -X GET "localhost:9200/_cat/indices"
yellow open ddi-2023.05.22 6mWQyvXlSuKExxEngKBjsw 5 1 4 0 25.7kb 25.7kb
yellow open ddi_index eWn5a2urRXKtP9-EFV4Uxg 5 1 0 0 1.2kb 1.2kb
yellow open .kibana wBrBaV9ZRGC41dbgk4JcaA 1 1 2 0 6.8kb 6.8kb
yellow open logstash-2023.05.22 31_uvd2KQvWLr6q30AcEZA 5 1 33 0 57.2kb 57.2kb
Let’s upgrade to version 7.17 now.
Upgrade Elasticsearch to version 7.17
So indices where created with version 5.6, cluster already upgraded one time to version 6.8. Let’s go for 7.17.
Elasticsearch upgraded, let’s check the service status.
[elastic@vmelastic app]# systemctl status elasticsearch.service -l
● elasticsearch.service - Elasticsearch
Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; disabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Thu 2023-05-25 10:59:35 CEST; 17s ago
Docs: https://www.elastic.co
Process: 9288 ExecStart=/usr/share/elasticsearch/bin/systemd-entrypoint -p ${PID_DIR}/elasticsearch.pid --quiet (code=exited, status=1/FAILURE)
Main PID: 9288 (code=exited, status=1/FAILURE)
May 25 10:59:34 vmelastic systemd-entrypoint[9288]: at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:77)
May 25 10:59:34 vmelastic systemd-entrypoint[9288]: at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:112)
May 25 10:59:34 vmelastic systemd-entrypoint[9288]: at org.elasticsearch.cli.Command.main(Command.java:77)
May 25 10:59:34 vmelastic systemd-entrypoint[9288]: at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:125)
May 25 10:59:34 vmelastic systemd-entrypoint[9288]: at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:80)
May 25 10:59:34 vmelastic systemd-entrypoint[9288]: For complete error details, refer to the log at /var/log/elasticsearch/elasticsearch.log
May 25 10:59:35 vmelastic systemd[1]: elasticsearch.service: main process exited, code=exited, status=1/FAILURE
May 25 10:59:35 vmelastic systemd[1]: Failed to start Elasticsearch.
May 25 10:59:35 vmelastic systemd[1]: Unit elasticsearch.service entered failed state.
May 25 10:59:35 vmelastic systemd[1]: elasticsearch.service failed.
[elastic@vmelastic app]# vi /var/log/elasticsearch/elasticsearch.log
...
[2023-05-25T11:03:34,346][ERROR][o.e.b.Bootstrap ] [vmelastic] Exception
java.lang.IllegalStateException: The index [ddi-2023.05.22/6mWQyvXlSuKExxEngKBjsw] was created with version [5.6.0] but the minimum compatible version is [6.0.0-beta1]. It should be re-indexed in Elasticsearch 6.x before upgrading to 7.17.10
Ok, my cluster fail to start, because there are some indices created with version 5.6 which is not compatible with version 7.17. and this is expected!
Conclusion
The aim of this blog is to make you aware about this compatibility story, be careful, check the compatibility before upgrade otherwise you will have to downgrade, reindex, aso.
In a next blog I will share the correct process to upgrade from 5.6 to 7.17, stay connected 😉