Skip to content

Elasticsearch – Snapshot Lifecycle Management (SLM)

In a previous blog, I spoke about snapshot, restore, and briefly about Snapshot Lifecycle Management. In this blog, I will go more in deep in SLM.

Snapshot lifecycle management (SLM) is the easiest way to regularly back up a cluster. An SLM policy automatically takes snapshots on a preset schedule. The policy can also delete snapshots based on retention rules you define.

Please note that Elasticsearch Service deployments automatically include the cloud-snapshot-policy SLM policy. Elasticsearch Service uses this policy to take periodic snapshots of your cluster.

SLM security

The following cluster privileges control access to the SLM actions when Elasticsearch security features are enabled:

  • manage_slm : Allows a user to perform all SLM actions, including creating and updating policies and starting and stopping SLM.
  • read_slm : Allows a user to perform all read-only SLM actions, such as getting policies and checking the SLM status.
  • cluster:admin/snapshot/* : Allows a user to take and delete snapshots of any index, whether or not they have access to that index.

You can create and manage roles to assign these privileges through Kibana Management.

Questions about your own setup? A dbi services consultant can look at your case.

Talk to an expert (opens in a new tab)

To grant the privileges necessary to create and manage SLM policies and snapshots, you can set up a role with the manage_slm and cluster:admin/snapshot/* cluster privileges and full access to the SLM history indices.

For example, the following request creates an slm-admin role:

POST _security/role/slm-admin
{
  "cluster": [ "manage_slm", "cluster:admin/snapshot/*" ],
  "indices": [
    {
      "names": [ ".slm-history-*" ],
      "privileges": [ "all" ]
    }
  ]
}

To grant read-only access to SLM policies and the snapshot history, you can set up a role with the read_slm cluster privilege and read access to the snapshot lifecycle management history indices.

For example, the following request creates a slm-read-only role:

POST _security/role/slm-read-only
{
  "cluster": [ "read_slm" ],
  "indices": [
    {
      "names": [ ".slm-history-*" ],
      "privileges": [ "read" ]
    }
  ]
}

Create an SLM policy

To manage SLM in Kibana, go to the main menu and click Stack Management > Snapshot and Restore > Policies. To create a policy, click Create policy.

You can also manage SLM using the SLM APIs. To create a policy, use the create SLM policy API.

The following request creates a policy that backs up the cluster state, all data streams, and all indices daily at 1:30 a.m. UTC

PUT _slm/policy/nightly-snapshots
{
  "schedule": "0 30 1 * * ?",       
  "name": "<nightly-snap-{now/d}>", 
  "repository": "my_repository",    
  "config": {
    "indices": "*",                 
    "include_global_state": true    
  },
  "retention": {                    
    "expire_after": "30d",
    "min_count": 5,
    "max_count": 50
  }
}
  • schedule: When to take snapshots, written in Cron syntax.
  • name: Supports date math. To prevent naming conflicts, the policy also appends a UUID to each snapshot name.
  • repository: Registered snapshot repository used to store the policy’s snapshots.
  • config: Data streams and indices to include in the policy’s snapshots.
  • include_global_state: If true, the policy’s snapshots include the cluster state.
  • retention: This configuration keeps snapshots for 30 days, retaining at least 5 and no more than 50 snapshots regardless of age.

Manually run an SLM policy

You can manually run an SLM policy to immediately create a snapshot. This is useful for testing a new policy or taking a snapshot before an upgrade. Manually running a policy doesn’t affect its snapshot schedule.

To run a policy in Kibana, go to the Policies page and click the run icon under the Actions column.

You can also use the execute SLM policy API.

POST _slm/policy/nightly-snapshots/_execute

Please note that the snapshot process runs in the background. To monitor any currently running snapshots, use the get snapshot API with the _current request path parameter.

GET _snapshot/my_repository/_current

SLM Retention

SLM snapshot retention is a cluster-level task that runs separately from a policy’s snapshot schedule. To control when the SLM retention task runs, configure the slm.retention_schedule cluster setting.

PUT _cluster/settings
{
  "persistent" : {
    "slm.retention_schedule" : "0 30 1 * * ?"
  }
}

To immediately run the retention task, execute:

POST _slm/_execute_retention

Please note that an SLM policy’s retention rules only apply to snapshots created using the policy. Other snapshots don’t count toward the policy’s retention limits.

These are the most important points to know about SLM, you know how to create, configure, execute, secure SLM. This is very helpful feature I hope this blog will be helpful, if you have any question, don’t hesitate to ask 🙂

Let’s work together

Ready to operate at
the next level?

Tell us about your challenge. We’ll bring the depth, the expertise, and the Swiss precision.

David Diab

David Diab

David Diab has more than 10 years of experience deploying and administering enterprise applications. His expertise spans Elastic Stack, WebLogic, JBoss, Tomcat, and Documentum, plus DevOps tooling (Kubernetes, CI/CD), scheduling solutions (Control-M, Dollar Universe), backup (NetBackup, Backup Exec), and file transfer (CFT). He holds certifications including Elastic Certified Engineer, OpenText Certified Documentum Administrator, Oracle Certified Professional WebLogic Server 12c, Certified Kubernetes Administrator, and ITIL V3 Foundation. Prior to dbi services, David worked as Integration and Operations Engineer at PSA Peugeot Citroën and at Atos. He holds an Engineering degree in Computer Science from UTBM. His industry experience covers automotive, banking, cosmetics, pharmaceutical, insurance, and chemical sectors.View Author posts

74 posts since January 2019

Most read in Elastic Stack

The Elastic Stack posts our readers open most: the all-time favourites, and what is trending right now.

Leave a Comment

Your email address will not be published. Required fields are marked *