Elasticsearch – Index, document, and search

I have recently wrote blogs about ELK Stack and more specifically around Elasticsearch, this blog comes as a complement of the previous blog (Data lifecycle management).

What is an index?

To understand what is an index in Elasticsearch, first we have to know that Elasticsearch is a distributed document store. Instead of storing information as rows of columnar data, Elasticsearch stores complex data structures that have been serialized as JSON documents, each document consists of fields and values. For example, if we want to store dbi services blogs in Elasticsearch, the below could be the document of my previous blog:

{
  "title": "Index data with Elastic",
  "category": "User Stories",
  "author": {
    "first_name": "David",
    "last_name": "Diab"
  }
}

Each document is indexed into an index, which is a logical way of grouping data.

So an index can be thought of as an optimized collection of documents and each document is a collection of fields, which are the key-value pairs that contain your data. By default, Elasticsearch indexes all data in every field and each indexed field has a dedicated, optimized data structure.

How to create an index?

To create an index you can send a REST request and use PUT.

curl -X PUT "localhost:9200/dbi-services-blogs?pretty"

Please note that if the Elasticsearch security features are enabled, you must have the create_index or manage index privilege for the target index.

But, using curl all the time can be a bit tedious, that’s why Kibana has Console, which is a developer tool for creating and submitting Elasticsearch requests more easily.

To access this tool from Kibana, from the left menu, click on Dev Tools.

Console will be the tab by default, I made a first query to get nodes list, result is visible on the right.

Now, you are familiar with the console tool, we can come back to our index creation.

The index dbi-services-blog has been created successfully, please note that here we keep default settings (shards, replicas, aso), don’t worry we will talk about in a separate blog.

How to create a document?

Let’s create a document inside our index, e.g. my last blog:

You noticed, the history of your queries stay until you delete them, which is really cool!

It is time to say that you don’t need to create an index before creating a document. In fact, create a document means index creation if not exists, let’s recall the last query change the name of the index to new-dbi-services-blogs:

How to find a document?

Now, my document (blog) has been indexed in dbi-services-blogs index, let’s do a quick search to retrieve it:

Easy, there is only one document in my index, but what to do in case of millions of documents? Here we are in the search side of Elasticsearch, you know it is for search 😉

There are some query options to search for a document, I will not go in deep in this blog to keep it readable. Below is a Domain Specific Language (DSL) query to search for any document having “Index” in its title filed.

Conclusion

In this blog we saw how to create an index, a document, and how to find it, we discovered the Kibana Console and some queries. Hope that this blog will help you to start with Elasticsearch 🙂

This is not the end of the blog series, it is just the beginning! See you in the next blog.