{"id":26329,"date":"2023-06-27T17:05:05","date_gmt":"2023-06-27T15:05:05","guid":{"rendered":"https:\/\/www.dbi-services.com\/blog\/?p=26329"},"modified":"2023-06-27T17:05:05","modified_gmt":"2023-06-27T15:05:05","slug":"logs-from-filebeat-to-logstash-to-elasticsearch-1","status":"publish","type":"post","link":"https:\/\/www.dbi-services.com\/blog\/logs-from-filebeat-to-logstash-to-elasticsearch-1\/","title":{"rendered":"Logs from Filebeat to Logstash to Elasticsearch -1"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">In a previous <a href=\"https:\/\/www.dbi-services.com\/blog\/elk-stack-filebeat-vs-logstash\/\" target=\"_blank\" rel=\"noreferrer noopener\">blog<\/a>, we saw together the difference between Filebeat and Logstash, we said also that both can cooperate together in some cases.  In this blog series we will deep into a use case: send logs using filebeat, transform it using Logstash, then send it to Elasticsearch, view it from Kibana.<\/p>\n\n\n\n\n\n\n\n<p class=\"wp-block-paragraph\">Let&#8217;s start by the first step, which is the installation of all components (Filebeat, Logstash, Elasticsearch, and Kibana). For that I will use an environment with 3 VMs as following:<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"319\" src=\"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/2023\/06\/image-61-1024x319.png\" alt=\"\" class=\"wp-image-26330\" srcset=\"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/2023\/06\/image-61-1024x319.png 1024w, https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/2023\/06\/image-61-300x93.png 300w, https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/2023\/06\/image-61-768x239.png 768w, https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/2023\/06\/image-61.png 1123w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">Filebeat Installation<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Let&#8217;s install Filebeat on the server we want to monitor (vmelastic1).<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">To download and install Filebeat, you can use tar package (other options are available <a href=\"https:\/\/www.elastic.co\/downloads\/beats\/filebeat\" target=\"_blank\" rel=\"noreferrer noopener\">here<\/a>):<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: plain; title: ; notranslate\" title=\"\">\n&#x5B;elastic@vmelastic1 app]# curl -L -O https:\/\/artifacts.elastic.co\/downloads\/beats\/filebeat\/filebeat-8.8.1-linux-x86_64.tar.gz\n  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n100 41.3M  100 41.3M    0     0   317k      0  0:02:13  0:02:13 --:--:-- 8579k\n&#x5B;elastic@vmelastic1 app]# tar xzvf filebeat-8.8.1-linux-x86_64.tar.gz\nfilebeat-8.8.1-linux-x86_64\/filebeat\nfilebeat-8.8.1-linux-x86_64\/fields.yml\n...\n&#x5B;elastic@vmelastic1 app]# ls -rtl filebeat-8.8.1-linux-x86_64\ntotal 140532\n-rw-r--r--  1 elastic elastic   2647144 Jun  5 21:56 NOTICE.txt\n-rw-r--r--  1 elastic elastic     13675 Jun  5 21:56 LICENSE.txt\ndrwxr-xr-x  4 elastic elastic        24 Jun  5 22:26 kibana\n-rw-------  1 elastic elastic      8622 Jun  5 22:26 filebeat.yml\n-rw-r--r--  1 elastic elastic    177676 Jun  5 22:26 filebeat.reference.yml\n-rw-r--r--  1 elastic elastic   3745648 Jun  5 22:26 fields.yml\ndrwxr-xr-x  2 elastic elastic      4096 Jun  5 22:26 modules.d\ndrwxr-xr-x 71 elastic elastic      4096 Jun  5 22:26 module\n-rwxr-xr-x  1 elastic elastic 137281944 Jun  5 22:36 filebeat\n-rw-r--r--  1 elastic elastic       809 Jun  5 22:36 README.md\n<\/pre><\/div>\n\n\n<p class=\"wp-block-paragraph\">We will configure Filebeat in a next step.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Logstash Installation<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Now, we will install Logstash on a separate server (vmelastic2).<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">To download and install Logstash, you can use tar package (other options are available <a href=\"https:\/\/www.elastic.co\/downloads\/logstash\" target=\"_blank\" rel=\"noreferrer noopener\">here<\/a>):<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: plain; title: ; notranslate\" title=\"\">\n&#x5B;elastic@vmelastic2 app]# curl -L -O https:\/\/artifacts.elastic.co\/downloads\/logstash\/logstash-8.8.1-linux-x86_64.tar.gz\n  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n100  327M  100  327M    0     0  2399k      0  0:02:19  0:02:19 --:--:-- 25.8M\n&#x5B;elastic@vmelastic2 app]# tar xvf logstash-8.8.1-linux-x86_64.tar.gz\n...\nlogstash-8.8.1\/jdk\/release\nlogstash-8.8.1\/LICENSE.txt\nlogstash-8.8.1\/logstash-core\/lib\/logstash\/build.rb\n&#x5B;elastic@vmelastic2 app]# ls -rtl logstash-8.8.1\ntotal 664\n-rw-r--r-- 1 elastic wheel 606491 Jun  2 19:42 NOTICE.TXT\n-rw-r--r-- 1 elastic wheel  13675 Jun  2 19:42 LICENSE.txt\n-rw-r--r-- 1 elastic wheel     16 Jun  2 19:42 JDK_VERSION\n-rw-r--r-- 1 elastic wheel  30037 Jun  2 19:42 Gemfile.lock\n-rw-r--r-- 1 elastic wheel   4101 Jun  2 19:42 Gemfile\ndrwxr-xr-x 2 elastic wheel      6 Jun  2 19:42 data\n-rw-r--r-- 1 elastic wheel   2276 Jun  2 19:42 CONTRIBUTORS\ndrwxr-xr-x 2 elastic elastic    4096 Jun 27 15:34 bin\ndrwxr-xr-x 2 elastic elastic     142 Jun 27 15:34 config\ndrwxr-xr-x 4 elastic elastic      55 Jun 27 15:34 modules\ndrwxr-xr-x 6 elastic elastic      84 Jun 27 15:34 lib\ndrwxr-xr-x 3 elastic elastic      30 Jun 27 15:34 tools\ndrwxr-xr-x 3 elastic elastic      86 Jun 27 15:34 logstash-core-plugin-api\ndrwxr-xr-x 4 elastic elastic      90 Jun 27 15:34 logstash-core\ndrwxr-xr-x 4 elastic elastic      33 Jun 27 15:34 vendor\ndrwxr-xr-x 9 elastic elastic     193 Jun 27 15:34 x-pack\ndrwxr-xr-x 9 elastic elastic     121 Jun 27 15:35 jdk\n<\/pre><\/div>\n\n\n<p class=\"wp-block-paragraph\">Logstash configuration will be done later.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Elasticsearch Installation<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">It is time to install Elasticsearch on a separate server (vmelastic3).<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">To download and install Elasticsearch, you can use tar package (other options are available <a href=\"https:\/\/www.elastic.co\/downloads\/elasticsearch\" target=\"_blank\" rel=\"noreferrer noopener\">here<\/a>):<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: plain; title: ; notranslate\" title=\"\">\n&#x5B;elastic@vmelastic3 app]# curl -L -O https:\/\/artifacts.elastic.co\/downloads\/elasticsearch\/elasticsearch-8.8.1-linux-x86_64.tar.gz\n  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n100  569M  100  569M    0     0  3927k      0  0:02:28  0:02:28 --:--:-- 27.7M\n&#x5B;elastic@vmelastic3 app]# tar xvf elasticsearch-8.8.1-linux-x86_64.tar.gz\nelasticsearch-8.8.1\/\nelasticsearch-8.8.1\/lib\/\n...\n&#x5B;elastic@vmelastic3 app]# ls -rtl elasticsearch-8.8.1\/\ntotal 2224\n-rw-r--r--  1 elastic elastic    8106 Jun  5 23:31 README.asciidoc\n-rw-r--r--  1 elastic elastic    3860 Jun  5 23:31 LICENSE.txt\ndrwxr-xr-x  2 elastic elastic       6 Jun  5 23:33 plugins\ndrwxr-xr-x  2 elastic elastic       6 Jun  5 23:33 logs\n-rw-r--r--  1 elastic elastic 2251526 Jun  5 23:33 NOTICE.txt\ndrwxr-xr-x  5 elastic elastic    4096 Jun  5 23:37 lib\ndrwxr-xr-x  8 elastic elastic      96 Jun  5 23:37 jdk\ndrwxr-xr-x  2 elastic elastic    4096 Jun  5 23:37 bin\ndrwxr-xr-x 74 elastic elastic    4096 Jun  5 23:37 modules\ndrwxr-xr-x  3 elastic elastic     210 Jun 27 16:05 config\n\n<\/pre><\/div>\n\n\n<p class=\"wp-block-paragraph\">Elasticsearch configuration will be done later.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Kibana Installation<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The last one is Kibana, will be installed on the same server as Elasticsearch (vmelastic3).<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">To download and install Elasticsearch, you can use tar package (other options are available <a href=\"https:\/\/www.elastic.co\/downloads\/kibana\" target=\"_blank\" rel=\"noreferrer noopener\">here<\/a>):<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: plain; title: ; notranslate\" title=\"\">\n&#x5B;elastic@vmelastic3 app]# curl -L -O https:\/\/artifacts.elastic.co\/downloads\/kibana\/kibana-8.8.1-linux-x86_64.tar.gz\n  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n100  263M  100  263M    0     0  1705k      0  0:02:38  0:02:38 --:--:-- 7537k\nkibana-8.8.1\/x-pack\/plugins\/fleet\/target\/\nkibana-8.8.1\/x-pack\/plugins\/fleet\/target\/agent_versions_list.json\n&#x5B;elastic@vmelastic3 app]# ls -rtl kibana-8.8.1\ntotal 1388\ndrwxr-xr-x   4 elastic elastic      95 Jun  6 00:19 x-pack\ndrwxr-xr-x  11 elastic elastic     191 Jun  6 00:19 src\n-rw-r--r--   1 elastic elastic    3966 Jun  6 00:19 README.txt\ndrwxr-xr-x   2 elastic elastic       6 Jun  6 00:19 plugins\ndrwxr-xr-x   5 elastic elastic      52 Jun  6 00:19 packages\n-rw-r--r--   1 elastic elastic     738 Jun  6 00:19 package.json\n-rw-r--r--   1 elastic elastic 1370699 Jun  6 00:19 NOTICE.txt\ndrwxr-xr-x 693 elastic elastic   20480 Jun  6 00:19 node_modules\ndrwxr-xr-x   2 elastic elastic       6 Jun  6 00:19 logs\n-rw-r--r--   1 elastic elastic    3860 Jun  6 00:19 LICENSE.txt\ndrwxr-xr-x   2 elastic elastic       6 Jun  6 00:19 data\ndrwxr-xr-x   2 elastic elastic      44 Jun  6 00:19 config\ndrwxr-xr-x   6 elastic elastic      71 Jun  6 00:19 node\ndrwxr-xr-x   2 elastic elastic     175 Jun  6 00:19 bin\n<\/pre><\/div>\n\n\n<h2 class=\"wp-block-heading\">Next steps<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">In a next blog, I will configure all these components to push logs from Filebeat to Logstash, make some transformation in Logstash, then send data to Elasticsearch, and see data from Kibana.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Stay connected \ud83d\ude09<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n","protected":false},"excerpt":{"rendered":"<p>In a previous blog, we saw together the difference between Filebeat and Logstash, we said also that both can cooperate together in some cases. In this blog series we will deep into a use case: send logs using filebeat, transform it using Logstash, then send it to Elasticsearch, view it from Kibana. Let&#8217;s start by&hellip;&nbsp;<\/p>\n","protected":false},"author":46,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"neve_meta_sidebar":"","neve_meta_container":"","neve_meta_enable_content_width":"","neve_meta_content_width":0,"neve_meta_title_alignment":"","neve_meta_author_avatar":"","neve_post_elements_order":"","neve_meta_disable_header":"","neve_meta_disable_footer":"","neve_meta_disable_title":"","neve_meta_reading_time":"","_themeisle_gutenberg_block_has_review":false,"footnotes":""},"categories":[3684],"tags":[86,87,889,88,90],"class_list":["post-26329","post","type-post","status-publish","format-standard","category-elastic-stack","tag-elasticsearch","tag-elk","tag-filebeat","tag-kibana","tag-logstash"],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v28.8 (Yoast SEO v28.9) - https:\/\/yoast.com\/product\/yoast-seo-premium-wordpress\/ -->\n<title>Logs from Filebeat to Logstash to Elasticsearch -1 - dbi services Blog<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.dbi-services.com\/blog\/logs-from-filebeat-to-logstash-to-elasticsearch-1\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Logs from Filebeat to Logstash to Elasticsearch -1\" \/>\n<meta property=\"og:description\" content=\"In a previous blog, we saw together the difference between Filebeat and Logstash, we said also that both can cooperate together in some cases. In this blog series we will deep into a use case: send logs using filebeat, transform it using Logstash, then send it to Elasticsearch, view it from Kibana. Let&#8217;s start by&hellip;&nbsp;\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.dbi-services.com\/blog\/logs-from-filebeat-to-logstash-to-elasticsearch-1\/\" \/>\n<meta property=\"og:site_name\" content=\"dbi services Blog\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/dbiservices\/\" \/>\n<meta property=\"article:published_time\" content=\"2023-06-27T15:05:05+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/2023\/06\/image-61.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1123\" \/>\n\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"David Diab\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@dbiservices\" \/>\n<meta name=\"twitter:site\" content=\"@dbiservices\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"David Diab\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/logs-from-filebeat-to-logstash-to-elasticsearch-1\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/logs-from-filebeat-to-logstash-to-elasticsearch-1\\\/\"},\"author\":{\"name\":\"David Diab\",\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/#\\\/schema\\\/person\\\/bd645301427dc811a22ddd641e945966\"},\"headline\":\"Logs from Filebeat to Logstash to Elasticsearch -1\",\"datePublished\":\"2023-06-27T15:05:05+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/logs-from-filebeat-to-logstash-to-elasticsearch-1\\\/\"},\"wordCount\":262,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/#organization\"},\"image\":{\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/logs-from-filebeat-to-logstash-to-elasticsearch-1\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/wp-content\\\/uploads\\\/2023\\\/06\\\/image-61-1024x319.png\",\"keywords\":[\"Elasticsearch\",\"ELK\",\"Filebeat\",\"Kibana\",\"Logstash\"],\"articleSection\":[\"Elastic Stack\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/logs-from-filebeat-to-logstash-to-elasticsearch-1\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/logs-from-filebeat-to-logstash-to-elasticsearch-1\\\/\",\"url\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/logs-from-filebeat-to-logstash-to-elasticsearch-1\\\/\",\"name\":\"Logs from Filebeat to Logstash to Elasticsearch -1 - dbi services Blog\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/logs-from-filebeat-to-logstash-to-elasticsearch-1\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/logs-from-filebeat-to-logstash-to-elasticsearch-1\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/wp-content\\\/uploads\\\/2023\\\/06\\\/image-61-1024x319.png\",\"datePublished\":\"2023-06-27T15:05:05+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/logs-from-filebeat-to-logstash-to-elasticsearch-1\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/logs-from-filebeat-to-logstash-to-elasticsearch-1\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/logs-from-filebeat-to-logstash-to-elasticsearch-1\\\/#primaryimage\",\"url\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/wp-content\\\/uploads\\\/2023\\\/06\\\/image-61.png\",\"contentUrl\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/wp-content\\\/uploads\\\/2023\\\/06\\\/image-61.png\",\"width\":1123,\"height\":350},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/logs-from-filebeat-to-logstash-to-elasticsearch-1\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Logs from Filebeat to Logstash to Elasticsearch -1\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/#website\",\"url\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/\",\"name\":\"dbi services Blog\",\"description\":\"#BeSharing\",\"publisher\":{\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/#organization\",\"name\":\"dbi services\",\"url\":\"https:\\\/\\\/www.dbi-services.com\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/wp-content\\\/uploads\\\/2026\\\/09\\\/dbi-services-logo.png\",\"contentUrl\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/wp-content\\\/uploads\\\/2026\\\/09\\\/dbi-services-logo.png\",\"width\":512,\"height\":512,\"caption\":\"dbi services\"},\"image\":{\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/#\\\/schema\\\/logo\\\/image\\\/\"},\"sameAs\":[\"https:\\\/\\\/www.facebook.com\\\/dbiservices\\\/\",\"https:\\\/\\\/x.com\\\/dbiservices\",\"https:\\\/\\\/www.linkedin.com\\\/company\\\/dbi-services\\\/\",\"https:\\\/\\\/www.youtube.com\\\/user\\\/dbiservices\"],\"description\":\"Swiss IT consulting and services company specialized in efficient data infrastructures and platforms: databases, application servers, cloud, DevOps, digitalization, analytics, data quality and monitoring. Headquartered in Del\u00e9mont with offices in Nyon, Basel, Bern and Zurich.\",\"email\":\"info@dbi-services.com\",\"telephone\":\"+41 32 422 96 00\",\"legalName\":\"dbi services SA\"},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/#\\\/schema\\\/person\\\/bd645301427dc811a22ddd641e945966\",\"name\":\"David Diab\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/wp-content\\\/uploads\\\/2022\\\/09\\\/DDI_web-min-scaled.jpg\",\"url\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/wp-content\\\/uploads\\\/2022\\\/09\\\/DDI_web-min-scaled.jpg\",\"contentUrl\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/wp-content\\\/uploads\\\/2022\\\/09\\\/DDI_web-min-scaled.jpg\",\"caption\":\"David Diab\"},\"description\":\"David Diab has more than 10 years of experience deploying and administering enterprise applications. His expertise spans Elastic Stack, WebLogic, JBoss, Tomcat, and Documentum, plus DevOps tooling (Kubernetes, CI\\\/CD), scheduling solutions (Control-M, Dollar Universe), backup (NetBackup, Backup Exec), and file transfer (CFT). He holds certifications including Elastic Certified Engineer, OpenText Certified Documentum Administrator, Oracle Certified Professional WebLogic Server 12c, Certified Kubernetes Administrator, and ITIL V3 Foundation. Prior to dbi services, David worked as Integration and Operations Engineer at PSA Peugeot Citro\u00ebn and at Atos. He holds an Engineering degree in Computer Science from UTBM. His industry experience covers automotive, banking, cosmetics, pharmaceutical, insurance, and chemical sectors.\",\"url\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/author\\\/david-diab\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Logs from Filebeat to Logstash to Elasticsearch -1 - dbi services Blog","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.dbi-services.com\/blog\/logs-from-filebeat-to-logstash-to-elasticsearch-1\/","og_locale":"en_US","og_type":"article","og_title":"Logs from Filebeat to Logstash to Elasticsearch -1","og_description":"In a previous blog, we saw together the difference between Filebeat and Logstash, we said also that both can cooperate together in some cases. In this blog series we will deep into a use case: send logs using filebeat, transform it using Logstash, then send it to Elasticsearch, view it from Kibana. Let&#8217;s start by&hellip;&nbsp;","og_url":"https:\/\/www.dbi-services.com\/blog\/logs-from-filebeat-to-logstash-to-elasticsearch-1\/","og_site_name":"dbi services Blog","article_publisher":"https:\/\/www.facebook.com\/dbiservices\/","article_published_time":"2023-06-27T15:05:05+00:00","og_image":[{"width":1123,"height":350,"url":"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/2023\/06\/image-61.png","type":"image\/png"}],"author":"David Diab","twitter_card":"summary_large_image","twitter_creator":"@dbiservices","twitter_site":"@dbiservices","twitter_misc":{"Written by":"David Diab","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.dbi-services.com\/blog\/logs-from-filebeat-to-logstash-to-elasticsearch-1\/#article","isPartOf":{"@id":"https:\/\/www.dbi-services.com\/blog\/logs-from-filebeat-to-logstash-to-elasticsearch-1\/"},"author":{"name":"David Diab","@id":"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/bd645301427dc811a22ddd641e945966"},"headline":"Logs from Filebeat to Logstash to Elasticsearch -1","datePublished":"2023-06-27T15:05:05+00:00","mainEntityOfPage":{"@id":"https:\/\/www.dbi-services.com\/blog\/logs-from-filebeat-to-logstash-to-elasticsearch-1\/"},"wordCount":262,"commentCount":0,"publisher":{"@id":"https:\/\/www.dbi-services.com\/#organization"},"image":{"@id":"https:\/\/www.dbi-services.com\/blog\/logs-from-filebeat-to-logstash-to-elasticsearch-1\/#primaryimage"},"thumbnailUrl":"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/2023\/06\/image-61-1024x319.png","keywords":["Elasticsearch","ELK","Filebeat","Kibana","Logstash"],"articleSection":["Elastic Stack"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.dbi-services.com\/blog\/logs-from-filebeat-to-logstash-to-elasticsearch-1\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.dbi-services.com\/blog\/logs-from-filebeat-to-logstash-to-elasticsearch-1\/","url":"https:\/\/www.dbi-services.com\/blog\/logs-from-filebeat-to-logstash-to-elasticsearch-1\/","name":"Logs from Filebeat to Logstash to Elasticsearch -1 - dbi services Blog","isPartOf":{"@id":"https:\/\/www.dbi-services.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.dbi-services.com\/blog\/logs-from-filebeat-to-logstash-to-elasticsearch-1\/#primaryimage"},"image":{"@id":"https:\/\/www.dbi-services.com\/blog\/logs-from-filebeat-to-logstash-to-elasticsearch-1\/#primaryimage"},"thumbnailUrl":"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/2023\/06\/image-61-1024x319.png","datePublished":"2023-06-27T15:05:05+00:00","breadcrumb":{"@id":"https:\/\/www.dbi-services.com\/blog\/logs-from-filebeat-to-logstash-to-elasticsearch-1\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.dbi-services.com\/blog\/logs-from-filebeat-to-logstash-to-elasticsearch-1\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.dbi-services.com\/blog\/logs-from-filebeat-to-logstash-to-elasticsearch-1\/#primaryimage","url":"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/2023\/06\/image-61.png","contentUrl":"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/2023\/06\/image-61.png","width":1123,"height":350},{"@type":"BreadcrumbList","@id":"https:\/\/www.dbi-services.com\/blog\/logs-from-filebeat-to-logstash-to-elasticsearch-1\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.dbi-services.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Logs from Filebeat to Logstash to Elasticsearch -1"}]},{"@type":"WebSite","@id":"https:\/\/www.dbi-services.com\/blog\/#website","url":"https:\/\/www.dbi-services.com\/blog\/","name":"dbi services Blog","description":"#BeSharing","publisher":{"@id":"https:\/\/www.dbi-services.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.dbi-services.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.dbi-services.com\/#organization","name":"dbi services","url":"https:\/\/www.dbi-services.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.dbi-services.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/2026\/09\/dbi-services-logo.png","contentUrl":"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/2026\/09\/dbi-services-logo.png","width":512,"height":512,"caption":"dbi services"},"image":{"@id":"https:\/\/www.dbi-services.com\/blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/dbiservices\/","https:\/\/x.com\/dbiservices","https:\/\/www.linkedin.com\/company\/dbi-services\/","https:\/\/www.youtube.com\/user\/dbiservices"],"description":"Swiss IT consulting and services company specialized in efficient data infrastructures and platforms: databases, application servers, cloud, DevOps, digitalization, analytics, data quality and monitoring. Headquartered in Del\u00e9mont with offices in Nyon, Basel, Bern and Zurich.","email":"info@dbi-services.com","telephone":"+41 32 422 96 00","legalName":"dbi services SA"},{"@type":"Person","@id":"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/bd645301427dc811a22ddd641e945966","name":"David Diab","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/2022\/09\/DDI_web-min-scaled.jpg","url":"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/2022\/09\/DDI_web-min-scaled.jpg","contentUrl":"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/2022\/09\/DDI_web-min-scaled.jpg","caption":"David Diab"},"description":"David Diab has more than 10 years of experience deploying and administering enterprise applications. His expertise spans Elastic Stack, WebLogic, JBoss, Tomcat, and Documentum, plus DevOps tooling (Kubernetes, CI\/CD), scheduling solutions (Control-M, Dollar Universe), backup (NetBackup, Backup Exec), and file transfer (CFT). He holds certifications including Elastic Certified Engineer, OpenText Certified Documentum Administrator, Oracle Certified Professional WebLogic Server 12c, Certified Kubernetes Administrator, and ITIL V3 Foundation. Prior to dbi services, David worked as Integration and Operations Engineer at PSA Peugeot Citro\u00ebn and at Atos. He holds an Engineering degree in Computer Science from UTBM. His industry experience covers automotive, banking, cosmetics, pharmaceutical, insurance, and chemical sectors.","url":"https:\/\/www.dbi-services.com\/blog\/author\/david-diab\/"}]}},"_links":{"self":[{"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/posts\/26329","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/users\/46"}],"replies":[{"embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/comments?post=26329"}],"version-history":[{"count":0,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/posts\/26329\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/media?parent=26329"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/categories?post=26329"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/tags?post=26329"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}