{"id":13789,"date":"2020-03-26T14:22:31","date_gmt":"2020-03-26T13:22:31","guid":{"rendered":"https:\/\/www.dbi-services.com\/blog\/sending-postgresql-data-to-aws-s3-by-using-aws-kinesis-data-streams\/"},"modified":"2020-03-26T14:22:31","modified_gmt":"2020-03-26T13:22:31","slug":"sending-postgresql-data-to-aws-s3-by-using-aws-kinesis-data-streams","status":"publish","type":"post","link":"https:\/\/www.dbi-services.com\/blog\/sending-postgresql-data-to-aws-s3-by-using-aws-kinesis-data-streams\/","title":{"rendered":"Sending PostgreSQL data to AWS S3 by using AWS Kinesis data streams"},"content":{"rendered":"

Before we really start with this post: This is just an experiment and you should not implement it like this in real life. The goal of this post is just to show what is possible and I am not saying that you should do it (the way it is implemented here will be catastrophic for your database performance and it is not really secure). As I am currently exploring a lot of AWS services<\/a> I wanted to check if there is an easy way to send data from PostgreSQL into an AWS Kinesis<\/a> data stream for testing purposes and it turned out that this is actually quite easy if you have the AWS Command Line Interface<\/a> installed and configured on the database server. <\/p>\n

<\/p>\n

Creating a new Kinesis stream in AWS is actually a matter of a few clicks (of course you can do that with the command line utilities as well):
\n\"\"<\/a>
\nWhat I want is a simple data stream where I can put data into:
\n\"\"<\/a>
\nObviously the new stream needs a name and as I will not do any performance or stress testing one shard is absolutely fine:
\n\"\"<\/a>
\nThat’s all what needs to be done, the new stream is ready:
\n\"\"<\/a><\/p>\n

An AWS Kinesis stream is not persistent by default. That means, if you want to permanently store the output of a stream you need to connect the stream to a consumer that processes, eventually transforms, and finally stores the data somewhere. For this you can use AWS Kinesis Firehose<\/a> and this is what I’ll be doing here:
\n\"\"<\/a>
\nAs I want to use AWS S3<\/a> as the target for my data I need to use a delivery stream:
\n\"\"<\/a>
\nThe delivery stream needs a name as well and we will use the stream just created above as the source:
\n\"\"<\/a>
\n\"\"<\/a>
\nWe could go ahead and transform the data with an AWS Lambda<\/a> function but we’re going to keep it simple for now and skip this option:
\n\"\"<\/a>
\n\"\"<\/a>
\nThe next screen is about the target for the data. This could be AWS Redshift<\/a>, AWS Elasticsearch<\/a>, Splunk<\/a> or AWS S3<\/a>, what we’ll be doing here:
\n\"\"<\/a>
\nFinally specifying the target S3 bucket:
\n\"\"<\/a>
\nThe settings for buffering at not really important for this test but will matter for real systems as these settings determine how fast your data is delivered to S3 (we also do not care about encryption and compression for now):
\n\"\"<\/a>
\nError logging should of course be enabled and we need an IAM role<\/a> with appropriate permissions:
\n\"\"<\/a>
\nFinal review:
\n\"\"<\/a>
\n\"\"<\/a>
\n… and the stream and delivery stream are ready to use:
\n\"\"<\/a><\/p>\n

That’s it for the setup on the AWS side and we can continue with configuring PostgreSQL to call the AWS command line utility to write data to the stream. Callling system commands from inside PostgreSQL can be done in various ways, we’ll be using pl\/Perl<\/a> for that, and even the untrusted<\/a> version so only superusers will be able to do that:<\/p>\n

\npostgres=# create extension plperlu;\nCREATE EXTENSION\npostgres=# \\dx\n                      List of installed extensions\n  Name   | Version |   Schema   |              Description               \n---------+---------+------------+----------------------------------------\n plperlu | 1.0     | pg_catalog | PL\/PerlU untrusted procedural language\n plpgsql | 1.0     | pg_catalog | PL\/pgSQL procedural language\n(2 rows)\n<\/pre>\n
Next we need a table that will contain the data we want to send to the stream:<\/p>\n
\npostgres=# create table stream_data ( id serial primary key\n                                    , stream text );\n<\/pre>\n
A trigger will fire each time a new row is inserted and the trigger function will call the AWS command line interface:<\/p>\n
\ncreate or replace function f_send_to_kinesis()\nreturns trigger\nlanguage plperlu\nAS $$\n     system('aws kinesis put-record --stream-name postgres-to-kinesis --partition-key 1 --data '.$_TD->{new}{stream});\n     return;\n$$;\n\ncreate trigger tr_test\nafter insert or update\non stream_data\nfor each row\nexecute procedure f_send_to_kinesis();\n<\/pre>\n
This is all we need. Let’s insert a row into the table and check if it arrives in AWS S3 (remember that it will take up to 300 seconds or 5MB of data):<\/p>\n
\npostgres=# insert into stream_data (stream) values ('aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa');\nINSERT 0 1\npostgres=# insert into stream_data (stream) values ('aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa');\nINSERT 0 1\npostgres=# insert into stream_data (stream) values ('aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa');\nINSERT 0 1\npostgres=# insert into stream_data (stream) values ('aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa');\nINSERT 0 1\npostgres=# insert into stream_data (stream) values ('aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa');\nINSERT 0 1\npostgres=# insert into stream_data (stream) values ('aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa');\nINSERT 0 1\npostgres=# insert into stream_data (stream) values ('aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa');\nINSERT 0 1\npostgres=# insert into stream_data (stream) values ('aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa');\nINSERT 0 1\npostgres=# insert into stream_data (stream) values ('aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa');\nINSERT 0 1\npostgres=# select * from stream_data;\n id |              stream              \n----+----------------------------------\n  1 | aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\n  2 | aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\n  3 | aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\n  4 | aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\n  5 | aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\n  6 | aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\n  7 | aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\n  8 | aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\n  9 | aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\n(9 rows)\n<\/pre>\n
You will also notice that the insert takes quite some time because calling the AWS command line utility and waiting for the result takes ages compared to a normal insert.<\/p>\n
While waiting for the data to arrive you can check the monitoring section of both, the stream and the delivery stream:
\n\"\"<\/a>
\n\"\"<\/a><\/p>\n
After a while the data appears in S3 and it is organized in [YEAR]\/[MONTH]\/[DAY]\/[HOUR]:
\n\"\"<\/a>
\n\"\"<\/a>
\n\"\"<\/a>
\n\"\"<\/a>
\n\"\"<\/a><\/p>\n
Looking at the file itself all our data is there:
\n\"\"<\/a><\/p>\n
So, actually it is quite easy to send data to an AWS Kinesis stream. If you really need to send data out of PostgreSQL I probably would go for listen<\/a>\/notify<\/a> to make the calls to the AWS command line utility not blocking the inserts or updates to the table that holds the data for the stream. Anyway, currently I am not aware of a good use case for sending streams of data out of PostgreSQL directly to AWS Kinesis. Maybe you do something like that and how?<\/p>\n","protected":false},"excerpt":{"rendered":"
Before we really start with this post: This is just an experiment and you should not implement it like this in real life. The goal of this post is just to show what is possible and I am not saying that you should do it (the way it is implemented here will be catastrophic for […]<\/p>\n","protected":false},"author":29,"featured_media":13790,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[955,229],"tags":[133,1881,1882,77],"type_dbi":[],"class_list":["post-13789","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-cloud","category-database-administration-monitoring","tag-aws","tag-firehose","tag-kinesis","tag-postgresql"],"acf":[],"yoast_head":"\nSending PostgreSQL data to AWS S3 by using AWS Kinesis data streams - dbi Blog<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.dbi-services.com\/blog\/sending-postgresql-data-to-aws-s3-by-using-aws-kinesis-data-streams\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Sending PostgreSQL data to AWS S3 by using AWS Kinesis data streams\" \/>\n<meta property=\"og:description\" content=\"Before we really start with this post: This is just an experiment and you should not implement it like this in real life. The goal of this post is just to show what is possible and I am not saying that you should do it (the way it is implemented here will be catastrophic for […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.dbi-services.com\/blog\/sending-postgresql-data-to-aws-s3-by-using-aws-kinesis-data-streams\/\" \/>\n<meta property=\"og:site_name\" content=\"dbi Blog\" \/>\n<meta property=\"article:published_time\" content=\"2020-03-26T13:22:31+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2022\/04\/aws_kinesis_1.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1000\" \/>\n\t<meta property=\"og:image:height\" content=\"526\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Daniel Westermann\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@westermanndanie\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Daniel Westermann\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/sending-postgresql-data-to-aws-s3-by-using-aws-kinesis-data-streams\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/sending-postgresql-data-to-aws-s3-by-using-aws-kinesis-data-streams\/\"},\"author\":{\"name\":\"Daniel Westermann\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/8d08e9bd996a89bd75c0286cbabf3c66\"},\"headline\":\"Sending PostgreSQL data to AWS S3 by using AWS Kinesis data streams\",\"datePublished\":\"2020-03-26T13:22:31+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/sending-postgresql-data-to-aws-s3-by-using-aws-kinesis-data-streams\/\"},\"wordCount\":741,\"commentCount\":0,\"image\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/sending-postgresql-data-to-aws-s3-by-using-aws-kinesis-data-streams\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2022\/04\/aws_kinesis_1.jpg\",\"keywords\":[\"AWS\",\"FireHose\",\"Kinesis\",\"PostgreSQL\"],\"articleSection\":[\"Cloud\",\"Database Administration & Monitoring\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/www.dbi-services.com\/blog\/sending-postgresql-data-to-aws-s3-by-using-aws-kinesis-data-streams\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/sending-postgresql-data-to-aws-s3-by-using-aws-kinesis-data-streams\/\",\"url\":\"https:\/\/www.dbi-services.com\/blog\/sending-postgresql-data-to-aws-s3-by-using-aws-kinesis-data-streams\/\",\"name\":\"Sending PostgreSQL data to AWS S3 by using AWS Kinesis data streams - dbi Blog\",\"isPartOf\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/sending-postgresql-data-to-aws-s3-by-using-aws-kinesis-data-streams\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/sending-postgresql-data-to-aws-s3-by-using-aws-kinesis-data-streams\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2022\/04\/aws_kinesis_1.jpg\",\"datePublished\":\"2020-03-26T13:22:31+00:00\",\"author\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/8d08e9bd996a89bd75c0286cbabf3c66\"},\"breadcrumb\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/sending-postgresql-data-to-aws-s3-by-using-aws-kinesis-data-streams\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.dbi-services.com\/blog\/sending-postgresql-data-to-aws-s3-by-using-aws-kinesis-data-streams\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/sending-postgresql-data-to-aws-s3-by-using-aws-kinesis-data-streams\/#primaryimage\",\"url\":\"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2022\/04\/aws_kinesis_1.jpg\",\"contentUrl\":\"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2022\/04\/aws_kinesis_1.jpg\",\"width\":1000,\"height\":526},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/sending-postgresql-data-to-aws-s3-by-using-aws-kinesis-data-streams\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Accueil\",\"item\":\"https:\/\/www.dbi-services.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Sending PostgreSQL data to AWS S3 by using AWS Kinesis data streams\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/#website\",\"url\":\"https:\/\/www.dbi-services.com\/blog\/\",\"name\":\"dbi Blog\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.dbi-services.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/8d08e9bd996a89bd75c0286cbabf3c66\",\"name\":\"Daniel Westermann\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/secure.gravatar.com\/avatar\/31350ceeecb1dd8986339a29bf040d4cd3cd087d410deccd8f55234466d6c317?s=96&d=mm&r=g\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/31350ceeecb1dd8986339a29bf040d4cd3cd087d410deccd8f55234466d6c317?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/31350ceeecb1dd8986339a29bf040d4cd3cd087d410deccd8f55234466d6c317?s=96&d=mm&r=g\",\"caption\":\"Daniel Westermann\"},\"description\":\"Daniel Westermann is Principal Consultant and Technology Leader Open Infrastructure at dbi services. He has more than 15 years of experience in management, engineering and optimization of databases and infrastructures, especially on Oracle and PostgreSQL. Since the beginning of his career, he has specialized in Oracle Technologies and is Oracle Certified Professional 12c and Oracle Certified Expert RAC\/GridInfra. Over time, Daniel has become increasingly interested in open source technologies, becoming \u201cTechnology Leader Open Infrastructure\u201d and PostgreSQL expert. \u00a0Based on community or EnterpriseDB tools, he develops and installs complex high available solutions with PostgreSQL. He is also a certified PostgreSQL Plus 9.0 Professional and a Postgres Advanced Server 9.4 Professional. He is a regular speaker at PostgreSQL conferences in Switzerland and Europe. Today Daniel is also supporting our customers on AWS services such as AWS RDS, database migrations into the cloud, EC2 and automated infrastructure management with AWS SSM (System Manager). He is a certified AWS Solutions Architect Professional. Prior to dbi services, Daniel was Management System Engineer at LC SYSTEMS-Engineering AG in Basel. Before that, he worked as Oracle Developper &\u00a0Project Manager at Delta Energy Solutions AG in Basel (today Powel AG). Daniel holds a diploma in Business Informatics (DHBW, Germany). His branch-related experience mainly covers the pharma industry, the financial sector, energy, lottery and telecommunications.\",\"sameAs\":[\"https:\/\/x.com\/westermanndanie\"],\"url\":\"https:\/\/www.dbi-services.com\/blog\/author\/daniel-westermann\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Sending PostgreSQL data to AWS S3 by using AWS Kinesis data streams - dbi Blog","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.dbi-services.com\/blog\/sending-postgresql-data-to-aws-s3-by-using-aws-kinesis-data-streams\/","og_locale":"en_US","og_type":"article","og_title":"Sending PostgreSQL data to AWS S3 by using AWS Kinesis data streams","og_description":"Before we really start with this post: This is just an experiment and you should not implement it like this in real life. The goal of this post is just to show what is possible and I am not saying that you should do it (the way it is implemented here will be catastrophic for […]","og_url":"https:\/\/www.dbi-services.com\/blog\/sending-postgresql-data-to-aws-s3-by-using-aws-kinesis-data-streams\/","og_site_name":"dbi Blog","article_published_time":"2020-03-26T13:22:31+00:00","og_image":[{"width":1000,"height":526,"url":"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2022\/04\/aws_kinesis_1.jpg","type":"image\/jpeg"}],"author":"Daniel Westermann","twitter_card":"summary_large_image","twitter_creator":"@westermanndanie","twitter_misc":{"Written by":"Daniel Westermann","Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.dbi-services.com\/blog\/sending-postgresql-data-to-aws-s3-by-using-aws-kinesis-data-streams\/#article","isPartOf":{"@id":"https:\/\/www.dbi-services.com\/blog\/sending-postgresql-data-to-aws-s3-by-using-aws-kinesis-data-streams\/"},"author":{"name":"Daniel Westermann","@id":"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/8d08e9bd996a89bd75c0286cbabf3c66"},"headline":"Sending PostgreSQL data to AWS S3 by using AWS Kinesis data streams","datePublished":"2020-03-26T13:22:31+00:00","mainEntityOfPage":{"@id":"https:\/\/www.dbi-services.com\/blog\/sending-postgresql-data-to-aws-s3-by-using-aws-kinesis-data-streams\/"},"wordCount":741,"commentCount":0,"image":{"@id":"https:\/\/www.dbi-services.com\/blog\/sending-postgresql-data-to-aws-s3-by-using-aws-kinesis-data-streams\/#primaryimage"},"thumbnailUrl":"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2022\/04\/aws_kinesis_1.jpg","keywords":["AWS","FireHose","Kinesis","PostgreSQL"],"articleSection":["Cloud","Database Administration & Monitoring"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.dbi-services.com\/blog\/sending-postgresql-data-to-aws-s3-by-using-aws-kinesis-data-streams\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.dbi-services.com\/blog\/sending-postgresql-data-to-aws-s3-by-using-aws-kinesis-data-streams\/","url":"https:\/\/www.dbi-services.com\/blog\/sending-postgresql-data-to-aws-s3-by-using-aws-kinesis-data-streams\/","name":"Sending PostgreSQL data to AWS S3 by using AWS Kinesis data streams - dbi Blog","isPartOf":{"@id":"https:\/\/www.dbi-services.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.dbi-services.com\/blog\/sending-postgresql-data-to-aws-s3-by-using-aws-kinesis-data-streams\/#primaryimage"},"image":{"@id":"https:\/\/www.dbi-services.com\/blog\/sending-postgresql-data-to-aws-s3-by-using-aws-kinesis-data-streams\/#primaryimage"},"thumbnailUrl":"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2022\/04\/aws_kinesis_1.jpg","datePublished":"2020-03-26T13:22:31+00:00","author":{"@id":"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/8d08e9bd996a89bd75c0286cbabf3c66"},"breadcrumb":{"@id":"https:\/\/www.dbi-services.com\/blog\/sending-postgresql-data-to-aws-s3-by-using-aws-kinesis-data-streams\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.dbi-services.com\/blog\/sending-postgresql-data-to-aws-s3-by-using-aws-kinesis-data-streams\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.dbi-services.com\/blog\/sending-postgresql-data-to-aws-s3-by-using-aws-kinesis-data-streams\/#primaryimage","url":"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2022\/04\/aws_kinesis_1.jpg","contentUrl":"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2022\/04\/aws_kinesis_1.jpg","width":1000,"height":526},{"@type":"BreadcrumbList","@id":"https:\/\/www.dbi-services.com\/blog\/sending-postgresql-data-to-aws-s3-by-using-aws-kinesis-data-streams\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Accueil","item":"https:\/\/www.dbi-services.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Sending PostgreSQL data to AWS S3 by using AWS Kinesis data streams"}]},{"@type":"WebSite","@id":"https:\/\/www.dbi-services.com\/blog\/#website","url":"https:\/\/www.dbi-services.com\/blog\/","name":"dbi Blog","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.dbi-services.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/8d08e9bd996a89bd75c0286cbabf3c66","name":"Daniel Westermann","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/31350ceeecb1dd8986339a29bf040d4cd3cd087d410deccd8f55234466d6c317?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/31350ceeecb1dd8986339a29bf040d4cd3cd087d410deccd8f55234466d6c317?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/31350ceeecb1dd8986339a29bf040d4cd3cd087d410deccd8f55234466d6c317?s=96&d=mm&r=g","caption":"Daniel Westermann"},"description":"Daniel Westermann is Principal Consultant and Technology Leader Open Infrastructure at dbi services. He has more than 15 years of experience in management, engineering and optimization of databases and infrastructures, especially on Oracle and PostgreSQL. Since the beginning of his career, he has specialized in Oracle Technologies and is Oracle Certified Professional 12c and Oracle Certified Expert RAC\/GridInfra. Over time, Daniel has become increasingly interested in open source technologies, becoming \u201cTechnology Leader Open Infrastructure\u201d and PostgreSQL expert. \u00a0Based on community or EnterpriseDB tools, he develops and installs complex high available solutions with PostgreSQL. He is also a certified PostgreSQL Plus 9.0 Professional and a Postgres Advanced Server 9.4 Professional. He is a regular speaker at PostgreSQL conferences in Switzerland and Europe. Today Daniel is also supporting our customers on AWS services such as AWS RDS, database migrations into the cloud, EC2 and automated infrastructure management with AWS SSM (System Manager). He is a certified AWS Solutions Architect Professional. Prior to dbi services, Daniel was Management System Engineer at LC SYSTEMS-Engineering AG in Basel. Before that, he worked as Oracle Developper &\u00a0Project Manager at Delta Energy Solutions AG in Basel (today Powel AG). Daniel holds a diploma in Business Informatics (DHBW, Germany). His branch-related experience mainly covers the pharma industry, the financial sector, energy, lottery and telecommunications.","sameAs":["https:\/\/x.com\/westermanndanie"],"url":"https:\/\/www.dbi-services.com\/blog\/author\/daniel-westermann\/"}]}},"_links":{"self":[{"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/posts\/13789","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/users\/29"}],"replies":[{"embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/comments?post=13789"}],"version-history":[{"count":0,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/posts\/13789\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/media\/13790"}],"wp:attachment":[{"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/media?parent=13789"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/categories?post=13789"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/tags?post=13789"},{"taxonomy":"type","embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/type_dbi?post=13789"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}