{"id":4743,"date":"2015-05-18T11:12:43","date_gmt":"2015-05-18T09:12:43","guid":{"rendered":"https:\/\/www.dbi-services.com\/blog\/the-fastest-way-to-load-1m-rows-in-postgresql\/"},"modified":"2015-05-18T11:12:43","modified_gmt":"2015-05-18T09:12:43","slug":"the-fastest-way-to-load-1m-rows-in-postgresql","status":"publish","type":"post","link":"https:\/\/www.dbi-services.com\/blog\/the-fastest-way-to-load-1m-rows-in-postgresql\/","title":{"rendered":"the fastest way to load 1m rows in postgresql"},"content":{"rendered":"<p><a class=\"easyblog-thumb-preview\" title=\"postreslogo.png\" href=\"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2022\/04\/postreslogo.png\"><img decoding=\"async\" title=\"postreslogo.png\" src=\"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2022\/04\/postreslogo.png\" alt=\"postreslogo.png\" \/><\/a><\/p>\n<p>There have been several posts on how to load 1m rows into a database in the last days:<br \/>\n<a href=\"http:\/\/dbi-services.com\/blog\/variations-on-1m-rows-insert-1-bulk-insert\/\" target=\"_blank\">Variations on 1M rows insert (1): bulk insert<\/a><br \/>\n<a href=\"http:\/\/dbi-services.com\/blog\/variations-on-1m-insert-2-write-commit\/\" target=\"_blank\">Variations on 1M rows insert(2): commit write<\/a><br \/>\n<a href=\"http:\/\/dbi-services.com\/blog\/variations-on-1m-rows-insert-1-bulk-insert-postgresql\/\" target=\"_blank\">Variations on 1M rows insert (1): bulk insert &#8211; PostgreSQL<\/a><br \/>\n<a title=\"Variations on 1M rows insert(2): commit write \u2013 PostgreSQL\" href=\"http:\/\/dbi-services.com\/blog\/variations-on-1m-rows-insert2-commit-write-postgresql\/\" target=\"_blank\">Variations on 1M rows insert(2): commit write &#8211; PostgreSQL<\/a><br \/>\n<a href=\"http:\/\/dbi-services.com\/blog\/variations-on-1m-rows-insert-3-timesten\/\" target=\"_blank\">Variations on 1M rows insert (3): TimesTen<\/a><\/p>\n<p>In this post I&#8217;ll focus on how to prepare a PostgreSQL database for bulk loading in more detail. The test system is the same which was used in the previous posts.<\/p>\n<pre class=\"brush: bash; gutter: true; first-line: 1\">cat \/etc\/oracle-release\nOracle Linux Server release 7.1\ncat \/proc\/cpuinfo | grep proc\nprocessor : 0\ncat \/proc\/meminfo | head -1\nMemTotal: 502612 kB<\/pre>\n<p class=\"brush: sql; gutter: true; first-line: 1\">I did not tweak any kernel parameters, just the default:<\/p>\n<pre class=\"brush: sql; gutter: true; first-line: 1\">cat \/usr\/lib\/sysctl.d\/00-system.conf | grep -v \"#\"\nnet.bridge.bridge-nf-call-ip6tables = 0\nnet.bridge.bridge-nf-call-iptables = 0\nnet.bridge.bridge-nf-call-arptables = 0\nkernel.shmmax = 4294967295\nkernel.shmall = 268435456<\/pre>\n<p class=\"brush: sql; gutter: true; first-line: 1\">All is on standard xfs filesystems. No tweaking there, too:<\/p>\n<pre class=\"brush: actionscript3; gutter: true; first-line: 1\">df -h\nFilesystem           Size  Used Avail Use% Mounted on\n\/dev\/mapper\/ol-root   48G  4.5G   44G  10% \/\ndevtmpfs             237M     0  237M   0% \/dev\ntmpfs                246M  4.0K  246M   1% \/dev\/shm\ntmpfs                246M  4.4M  242M   2% \/run\ntmpfs                246M     0  246M   0% \/sys\/fs\/cgroup\n\/dev\/sda1            497M  255M  243M  52% \/boot\nmount | grep \/dev\/mapper\/ol-root\n\/dev\/mapper\/ol-root on \/ type xfs (rw,relatime,seclabel,attr2,inode64,noquota)\n<\/pre>\n<p class=\"brush: sql; gutter: true; first-line: 1\">The postgresql version still is 9.4.1:<\/p>\n<pre class=\"brush: actionscript3; gutter: true; first-line: 1\">postgres=# select version();\n                                                         version                                                          \n--------------------------------------------------------------------------------------------------------------------------\n PostgreSQL 9.4.1dbi services on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 4.8.3 20140911 (Red Hat 4.8.3-9), 64-bit\n(1 row)\n<\/pre>\n<p class=\"brush: sql; gutter: true; first-line: 1\">I&#8217;ll adjust several postgresql parameters to get best performance:<\/p>\n<pre class=\"brush: actionscript3; gutter: true; first-line: 1\">fsync='off'\nsynchronous_commit='off'\nfull_page_writes='off'\nbgwriter_lru_maxpages=0\nwal_level='minimal'\narchive_mode='off'\nwork_mem='64MB'\ncheckpoint_segments=32\nmax_wal_senders=0\nmaintenance_work_mem='64MB'\nshared_buffers='128MB'\n<\/pre>\n<p class=\"brush: sql; gutter: true; first-line: 1\">For a detailed description of the parameters head over to the <a href=\"http:\/\/www.postgresql.org\/docs\/9.4\/static\/runtime-config-wal.html\" target=\"_blank\">official documentation<\/a>. Just to be clear again: Turning fsync and synchronous_commit off is not crash safe. This means if the server crashes during the load for some reason the database will be corrupted. See the reference section at the end of this post for more details. The fastest method to load or unload data in PostgreSQL is the <a href=\"http:\/\/www.postgresql.org\/docs\/9.4\/static\/sql-copy.html\" target=\"_blank\">copy<\/a> command. I&#8217;ll use the same file to load the data as in <a href=\"http:\/\/dbi-services.com\/blog\/variations-on-1m-rows-insert-1-bulk-insert-postgresql\/\" target=\"_blank\">the previous post<\/a>. Some notes about indexes and foreign keys: It is usually faster to create the indexes and foreign keys after the bulk load and to delete or disable them before the load. I&#8217;ll do a test case on this below.<br \/>\nLets prepare the system:<\/p>\n<pre class=\"brush: actionscript3; gutter: true; first-line: 1\">postgres=# alter system set fsync='off';\nALTER SYSTEM\npostgres=# alter system set synchronous_commit='off';\nALTER SYSTEM\npostgres=# alter system set full_page_writes='off';\nALTER SYSTEM\npostgres=# alter system set bgwriter_lru_maxpages=0;\nALTER SYSTEM\npostgres=# alter system set wal_level='minimal';\nALTER SYSTEM\npostgres=# alter system set archive_mode='off';\nALTER SYSTEM\npostgres=# alter system set work_mem='64MB';\nALTER SYSTEM\npostgres=# alter system set checkpoint_segments=32;\nALTER SYSTEM\npostgres=# alter system set max_wal_senders=0;\nALTER SYSTEM\npostgres=# alter system set maintenance_work_mem='64MB';\nALTER SYSTEM\npostgres=# alter system set shared_buffers='128MB';\nALTER SYSTEM\n<\/pre>\n<p class=\"brush: sql; gutter: true; first-line: 1\">As some of the parameters require a restart of the database lets do this:<\/p>\n<p class=\"brush: actionscript3; gutter: true; first-line: 1\">pg_ctl -D \/u02\/pgdata\/PG1\/ restart -m fast<\/p>\n<p class=\"brush: sql; gutter: true; first-line: 1\">Ready for the test cases. As said earlier I&#8217;ll use the same file to load the data (1000001 records):<\/p>\n<pre class=\"brush: actionscript3; gutter: true; first-line: 1\">postgres@oel7:\/home\/postgres\/ [PG1] ls -la demo.txt \n-rw-r--r--. 1 postgres postgres 16690526 May  4 15:53 demo.txt\npostgres@oel7:\/home\/postgres\/ [PG1] cat demo.txt | wc -l\n1000001\npostgres@oel7:\/home\/postgres\/ [PG1] head -2 demo.txt \n0\tMarc\t564\n1\tBill\t345\n<\/pre>\n<h3>First test case: No indexes<\/h3>\n<p class=\"brush: sql; gutter: true; first-line: 1\">The first test case will be without any indexes and looks like this:<\/p>\n<pre class=\"brush: actionscript3; gutter: true; first-line: 1\">\\timing off\ndrop table demo;\ncreate table demo ( id int\n                  , text varchar(15)\n                  , number int\n                  );\n\\timing on\ncopy DEMO from '\/home\/postgres\/demo.txt';\n<\/pre>\n<p class=\"brush: sql; gutter: true; first-line: 1\">How long does it take? I&#8217;ll execute the test several times to get an average:<\/p>\n<pre class=\"brush: actionscript3; gutter: true; first-line: 1\">postgres=# \\i 1mio_copy_no_indexes.sql\nTiming is off.\nDROP TABLE\nCREATE TABLE\nTiming is on.\nCOPY 1000001\nTime: 467.219 ms\npostgres=# \\i 1mio_copy_no_indexes.sql\nTiming is off.\nDROP TABLE\nCREATE TABLE\nTiming is on.\nCOPY 1000001\nTime: 460.730 ms\npostgres=# \\i 1mio_copy_no_indexes.sql\nTiming is off.\nDROP TABLE\nCREATE TABLE\nTiming is on.\nCOPY 1000001\nTime: 680.538 ms\npostgres=# \\i 1mio_copy_no_indexes.sql\nTiming is off.\nDROP TABLE\nCREATE TABLE\nTiming is on.\nCOPY 1000001\nTime: 444.932 ms\npostgres=# select count(*) from demo;\n  count  \n---------\n 1000001\n(1 row)\n\nTime: 108.792 ms\n<\/pre>\n<p class=\"brush: sql; gutter: true; first-line: 1\">Between 0,4 and and 0,7 seconds for 1000001 rows.<\/p>\n<h3>Second test case: Primary key exists prior to the load<\/h3>\n<p class=\"brush: sql; gutter: true; first-line: 1\">This test case creates a primary key (and thus an index) before the load is started:<\/p>\n<pre class=\"brush: actionscript3; gutter: true; first-line: 1\">\\timing off\ndrop table demo;\ncreate table demo ( id int\n                  , text varchar(15)\n                  , number int\n                  , constraint demo_pk primary key ( id )\n                  );\n\\timing on\ncopy DEMO from '\/home\/postgres\/demo.txt';\n<\/pre>\n<p class=\"brush: sql; gutter: true; first-line: 1\">The results ( Again several executions to get an average ):<\/p>\n<pre class=\"brush: actionscript3; gutter: true; first-line: 1\">postgres=# \\i 1mio_copy_pk_before_load.sql\nTiming is off.\nDROP TABLE\nCREATE TABLE\nTiming is on.\nCOPY 1000001\nTime: 5721.706 ms\npostgres=# \\i 1mio_copy_pk_before_load.sql\nTiming is off.\nDROP TABLE\nCREATE TABLE\nTiming is on.\nCOPY 1000001\nTime: 6653.489 ms\npostgres=# \\i 1mio_copy_pk_before_load.sql\nTiming is off.\nDROP TABLE\nCREATE TABLE\nTiming is on.\nCOPY 1000001\nTime: 3979.691 ms\n<\/pre>\n<p class=\"brush: sql; gutter: true; first-line: 1\">Something between 3,9 and 6,6 seconds.<\/p>\n<h3>Third test case: Primary key after the load<\/h3>\n<p class=\"brush: sql; gutter: true; first-line: 1\">In this case the primary key will be added after the load:<\/p>\n<pre class=\"brush: bash; gutter: true; first-line: 1\">\\timing off drop table demo;\ncreate table demo ( id int , text varchar(15) , number int );\n\\timing on copy DEMO from '\/home\/postgres\/demo.txt';\nalter table demo add constraint demo_pk primary key (id);<\/pre>\n<p class=\"brush: sql; gutter: true; first-line: 1\">\n<p class=\"brush: sql; gutter: true; first-line: 1\">The results ( Again several executions to get an average ):<\/p>\n<pre class=\"brush: actionscript3; gutter: true; first-line: 1\">postgres=# \\i 1mio_copy_pk_after_load.sql\nTiming is off.\nDROP TABLE\nCREATE TABLE\nTiming is on.\nCOPY 1000001\nTime: 519.758 ms\nALTER TABLE\nTime: 555.851 ms\npostgres=# \\i 1mio_copy_pk_after_load.sql\nTiming is off.\nDROP TABLE\nCREATE TABLE\nTiming is on.\nCOPY 1000001\nTime: 626.097 ms\nALTER TABLE\nTime: 654.467 ms\npostgres=# \\i 1mio_copy_pk_after_load.sql\nTiming is off.\nDROP TABLE\nCREATE TABLE\nTiming is on.\nCOPY 1000001\nTime: 756.139 ms\nALTER TABLE\nTime: 694.572 ms\n<\/pre>\n<p class=\"brush: sql; gutter: true; first-line: 1\">Something between 1,2 and 1,5 seconds.<br \/>\n<strong><br \/>\nConclusion<\/strong>:<br \/>\nIf you need indexes (and who does not) it is definitely faster do drop or disable from before the load and create them afterwards.<\/p>\n<p>References:<br \/>\n<a href=\"http:\/\/www.postgresql.org\/docs\/9.4\/static\/populate.html\" target=\"_blank\">Populating a Database<\/a><br \/>\n<a href=\"http:\/\/rhaas.blogspot.ae\/2010\/06\/postgresql-as-in-memory-only-database_24.html\" target=\"_blank\">PostgreSQL as an In-Memory Only Database<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>There have been several posts on how to load 1m rows into a database in the last days: Variations on 1M rows insert (1): bulk insert Variations on 1M rows insert(2): commit write Variations on 1M rows insert (1): bulk insert &#8211; PostgreSQL Variations on 1M rows insert(2): commit write &#8211; PostgreSQL Variations on 1M [&hellip;]<\/p>\n","protected":false},"author":29,"featured_media":4744,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[368],"tags":[77],"type_dbi":[],"class_list":["post-4743","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-development-performance","tag-postgresql"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v27.2 (Yoast SEO v27.2) - https:\/\/yoast.com\/product\/yoast-seo-premium-wordpress\/ -->\n<title>the fastest way to load 1m rows in postgresql - dbi Blog<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.dbi-services.com\/blog\/the-fastest-way-to-load-1m-rows-in-postgresql\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"the fastest way to load 1m rows in postgresql\" \/>\n<meta property=\"og:description\" content=\"There have been several posts on how to load 1m rows into a database in the last days: Variations on 1M rows insert (1): bulk insert Variations on 1M rows insert(2): commit write Variations on 1M rows insert (1): bulk insert &#8211; PostgreSQL Variations on 1M rows insert(2): commit write &#8211; PostgreSQL Variations on 1M [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.dbi-services.com\/blog\/the-fastest-way-to-load-1m-rows-in-postgresql\/\" \/>\n<meta property=\"og:site_name\" content=\"dbi Blog\" \/>\n<meta property=\"article:published_time\" content=\"2015-05-18T09:12:43+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2022\/04\/postreslogo.png\" \/>\n\t<meta property=\"og:image:width\" content=\"206\" \/>\n\t<meta property=\"og:image:height\" content=\"80\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Daniel Westermann\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@westermanndanie\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Daniel Westermann\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/the-fastest-way-to-load-1m-rows-in-postgresql\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/the-fastest-way-to-load-1m-rows-in-postgresql\/\"},\"author\":{\"name\":\"Daniel Westermann\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/8d08e9bd996a89bd75c0286cbabf3c66\"},\"headline\":\"the fastest way to load 1m rows in postgresql\",\"datePublished\":\"2015-05-18T09:12:43+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/the-fastest-way-to-load-1m-rows-in-postgresql\/\"},\"wordCount\":452,\"commentCount\":0,\"image\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/the-fastest-way-to-load-1m-rows-in-postgresql\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2022\/04\/postreslogo.png\",\"keywords\":[\"PostgreSQL\"],\"articleSection\":[\"Development &amp; Performance\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/www.dbi-services.com\/blog\/the-fastest-way-to-load-1m-rows-in-postgresql\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/the-fastest-way-to-load-1m-rows-in-postgresql\/\",\"url\":\"https:\/\/www.dbi-services.com\/blog\/the-fastest-way-to-load-1m-rows-in-postgresql\/\",\"name\":\"the fastest way to load 1m rows in postgresql - dbi Blog\",\"isPartOf\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/the-fastest-way-to-load-1m-rows-in-postgresql\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/the-fastest-way-to-load-1m-rows-in-postgresql\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2022\/04\/postreslogo.png\",\"datePublished\":\"2015-05-18T09:12:43+00:00\",\"author\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/8d08e9bd996a89bd75c0286cbabf3c66\"},\"breadcrumb\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/the-fastest-way-to-load-1m-rows-in-postgresql\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.dbi-services.com\/blog\/the-fastest-way-to-load-1m-rows-in-postgresql\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/the-fastest-way-to-load-1m-rows-in-postgresql\/#primaryimage\",\"url\":\"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2022\/04\/postreslogo.png\",\"contentUrl\":\"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2022\/04\/postreslogo.png\",\"width\":206,\"height\":80},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/the-fastest-way-to-load-1m-rows-in-postgresql\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Accueil\",\"item\":\"https:\/\/www.dbi-services.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"the fastest way to load 1m rows in postgresql\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/#website\",\"url\":\"https:\/\/www.dbi-services.com\/blog\/\",\"name\":\"dbi Blog\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.dbi-services.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/8d08e9bd996a89bd75c0286cbabf3c66\",\"name\":\"Daniel Westermann\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/secure.gravatar.com\/avatar\/31350ceeecb1dd8986339a29bf040d4cd3cd087d410deccd8f55234466d6c317?s=96&d=mm&r=g\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/31350ceeecb1dd8986339a29bf040d4cd3cd087d410deccd8f55234466d6c317?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/31350ceeecb1dd8986339a29bf040d4cd3cd087d410deccd8f55234466d6c317?s=96&d=mm&r=g\",\"caption\":\"Daniel Westermann\"},\"description\":\"Daniel Westermann is Principal Consultant and Technology Leader Open Infrastructure at dbi services. He has more than 15 years of experience in management, engineering and optimization of databases and infrastructures, especially on Oracle and PostgreSQL. Since the beginning of his career, he has specialized in Oracle Technologies and is Oracle Certified Professional 12c and Oracle Certified Expert RAC\/GridInfra. Over time, Daniel has become increasingly interested in open source technologies, becoming \u201cTechnology Leader Open Infrastructure\u201d and PostgreSQL expert. \u00a0Based on community or EnterpriseDB tools, he develops and installs complex high available solutions with PostgreSQL. He is also a certified PostgreSQL Plus 9.0 Professional and a Postgres Advanced Server 9.4 Professional. He is a regular speaker at PostgreSQL conferences in Switzerland and Europe. Today Daniel is also supporting our customers on AWS services such as AWS RDS, database migrations into the cloud, EC2 and automated infrastructure management with AWS SSM (System Manager). He is a certified AWS Solutions Architect Professional. Prior to dbi services, Daniel was Management System Engineer at LC SYSTEMS-Engineering AG in Basel. Before that, he worked as Oracle Developper &amp;\u00a0Project Manager at Delta Energy Solutions AG in Basel (today Powel AG). Daniel holds a diploma in Business Informatics (DHBW, Germany). His branch-related experience mainly covers the pharma industry, the financial sector, energy, lottery and telecommunications.\",\"sameAs\":[\"https:\/\/x.com\/westermanndanie\"],\"url\":\"https:\/\/www.dbi-services.com\/blog\/author\/daniel-westermann\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"the fastest way to load 1m rows in postgresql - dbi Blog","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.dbi-services.com\/blog\/the-fastest-way-to-load-1m-rows-in-postgresql\/","og_locale":"en_US","og_type":"article","og_title":"the fastest way to load 1m rows in postgresql","og_description":"There have been several posts on how to load 1m rows into a database in the last days: Variations on 1M rows insert (1): bulk insert Variations on 1M rows insert(2): commit write Variations on 1M rows insert (1): bulk insert &#8211; PostgreSQL Variations on 1M rows insert(2): commit write &#8211; PostgreSQL Variations on 1M [&hellip;]","og_url":"https:\/\/www.dbi-services.com\/blog\/the-fastest-way-to-load-1m-rows-in-postgresql\/","og_site_name":"dbi Blog","article_published_time":"2015-05-18T09:12:43+00:00","og_image":[{"width":206,"height":80,"url":"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2022\/04\/postreslogo.png","type":"image\/png"}],"author":"Daniel Westermann","twitter_card":"summary_large_image","twitter_creator":"@westermanndanie","twitter_misc":{"Written by":"Daniel Westermann","Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.dbi-services.com\/blog\/the-fastest-way-to-load-1m-rows-in-postgresql\/#article","isPartOf":{"@id":"https:\/\/www.dbi-services.com\/blog\/the-fastest-way-to-load-1m-rows-in-postgresql\/"},"author":{"name":"Daniel Westermann","@id":"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/8d08e9bd996a89bd75c0286cbabf3c66"},"headline":"the fastest way to load 1m rows in postgresql","datePublished":"2015-05-18T09:12:43+00:00","mainEntityOfPage":{"@id":"https:\/\/www.dbi-services.com\/blog\/the-fastest-way-to-load-1m-rows-in-postgresql\/"},"wordCount":452,"commentCount":0,"image":{"@id":"https:\/\/www.dbi-services.com\/blog\/the-fastest-way-to-load-1m-rows-in-postgresql\/#primaryimage"},"thumbnailUrl":"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2022\/04\/postreslogo.png","keywords":["PostgreSQL"],"articleSection":["Development &amp; Performance"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.dbi-services.com\/blog\/the-fastest-way-to-load-1m-rows-in-postgresql\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.dbi-services.com\/blog\/the-fastest-way-to-load-1m-rows-in-postgresql\/","url":"https:\/\/www.dbi-services.com\/blog\/the-fastest-way-to-load-1m-rows-in-postgresql\/","name":"the fastest way to load 1m rows in postgresql - dbi Blog","isPartOf":{"@id":"https:\/\/www.dbi-services.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.dbi-services.com\/blog\/the-fastest-way-to-load-1m-rows-in-postgresql\/#primaryimage"},"image":{"@id":"https:\/\/www.dbi-services.com\/blog\/the-fastest-way-to-load-1m-rows-in-postgresql\/#primaryimage"},"thumbnailUrl":"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2022\/04\/postreslogo.png","datePublished":"2015-05-18T09:12:43+00:00","author":{"@id":"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/8d08e9bd996a89bd75c0286cbabf3c66"},"breadcrumb":{"@id":"https:\/\/www.dbi-services.com\/blog\/the-fastest-way-to-load-1m-rows-in-postgresql\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.dbi-services.com\/blog\/the-fastest-way-to-load-1m-rows-in-postgresql\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.dbi-services.com\/blog\/the-fastest-way-to-load-1m-rows-in-postgresql\/#primaryimage","url":"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2022\/04\/postreslogo.png","contentUrl":"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2022\/04\/postreslogo.png","width":206,"height":80},{"@type":"BreadcrumbList","@id":"https:\/\/www.dbi-services.com\/blog\/the-fastest-way-to-load-1m-rows-in-postgresql\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Accueil","item":"https:\/\/www.dbi-services.com\/blog\/"},{"@type":"ListItem","position":2,"name":"the fastest way to load 1m rows in postgresql"}]},{"@type":"WebSite","@id":"https:\/\/www.dbi-services.com\/blog\/#website","url":"https:\/\/www.dbi-services.com\/blog\/","name":"dbi Blog","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.dbi-services.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/8d08e9bd996a89bd75c0286cbabf3c66","name":"Daniel Westermann","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/31350ceeecb1dd8986339a29bf040d4cd3cd087d410deccd8f55234466d6c317?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/31350ceeecb1dd8986339a29bf040d4cd3cd087d410deccd8f55234466d6c317?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/31350ceeecb1dd8986339a29bf040d4cd3cd087d410deccd8f55234466d6c317?s=96&d=mm&r=g","caption":"Daniel Westermann"},"description":"Daniel Westermann is Principal Consultant and Technology Leader Open Infrastructure at dbi services. He has more than 15 years of experience in management, engineering and optimization of databases and infrastructures, especially on Oracle and PostgreSQL. Since the beginning of his career, he has specialized in Oracle Technologies and is Oracle Certified Professional 12c and Oracle Certified Expert RAC\/GridInfra. Over time, Daniel has become increasingly interested in open source technologies, becoming \u201cTechnology Leader Open Infrastructure\u201d and PostgreSQL expert. \u00a0Based on community or EnterpriseDB tools, he develops and installs complex high available solutions with PostgreSQL. He is also a certified PostgreSQL Plus 9.0 Professional and a Postgres Advanced Server 9.4 Professional. He is a regular speaker at PostgreSQL conferences in Switzerland and Europe. Today Daniel is also supporting our customers on AWS services such as AWS RDS, database migrations into the cloud, EC2 and automated infrastructure management with AWS SSM (System Manager). He is a certified AWS Solutions Architect Professional. Prior to dbi services, Daniel was Management System Engineer at LC SYSTEMS-Engineering AG in Basel. Before that, he worked as Oracle Developper &amp;\u00a0Project Manager at Delta Energy Solutions AG in Basel (today Powel AG). Daniel holds a diploma in Business Informatics (DHBW, Germany). His branch-related experience mainly covers the pharma industry, the financial sector, energy, lottery and telecommunications.","sameAs":["https:\/\/x.com\/westermanndanie"],"url":"https:\/\/www.dbi-services.com\/blog\/author\/daniel-westermann\/"}]}},"_links":{"self":[{"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/posts\/4743","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/users\/29"}],"replies":[{"embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/comments?post=4743"}],"version-history":[{"count":0,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/posts\/4743\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/media\/4744"}],"wp:attachment":[{"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/media?parent=4743"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/categories?post=4743"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/tags?post=4743"},{"taxonomy":"type","embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/type_dbi?post=4743"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}