{"id":11042,"date":"2018-03-23T09:53:06","date_gmt":"2018-03-23T08:53:06","guid":{"rendered":"https:\/\/www.dbi-services.com\/blog\/what-is-the-maximum-in-list-size-in-postgresql\/"},"modified":"2018-03-23T09:53:06","modified_gmt":"2018-03-23T08:53:06","slug":"what-is-the-maximum-in-list-size-in-postgresql","status":"publish","type":"post","link":"https:\/\/www.dbi-services.com\/blog\/what-is-the-maximum-in-list-size-in-postgresql\/","title":{"rendered":"What is the maximum in list size in PostgreSQL?"},"content":{"rendered":"<p>Yesterday, while being at a customer, an interesting question popped up: What is the maximum of in list values in PostgreSQL? I couldn&#8217;t answer although I never read somewhere that there is a limit. The following is for fun only and I am not saying that creating huge in lists is a good idea. Lets go.<\/p>\n<p><!--more--><\/p>\n<p>The version I tested is PostgreSQL 10:<\/p>\n<pre class=\"brush: sql; gutter: true; first-line: 1\">\npostgres=# select version(), now();\n                                                  version                                                   |              now              \n------------------------------------------------------------------------------------------------------------+-------------------------------\n PostgreSQL 10.0 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-16), 64-bit | 2018-03-21 18:29:50.269369+01\n(1 row)\n\npostgres=# create table t1 ( a varchar(10));\nCREATE TABLE\n<\/pre>\n<p>We will use a very simple bash script to build the in list and execute the result in PostgreSQL:<\/p>\n<pre class=\"brush: bash; gutter: true; first-line: 1\">\npostgres@pgbox:\/home\/postgres\/ [PG10] cat t.sh \n#!\/bin\/bash\ncount=$1\n\nstatement='select * from t1 where a in ('\n\nfor (( i=1; i&lt;=$count; i++ ))\ndo  \n    if [ &quot;${i}&quot; -lt &quot;${count}&quot; ]; then\n        statement=&quot;${statement} &#039;${i}&#039;,&quot;\n    elif [ &quot;${i}&quot; == &quot;${count}&quot; ]; then\n        statement=&quot;${statement} &#039;${i}&#039;);&quot;\n    fi\ndone\n\npsql -c &quot;${statement}&quot; postgres\n<\/pre>\n<p>Lets start with 100:<\/p>\n<pre class=\"brush: bash; gutter: true; first-line: 1\">\npostgres@pgbox:\/home\/postgres\/ [PG10] .\/t.sh 100\n a \n---\n(0 rows)\n\nTime: 0.983 ms\n<\/pre>\n<p>1000:<\/p>\n<pre class=\"brush: bash; gutter: true; first-line: 1\">\npostgres@pgbox:\/home\/postgres\/ [PG10] .\/t.sh 1000\n a \n---\n(0 rows)\n\nTime: 1.525 ms\n<\/pre>\n<p>10000:<\/p>\n<pre class=\"brush: bash; gutter: true; first-line: 1\">\npostgres@pgbox:\/home\/postgres\/ [PG10] .\/t.sh 10000\n a \n---\n(0 rows)\n\nTime: 11.549 ms\n<\/pre>\n<p>&#8230; takes a much longer (because of the bash script which fully occupies my virtual core):<\/p>\n<pre class=\"brush: bash; gutter: true; first-line: 1\">\n.\/t.sh: line 15: \/u01\/app\/postgres\/product\/10\/db_0\/bin\/psql: Argument list too long\n<\/pre>\n<p>So there at least is a limit with psql. Lets try by creating a sql script and execute that instead:<\/p>\n<pre class=\"brush: sql; gutter: true; first-line: 1\">\npostgres@pgbox:\/home\/postgres\/ [PG10] cat t.sh \n#!\/bin\/bash\ncount=$1\n\necho 'select * from t1 where a in (' &gt; ttt.sql\n\nfor (( i=1; i&lt;=$count; i++ ))\ndo  \n    if [ &quot;${i}&quot; -lt &quot;${count}&quot; ]; then\n        echo  \"&#039;${i}&#039;,\" &gt;&gt; ttt.sql\n    elif [ &quot;${i}&quot; == &quot;${count}&quot; ]; then\n        echo \"&#039;${i}&#039;);&quot; &gt;&gt; ttt.sql\n    fi\ndone\n\npsql -f ttt.sql postgres\n<\/pre>\n<p>This way of stringing together the statement is much more efficient than building the list by concatenating everything into one variable. Does it still work?<\/p>\n<pre class=\"brush: sql; gutter: true; first-line: 1\">\npostgres@pgbox:\/home\/postgres\/ [PG10] .\/t.sh 100000\n a \n---\n(0 rows)\n\nTime: 155.986 ms\n<\/pre>\n<p>Not a problem, one more:<\/p>\n<pre class=\"brush: sql; gutter: true; first-line: 1\">\npostgres@pgbox:\/home\/postgres\/ [PG10] .\/t.sh 1000000\n a \n---\n(0 rows)\n\nTime: 14211.766 ms (00:14.212)\n<\/pre>\n<p>Still works. So now we could say: lets stop, who in the world will pass one million values into an in list. On the other hand, lets have fun and double:<\/p>\n<pre class=\"brush: sql; gutter: true; first-line: 1\">\npostgres@pgbox:\/home\/postgres\/ [PG10] .\/t.sh 2000000\n a \n---\n(0 rows)\n\nTime: 3993.091 ms (00:03.993)\n<\/pre>\n<p>One more:<\/p>\n<pre class=\"brush: sql; gutter: true; first-line: 1\">\npostgres@pgbox:\/home\/postgres\/ [PG10] .\/t.sh 3000000\npsql:ttt.sql:3000001: ERROR:  out of memory\nDETAIL:  Failed on request of size 524288.\nTime: 3026.925 ms (00:03.027)\n<\/pre>\n<p>Ok, now I am hitting some limits, but probably not those of PostgreSQL. I&#8217;ll test further when I have more time for that \ud83d\ude42<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Yesterday, while being at a customer, an interesting question popped up: What is the maximum of in list values in PostgreSQL? I couldn&#8217;t answer although I never read somewhere that there is a limit. The following is for fun only and I am not saying that creating huge in lists is a good idea. Lets [&hellip;]<\/p>\n","protected":false},"author":29,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[229],"tags":[77],"type_dbi":[],"class_list":["post-11042","post","type-post","status-publish","format-standard","hentry","category-database-administration-monitoring","tag-postgresql"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v27.2 (Yoast SEO v27.2) - https:\/\/yoast.com\/product\/yoast-seo-premium-wordpress\/ -->\n<title>What is the maximum in list size in PostgreSQL? - dbi Blog<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.dbi-services.com\/blog\/what-is-the-maximum-in-list-size-in-postgresql\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"What is the maximum in list size in PostgreSQL?\" \/>\n<meta property=\"og:description\" content=\"Yesterday, while being at a customer, an interesting question popped up: What is the maximum of in list values in PostgreSQL? I couldn&#8217;t answer although I never read somewhere that there is a limit. The following is for fun only and I am not saying that creating huge in lists is a good idea. Lets [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.dbi-services.com\/blog\/what-is-the-maximum-in-list-size-in-postgresql\/\" \/>\n<meta property=\"og:site_name\" content=\"dbi Blog\" \/>\n<meta property=\"article:published_time\" content=\"2018-03-23T08:53:06+00:00\" \/>\n<meta name=\"author\" content=\"Daniel Westermann\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@westermanndanie\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Daniel Westermann\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/what-is-the-maximum-in-list-size-in-postgresql\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/what-is-the-maximum-in-list-size-in-postgresql\/\"},\"author\":{\"name\":\"Daniel Westermann\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/8d08e9bd996a89bd75c0286cbabf3c66\"},\"headline\":\"What is the maximum in list size in PostgreSQL?\",\"datePublished\":\"2018-03-23T08:53:06+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/what-is-the-maximum-in-list-size-in-postgresql\/\"},\"wordCount\":216,\"commentCount\":0,\"keywords\":[\"PostgreSQL\"],\"articleSection\":[\"Database Administration &amp; Monitoring\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/www.dbi-services.com\/blog\/what-is-the-maximum-in-list-size-in-postgresql\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/what-is-the-maximum-in-list-size-in-postgresql\/\",\"url\":\"https:\/\/www.dbi-services.com\/blog\/what-is-the-maximum-in-list-size-in-postgresql\/\",\"name\":\"What is the maximum in list size in PostgreSQL? - dbi Blog\",\"isPartOf\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/#website\"},\"datePublished\":\"2018-03-23T08:53:06+00:00\",\"author\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/8d08e9bd996a89bd75c0286cbabf3c66\"},\"breadcrumb\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/what-is-the-maximum-in-list-size-in-postgresql\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.dbi-services.com\/blog\/what-is-the-maximum-in-list-size-in-postgresql\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/what-is-the-maximum-in-list-size-in-postgresql\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Accueil\",\"item\":\"https:\/\/www.dbi-services.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"What is the maximum in list size in PostgreSQL?\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/#website\",\"url\":\"https:\/\/www.dbi-services.com\/blog\/\",\"name\":\"dbi Blog\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.dbi-services.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/8d08e9bd996a89bd75c0286cbabf3c66\",\"name\":\"Daniel Westermann\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/secure.gravatar.com\/avatar\/31350ceeecb1dd8986339a29bf040d4cd3cd087d410deccd8f55234466d6c317?s=96&d=mm&r=g\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/31350ceeecb1dd8986339a29bf040d4cd3cd087d410deccd8f55234466d6c317?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/31350ceeecb1dd8986339a29bf040d4cd3cd087d410deccd8f55234466d6c317?s=96&d=mm&r=g\",\"caption\":\"Daniel Westermann\"},\"description\":\"Daniel Westermann is Principal Consultant and Technology Leader Open Infrastructure at dbi services. He has more than 15 years of experience in management, engineering and optimization of databases and infrastructures, especially on Oracle and PostgreSQL. Since the beginning of his career, he has specialized in Oracle Technologies and is Oracle Certified Professional 12c and Oracle Certified Expert RAC\/GridInfra. Over time, Daniel has become increasingly interested in open source technologies, becoming \u201cTechnology Leader Open Infrastructure\u201d and PostgreSQL expert. \u00a0Based on community or EnterpriseDB tools, he develops and installs complex high available solutions with PostgreSQL. He is also a certified PostgreSQL Plus 9.0 Professional and a Postgres Advanced Server 9.4 Professional. He is a regular speaker at PostgreSQL conferences in Switzerland and Europe. Today Daniel is also supporting our customers on AWS services such as AWS RDS, database migrations into the cloud, EC2 and automated infrastructure management with AWS SSM (System Manager). He is a certified AWS Solutions Architect Professional. Prior to dbi services, Daniel was Management System Engineer at LC SYSTEMS-Engineering AG in Basel. Before that, he worked as Oracle Developper &amp;\u00a0Project Manager at Delta Energy Solutions AG in Basel (today Powel AG). Daniel holds a diploma in Business Informatics (DHBW, Germany). His branch-related experience mainly covers the pharma industry, the financial sector, energy, lottery and telecommunications.\",\"sameAs\":[\"https:\/\/x.com\/westermanndanie\"],\"url\":\"https:\/\/www.dbi-services.com\/blog\/author\/daniel-westermann\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"What is the maximum in list size in PostgreSQL? - dbi Blog","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.dbi-services.com\/blog\/what-is-the-maximum-in-list-size-in-postgresql\/","og_locale":"en_US","og_type":"article","og_title":"What is the maximum in list size in PostgreSQL?","og_description":"Yesterday, while being at a customer, an interesting question popped up: What is the maximum of in list values in PostgreSQL? I couldn&#8217;t answer although I never read somewhere that there is a limit. The following is for fun only and I am not saying that creating huge in lists is a good idea. Lets [&hellip;]","og_url":"https:\/\/www.dbi-services.com\/blog\/what-is-the-maximum-in-list-size-in-postgresql\/","og_site_name":"dbi Blog","article_published_time":"2018-03-23T08:53:06+00:00","author":"Daniel Westermann","twitter_card":"summary_large_image","twitter_creator":"@westermanndanie","twitter_misc":{"Written by":"Daniel Westermann","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.dbi-services.com\/blog\/what-is-the-maximum-in-list-size-in-postgresql\/#article","isPartOf":{"@id":"https:\/\/www.dbi-services.com\/blog\/what-is-the-maximum-in-list-size-in-postgresql\/"},"author":{"name":"Daniel Westermann","@id":"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/8d08e9bd996a89bd75c0286cbabf3c66"},"headline":"What is the maximum in list size in PostgreSQL?","datePublished":"2018-03-23T08:53:06+00:00","mainEntityOfPage":{"@id":"https:\/\/www.dbi-services.com\/blog\/what-is-the-maximum-in-list-size-in-postgresql\/"},"wordCount":216,"commentCount":0,"keywords":["PostgreSQL"],"articleSection":["Database Administration &amp; Monitoring"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.dbi-services.com\/blog\/what-is-the-maximum-in-list-size-in-postgresql\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.dbi-services.com\/blog\/what-is-the-maximum-in-list-size-in-postgresql\/","url":"https:\/\/www.dbi-services.com\/blog\/what-is-the-maximum-in-list-size-in-postgresql\/","name":"What is the maximum in list size in PostgreSQL? - dbi Blog","isPartOf":{"@id":"https:\/\/www.dbi-services.com\/blog\/#website"},"datePublished":"2018-03-23T08:53:06+00:00","author":{"@id":"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/8d08e9bd996a89bd75c0286cbabf3c66"},"breadcrumb":{"@id":"https:\/\/www.dbi-services.com\/blog\/what-is-the-maximum-in-list-size-in-postgresql\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.dbi-services.com\/blog\/what-is-the-maximum-in-list-size-in-postgresql\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.dbi-services.com\/blog\/what-is-the-maximum-in-list-size-in-postgresql\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Accueil","item":"https:\/\/www.dbi-services.com\/blog\/"},{"@type":"ListItem","position":2,"name":"What is the maximum in list size in PostgreSQL?"}]},{"@type":"WebSite","@id":"https:\/\/www.dbi-services.com\/blog\/#website","url":"https:\/\/www.dbi-services.com\/blog\/","name":"dbi Blog","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.dbi-services.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/8d08e9bd996a89bd75c0286cbabf3c66","name":"Daniel Westermann","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/31350ceeecb1dd8986339a29bf040d4cd3cd087d410deccd8f55234466d6c317?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/31350ceeecb1dd8986339a29bf040d4cd3cd087d410deccd8f55234466d6c317?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/31350ceeecb1dd8986339a29bf040d4cd3cd087d410deccd8f55234466d6c317?s=96&d=mm&r=g","caption":"Daniel Westermann"},"description":"Daniel Westermann is Principal Consultant and Technology Leader Open Infrastructure at dbi services. He has more than 15 years of experience in management, engineering and optimization of databases and infrastructures, especially on Oracle and PostgreSQL. Since the beginning of his career, he has specialized in Oracle Technologies and is Oracle Certified Professional 12c and Oracle Certified Expert RAC\/GridInfra. Over time, Daniel has become increasingly interested in open source technologies, becoming \u201cTechnology Leader Open Infrastructure\u201d and PostgreSQL expert. \u00a0Based on community or EnterpriseDB tools, he develops and installs complex high available solutions with PostgreSQL. He is also a certified PostgreSQL Plus 9.0 Professional and a Postgres Advanced Server 9.4 Professional. He is a regular speaker at PostgreSQL conferences in Switzerland and Europe. Today Daniel is also supporting our customers on AWS services such as AWS RDS, database migrations into the cloud, EC2 and automated infrastructure management with AWS SSM (System Manager). He is a certified AWS Solutions Architect Professional. Prior to dbi services, Daniel was Management System Engineer at LC SYSTEMS-Engineering AG in Basel. Before that, he worked as Oracle Developper &amp;\u00a0Project Manager at Delta Energy Solutions AG in Basel (today Powel AG). Daniel holds a diploma in Business Informatics (DHBW, Germany). His branch-related experience mainly covers the pharma industry, the financial sector, energy, lottery and telecommunications.","sameAs":["https:\/\/x.com\/westermanndanie"],"url":"https:\/\/www.dbi-services.com\/blog\/author\/daniel-westermann\/"}]}},"_links":{"self":[{"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/posts\/11042","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/users\/29"}],"replies":[{"embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/comments?post=11042"}],"version-history":[{"count":0,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/posts\/11042\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/media?parent=11042"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/categories?post=11042"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/tags?post=11042"},{"taxonomy":"type","embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/type_dbi?post=11042"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}