{"id":16674,"date":"2021-09-09T15:52:42","date_gmt":"2021-09-09T13:52:42","guid":{"rendered":"https:\/\/www.dbi-services.com\/blog\/how-much-shared-memory-is-postgresql-using-2-details\/"},"modified":"2021-09-09T15:52:42","modified_gmt":"2021-09-09T13:52:42","slug":"how-much-shared-memory-is-postgresql-using-2-details","status":"publish","type":"post","link":"https:\/\/www.dbi-services.com\/blog\/how-much-shared-memory-is-postgresql-using-2-details\/","title":{"rendered":"How much shared memory is PostgreSQL using? (2) – details"},"content":{"rendered":"

In the last post in this two post series<\/a> we’ve briefly looked at configuring huge pages on Linux for PostgreSQL. It became clear, that calculating the number of huge pages purely based on the setting of shared_buffers<\/a> will not work. PostgreSQL requires more shared memory, but what exactly is it? Well, this is the goal of this post and I’ll introduce a new feature of PostgreSQL 15 which helps in identifying those bits without digging into the operating systems details about memory management.<\/p>\n

<\/p>\n

Let’s quickly review the current situation. We have shared_buffers set to 128MB:<\/p>\n

\npostgres=# show shared_buffers;\n shared_buffers \n----------------\n 128MB\n(1 row)\n<\/pre>\n
Using the official way of calculating the huge pages we see that PostgreSQL currently requires this amount of shared memory (huge pages in PostgreSQL need to be disabled for this):<\/p>\n
\npostgres@debian11pg:\/home\/postgres\/ [pgdev] head -1 $PGDATA\/postmaster.pid\n826\npostgres@debian11pg:\/home\/postgres\/ [pgdev] pmap 826 | awk '\/rw-s\/ && \/zero\/ {print $2}'\n145360K\n<\/pre>\n
As we already have seen in the last post this is more than the 128MB configured for shared_buffers. What else could change the requirement for shared memory. What about wal_buffers<\/a>? In the default configuration (-1) this consumes 128\/32 MB:<\/p>\n
\npostgres=# show wal_buffers;\n wal_buffers \n-------------\n 4MB\n(1 row)\n<\/pre>\n
What happens if we change it?<\/p>\n
\npostgres@debian11pg:\/home\/postgres\/ [pgdev] psql -c \"alter system set wal_buffers='16MB'\"\nALTER SYSTEM\npostgres@debian11pg:\/home\/postgres\/ [pgdev] pg_ctl restart\npostgres@debian11pg:\/home\/postgres\/ [pgdev] head -1 $PGDATA\/postmaster.pid\n858\npostgres@debian11pg:\/home\/postgres\/ [pgdev] pmap 858 | awk '\/rw-s\/ && \/zero\/ {print $2}'\n157664K\n<\/pre>\n
This increases shared memory usage from 145360K to 157664K. So, yes, wal_buffers definitely impact the usage of shared memory. What about extensions<\/a>, could they have an impact as well? <\/p>\n
\npostgres@debian11pg:\/home\/postgres\/ [pgdev] psql -c \"alter system set shared_preload_libraries=pg_stat_statements\"\nALTER SYSTEM\npostgres@debian11pg:\/home\/postgres\/ [pgdev] pg_ctl restart\npostgres@debian11pg:\/home\/postgres\/ [pgdev] head -1 $PGDATA\/postmaster.pid\n882\npostgres@debian11pg:\/home\/postgres\/ [pgdev] pmap 882 | awk '\/rw-s\/ && \/zero\/ {print $2}'\n159176K\n<\/pre>\n
We’ve again increased shared memory usage from 157664K to 159176K. Extensions also might consume shared memory. What about max_connections<\/a>?<\/p>\n
\npostgres@debian11pg:\/home\/postgres\/ [pgdev] psql -c \"show max_connections\"\n max_connections \n-----------------\n 100\n(1 row)\n\npostgres@debian11pg:\/home\/postgres\/ [pgdev] psql -c \"alter system set max_connections=1000\"\nALTER SYSTEM\npostgres@debian11pg:\/home\/postgres\/ [pgdev] pg_ctl restart\npostgres@debian11pg:\/home\/postgres\/ [pgdev] head -1 $PGDATA\/postmaster.pid\n900\npostgres@debian11pg:\/home\/postgres\/ [pgdev] pmap 900 | awk '\/rw-s\/ && \/zero\/ {print $2}'\n201288K\n<\/pre>\n
Next increase, this time from 159176K to 201288K. Without going any further: There was no way to see the real shared memory consumption from inside PostgreSQL and this is why  this discussion<\/a> started. The result is this:<\/p>\n
\npostgres=# show shared_buffers;\n shared_buffers \n----------------\n 128MB\n(1 row)\n\npostgres=# show shared_memory_size;\n shared_memory_size \n--------------------\n 197MB\n(1 row)\n<\/pre>\n
… which is pretty close to 201288K. Really nice.<\/p>\n","protected":false},"excerpt":{"rendered":"
In the last post in this two post series we’ve briefly looked at configuring huge pages on Linux for PostgreSQL. It became clear, that calculating the number of huge pages purely based on the setting of shared_buffers will not work. PostgreSQL requires more shared memory, but what exactly is it? Well, this is the goal […]<\/p>\n","protected":false},"author":29,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[229],"tags":[77],"type_dbi":[],"class_list":["post-16674","post","type-post","status-publish","format-standard","hentry","category-database-administration-monitoring","tag-postgresql"],"acf":[],"yoast_head":"\nHow much shared memory is PostgreSQL using? (2) - details - dbi Blog<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.dbi-services.com\/blog\/how-much-shared-memory-is-postgresql-using-2-details\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"How much shared memory is PostgreSQL using? (2) - details\" \/>\n<meta property=\"og:description\" content=\"In the last post in this two post series we’ve briefly looked at configuring huge pages on Linux for PostgreSQL. It became clear, that calculating the number of huge pages purely based on the setting of shared_buffers will not work. PostgreSQL requires more shared memory, but what exactly is it? Well, this is the goal […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.dbi-services.com\/blog\/how-much-shared-memory-is-postgresql-using-2-details\/\" \/>\n<meta property=\"og:site_name\" content=\"dbi Blog\" \/>\n<meta property=\"article:published_time\" content=\"2021-09-09T13:52:42+00:00\" \/>\n<meta name=\"author\" content=\"Daniel Westermann\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@westermanndanie\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Daniel Westermann\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/how-much-shared-memory-is-postgresql-using-2-details\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/how-much-shared-memory-is-postgresql-using-2-details\/\"},\"author\":{\"name\":\"Daniel Westermann\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/8d08e9bd996a89bd75c0286cbabf3c66\"},\"headline\":\"How much shared memory is PostgreSQL using? (2) – details\",\"datePublished\":\"2021-09-09T13:52:42+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/how-much-shared-memory-is-postgresql-using-2-details\/\"},\"wordCount\":280,\"commentCount\":0,\"keywords\":[\"PostgreSQL\"],\"articleSection\":[\"Database Administration & Monitoring\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/www.dbi-services.com\/blog\/how-much-shared-memory-is-postgresql-using-2-details\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/how-much-shared-memory-is-postgresql-using-2-details\/\",\"url\":\"https:\/\/www.dbi-services.com\/blog\/how-much-shared-memory-is-postgresql-using-2-details\/\",\"name\":\"How much shared memory is PostgreSQL using? (2) - details - dbi Blog\",\"isPartOf\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/#website\"},\"datePublished\":\"2021-09-09T13:52:42+00:00\",\"author\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/8d08e9bd996a89bd75c0286cbabf3c66\"},\"breadcrumb\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/how-much-shared-memory-is-postgresql-using-2-details\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.dbi-services.com\/blog\/how-much-shared-memory-is-postgresql-using-2-details\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/how-much-shared-memory-is-postgresql-using-2-details\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Accueil\",\"item\":\"https:\/\/www.dbi-services.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"How much shared memory is PostgreSQL using? (2) – details\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/#website\",\"url\":\"https:\/\/www.dbi-services.com\/blog\/\",\"name\":\"dbi Blog\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.dbi-services.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/8d08e9bd996a89bd75c0286cbabf3c66\",\"name\":\"Daniel Westermann\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/secure.gravatar.com\/avatar\/31350ceeecb1dd8986339a29bf040d4cd3cd087d410deccd8f55234466d6c317?s=96&d=mm&r=g\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/31350ceeecb1dd8986339a29bf040d4cd3cd087d410deccd8f55234466d6c317?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/31350ceeecb1dd8986339a29bf040d4cd3cd087d410deccd8f55234466d6c317?s=96&d=mm&r=g\",\"caption\":\"Daniel Westermann\"},\"description\":\"Daniel Westermann is Principal Consultant and Technology Leader Open Infrastructure at dbi services. He has more than 15 years of experience in management, engineering and optimization of databases and infrastructures, especially on Oracle and PostgreSQL. Since the beginning of his career, he has specialized in Oracle Technologies and is Oracle Certified Professional 12c and Oracle Certified Expert RAC\/GridInfra. Over time, Daniel has become increasingly interested in open source technologies, becoming \u201cTechnology Leader Open Infrastructure\u201d and PostgreSQL expert. \u00a0Based on community or EnterpriseDB tools, he develops and installs complex high available solutions with PostgreSQL. He is also a certified PostgreSQL Plus 9.0 Professional and a Postgres Advanced Server 9.4 Professional. He is a regular speaker at PostgreSQL conferences in Switzerland and Europe. Today Daniel is also supporting our customers on AWS services such as AWS RDS, database migrations into the cloud, EC2 and automated infrastructure management with AWS SSM (System Manager). He is a certified AWS Solutions Architect Professional. Prior to dbi services, Daniel was Management System Engineer at LC SYSTEMS-Engineering AG in Basel. Before that, he worked as Oracle Developper &\u00a0Project Manager at Delta Energy Solutions AG in Basel (today Powel AG). Daniel holds a diploma in Business Informatics (DHBW, Germany). His branch-related experience mainly covers the pharma industry, the financial sector, energy, lottery and telecommunications.\",\"sameAs\":[\"https:\/\/x.com\/westermanndanie\"],\"url\":\"https:\/\/www.dbi-services.com\/blog\/author\/daniel-westermann\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"How much shared memory is PostgreSQL using? (2) - details - dbi Blog","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.dbi-services.com\/blog\/how-much-shared-memory-is-postgresql-using-2-details\/","og_locale":"en_US","og_type":"article","og_title":"How much shared memory is PostgreSQL using? (2) - details","og_description":"In the last post in this two post series we’ve briefly looked at configuring huge pages on Linux for PostgreSQL. It became clear, that calculating the number of huge pages purely based on the setting of shared_buffers will not work. PostgreSQL requires more shared memory, but what exactly is it? Well, this is the goal […]","og_url":"https:\/\/www.dbi-services.com\/blog\/how-much-shared-memory-is-postgresql-using-2-details\/","og_site_name":"dbi Blog","article_published_time":"2021-09-09T13:52:42+00:00","author":"Daniel Westermann","twitter_card":"summary_large_image","twitter_creator":"@westermanndanie","twitter_misc":{"Written by":"Daniel Westermann","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.dbi-services.com\/blog\/how-much-shared-memory-is-postgresql-using-2-details\/#article","isPartOf":{"@id":"https:\/\/www.dbi-services.com\/blog\/how-much-shared-memory-is-postgresql-using-2-details\/"},"author":{"name":"Daniel Westermann","@id":"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/8d08e9bd996a89bd75c0286cbabf3c66"},"headline":"How much shared memory is PostgreSQL using? (2) – details","datePublished":"2021-09-09T13:52:42+00:00","mainEntityOfPage":{"@id":"https:\/\/www.dbi-services.com\/blog\/how-much-shared-memory-is-postgresql-using-2-details\/"},"wordCount":280,"commentCount":0,"keywords":["PostgreSQL"],"articleSection":["Database Administration & Monitoring"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.dbi-services.com\/blog\/how-much-shared-memory-is-postgresql-using-2-details\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.dbi-services.com\/blog\/how-much-shared-memory-is-postgresql-using-2-details\/","url":"https:\/\/www.dbi-services.com\/blog\/how-much-shared-memory-is-postgresql-using-2-details\/","name":"How much shared memory is PostgreSQL using? (2) - details - dbi Blog","isPartOf":{"@id":"https:\/\/www.dbi-services.com\/blog\/#website"},"datePublished":"2021-09-09T13:52:42+00:00","author":{"@id":"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/8d08e9bd996a89bd75c0286cbabf3c66"},"breadcrumb":{"@id":"https:\/\/www.dbi-services.com\/blog\/how-much-shared-memory-is-postgresql-using-2-details\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.dbi-services.com\/blog\/how-much-shared-memory-is-postgresql-using-2-details\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.dbi-services.com\/blog\/how-much-shared-memory-is-postgresql-using-2-details\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Accueil","item":"https:\/\/www.dbi-services.com\/blog\/"},{"@type":"ListItem","position":2,"name":"How much shared memory is PostgreSQL using? (2) – details"}]},{"@type":"WebSite","@id":"https:\/\/www.dbi-services.com\/blog\/#website","url":"https:\/\/www.dbi-services.com\/blog\/","name":"dbi Blog","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.dbi-services.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/8d08e9bd996a89bd75c0286cbabf3c66","name":"Daniel Westermann","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/31350ceeecb1dd8986339a29bf040d4cd3cd087d410deccd8f55234466d6c317?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/31350ceeecb1dd8986339a29bf040d4cd3cd087d410deccd8f55234466d6c317?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/31350ceeecb1dd8986339a29bf040d4cd3cd087d410deccd8f55234466d6c317?s=96&d=mm&r=g","caption":"Daniel Westermann"},"description":"Daniel Westermann is Principal Consultant and Technology Leader Open Infrastructure at dbi services. He has more than 15 years of experience in management, engineering and optimization of databases and infrastructures, especially on Oracle and PostgreSQL. Since the beginning of his career, he has specialized in Oracle Technologies and is Oracle Certified Professional 12c and Oracle Certified Expert RAC\/GridInfra. Over time, Daniel has become increasingly interested in open source technologies, becoming \u201cTechnology Leader Open Infrastructure\u201d and PostgreSQL expert. \u00a0Based on community or EnterpriseDB tools, he develops and installs complex high available solutions with PostgreSQL. He is also a certified PostgreSQL Plus 9.0 Professional and a Postgres Advanced Server 9.4 Professional. He is a regular speaker at PostgreSQL conferences in Switzerland and Europe. Today Daniel is also supporting our customers on AWS services such as AWS RDS, database migrations into the cloud, EC2 and automated infrastructure management with AWS SSM (System Manager). He is a certified AWS Solutions Architect Professional. Prior to dbi services, Daniel was Management System Engineer at LC SYSTEMS-Engineering AG in Basel. Before that, he worked as Oracle Developper &\u00a0Project Manager at Delta Energy Solutions AG in Basel (today Powel AG). Daniel holds a diploma in Business Informatics (DHBW, Germany). His branch-related experience mainly covers the pharma industry, the financial sector, energy, lottery and telecommunications.","sameAs":["https:\/\/x.com\/westermanndanie"],"url":"https:\/\/www.dbi-services.com\/blog\/author\/daniel-westermann\/"}]}},"_links":{"self":[{"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/posts\/16674","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/users\/29"}],"replies":[{"embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/comments?post=16674"}],"version-history":[{"count":0,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/posts\/16674\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/media?parent=16674"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/categories?post=16674"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/tags?post=16674"},{"taxonomy":"type","embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/type_dbi?post=16674"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}