{"id":15353,"date":"2020-12-07T14:54:03","date_gmt":"2020-12-07T13:54:03","guid":{"rendered":"https:\/\/www.dbi-services.com\/blog\/incremental-materialized-view-maintenance-for-postgresql-14\/"},"modified":"2020-12-07T14:54:03","modified_gmt":"2020-12-07T13:54:03","slug":"incremental-materialized-view-maintenance-for-postgresql-14","status":"publish","type":"post","link":"https:\/\/www.dbi-services.com\/blog\/incremental-materialized-view-maintenance-for-postgresql-14\/","title":{"rendered":"Incremental materialized view maintenance for PostgreSQL 14?"},"content":{"rendered":"<p>Since <a href=\"https:\/\/www.postgresql.org\/docs\/9.3\/release-9-3.html\" target=\"_blank\" rel=\"noopener noreferrer\">PostgreSQL 9.3<\/a> there is the possibility to create materialized views in PostgreSQL. <a href=\"https:\/\/www.postgresql.org\/docs\/9.4\/sql-refreshmaterializedview.html\" target=\"_blank\" rel=\"noopener noreferrer\">PostgreSQL 9.4 (one year later)<\/a> brought concurrent refresh which already is a major step forward as this allowed querying the materialized view while it is being refreshed. What still is missing are materialized views which refresh themselves, as soon as there are changed to the underlying tables. This might change with PostgreSQL 14, as <a href=\"https:\/\/commitfest.postgresql.org\/32\/2138\/\" target=\"_blank\" rel=\"noopener noreferrer\">this patch<\/a> is in active development (at least since middle of 2019). Lets have a look at how that currently works and what the limitations are. If you want to play with this for yourself and do not want to apply the patches: <a href=\"https:\/\/hub.docker.com\/r\/yugonagata\/postgresql-ivm\" target=\"_blank\" rel=\"noopener noreferrer\">There is a Docker container you can use for your testing as well<\/a>.<\/p>\n<p><!--more--><\/p>\n<p>If you want to have a materialized view that is incrementally updated you need to specify this when the materialized view is created:<\/p>\n<pre class=\"brush: sql; gutter: true; first-line: 1; highlight: [5]\">\npostgres=# h create materialized view\nCommand:     CREATE MATERIALIZED VIEW\nDescription: define a new materialized view\nSyntax:\nCREATE [ INCREMENTAL ] MATERIALIZED VIEW [ IF NOT EXISTS ] table_name\n    [ (column_name [, ...] ) ]\n    [ USING method ]\n    [ WITH ( storage_parameter [= value] [, ... ] ) ]\n    [ TABLESPACE tablespace_name ]\n    AS query\n    [ WITH [ NO ] DATA ]\n\nURL: https:\/\/www.postgresql.org\/docs\/devel\/sql-creatematerializedview.html\n<\/pre>\n<p>If you skip &#8220;INCREMENTAL&#8221;, the materialized view will not be updated automatically and you get the behavior as it is now. As we want to have a look at the new feature lets create a base table and then add an incrementally updated materialized view on top of it:<\/p>\n<pre class=\"brush: sql; gutter: true; first-line: 1\">\npostgres=# create table t1 ( a int, b text, c date );\nCREATE TABLE\npostgres=# insert into t1 select x, x::text, now() from generate_series(1,1000000) x;\nINSERT 0 1000000\npostgres=# create incremental materialized view mv1 as select * from t1 with data;\nSELECT 1000000\npostgres=# \n<\/pre>\n<p>&#8220;d+&#8221; will show you that this materialized view is incrementally updated:<\/p>\n<pre class=\"brush: sql; gutter: true; first-line: 1; highlight: [14]\">\npostgres=# d+ mv1\n                              Materialized view \"public.mv1\"\n Column |  Type   | Collation | Nullable | Default | Storage  | Stats target | Description \n--------+---------+-----------+----------+---------+----------+--------------+-------------\n a      | integer |           |          |         | plain    |              | \n b      | text    |           |          |         | extended |              | \n c      | date    |           |          |         | plain    |              | \nView definition:\n SELECT t1.a,\n    t1.b,\n    t1.c\n   FROM t1;\nAccess method: heap\nIncremental view maintenance: yes\n<\/pre>\n<p>If we update the underlying table, the materialized view gets updated automatically:<\/p>\n<pre class=\"brush: sql; gutter: true; first-line: 1\">\npostgres=# insert into t1 (a,b,c) values(-1,'aaa',now());\nINSERT 0 1\npostgres=# select * from mv1 where a = -1;\n a  |  b  |     c      \n----+-----+------------\n -1 | aaa | 2020-11-23\n(1 row)\n\npostgres=# update t1 set a = -2 where a = -1;\nUPDATE 1\npostgres=# select * from mv1 where a = -2;\n a  |  b  |     c      \n----+-----+------------\n -2 | aaa | 2020-11-23\n(1 row)\n\npostgres=# \n<\/pre>\n<p>That&#8217;s really cool but you need to be aware that this comes with a cost: Modifying (insert\/update\/delete) the underlying table(s) becomes more expensive. Lets compare a small bulk load into a table without a materialized view on top of it against the same load into a table with a materialized view on top:<\/p>\n<pre class=\"brush: sql; gutter: true; first-line: 1\">\npostgres=# truncate table t1;\nTRUNCATE TABLE\npostgres=# create table t2 ( a int, b text, c date );\nCREATE TABLE\npostgres=# timing\nTiming is on.\npostgres=# insert into t1 select x, x::text, now() from generate_series(1,1000000) x;\nINSERT 0 1000000\nTime: 3214.712 ms (00:03.215)\npostgres=# insert into t2 select x, x::text, now() from generate_series(1,1000000) x;\nINSERT 0 1000000\nTime: 1285.578 ms (00:01.286)\npostgres=# insert into t1 select x, x::text, now() from generate_series(1,1000000) x;\nINSERT 0 1000000\nTime: 4117.097 ms (00:04.117)\npostgres=# insert into t2 select x, x::text, now() from generate_series(1,1000000) x;\nINSERT 0 1000000\nTime: 1511.681 ms (00:01.512)\npostgres=# insert into t1 select x, x::text, now() from generate_series(1,1000000) x;\nINSERT 0 1000000\nTime: 3844.273 ms (00:03.844)\npostgres=# insert into t2 select x, x::text, now() from generate_series(1,1000000) x;\nINSERT 0 1000000\nTime: 1463.377 ms (00:01.463)\n<\/pre>\n<p>Without a materialized view, the load time is around 3 times faster, so you have to decide what is more important to you: Fast loading or up to date materialized views. <\/p>\n<p>Finally: <a href=\"https:\/\/wiki.postgresql.org\/wiki\/Incremental_View_Maintenance\" target=\"_blank\" rel=\"noopener noreferrer\">Here is the Wiki page that summarizes the feature and also lists some limitations.<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Since PostgreSQL 9.3 there is the possibility to create materialized views in PostgreSQL. PostgreSQL 9.4 (one year later) brought concurrent refresh which already is a major step forward as this allowed querying the materialized view while it is being refreshed. What still is missing are materialized views which refresh themselves, as soon as there are [&hellip;]<\/p>\n","protected":false},"author":29,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[229],"tags":[77],"type_dbi":[],"class_list":["post-15353","post","type-post","status-publish","format-standard","hentry","category-database-administration-monitoring","tag-postgresql"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v27.2 (Yoast SEO v27.2) - https:\/\/yoast.com\/product\/yoast-seo-premium-wordpress\/ -->\n<title>Incremental materialized view maintenance for PostgreSQL 14? - dbi Blog<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.dbi-services.com\/blog\/incremental-materialized-view-maintenance-for-postgresql-14\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Incremental materialized view maintenance for PostgreSQL 14?\" \/>\n<meta property=\"og:description\" content=\"Since PostgreSQL 9.3 there is the possibility to create materialized views in PostgreSQL. PostgreSQL 9.4 (one year later) brought concurrent refresh which already is a major step forward as this allowed querying the materialized view while it is being refreshed. What still is missing are materialized views which refresh themselves, as soon as there are [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.dbi-services.com\/blog\/incremental-materialized-view-maintenance-for-postgresql-14\/\" \/>\n<meta property=\"og:site_name\" content=\"dbi Blog\" \/>\n<meta property=\"article:published_time\" content=\"2020-12-07T13:54:03+00:00\" \/>\n<meta name=\"author\" content=\"Daniel Westermann\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@westermanndanie\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Daniel Westermann\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/incremental-materialized-view-maintenance-for-postgresql-14\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/incremental-materialized-view-maintenance-for-postgresql-14\/\"},\"author\":{\"name\":\"Daniel Westermann\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/8d08e9bd996a89bd75c0286cbabf3c66\"},\"headline\":\"Incremental materialized view maintenance for PostgreSQL 14?\",\"datePublished\":\"2020-12-07T13:54:03+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/incremental-materialized-view-maintenance-for-postgresql-14\/\"},\"wordCount\":322,\"commentCount\":0,\"keywords\":[\"PostgreSQL\"],\"articleSection\":[\"Database Administration &amp; Monitoring\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/www.dbi-services.com\/blog\/incremental-materialized-view-maintenance-for-postgresql-14\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/incremental-materialized-view-maintenance-for-postgresql-14\/\",\"url\":\"https:\/\/www.dbi-services.com\/blog\/incremental-materialized-view-maintenance-for-postgresql-14\/\",\"name\":\"Incremental materialized view maintenance for PostgreSQL 14? - dbi Blog\",\"isPartOf\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/#website\"},\"datePublished\":\"2020-12-07T13:54:03+00:00\",\"author\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/8d08e9bd996a89bd75c0286cbabf3c66\"},\"breadcrumb\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/incremental-materialized-view-maintenance-for-postgresql-14\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.dbi-services.com\/blog\/incremental-materialized-view-maintenance-for-postgresql-14\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/incremental-materialized-view-maintenance-for-postgresql-14\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Accueil\",\"item\":\"https:\/\/www.dbi-services.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Incremental materialized view maintenance for PostgreSQL 14?\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/#website\",\"url\":\"https:\/\/www.dbi-services.com\/blog\/\",\"name\":\"dbi Blog\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.dbi-services.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/8d08e9bd996a89bd75c0286cbabf3c66\",\"name\":\"Daniel Westermann\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/secure.gravatar.com\/avatar\/31350ceeecb1dd8986339a29bf040d4cd3cd087d410deccd8f55234466d6c317?s=96&d=mm&r=g\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/31350ceeecb1dd8986339a29bf040d4cd3cd087d410deccd8f55234466d6c317?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/31350ceeecb1dd8986339a29bf040d4cd3cd087d410deccd8f55234466d6c317?s=96&d=mm&r=g\",\"caption\":\"Daniel Westermann\"},\"description\":\"Daniel Westermann is Principal Consultant and Technology Leader Open Infrastructure at dbi services. He has more than 15 years of experience in management, engineering and optimization of databases and infrastructures, especially on Oracle and PostgreSQL. Since the beginning of his career, he has specialized in Oracle Technologies and is Oracle Certified Professional 12c and Oracle Certified Expert RAC\/GridInfra. Over time, Daniel has become increasingly interested in open source technologies, becoming \u201cTechnology Leader Open Infrastructure\u201d and PostgreSQL expert. \u00a0Based on community or EnterpriseDB tools, he develops and installs complex high available solutions with PostgreSQL. He is also a certified PostgreSQL Plus 9.0 Professional and a Postgres Advanced Server 9.4 Professional. He is a regular speaker at PostgreSQL conferences in Switzerland and Europe. Today Daniel is also supporting our customers on AWS services such as AWS RDS, database migrations into the cloud, EC2 and automated infrastructure management with AWS SSM (System Manager). He is a certified AWS Solutions Architect Professional. Prior to dbi services, Daniel was Management System Engineer at LC SYSTEMS-Engineering AG in Basel. Before that, he worked as Oracle Developper &amp;\u00a0Project Manager at Delta Energy Solutions AG in Basel (today Powel AG). Daniel holds a diploma in Business Informatics (DHBW, Germany). His branch-related experience mainly covers the pharma industry, the financial sector, energy, lottery and telecommunications.\",\"sameAs\":[\"https:\/\/x.com\/westermanndanie\"],\"url\":\"https:\/\/www.dbi-services.com\/blog\/author\/daniel-westermann\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Incremental materialized view maintenance for PostgreSQL 14? - dbi Blog","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.dbi-services.com\/blog\/incremental-materialized-view-maintenance-for-postgresql-14\/","og_locale":"en_US","og_type":"article","og_title":"Incremental materialized view maintenance for PostgreSQL 14?","og_description":"Since PostgreSQL 9.3 there is the possibility to create materialized views in PostgreSQL. PostgreSQL 9.4 (one year later) brought concurrent refresh which already is a major step forward as this allowed querying the materialized view while it is being refreshed. What still is missing are materialized views which refresh themselves, as soon as there are [&hellip;]","og_url":"https:\/\/www.dbi-services.com\/blog\/incremental-materialized-view-maintenance-for-postgresql-14\/","og_site_name":"dbi Blog","article_published_time":"2020-12-07T13:54:03+00:00","author":"Daniel Westermann","twitter_card":"summary_large_image","twitter_creator":"@westermanndanie","twitter_misc":{"Written by":"Daniel Westermann","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.dbi-services.com\/blog\/incremental-materialized-view-maintenance-for-postgresql-14\/#article","isPartOf":{"@id":"https:\/\/www.dbi-services.com\/blog\/incremental-materialized-view-maintenance-for-postgresql-14\/"},"author":{"name":"Daniel Westermann","@id":"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/8d08e9bd996a89bd75c0286cbabf3c66"},"headline":"Incremental materialized view maintenance for PostgreSQL 14?","datePublished":"2020-12-07T13:54:03+00:00","mainEntityOfPage":{"@id":"https:\/\/www.dbi-services.com\/blog\/incremental-materialized-view-maintenance-for-postgresql-14\/"},"wordCount":322,"commentCount":0,"keywords":["PostgreSQL"],"articleSection":["Database Administration &amp; Monitoring"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.dbi-services.com\/blog\/incremental-materialized-view-maintenance-for-postgresql-14\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.dbi-services.com\/blog\/incremental-materialized-view-maintenance-for-postgresql-14\/","url":"https:\/\/www.dbi-services.com\/blog\/incremental-materialized-view-maintenance-for-postgresql-14\/","name":"Incremental materialized view maintenance for PostgreSQL 14? - dbi Blog","isPartOf":{"@id":"https:\/\/www.dbi-services.com\/blog\/#website"},"datePublished":"2020-12-07T13:54:03+00:00","author":{"@id":"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/8d08e9bd996a89bd75c0286cbabf3c66"},"breadcrumb":{"@id":"https:\/\/www.dbi-services.com\/blog\/incremental-materialized-view-maintenance-for-postgresql-14\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.dbi-services.com\/blog\/incremental-materialized-view-maintenance-for-postgresql-14\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.dbi-services.com\/blog\/incremental-materialized-view-maintenance-for-postgresql-14\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Accueil","item":"https:\/\/www.dbi-services.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Incremental materialized view maintenance for PostgreSQL 14?"}]},{"@type":"WebSite","@id":"https:\/\/www.dbi-services.com\/blog\/#website","url":"https:\/\/www.dbi-services.com\/blog\/","name":"dbi Blog","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.dbi-services.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/8d08e9bd996a89bd75c0286cbabf3c66","name":"Daniel Westermann","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/31350ceeecb1dd8986339a29bf040d4cd3cd087d410deccd8f55234466d6c317?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/31350ceeecb1dd8986339a29bf040d4cd3cd087d410deccd8f55234466d6c317?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/31350ceeecb1dd8986339a29bf040d4cd3cd087d410deccd8f55234466d6c317?s=96&d=mm&r=g","caption":"Daniel Westermann"},"description":"Daniel Westermann is Principal Consultant and Technology Leader Open Infrastructure at dbi services. He has more than 15 years of experience in management, engineering and optimization of databases and infrastructures, especially on Oracle and PostgreSQL. Since the beginning of his career, he has specialized in Oracle Technologies and is Oracle Certified Professional 12c and Oracle Certified Expert RAC\/GridInfra. Over time, Daniel has become increasingly interested in open source technologies, becoming \u201cTechnology Leader Open Infrastructure\u201d and PostgreSQL expert. \u00a0Based on community or EnterpriseDB tools, he develops and installs complex high available solutions with PostgreSQL. He is also a certified PostgreSQL Plus 9.0 Professional and a Postgres Advanced Server 9.4 Professional. He is a regular speaker at PostgreSQL conferences in Switzerland and Europe. Today Daniel is also supporting our customers on AWS services such as AWS RDS, database migrations into the cloud, EC2 and automated infrastructure management with AWS SSM (System Manager). He is a certified AWS Solutions Architect Professional. Prior to dbi services, Daniel was Management System Engineer at LC SYSTEMS-Engineering AG in Basel. Before that, he worked as Oracle Developper &amp;\u00a0Project Manager at Delta Energy Solutions AG in Basel (today Powel AG). Daniel holds a diploma in Business Informatics (DHBW, Germany). His branch-related experience mainly covers the pharma industry, the financial sector, energy, lottery and telecommunications.","sameAs":["https:\/\/x.com\/westermanndanie"],"url":"https:\/\/www.dbi-services.com\/blog\/author\/daniel-westermann\/"}]}},"_links":{"self":[{"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/posts\/15353","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/users\/29"}],"replies":[{"embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/comments?post=15353"}],"version-history":[{"count":0,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/posts\/15353\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/media?parent=15353"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/categories?post=15353"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/tags?post=15353"},{"taxonomy":"type","embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/type_dbi?post=15353"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}