{"id":4058,"date":"2025-09-02T20:31:50","date_gmt":"2025-09-03T01:31:50","guid":{"rendered":"https:\/\/promincproductions.com\/blog\/?p=4058"},"modified":"2025-09-02T20:32:21","modified_gmt":"2025-09-03T01:32:21","slug":"update-postgres-with-data-migration","status":"publish","type":"post","link":"https:\/\/promincproductions.com\/blog\/update-postgres-with-data-migration\/","title":{"rendered":"Update Postgres with Data Migration"},"content":{"rendered":"<p>A tutorial on how to upgrade Postgres containers installed on a QNAP NAS via Container Station 3.  Some major version upgrades of Postgres aren&#8217;t directly possible as there are data structure and other critical changes that need to be handled via upgrade tools.  This can cause additional steps and effort and complexity.  It&#8217;s all possible to get around &#8211; as long as you know the correct sequence of steps.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full is-resized\"><img decoding=\"async\" width=\"600\" height=\"600\" src=\"https:\/\/promincproductions.com\/blog\/wp-content\/uploads\/2025\/09\/qnap-container-station-postgres-upgrade.png\" alt=\"QNAP Container Station Upgrade PostgreSQL Container\" class=\"wp-image-4067\" style=\"width:362px;height:auto\" srcset=\"https:\/\/promincproductions.com\/blog\/wp-content\/uploads\/2025\/09\/qnap-container-station-postgres-upgrade.png 600w, https:\/\/promincproductions.com\/blog\/wp-content\/uploads\/2025\/09\/qnap-container-station-postgres-upgrade-500x500.png 500w, https:\/\/promincproductions.com\/blog\/wp-content\/uploads\/2025\/09\/qnap-container-station-postgres-upgrade-150x150.png 150w, https:\/\/promincproductions.com\/blog\/wp-content\/uploads\/2025\/09\/qnap-container-station-postgres-upgrade-450x450.png 450w\" sizes=\"(max-width: 600px) 100vw, 600px\" \/><\/figure>\n<\/div>\n\n\n<h2 class=\"wp-block-heading\">Background: Why the Upgrade Didn&#8217;t Work Directly<\/h2>\n\n\n\n<p>I went to update version of<a href=\"https:\/\/promincproductions.com\/blog\/install-mattermost-on-qnap-nas-with-container-station\/\" data-lasso-id=\"1013\"> Mattermost installed via Container Station on a QNAP NAS<\/a> and ran into an issue.  The version of the Postgres database server was too old, not supported by the new version of Mattermost.  Thus triggering the need to update the Postgres version as well.<\/p>\n\n\n\n<p>On the surface this didn&#8217;t seem like a big issue &#8211; just update the version of the container.  Sure &#8211; that can be done.  In fact I already have a whole guide on how to do this &#8211; <a href=\"https:\/\/promincproductions.com\/blog\/update-container-version-in-qnap-container-station-3\/\" data-lasso-id=\"1014\">Update Container Version in QNAP Container Station 3<\/a>.  But yet another issue ran into&#8230;  I was upgrading from Postgres 11 to Postgres 17 isn&#8217;t directly possible.  There were enough changes between these two versions that upgrade tools need to be run and so progress was halted here.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">How to Correctly Upgrade Postgres Installed via Containers<\/h2>\n\n\n\n<p>Research indicates that there is a <code>pg_upgrade<\/code> utility for this purpose.  However given that Postgres has been installed via Container Station (Docker containers) the upgrade doesn&#8217;t happen within the machine like a direct installation and thus this wasn&#8217;t a feasible option.<\/p>\n\n\n\n<p>Instead an alternate approach is needed here.  First the data from the older version needs to be exported and then imported into the new container.  All in all, fairly straightforward &#8211; assuming you know the correct commands and procedures.  Read on to see what worked for my upgrade.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Postgres Installed via Containers Upgrade Process<\/h2>\n\n\n\n<p>Follow these steps to upgrade major versions of Postgres installed via Container Station 3 on a QNAP NAS via a data export \/ import process.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Connect to QNAP NAS via SSH<\/h3>\n\n\n\n<p>This process will require SSH access to the QNAP NAS.  SSH allows for command line (terminal) access directly to the server to run commands to process the upgrade.  If you don\u2019t have SSH access to the QNAP server, follow the\u00a0<a href=\"https:\/\/www.qnap.com\/en\/how-to\/faq\/article\/how-do-i-access-my-qnap-nas-using-ssh\" target=\"_blank\" rel=\"noopener\" data-lasso-id=\"1015\">QNAP SSH tutorial<\/a>\u00a0for how to setup access.<\/p>\n\n\n\n<p>DISCLAIMER: SSH access does give you access to the server that if used incorrectly can cause damage to the system.  Use at your own risk.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Expose Docker Command<\/h3>\n\n\n\n<p>On a QNAP NAS the <code>docker<\/code> command isn&#8217;t natively exposed which makes completing this task more difficult.  For convenience you can expose this command to allow for ease of completing this project.  Running this only exposes the <code>docker<\/code> command for this one SSH connection and will need to be re-run later on.  More details on how to do this can be found at <a href=\"https:\/\/promincproductions.com\/blog\/access-docker-command-via-ssh-on-qnap-ts-251\/\" data-lasso-id=\"1016\">this blog post<\/a>, but simply run this command once connected to the QNAP NAS via SSH.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>export PATH=$PATH\":\/share\/CACHEDEV1_DATA\/.qpkg\/container-station\/bin\/:\"<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">Start the Old Version of Postgres<\/h3>\n\n\n\n<p>The current \/ old version of Postgres needs to be running so that the data can be exported from it.  In most cases this is already happening.  The easiest way to ensure the old Postgres server is running is using the QNAP web admin via Container Station.  Find the Postgres Application \/ Container and ensure that it has the <code>Running<\/code> status.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"obtain-container-id-old\">Obtain the ID of the Old Postgres Container<\/h3>\n\n\n\n<p>Now is when it is required to connect to the QNAP NAS via SSH.  The ID of the Postgres Docker container is needed to proceed.  It can be obtained by running this command:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>docker ps -a | grep postgres<\/code><\/pre>\n\n\n\n<p>This command will produce output similar to this:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>123456abcdef  postgres:11.6  \"docker-entrypoint.s\u2026\"   2 months ago  Up 5 hours                                     postgresql-postgres-1\n<\/code><\/pre>\n\n\n\n<p>There might be multiple rows depending on what containers have been installed on your server.  Notice though the name and version of <code>postgres:11.6<\/code>.  That&#8217;s a good clue of what entry matters here.  Also notice <code>Up 5 hours<\/code>.  <code>Up<\/code> means that the container server is running.  Bingo &#8211; that&#8217;s what we are looking for!  The first column &#8211; <code>123456abcdef<\/code> is the ID we are after here.  This is used in future commands to tell the command this is the container we want to act upon.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Create a Backup Dump of the Database from the Old Postgres Server<\/h3>\n\n\n\n<p>The data from the current\/old Postgres database server needs to be saved out to a dump file.  This will later be used to import into the server with the new version of Postgres.<\/p>\n\n\n\n<p>First pick a destination \/ create a directory to save the data to.  <\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>## create a directory to save the data to\nmkdir -p Backups\/postgresql\/&lt;date&gt;\/\n\n## Execute the export\ndocker exec -it 123456abcdef pg_dump -U &lt;postgres_username&gt; -d &lt;database_name&gt; -c -C &gt; Backups\/postgresql\/&lt;date&gt;\/postgres-v11-mattermost.dump<\/code><\/pre>\n\n\n\n<p><code>pg_dump<\/code> arguments:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><code>-U<\/code>: username.  The username of the current\/old Postgres database user<\/li>\n\n\n\n<li><code>-d<\/code>: dbname.  The name of the database to export.  Typically for this project, it is <code>mattermost<\/code><\/li>\n\n\n\n<li><code>-c<\/code>: clean.  This will drop the database in the new version of Postgres if it exists before running the import.<\/li>\n\n\n\n<li><code>-C<\/code>: create.  This will ensure a create statement is added to the data dump file.<\/li>\n<\/ul>\n\n\n\n<p>The <code>&gt;<\/code> then redirects the output of the <code>pg_dump<\/code> command to a file.  Name the path and file accordingly to your needs.<\/p>\n\n\n\n<p>[<a href=\"https:\/\/www.postgresql.org\/docs\/current\/app-pgdump.html\" data-lasso-id=\"1017\"><code>pg_dump<\/code> reference<\/a>]<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Create an updated version of the Postgres Application<\/h3>\n\n\n\n<p>Moving back into Container Station 3 in the QNAP Web Admin it is time to create the Application for the updated Postgres server.<\/p>\n\n\n\n<p>NOTE: It is best to keep the same username\/password as the prior Postgres application so that Mattermost (or other services that use Postgres) can connect seamlessly.  The URL\/IP address and port will be the same &#8211; just the container running Postgres and the image are what will be different.<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Login to the QNAP Web Admin<\/li>\n\n\n\n<li>Open <strong>Container Station 3<\/strong><\/li>\n\n\n\n<li>Click on <strong>Applications<\/strong> in the left hand menu<\/li>\n\n\n\n<li>Click the <strong>+ Create<\/strong> button<\/li>\n\n\n\n<li>Enter an <strong>Application Name<\/strong> (example: <code>postgresql-17_6<\/code>)<\/li>\n\n\n\n<li>Enter the following YAML configuration code:<\/li>\n<\/ol>\n\n\n\n<pre class=\"wp-block-code\"><code>version: '3'\n\nservices:\n  postgres:\n    image: postgres:17.6\n    restart: on-failure\n    ports:\n      - 127.0.0.1:5432:5432\n    volumes:\n      - \/share\/Container\/postgresdb_17:\/var\/lib\/postgresql\/data\n    environment:\n      - POSTGRES_USER=&lt;postgres_username&gt;\n      - POSTGRES_PASSWORD=&lt;postgres_password&gt;\n\n  pgadmin4:\n    image: dpage\/pgadmin4:latest\n    restart: on-failure\n    ports:\n      - 7070:80\n      - 7443:443\n    depends_on:\n      - postgres     \n    environment:\n      - PGADMIN_DEFAULT_EMAIL&lt;your_email_for_admin_account&gt;\n      - PGADMIN_DEFAULT_PASSWORD=&lt;your_password_for_admin_account&gt;<\/code><\/pre>\n\n\n\n<ol start=\"7\" class=\"wp-block-list\">\n<li>Click the <strong>Create<\/strong> button<\/li>\n\n\n\n<li>Wait for the container to start<\/li>\n<\/ol>\n\n\n\n<h4 class=\"wp-block-heading\">Notes:<\/h4>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Set the <code>image<\/code> to the version desired.  The list of versions available can be found on the <a href=\"https:\/\/hub.docker.com\/_\/postgres\" data-lasso-id=\"1018\">Postgres Docker Hub page<\/a>.<\/li>\n\n\n\n<li>The <code>volume<\/code> is wise to use a volume specific to this version so that there isn&#8217;t a conflict with the current\/old version of Postgres.  If there is an issue with the update and you need to roll-back or anything it&#8217;s wise to keep these separate.<\/li>\n\n\n\n<li>The <code>pgadmin4<\/code> section is optional.  It provides a web-based admin interface for managing the database.  It can be removed if desired.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Obtain the ID of the New Postgres Container<\/h3>\n\n\n\n<p>Just like <a href=\"#obtain-container-id-old\">above<\/a>, the container ID for the new Postgres container is needed.  We again run this command:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code><code>docker ps -a | grep postgres<\/code><\/code><\/pre>\n\n\n\n<p>Here again, the alpha-numeric ID from the first column is what is needed but for the new container &#8211; the container that is running of course.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>123456abcdef  postgres:11.6  \"docker-entrypoint.s\u2026\"   2 months ago  Exited (0) 15 hours ago                                   postgresql-postgres-1\n987654zyxwvu  postgres:17.6  \"docker-entrypoint.s\u2026\"   3 minutes ago  Up 3 minutes                    127.0.0.1:5432-&gt;5432\/tcp   postgresql-17_6-postgres-1<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">Import the Data Dump from the Old Postgres Server Into the New Postgres Server<\/h3>\n\n\n\n<p>It&#8217;s now time to take the data dump from the old Postgres server and import it into the new Postgres server.  Since the import is being run on the new version any data structure and collation changes needed will be processed successfully at this time.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>docker exec -i 987654zyxwvu psql -U &lt;postgres_username&gt; &lt; Backups\/postgresql\/&lt;date&gt;\/postgres-v11-mattermost.dump<\/code><\/pre>\n\n\n\n<p>Technically at this point all should be good.  The new server is up and running and should have the old data imported and restructured as needed.  There are a few more steps suggested however to ensure that all is running as expected.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Open Postgres Command Line Interface<\/h3>\n\n\n\n<p>Run Postgres via the command line with this command.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>docker exec -it 987654zyxwvu psql -U &lt;postgres_username&gt; -d mattermost<\/code><\/pre>\n\n\n\n<ul class=\"wp-block-list\">\n<li><code>-d<\/code>: database name.  (typically is <code>mattermost<\/code> for this project)<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Confirm Mattermost Tables Exist<\/h3>\n\n\n\n<p>A quick sanity check that the data imported is to see if tables exist.  This command to list the tables should return ~20+ tables.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>\\dt # list tables<\/code><\/pre>\n\n\n\n<ul class=\"wp-block-list\">\n<li><code>spacebar<\/code>: page down to the next table name<\/li>\n\n\n\n<li><code>q<\/code>: close out of the table list mode<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Confirm Past Mattermost Posts Exist<\/h3>\n\n\n\n<p>This query will show the last 5 posts made in the chat history.  This can be used as a sanity check that the data has been imported as expected.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>SELECT * FROM posts ORDER BY createat DESC LIMIT 5;<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">Database Maintenance<\/h3>\n\n\n\n<p>Assuming that the checks above look as expected, it&#8217;s time to do a bit of maintenance functions and then be done with this upgrade.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Switch to the Postgres Database<\/h4>\n\n\n\n<p>The <code>mattermost<\/code> database used above holds the data for the application.  There is also a database that holds some of the root Postgres data &#8211; that database is named <code>postgres<\/code>.  Switch to that database with the following command:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>\\c postgres<\/code><\/pre>\n\n\n\n<p>Upon switching to the <code>postgres<\/code> database there might be an error displayed such as:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>WARNING: database \"postgres\" has a collation version mismatch<br>DETAIL: The database was created using collation version 2.36, but the operating system provides version 2.41.<br>HINT: Rebuild all objects in this database that use the default collation and run ALTER DATABASE postgres REFRESH COLLATION VERSION, or build PostgreSQL with the right library version.<\/code><\/pre>\n\n\n\n<p>The error message gives the clues as to how to resolve this hurdle.  Run the query provided &#8211; ensure to add a <code>;<\/code> at the end to tell the command line it is time to execute the command.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>ALTER DATABASE postgres REFRESH COLLATION VERSION;<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\">Reset Database User Password<\/h4>\n\n\n\n<p>In some cases the password for the database isn&#8217;t correct.  This could be due to miss-configuration from the old to the new container.  It could also be due to the encryption method of the password from the old to the new server changing.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>ALTER USER &lt;postgres_username&gt; WITH PASSWORD '&lt;postgres_password&gt;';<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\">Garbage Collection and Recollect Stats<\/h4>\n\n\n\n<p>With the data structure changing from the old to new version of Postgres, it&#8217;s good practice to run the <a href=\"https:\/\/www.postgresql.org\/docs\/current\/sql-vacuum.html\" target=\"_blank\" rel=\"noopener\" data-lasso-id=\"1019\">VACUUM<\/a> and <a href=\"https:\/\/www.postgresql.org\/docs\/current\/sql-analyze.html\" data-lasso-id=\"1020\">ANALYZE<\/a> commands.  These clean up any dead space left behind by data manipulations and reset the stats tables in Postgres.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>VACUUM ANALYZE;<\/code><\/pre>\n\n\n\n<h2 class=\"wp-block-heading\">Conclusion<\/h2>\n\n\n\n<p>Migrating between major versions of Postgres (ie. version 11 to version 17) using QNAP Container Station 3 (as well as Docker containers) is a straightforward and rather quick task assuming the correct steps are followed.  The process involves exporting the desired data, shutting down the Docker container for the old version of Postgres, starting the Docker container for the new version of Postgres, and importing the data.  Lastly a few Postgres maintenance tasks need to be run to ensure the transition is complete.<\/p>","protected":false},"excerpt":{"rendered":"<p>A tutorial on how to upgrade Postgres containers installed on a QNAP NAS via Container Station 3. Some [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":4067,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"wprm-recipe-roundup-name":"","wprm-recipe-roundup-description":"","_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_post_was_ever_published":false},"categories":[384],"tags":[804,803,330],"class_list":["post-4058","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-qnap","tag-database","tag-postgres","tag-qnap"],"jetpack_featured_media_url":"https:\/\/promincproductions.com\/blog\/wp-content\/uploads\/2025\/09\/qnap-container-station-postgres-upgrade.png","jetpack_shortlink":"https:\/\/wp.me\/p4BbcR-13s","jetpack_sharing_enabled":true,"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/promincproductions.com\/blog\/wp-json\/wp\/v2\/posts\/4058","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/promincproductions.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/promincproductions.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/promincproductions.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/promincproductions.com\/blog\/wp-json\/wp\/v2\/comments?post=4058"}],"version-history":[{"count":8,"href":"https:\/\/promincproductions.com\/blog\/wp-json\/wp\/v2\/posts\/4058\/revisions"}],"predecessor-version":[{"id":4068,"href":"https:\/\/promincproductions.com\/blog\/wp-json\/wp\/v2\/posts\/4058\/revisions\/4068"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/promincproductions.com\/blog\/wp-json\/wp\/v2\/media\/4067"}],"wp:attachment":[{"href":"https:\/\/promincproductions.com\/blog\/wp-json\/wp\/v2\/media?parent=4058"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/promincproductions.com\/blog\/wp-json\/wp\/v2\/categories?post=4058"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/promincproductions.com\/blog\/wp-json\/wp\/v2\/tags?post=4058"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}