From 5dec9630333406db022149455507847e1f43534e Mon Sep 17 00:00:00 2001 From: "QUALISYSTEMS\\nahum-t" Date: Tue, 6 Oct 2026 16:26:26 +0300 Subject: [PATCH 1/2] Document SQL Server to PostgreSQL migration and database sanitization - New how-to: Migrating the Databases from SQL Server to PostgreSQL. The migration tool ships in CloudShell\Utilities\SqlServerToPostgresqlMigrator starting with 2026.2 (TFS CS #190053/#190054); earlier versions get it from Support. - New troubleshooting page: Sharing Your Databases with Quali Support, covering the sanitization scripts. - Database Prerequisites: point to the wizard and the migration page instead of "contact Support for components and migration guidance". Ticket #67440. Co-Authored-By: Claude Opus 5.5 --- .../migrate-sql-server-to-postgresql.md | 87 +++++++++++++++++++ .../cs-system-requirements/database-prereq.md | 4 +- .../share-database-with-support.md | 25 ++++++ 3 files changed, 114 insertions(+), 2 deletions(-) create mode 100644 docs/install-configure/cloudshell-suite/backup-restore/migrate-sql-server-to-postgresql.md create mode 100644 docs/troubleshooting/database-and-permissions/share-database-with-support.md diff --git a/docs/install-configure/cloudshell-suite/backup-restore/migrate-sql-server-to-postgresql.md b/docs/install-configure/cloudshell-suite/backup-restore/migrate-sql-server-to-postgresql.md new file mode 100644 index 0000000000..2e4f540a44 --- /dev/null +++ b/docs/install-configure/cloudshell-suite/backup-restore/migrate-sql-server-to-postgresql.md @@ -0,0 +1,87 @@ +--- +sidebar_position: 4 +--- + +# Migrating the Databases from SQL Server to PostgreSQL + +This article explains how to move an existing CloudShell from Microsoft SQL Server to PostgreSQL, using the CloudShell database migration tool. The tool copies the three CloudShell relational databases. MongoDB isn't affected: CloudShell keeps using the same MongoDB after the migration. + +| Database (default name) | SQL Server connection name | PostgreSQL connection name | +|---|---|---| +| Quali | TestShell | TestShellPostgres | +| QualiResults | TestShellResult | TestShellResultPostgres | +| QualiInsight | InSightResult | InSightResultPostgres | + +:::tip +Migrate a copy of your environment first, and keep your SQL Server databases until you have validated CloudShell on PostgreSQL. +::: + +## Getting the tool + +The tool's version must match your installed CloudShell version exactly. The tool prints its version when it starts, but it doesn't block a mismatch, and a mismatched tool creates a database schema that doesn't match your Quali Server. + +- **CloudShell 2026.2 and later:** the CloudShell installation package includes the matching tool, in `CloudShell\Utilities\SqlServerToPostgresqlMigrator`. +- **Earlier versions:** contact Quali Support for the tool that matches your version. + +## Before you start + +1. Make sure CloudShell works normally on SQL Server, and that the Quali Server Configuration Wizard has completed. +2. Back up the SQL Server databases and the MongoDB databases. See [Backing Up CloudShell Databases](./backup-cs-db.md). +3. Stop the Quali Server service, and close any running Configuration Wizard. +4. On a PostgreSQL 12 or later server, create three new, empty databases, owned by the login CloudShell will use. Use lowercase names. For example, in `psql`: + + ```sql + CREATE DATABASE quali WITH OWNER = cloudshell ENCODING = 'UTF8'; + CREATE DATABASE qualiresults WITH OWNER = cloudshell ENCODING = 'UTF8'; + CREATE DATABASE qualiinsight WITH OWNER = cloudshell ENCODING = 'UTF8'; + ``` + + Don't create the databases with the Configuration Wizard. The tool expects them to be empty. + +5. Give that login superuser rights for the duration of the migration. The tool needs them to copy the data. Run the tool with the same login CloudShell will use, so CloudShell owns the tables: + + ```sql + ALTER ROLE cloudshell SUPERUSER; + ``` + +6. Make sure the PostgreSQL server has enough disk space for the full data size of the SQL Server databases. +7. Run the tool from a machine with .NET Framework 4.8 and a fast, reliable connection to both database servers. The Quali Server machine is usually the best choice. + +## Running the migration + +1. Copy the `SqlServerToPostgresqlMigrator` folder to the machine, and set the six connection strings in `connections.config`: the three SQL Server databases and the three PostgreSQL databases. + + ```xml + + + ``` + + Use the same SQL Server connection strings as your Quali Server. + +2. Run `QualiSystems.SqlServerToPostgresqlMigrator.exe` from a command prompt and follow the prompts. The tool: + 1. Tests the connections. If some databases can't be reached, it offers to migrate only the others. InSight can only be migrated together with Results. + 2. Waits until no Quali Server is using the Quali database. + 3. Creates the CloudShell schema on PostgreSQL and compares it with SQL Server. + 4. Empties the PostgreSQL tables, copies all rows, and shows the progress. + 5. Resets the PostgreSQL ID sequences. + + :::warning + If the tool reports that your SQL Server databases haven't been upgraded to its version and offers to upgrade them, don't accept. Press Ctrl+C, run the Quali Server Configuration Wizard on SQL Server, and then run the tool again. + ::: + +3. When the tool prints `Migration process is finished`, remove the superuser rights: + + ```sql + ALTER ROLE cloudshell NOSUPERUSER; + ``` + +4. Run the Quali Server Configuration Wizard. In the database step, select **PostgreSQL**, point it to the migrated databases, and don't create new databases. +5. Start the Quali Server, log in to CloudShell, and check your blueprints, resources, sandboxes and users. + +## If the migration fails + +- Your SQL Server data isn't changed, unless you accepted the upgrade offer above. Fix the cause and run the tool again. Each run copies everything from the start. +- If the tool reports differences between the SQL Server and PostgreSQL catalogs, check that the tool's version matches your CloudShell version, and contact Quali Support with the full console output. +- If the copy fails with a permission error on `session_replication_role`, the PostgreSQL login doesn't have superuser rights. diff --git a/docs/install-configure/cs-system-requirements/database-prereq.md b/docs/install-configure/cs-system-requirements/database-prereq.md index 89b1b85978..f54f8834be 100644 --- a/docs/install-configure/cs-system-requirements/database-prereq.md +++ b/docs/install-configure/cs-system-requirements/database-prereq.md @@ -98,12 +98,12 @@ Quali Server, SQL server or SQL Express, and the CloudShell applications can be Starting with CloudShell 2025.1, PostgreSQL is supported as an alternative relational database backend, **in Beta (evaluation only)**. It is intended for evaluation and is not yet recommended for production deployments. Microsoft SQL Server remains the default, fully supported relational database backend. ::: -To evaluate CloudShell on PostgreSQL, or to migrate an existing CloudShell SQL Server database to PostgreSQL, **contact Quali Support for evaluation access**. Quali Support will provide the required components, supported-version details, and setup and migration guidance for your CloudShell version. +To install CloudShell on PostgreSQL, select **PostgreSQL** in the database step of the Quali Server Configuration Wizard. To move an existing CloudShell from SQL Server to PostgreSQL, see [Migrating the Databases from SQL Server to PostgreSQL](../cloudshell-suite/backup-restore/migrate-sql-server-to-postgresql.md). Notes: - PostgreSQL replaces only the SQL Server relational backend. CloudShell still uses MongoDB for the data described in [Select the Database Type](../cloudshell-suite/complete-install/install-cloudshell/select-database-type/index.md), regardless of whether the relational backend is SQL Server or PostgreSQL. -- Because this feature is in Beta, the exact supported PostgreSQL versions and configuration steps depend on your CloudShell version and are provided by Quali Support as part of evaluation access. +- For the PostgreSQL versions supported by your CloudShell version, contact Quali Support. ## Additional requirements diff --git a/docs/troubleshooting/database-and-permissions/share-database-with-support.md b/docs/troubleshooting/database-and-permissions/share-database-with-support.md new file mode 100644 index 0000000000..6bbf874271 --- /dev/null +++ b/docs/troubleshooting/database-and-permissions/share-database-with-support.md @@ -0,0 +1,25 @@ +--- +sidebar_position: 6 +--- + +# Sharing Your Databases with Quali Support + +Some issues can only be investigated with a copy of your CloudShell databases. Before you share them, remove personal data and secrets with the CloudShell sanitization scripts. Quali Support provides the scripts on request. + +The scripts cover the Quali, QualiResults and QualiInsight databases, on SQL Server or PostgreSQL, and the CloudShell MongoDB databases. They: + +- Replace user names and display names with `user`, consistently across all the databases, and user emails with `user@example.invalid`. +- Replace email addresses in free text, such as the event log, command output, and reservation descriptions. +- Clear stored passwords, API tokens, access keys, repository credentials, and machine names. +- Set the password of `admin` and of every anonymized user to a known value, so Quali Support can log in as any user to reproduce the issue. + +:::danger +Run the scripts only on restored copies of your databases, never on your production databases. The changes can't be undone. The scripts refuse to run until you confirm the databases are copies, or while a Quali Server uses them. +::: + +**To share your databases:** + +1. Back up the databases. See [Backing Up CloudShell Databases](../../install-configure/cloudshell-suite/backup-restore/backup-cs-db.md). +2. Restore the backups under new names, on a server that no Quali Server is connected to. +3. Run the sanitization scripts on the restored copies, as explained in the README that comes with them. +4. Back up the sanitized copies, and send the backups to Quali Support. From 2d995ee1c0d61ef2f14ae87ff37deedb2bb91af3 Mon Sep 17 00:00:00 2001 From: "QUALISYSTEMS\\nahum-t" Date: Tue, 6 Oct 2026 16:28:31 +0300 Subject: [PATCH 2/2] Spellcheck: backtick connection names, add anonymized/sanitization/InSight to wordlist Co-Authored-By: Claude Opus 5.5 --- .wordlist.txt | 5 +- .../migrate-sql-server-to-postgresql.md | 174 +++++++++--------- 2 files changed, 91 insertions(+), 88 deletions(-) diff --git a/.wordlist.txt b/.wordlist.txt index b41cf2871f..25f0619c4b 100644 --- a/.wordlist.txt +++ b/.wordlist.txt @@ -1665,4 +1665,7 @@ Unresolving affordance deployable plugable -unresolve \ No newline at end of file +unresolve +anonymized +sanitization +InSight diff --git a/docs/install-configure/cloudshell-suite/backup-restore/migrate-sql-server-to-postgresql.md b/docs/install-configure/cloudshell-suite/backup-restore/migrate-sql-server-to-postgresql.md index 2e4f540a44..de20c6c1c6 100644 --- a/docs/install-configure/cloudshell-suite/backup-restore/migrate-sql-server-to-postgresql.md +++ b/docs/install-configure/cloudshell-suite/backup-restore/migrate-sql-server-to-postgresql.md @@ -1,87 +1,87 @@ ---- -sidebar_position: 4 ---- - -# Migrating the Databases from SQL Server to PostgreSQL - -This article explains how to move an existing CloudShell from Microsoft SQL Server to PostgreSQL, using the CloudShell database migration tool. The tool copies the three CloudShell relational databases. MongoDB isn't affected: CloudShell keeps using the same MongoDB after the migration. - -| Database (default name) | SQL Server connection name | PostgreSQL connection name | -|---|---|---| -| Quali | TestShell | TestShellPostgres | -| QualiResults | TestShellResult | TestShellResultPostgres | -| QualiInsight | InSightResult | InSightResultPostgres | - -:::tip -Migrate a copy of your environment first, and keep your SQL Server databases until you have validated CloudShell on PostgreSQL. -::: - -## Getting the tool - -The tool's version must match your installed CloudShell version exactly. The tool prints its version when it starts, but it doesn't block a mismatch, and a mismatched tool creates a database schema that doesn't match your Quali Server. - -- **CloudShell 2026.2 and later:** the CloudShell installation package includes the matching tool, in `CloudShell\Utilities\SqlServerToPostgresqlMigrator`. -- **Earlier versions:** contact Quali Support for the tool that matches your version. - -## Before you start - -1. Make sure CloudShell works normally on SQL Server, and that the Quali Server Configuration Wizard has completed. -2. Back up the SQL Server databases and the MongoDB databases. See [Backing Up CloudShell Databases](./backup-cs-db.md). -3. Stop the Quali Server service, and close any running Configuration Wizard. -4. On a PostgreSQL 12 or later server, create three new, empty databases, owned by the login CloudShell will use. Use lowercase names. For example, in `psql`: - - ```sql - CREATE DATABASE quali WITH OWNER = cloudshell ENCODING = 'UTF8'; - CREATE DATABASE qualiresults WITH OWNER = cloudshell ENCODING = 'UTF8'; - CREATE DATABASE qualiinsight WITH OWNER = cloudshell ENCODING = 'UTF8'; - ``` - - Don't create the databases with the Configuration Wizard. The tool expects them to be empty. - -5. Give that login superuser rights for the duration of the migration. The tool needs them to copy the data. Run the tool with the same login CloudShell will use, so CloudShell owns the tables: - - ```sql - ALTER ROLE cloudshell SUPERUSER; - ``` - -6. Make sure the PostgreSQL server has enough disk space for the full data size of the SQL Server databases. -7. Run the tool from a machine with .NET Framework 4.8 and a fast, reliable connection to both database servers. The Quali Server machine is usually the best choice. - -## Running the migration - -1. Copy the `SqlServerToPostgresqlMigrator` folder to the machine, and set the six connection strings in `connections.config`: the three SQL Server databases and the three PostgreSQL databases. - - ```xml - - - ``` - - Use the same SQL Server connection strings as your Quali Server. - -2. Run `QualiSystems.SqlServerToPostgresqlMigrator.exe` from a command prompt and follow the prompts. The tool: - 1. Tests the connections. If some databases can't be reached, it offers to migrate only the others. InSight can only be migrated together with Results. - 2. Waits until no Quali Server is using the Quali database. - 3. Creates the CloudShell schema on PostgreSQL and compares it with SQL Server. - 4. Empties the PostgreSQL tables, copies all rows, and shows the progress. - 5. Resets the PostgreSQL ID sequences. - - :::warning - If the tool reports that your SQL Server databases haven't been upgraded to its version and offers to upgrade them, don't accept. Press Ctrl+C, run the Quali Server Configuration Wizard on SQL Server, and then run the tool again. - ::: - -3. When the tool prints `Migration process is finished`, remove the superuser rights: - - ```sql - ALTER ROLE cloudshell NOSUPERUSER; - ``` - -4. Run the Quali Server Configuration Wizard. In the database step, select **PostgreSQL**, point it to the migrated databases, and don't create new databases. -5. Start the Quali Server, log in to CloudShell, and check your blueprints, resources, sandboxes and users. - -## If the migration fails - -- Your SQL Server data isn't changed, unless you accepted the upgrade offer above. Fix the cause and run the tool again. Each run copies everything from the start. -- If the tool reports differences between the SQL Server and PostgreSQL catalogs, check that the tool's version matches your CloudShell version, and contact Quali Support with the full console output. -- If the copy fails with a permission error on `session_replication_role`, the PostgreSQL login doesn't have superuser rights. +--- +sidebar_position: 4 +--- + +# Migrating the Databases from SQL Server to PostgreSQL + +This article explains how to move an existing CloudShell from Microsoft SQL Server to PostgreSQL, using the CloudShell database migration tool. The tool copies the three CloudShell relational databases. MongoDB isn't affected: CloudShell keeps using the same MongoDB after the migration. + +| Database (default name) | SQL Server connection name | PostgreSQL connection name | +|---|---|---| +| Quali | `TestShell` | `TestShellPostgres` | +| QualiResults | `TestShellResult` | `TestShellResultPostgres` | +| QualiInsight | `InSightResult` | `InSightResultPostgres` | + +:::tip +Migrate a copy of your environment first, and keep your SQL Server databases until you have validated CloudShell on PostgreSQL. +::: + +## Getting the tool + +The tool's version must match your installed CloudShell version exactly. The tool prints its version when it starts, but it doesn't block a mismatch, and a mismatched tool creates a database schema that doesn't match your Quali Server. + +- **CloudShell 2026.2 and later:** the CloudShell installation package includes the matching tool, in `CloudShell\Utilities\SqlServerToPostgresqlMigrator`. +- **Earlier versions:** contact Quali Support for the tool that matches your version. + +## Before you start + +1. Make sure CloudShell works normally on SQL Server, and that the Quali Server Configuration Wizard has completed. +2. Back up the SQL Server databases and the MongoDB databases. See [Backing Up CloudShell Databases](./backup-cs-db.md). +3. Stop the Quali Server service, and close any running Configuration Wizard. +4. On a PostgreSQL 12 or later server, create three new, empty databases, owned by the login CloudShell will use. Use lowercase names. For example, in `psql`: + + ```sql + CREATE DATABASE quali WITH OWNER = cloudshell ENCODING = 'UTF8'; + CREATE DATABASE qualiresults WITH OWNER = cloudshell ENCODING = 'UTF8'; + CREATE DATABASE qualiinsight WITH OWNER = cloudshell ENCODING = 'UTF8'; + ``` + + Don't create the databases with the Configuration Wizard. The tool expects them to be empty. + +5. Give that login superuser rights for the duration of the migration. The tool needs them to copy the data. Run the tool with the same login CloudShell will use, so CloudShell owns the tables: + + ```sql + ALTER ROLE cloudshell SUPERUSER; + ``` + +6. Make sure the PostgreSQL server has enough disk space for the full data size of the SQL Server databases. +7. Run the tool from a machine with .NET Framework 4.8 and a fast, reliable connection to both database servers. The Quali Server machine is usually the best choice. + +## Running the migration + +1. Copy the `SqlServerToPostgresqlMigrator` folder to the machine, and set the six connection strings in `connections.config`: the three SQL Server databases and the three PostgreSQL databases. + + ```xml + + + ``` + + Use the same SQL Server connection strings as your Quali Server. + +2. Run `QualiSystems.SqlServerToPostgresqlMigrator.exe` from a command prompt and follow the prompts. The tool: + 1. Tests the connections. If some databases can't be reached, it offers to migrate only the others. InSight can only be migrated together with Results. + 2. Waits until no Quali Server is using the Quali database. + 3. Creates the CloudShell schema on PostgreSQL and compares it with SQL Server. + 4. Empties the PostgreSQL tables, copies all rows, and shows the progress. + 5. Resets the PostgreSQL ID sequences. + + :::warning + If the tool reports that your SQL Server databases haven't been upgraded to its version and offers to upgrade them, don't accept. Press Ctrl+C, run the Quali Server Configuration Wizard on SQL Server, and then run the tool again. + ::: + +3. When the tool prints `Migration process is finished`, remove the superuser rights: + + ```sql + ALTER ROLE cloudshell NOSUPERUSER; + ``` + +4. Run the Quali Server Configuration Wizard. In the database step, select **PostgreSQL**, point it to the migrated databases, and don't create new databases. +5. Start the Quali Server, log in to CloudShell, and check your blueprints, resources, sandboxes and users. + +## If the migration fails + +- Your SQL Server data isn't changed, unless you accepted the upgrade offer above. Fix the cause and run the tool again. Each run copies everything from the start. +- If the tool reports differences between the SQL Server and PostgreSQL catalogs, check that the tool's version matches your CloudShell version, and contact Quali Support with the full console output. +- If the copy fails with a permission error on `session_replication_role`, the PostgreSQL login doesn't have superuser rights.