Database Migration Manager

Version: 1.9.0

Overview

Database Migration Manager is a containerized, interactive database migration tool designed to simplify the process of moving data between database servers. Whether you're migrating between different hosts, creating database backups, or replicating databases for testing environments, this tool provides a unified, user-friendly interface for managing migrations across multiple database engines.

Built entirely with Bash and Dialog, the tool runs inside a Docker container with a text-based user interface (TUI), eliminating the need for complex installations or dependencies on your host system. All database operations are executed using official Docker images (MySQL, PostgreSQL, SQL Server), ensuring consistency and reliability across different environments.

The tool handles the complete migration workflow: connecting to source and destination databases, exporting data in appropriate formats, and importing it safely with proper validation at each step. All dump files are stored in a persistent Docker volume, ensuring your data remains safe even after container restarts.

Key Features

Multi-Database Engine Support: MySQL 8.0, PostgreSQL 16, and SQL Server (via SqlPackage)
Interactive TUI: User-friendly dialog-based interface for configuration and operation
Docker-First Architecture: Runs entirely in containers, no local database tools required
Persistent Storage: Dumps stored in named Docker volumes for data safety
Complete Migration Workflow: One-click dump and restore operations
Connection Testing: Validate database credentials before migration
Configuration Management: Save and reuse database connection profiles
Cross-Platform: Works on Linux, macOS, and Windows (via Git Bash or WSL)

Architecture

The tool uses a nested Docker architecture:

Main Container (database-migration-manager): Runs the TUI application and orchestrates migrations
Database Containers: Spawned dynamically for each operation (dump/load) using official images
Shared Resources:
- /dumps volume: Persistent storage for database dumps
- Docker socket: Allows main container to spawn database containers
- Host networking: Direct access to database servers

Requirements

Docker: Version 20.10 or higher
Docker Compose (optional): For advanced setups
Operating System:
- Linux/Unix: Native support
- Windows: Git Bash, WSL, or MSYS2
- macOS: Native support

Installation

Clone the repository:

git clone <repository-url>
cd Database-Migration-Manager

Ensure Docker is running:
```
docker --version
docker ps
```
Set up SQL Server dependencies (if using SQL Server migrations):
- Download SqlPackage from Microsoft's official site
- Extract to dependencies/sqlpackage/ directory
- The structure should be: dependencies/sqlpackage/sqlpackage.dll
Make script executable:
```
chmod +x run.sh
```

Usage

Universal Command (Auto-detects OS)

./run.sh

The script automatically detects your operating system and runs the appropriate launcher:

Linux/macOS → Uses Unix launcher
Windows (Git Bash/MSYS/MinGW) → Uses Windows launcher

Manual Execution (Optional)

If you prefer to run the OS-specific launchers directly:

Linux/Unix/macOS:

./lib/run-docker-unix.lib.sh

Windows (Git Bash):

./lib/run-docker-windows.lib.sh

First Run

On first execution, the tool will:

Detect your operating system
Build the Docker image (takes 2-3 minutes)
Create the db-migration-dumps volume
Launch the interactive TUI

Main Menu Options

1. MIGRATE - Complete Migration

Performs a full database migration (dump + restore) in one operation:

Select source database type (MySQL, PostgreSQL, SQL Server)
Enter source connection details (host, port, user, password, database)
Select destination database type
Enter destination connection details
Tool automatically:
- Exports source database to dump file
- Imports dump into destination database
- Reports success/failure

Use case: Quick migration between two servers

2. DUMP - Export Database

Exports a database to a dump file with flexible options:

Dump Type Selection (MySQL & PostgreSQL): Before creating the dump, you can choose what to include:

Both (Recommended): Complete dump with structure and data
Structure only: Schema, tables, views, procedures, triggers (no data)
Data only: Table data only (no schema definitions)

Note: SQL Server always dumps both structure and data (BACPAC format limitation)

Output Formats:

MySQL: Creates .sql dump using mysqldump
PostgreSQL: Creates custom format dump using pg_dump -F c
SQL Server: Creates .bacpac file using SqlPackage

Dumps are stored in /dumps (Docker volume db-migration-dumps) with descriptive filenames:

mysql-mydb-full-20260217-143022.txt (both)
mysql-mydb-structure-20260217-143022.txt (structure only)
mysql-mydb-data-20260217-143022.txt (data only)

Use cases:

Structure only: Schema comparisons, deploying empty databases, documentation
Data only: Migrating data to existing schema, data updates
Both: Full backups, complete migrations, versioning

3. LOAD - Import Database

Imports a previously created dump file:

Select dump file from list
Choose destination database type
Enter destination connection details
Important: For MySQL, the destination database must already exist
Tool imports data safely

Use case: Restore backup, clone database, or import to different engine

4. TEST - Test Connection

Validates database credentials without performing any operations:

Tests network connectivity
Verifies authentication
Lists available databases (for PostgreSQL/MySQL)

Use case: Debugging connection issues before migration

5. CONFIG - Configuration

Manages database connection settings:

Database Type: Choose MySQL, PostgreSQL, or SQL Server
SOURCE Configuration: Connection details for source database
DESTINATION Configuration: Connection details for destination database
Complete Setup: Step-by-step wizard for all settings
View Configuration: Display current settings

Note: Dump files are automatically stored in /dumps (fixed Docker volume location) and cannot be changed. Files are auto-named as <db-engine>-<timestamp>.txt.

Configuration File

Settings are stored in .config file (automatically created):

DB_TYPE=mysql
SRC_HOST=localhost
SRC_PORT=3306
SRC_USER=root
SRC_PASS=password
SRC_DB=mydb
DST_HOST=localhost
DST_PORT=3306
DST_USER=root
DST_PASS=password
DST_DB=mydb_copy

Important: The dump directory is always /dumps and is not stored in the configuration file. This is the Docker volume mount point and ensures all dumps persist in the db-migration-dumps volume.

Dump File Formats

MySQL

Format: SQL text file (.sql)
Tool: mysqldump
Options:
- Structure only: --no-data --routines --triggers --events
- Data only: --no-create-info --skip-triggers --skip-routines --skip-events --complete-insert
- Both: --single-transaction --routines --triggers --events

PostgreSQL

Format: Custom format (.dump)
Tool: pg_dump -F c
Options:
- Structure only: --schema-only
- Data only: --data-only
- Both: Default behavior (all included)
Features: Binary format, includes --clean and --if-exists on restore

SQL Server

Format: BACPAC (.bacpac)
Tool: SqlPackage
Options: Always includes both structure and data (BACPAC format limitation)
Features: Schema + data, portable across SQL Server versions
Note: Creates .txt reference file containing path to actual .bacpac

Volume Management

List Dumps

docker volume inspect db-migration-dumps
docker run --rm -v db-migration-dumps:/dumps alpine ls -lh /dumps

Backup Volume

docker run --rm -v db-migration-dumps:/dumps -v $(pwd):/backup alpine \
  tar czf /backup/dumps-backup.tar.gz /dumps

Restore Volume

docker run --rm -v db-migration-dumps:/dumps -v $(pwd):/backup alpine \
  tar xzf /backup/dumps-backup.tar.gz -C /

Delete Volume

docker volume rm db-migration-dumps

Troubleshooting

"Docker not found"

Ensure Docker is installed and running
Check with: docker ps

"Permission denied on /var/run/docker.sock"

Add your user to docker group: sudo usermod -aG docker $USER
Log out and back in

"SQLPACKAGE_DIR environment variable not set"

Ensure SqlPackage is in dependencies/sqlpackage/
Check launcher scripts pass -e SQLPACKAGE_DIR

"Dump directory must be /dumps"

Configuration auto-corrects on load
Never manually set DUMP_DIR to non-volume paths

MySQL database does not exist on import

MySQL imports require the destination database to already exist
Create it manually: CREATE DATABASE dbname;
Then run the import operation

MySQL/PostgreSQL connection fails

Verify --network host is working
Test connection from host: mysql -h <host> -P <port> -u <user> -p
Check firewall rules

SQL Server BACPAC import fails

Ensure destination server has sufficient space
Check SQL Server edition compatibility
Review error logs in container output

Changelog

Version 1.9.0

Removed: All emojis from TUI dialog menus and log output for cleaner, more compatible terminal display

Version 1.6.0

New Feature: Dump type selection for MySQL and PostgreSQL
- Choose between structure only, data only, or both before dumping
- Interactive dialog menu for easy selection
- Filenames now include dump type for better organization
Improved: More descriptive dump filenames (-structure-, -data-, -full-)
Enhanced: Better logging messages indicating what's being dumped

Version 1.5.1

Initial stable release
Multi-database support (MySQL, PostgreSQL, SQL Server)
Interactive TUI with Dialog
Docker-first architecture

Project Structure

Database-Migration-Manager/
├── run.sh                     # Universal launcher (auto-detects OS)
├── db-manager.sh              # Main TUI application
├── Dockerfile                 # Container build definition
├── lib/
│   ├── metadata.lib.sh        # Project version and name
│   ├── log.lib.sh             # Logging functions
│   ├── run-docker-unix.lib.sh # Unix/Linux/macOS launcher
│   └── run-docker-windows.lib.sh # Windows launcher
├── operation/
│   ├── mysql-dump.operation.sh
│   ├── mysql-load.operation.sh
│   ├── postgres-dump.operation.sh
│   ├── postgres-load.operation.sh
│   ├── sqlserver-dump.operation.sh
│   └── sqlserver-load.operation.sh
├── dependencies/
│   ├── dialog/                # Dialog binary (bundled)
│   └── sqlpackage/            # SqlPackage (user-provided)
└── README.md

Environment Variables

The following environment variables are automatically set by launch scripts:

DUMPS_VOLUME: Name of Docker volume for dumps (db-migration-dumps)
SQLPACKAGE_DIR: Host path to SqlPackage directory (SQL Server only)
LANG=C.UTF-8: Ensures proper character encoding
LC_ALL=C.UTF-8: Locale settings for internationalization

Security Considerations

Passwords: Passed via environment variables to containers (not stored on disk)
Network: Uses --network host for database access (consider VPN in production)
Docker Socket: Main container has full Docker access (run only trusted code)
Volumes: Dumps stored in named volume (backed up separately)

Limitations

Large Databases: Very large dumps (>10GB) may be slow via stdout/stdin
SQL Server: Requires SqlPackage download (not bundled due to licensing)
Windows Paths: Use Git Bash/WSL, avoid cmd.exe
Network: Requires direct network access to database servers

Contributing

Contributions welcome! Please:

Fork the repository
Create a feature branch
Follow existing code style (Bash best practices)
Test with all three database engines
Submit pull request with clear description

License

[Specify your license here - MIT, Apache 2.0, GPL, etc.]

Support

For issues, questions, or feature requests:

Open an issue on GitHub
Provide Docker version, OS, and error logs
Include .config file contents (redact passwords)

Acknowledgments

Built with Dialog for TUI
Uses official Docker images: MySQL, PostgreSQL, .NET Runtime
SqlPackage by Microsoft

Happy Migrating!

Name		Name	Last commit message	Last commit date
dependencies		dependencies
lib		lib
operation		operation
.gitignore		.gitignore
Dockerfile		Dockerfile
README.md		README.md
db-manager.sh		db-manager.sh
run.sh		run.sh

Latest commit

History

Folders and files

Repository files navigation