Environment
- 64 bit Operating system: Windows Server 2022+ (Clean build recommended)
- Microsoft .NET 10 Hosting Bundle (the installer adds this if it is missing)
- PostgreSQL 18 is included in the installer. To use your own PostgreSQL server instead, it must be version 17.4 or later
- Recommended system specification - Primary Server:
- 3GHz 8 Core Processor or better
- 200+GB Disk space
- 16+GB Memory
- Recommended system specification - Secondary Server(s):
- 3GHz 4 Core Processor or better
- 100GB Disk space
- 8+GB Memory
Secondary servers keep migration trace logs on disk and temporarily store large files while they are exported, including Google Vault export files. For large migrations, allow 200GB or more of disk on each secondary server. To keep trace logs off the servers, see Centralised Self-Hosted Logging.
Any available Windows updates should be installed and the machine rebooted prior to installation, and a full system administrator is recommended for installation.
Large Migrations Infrastructure
A 'large migration' is considered to be a migration of over 25,000 users or 10 million objects. Please ensure you refer to the following article for additional considerations:
Contact the support team if you are storing Drive document mappings and running a large migration.
If your project is much larger than the figures in this article, or you plan to raise the number of users per secondary server above the default of 20, contact the team for guidance on your infrastructure.
Separate PostgreSQL and Redis servers
For large migrations we recommend moving PostgreSQL and Redis off the primary server and onto their own servers. Either can run on Windows or Linux. The figures below are starting points. What you need depends on how many items you migrate and how many users you migrate at the same time, so monitor disk and memory during your first batches and adjust.
PostgreSQL server
PostgreSQL stores your project configuration, user lists, reports and a result record for every item migrated. Its disk use grows with the total number of items in the migration.
- 8 core processor or better
- 16+GB memory
- SSD storage: allow 50GB, plus 3GB for every million items you plan to migrate
| Items migrated | Suggested disk |
|---|---|
| 10 million | 80GB |
| 50 million | 200GB |
| 100 million | 350GB |
Migrations with a high failure rate, or Drive and SharePoint content with very long file paths, use more space per item. Leave headroom in those cases.
These figures cover one run of each batch. CloudM Migrate also keeps a copy of each batch's results for its most recent runs, set by Run History Retention Count in the general advanced settings (default 5). Each run kept adds roughly 1.5GB per million items, so a batch that has been run five or more times needs up to about 10GB per million items. Lower the setting to reduce this, or set it to 0 to keep no run history.
Redis server
Redis holds working data for the users currently being migrated. When a user's migration finishes, their data is written back to the primary server and removed from Redis. So Redis memory depends on how many users you migrate at the same time and how many items each of those users has, not on the size of the whole migration. Each secondary server migrates 20 users at a time by default.
- 4 core processor or better
- Memory: allow 2GB for every 10 million items being migrated at the same time, with a minimum of 8GB. Items being migrated at the same time is the number of users migrating at once multiplied by the items each of those users has
- 50GB disk space
Allow more disk if you give Redis more than 24GB of memory (at least twice the memory), if you turn on Redis append-only persistence (at least three times the memory), or if the server runs anything other than Redis.
| Secondary servers | Users migrating at once | Items per user | Suggested memory |
|---|---|---|---|
| 1 | 20 | 500,000 | 8GB |
| 3 | 60 | 500,000 | 8GB |
| 6 | 120 | 500,000 | 16GB |
| 10 | 200 | 500,000 | 24GB |
| 10 | 200 | 1 million | 48GB |
If you set a memory limit on Redis, keep the eviction policy at noeviction, which is the Redis default. Any other policy lets Redis discard migration data when it is full, which can cause items to be migrated again on the next run.
Always set a password on a separate Redis server. For setup steps, see Installing a separate Redis server (GCP).
CloudM Migrate migrates data over HTTPS, and therefore, port 443 must be open.
Please refer to the following article for more information regarding Ports: here
For how the servers fit together, what runs on each one and where your data goes during a migration, see CloudM Migrate Self Hosted Architecture & Security.