Google Cloud Storage Setup for CloudM Archive

Prior to attempting these steps, please ensure that you have a valid Google Cloud Billing account and the permissions necessary to create or manage a GCP project.

Also, see here for more information on Cloud Storage pricing.

You should also make sure you have followed all of our Archive Prerequisites stated for Google. 

• Powershell Method
• Manual Method
• CloudM Archive GCS storage bucket requirements

Powershell Method

This method uses a Powershell script to automate the majority of the process required to configure a Google Cloud Storage bucket for use in CloudM Archive.

It is easier, quicker and less error prone than the

Before you start, you will need:

• An account in GCP with permissions to create a project (resourcemanager.projects.create role) or owner on existing project,
• The ability to run Powershell Script as Administrator,
• A browser window open and authenticated into the GCP tenant. This must be the last browser tab you have used.

To run the Powershell:

1. Install Google Cloud SDK using the instructions provided by Google here,
2. Ensure that the Google Cloud SDK is initialized by running the “gcloud init” command, and follow the instructions.
3. Once Google Cloud SDK has finished initializing, download the GCP_Storage_Configuration.ps1 file  to your desktop.
   • You can also copy the script into a text editor, saving it as GCP_Storage_Configuration and applying the Windows Powershell file type to it.
4. Click on your Desktop Search icon (next to the Start Icon) and search for Windows Powershell.
5. Select Run as Administrator.
6. Locate the GCP_Storage_Configuration.ps1 file you previously downloaded and copy the folder path to the file.
7. In the Windows Powershell window, enter CD and a space, paste the folder path from the previous step. It will look similar to:
   • CD C:\Users\(your name)\Desktop
8. Enter the text .\GCP_Storage_Configuration.ps1 and press enter.

9. On the Project ID line, enter a unique Project ID name.
   • ProjectId must be a unique string of 6 to 30 lowercase letters, digits, or hyphens. It must start with a lower case letter, followed by one or more lower case alphanumerical characters that can be separated by hyphens. It cannot have a trailing hyphen.
10. On the Service Account ID line, enter a unique Service Account name. You can use the same name as the Project ID, or use the same naming conventions.
11. On the Region line, enter either us-central1 or europe-west1, depending on the region that you want to store your data in.
12. On the BucketName line, enter a name for your storage bucket, adhering to the naming conventions outlined in this article from Google. You will need to remember the Bucket Name later to configure the Archive features within CloudM Manage.
13. The Powershell script will now create the Service Account and Bucket. This may take a few minutes.
14. Once the Powershell has stopped, you can add a KeyName. This step is optional, but, if you do enter a Key Name, it must be between 6 and 30 letters, digits, hyphens or underscores. It must start with a lower case letter, followed by one or more alphanumerical characters that can be separated by hyphens or underscores. It cannot have a trailing hyphen or underscore. 
15. Optionally, set the StorageClass for the Bucket Storage. It must be one of ‘STANDARD’, ‘NEARLINE’, ‘COLDLINE’ or ‘ARCHIVE’.
16. Optionally, set the ServiceAccountKeyType. It must be ‘json’.
17. Now, on the Output Path line, specify where the JSON Key and Log will be exported to on your computer (e.g. C:\\CloudM GCPConfig). The path will default to “$Home\GCPConfig”.
18. The Powershell will run and provide the following details (that you should note down)
    • Service Account Email Address
    • Path to Service Account Json key
    • Bucket Url
    • KMs Key Path
19. Follow the additional steps given in the output of the Powershell script.

Manual Process

Obtaining the Service Account Key File

1. Go to https://console.cloud.google.com/
2. Ensure your project is set at the top of the screen.
3. To create the Service Account Key File, go to IAM & Admin > Service Accounts from the left menu
4. Go to any active service account (preferable) or create a new one.
5. Select the Keys tab.
6. Select Add Key > Create New Key > JSON

• You will need to upload the Service Account JSON key file later when configuring the Archive feature in CloudM Automate. Keep the file confidential as it allows full access to your archive.

Creating a Key Ring and Key (optional)

1. Search for KMS in the search field, or select Security > Key Management
2. Select Create Key Ring. The name can be set to the same as the bucket name.
3. Ensure the keyring location matches the bucket location (europe-west1 or us-central1), and remember which location you set as you will need it when configuring Archive in CloudM Manage
4. Click Next, 
5. On the Create Key screen, use the same Key name as the Key ring name (optional),
6. Leave all the other settings as default except Rotation Period,
7. Set Rotation Period to Never (manual rotation) and select Create.
8. Copy the Resource name of the KMS key that you have just created (by selecting the 3 dot ellipsis under Actions and clicking Copy resource name) 

• You will asked for the Resource name later to configure the Archive feature within CloudM Manage (if you create a Key Ring and Key)

The key ring and key are used to encrypt the blob storage and should not be removed or deleted at any point. If they are removed or deleted, the blobs in the storage bucket will become inaccessible.

Creating a Bucket

1. From the Navigation menu (accessed by selecting the "Hamburger" Menu icon in the top left of the screen), go to Cloud Storage > Bucket and select Create Bucket > Set to specific region (europe-west1 or us-central1), as set in step 3 of the Creating a Key Ring section above (if completed).
2. Make sure to use the “archive” prefix for the bucket name (e.g. archive-test) so that you can quickly identify the bucket.

• You will need the bucket name later to configure the Archive feature within CloudM Manage,

Adding permissions to the Service Account

The owner is the only one with permissions to add members, and you will need someone to do this for you if you do not have the relevant permissions.

1. Go to IAM & Admin > Service Accounts and select the service account that you created the Service Account JSON key file on,
2. Copy the Email address in the Service Account Details section,
3. Go to Cloud Storage > Bucket and then select the bucket you created earlier,
4. Click on the Permissions tab and select Add a permission,
5. Paste the email from step 1 in to the members field,
6. Add Storage Admin and Storage Object Admin roles and Save,
7. For CloudM Backup, you will need to add an extra role (Monitoring Viewer) to the Service Account.
   • Go to IAM & Admin > IAM,
   • Select the edit icon next to the required Service Account,
   • Add the Monitoring Viewer role, if it does not already exist.
   • Select Save to confirm.

Adding permissions to the Storage Bucket and KMS CryptoKey (optional)

The owner is the only one with permissions to add members, and you will need someone to do this for you if you do not have the relevant permissions.

1. Go to Cloud Storage > Settings,
2. Copy the Service Account email (under the Cloud Storage Service Account section) and add the roles in the previous step to this email as well,
3. Click on the KMS key you created in Security > Cryptographic Keys. On the next page, where only the specified KMS Key should be listed, click on it again. 
4. Click on Permissions > Add Member, in the panel on the right side of the screen.

• Click on the Show Info Panel option if you cannot see the panel.

CloudM Archive GCS storage bucket requirements

Your CloudM Archive storage bucket needs to be either US or Europe and it has to be in the same region as your Google Workspace Tenant. It cannot be the same bucket that you use for CloudM Backup. 

CloudM Archive supported GCS regions (as of 16th Nov 2023):
US // multi-region
NAM4 // dual region
US-CENTRAL1
US-WEST1 
US-WEST2
US-WEST3
US-WEST4
US-EAST1
US-EAST4
US-EAST5
US-SOUTH1
NORTHAMERICA-NORTHEAST1
NORTHAMERICA-NORTHEAST2
EU // multi-region
EUR4 // dual region
EUROPE-WEST1
EUROPE-WEST2
EUROPE-WEST3
EUROPE-WEST4
EUROPE-WEST6
EUROPE-WEST8 
EUROPE-WEST9
EUROPE-WEST12
EUROPE-SOUTHWEST1
EUROPE-CENTRAL2
EUROPE-NORTH1