FAQ: Discovering Deleted but Retained Data in Google Vault
CloudM Archive and CloudM Archive Vault cover different data, and are scoped separately:
- Archive archives the Gmail, Drive and Chat data held in the active account to your Google Cloud Storage bucket. Anything the user can still reach, including items sitting in Trash, is covered here.
- Archive Vault archives only the retained deleted data: mail and Drive items the user has cleared from Trash and can no longer reach, which Google Vault may still be holding, depending on the retention rules and holds configured in that environment.
The two do not overlap, so before adding the Archive Vault step to a workflow it is worth establishing two things:
- Whether there are any users whose data could be Vault archived at all.
- How much retained deleted data those users actually hold.
This covers both.
Step 1: Identify Vault-licensed users with GAM
GAM (Google Apps Manager) is the best way to get this list. The Admin Console does not give you a clean, exportable list of who holds a Vault license, so anything assembled by hand will be incomplete on a tenant of any size.
The command depends on how Vault is licensed, so check the subscription first.
Enterprise customers, where Vault is bundled into the core license:
gam redirect csv ./vault_users.csv print licenses products Google-Apps,101034
Non-Enterprise customers, where Vault is a separate add-on:
gam redirect csv ./vault_users.csv print licenses products Google-Vault
The ./ writes vault_users.csv into GAM’s working directory, which is the GAM folder specified when you installed GAM.
Note: running the wrong command for the subscription type returns an empty or partial file rather than an error. Check the row count looks right for the tenant before going further. If the file is empty on both commands, nobody is Vault-licensed and there is nothing for Archive Vault to do.
Step 2: Prepare the user list
The Google Vault Matter search expects every address on a single line, as one comma-separated list. Open vault_users.csv in Google Sheets or Excel and join the email column with =TEXTJOIN(", ", TRUE, A2:A500), adjusting the range to match. Copy the resulting string.
Step 3: Search the Google Vault Matter for retained deleted data
The same filter works for both Gmail and Drive, but the services are searched separately, so run this twice.
- In Google Vault, open Matters and create a Matter, or open an existing one.
- Set the service to Gmail, or to Drive on the second pass.
- Set the source to All data.
- Set the entity to Specific accounts and paste the list from Step 2.
- Add
label:^deletedto the search terms. This returns items deleted and cleared from Trash that Vault still retains, which is exactly the data the Archive Vault step captures. - Run the search.
Note for the Drive pass: set Shared drives to Not included. Shared Drive data is not in scope for CloudM Archive, so leaving it in inflates the figure with data the Archive Vault step will never touch.
Step 4: Turn the results into numbers
There is no Count button in the Google Vault Matter search. Counting is done one of two ways.
Via the Vault API. The matters.count method returns the number of matching messages without running an export. It is considerably faster and avoids the export overhead entirely, but it supports Gmail and Groups only (Groups is not in scope for CloudM Archive), not Drive.
Via an export. This is the only route for Drive, and the only route at all if you are not using the API.
- With your search results on screen, click Export.
- Go to the Exports tab inside the Matter. Exports are built in the background, so it will not be ready immediately.
- Once the status shows it as complete, click Download.
- Open the metadata file included with the export in Google Sheets or Excel. It lists one row per exported item.
- Select the first populated column and read the row count, remembering to discount the header row.
=COUNTA(A2:A)gives the same figure.
Note: exporting purely to obtain a count can be expensive. Large exports are slow and are subject to Vault API quotas that cap concurrent exports across the whole organisation, so scope the accounts and the date range down first. See Google’s Vault API limits, and Archive Vault FAQ: Google Vault Export Limits before running several at once.
Things to be aware of
- Retention is what makes any of this exist. Vault defers the purge of a deleted item only while a retention rule or a hold covers it. A search returning nothing usually means nothing is retained in that environment rather than that the search is wrong. Retention configuration is the customer’s own.
- Vault reports item counts, never data size. Neither the interface nor the API returns a volume figure, so anything needing GB has to come from the Admin Console usage columns. See Google Data Discovery.
- Drive counts files, not folders. Vault search excludes folders and Drive shortcuts, so the figure will not reconcile with a folder-aware count taken elsewhere.
- Expect large result sets. Google notes that searches on system-generated labels can return an excessive number of results. Narrow with a date range if the numbers are unmanageable.
-
A related term worth knowing:
label:^vault_deletionfinds mail removed from mailboxes because a retention rule expired, rather than deleted by the user.
See also: Google Vault Data Discovery | Archive Vault FAQ: Google Vault Export Limits