- 30 Sep 2024
- 11 Minutes to read
- PDF
Integrate Google Cloud Platform (GCP) with Red Canary
- Updated on 30 Sep 2024
- 11 Minutes to read
- PDF
Integrating Google Cloud Platform (GCP) with Red Canary provides enhanced threat detection and response capabilities for cloud environments. You can gain deeper visibility into cloud activities, identify potential threats faster, and mitigate risks more effectively. To integrate Google Cloud Platform (GCP) with Red Canary, follow the procedure below from beginning to end. Once all steps are completed successfully, the data should be ingested into Red Canary within ~20 minutes.
Note: You can integrate GCP with Red Canary at an organization or project level. Steps are provided for both organizational and project-level integrations.
Prerequisites
You must be subscribed to Red Canary’s Cloud Control Plane license
You must have the following roles in GCP (which are encompassed by Editor and Owner in most cases):
Organization Administrator or Organization Role Administrator
Role Administrator or IAM Admin
Deployment Manager Editor
Integration Resources
Throughout this integration, you will create resources within your GCP environment. These resources enable Red Canary to acquire your logs and monitor your systems.
Pub/Sub topic. This topic, named red-canary-log-export, receives all published messages.
Pub/Sub topic subscription. A red-canary-log-ingest subscription is attached to the ‘red-canary-log-export’ topic and is consumed by Red Canary.
A Google Logging Sink named red-canary-log-sink receives all messages from the GCP Logging Service and publishes them on the red-canary-log-export topic. This Sink will receive messages from all the children’s projects at an organizational level.
A new custom role named red_canary_ingestor provides the necessary permissions for the Red Canary Integration.
A new policy binding assigns the red_canary_ingestor role to Red Canary’s red-canary-gcp-log-ingest@red-canary-platform-production.iam.gserviceaccount.com service account for your organization or project.
A second binding policy assigns the built-in roles/pubsub.subscriber role to the Red Canary service account, enabling Red Canary to subscribe to the subscription.
Step 1: Collect information required for integration
To integrate your Organization with Red Canary, you’ll need to collect a few pieces of information:
Organization ID
Provisioning Project ID
Provisioning Project Number
From your Red Canary homepage, click Integrations.
From the Integrations section, locate and then click Google Cloud Platform.
Enter a name for your new GCP integration.
Choose the scope of your integration by selecting Organization or Project.
Note: If you select Project, click here to continue the Project integration process.
Organization
If you selected Organization, enter your Organization ID.
To locate your Organization ID, navigate to your GCP cloud console and then select your Organization from the resource browser.
Copy your Organization ID, and then paste it into Red Canary.
Enter your Provisioning Project ID.
Note: The Provisioning Project can be any Project within your Organization; you can use an existing Project or create a new one. This setup provisions a Pub/Sub Topic and Subscription within this Project to use as the destination for log routing.
To locate your Provisioning Project ID, navigate to your GCP cloud console and select your Project in the resource explorer.
Copy your Project ID, and then paste it into Red Canary.
From Red Canary, enter any Excluded Projects; this step is optional.
Note: Excluded Projects is a comma-separated list of project ID(s) that will be excluded from the Red Canary scan.
From Red Canary, click Next.
Project
If you selected Project, enter your Project ID.
To locate your Provisioning Project ID, navigate to your GCP cloud console and then select your Project in the resource explorer.
Copy your Project ID, and then paste it into Red Canary.
From Red Canary, click Next.
Step 2: Enable Data Access Logs
Enabling Data Access logs ensures Red Canary can monitor activity in your GCP environment.
From Red Canary, select I need to enable Data Access Logs.
To complete this process, navigate to your GCP Audit Logs.
Select your Organization or Project from the resource browser.
Click Set Default Configuration.
Select Admin Read, Data Read, and Data Write.
Click Save.
From Red Canary, select I’ve enabled Data Access logs.
Note: To prevent unnecessary charges from GCP, ensure to opt out of data storage. GCP automatically directs logs to a default storage sink for storage. For our purposes, we require only the routing of these logs, not their storage.
Click Next.
Step 3: Configure Telemetry routing to Red Canary
Red Canary provides setup files to provision resources in your GCP environment to enable the integration process.
The integration process can be completed in three different ways:
From Red Canary, choose the scope of your integration by selecting Bash or Terraform.
Bash
Click setup files to download the files that Red Canary has provided.
Unzip the files you downloaded in the previous step.
Navigate to your GCP Cloud Shell, and then click Activate Cloud Shell.
Upload the setup files to your GCP Cloud Shell.
Use the following command to grant execution permissions to the setup file:
For Organization
chmod +x organization_provisioning.sh
For Project
chmod +x project_provisioning.sh
Initiate the setup files with the following command to allocate the necessary resources for this integration:
For Organization
./organization_provisioning.sh --organization-id ORGANIZATION_ID --provisioning-project-id PROVISIONING_PROJECT_ID
For Project
./project_provisioning.sh --provisioning-project-id PROJECT_ID
If you receive an Authorization confirmation pop-up, click Authorize.
From Red Canary, select I’ve completed the provisioning steps.
Click Save.
Terraform
If you selected Terraform, click setup files to download the files that Red Canary has provided.
Integrate the Terraform template downloaded in the previous step into your Infrastructure as Code (IaaC) pipeline. Follow your pipeline's standard process for deploying Terraform templates.
It’s essential to edit the terraform files by replacing all instances of information contained in ‘<>’ with the necessary information. This step is crucial as it ensures the correct configuration for the organization or project-level integration.
This process includes:
- The Project ID of your provisioning Project
- The region where resources will be created in your provisioning Project
- Organization ID or Project ID
- Project NumberAdd the Terraform template to your IAC.
From your command line, enter the code below:
terraform plan
Note: When you plan your terraform, ensure that there are no unexpected errors, resources being created, or resources being deployed.
From your command line, enter the code below:
terraform apply
From Red Canary, select I’ve completed the provisioning steps.
Click Save.
GCP Deployment Manager
Click setup files to download the files that Red Canary has provided.
Navigate to your GCP Cloud Shell.
Upload the setup files to your GCP Cloud Shell.
Upload the following three files:
For Organization
Pub/Sub template: rc_pubsub.jinja
PubSub deployment: rc_pubsub.yaml
Custom IAM role for Organization integration: organization_rc_ingest_custom_role.json
For Project
Pub/Sub template: rc_pubsub.jinja
PubSub deployment: rc_pubsub.yaml
Custom IAM role for Project integration: project_rc_ingest_custom_role.json
Click Upload.
From your text editor, execute the following commands:
Note: This step requires you to substitute the ORGANIZATION_ID, PROVISIONING_PROJECT_ID, and PROVISIONING_PROJECT_NUMBER using the information you collected in Step 1.
Switch to your Provisioning Project.
For Organization
gcloud config set project PROVISIONING_PROJECT_ID
For Project
gcloud config set project PROJECT_ID
Enable services required for this integration.
For Organization & Project
gcloud services enable deploymentmanager.googleapis.com pubsub.googleapis.com
Create the Red Canary custom IAM role.
For Organization
gcloud iam roles create red_canary_ingestor --organization=ORGANIZATION_ID --file=organization_rc_ingest_custom_role.json
For Project
gcloud iam roles create red_canary_ingestor --project=PROJECT_ID --file=project_rc_ingest_custom_role.json
Assign the custom role to Red Canary’s service account.
For Organization
gcloud organizations add-iam-policy-binding ORGANIZATION_ID --member=serviceAccount:red-canary-gcp-log-ingest@red-canary-platform-production.iam.gserviceaccount.com --role=organizations/ORGANIZATION_ID/roles/red_canary_ingestor
For Project
gcloud projects add-iam-policy-binding PROJECT_ID --member=serviceAccount:red-canary-gcp-log-ingest@red-canary-platform-production.iam.gserviceaccount.com --role=projects/PROJECT_ID/roles/red_canary_ingestor
Assign the Owner role to your Provisioning Project’s Cloud Services service account.
For Organization
gcloud projects add-iam-policy-binding PROVISIONING_PROJECT_ID --member serviceAccount:PROVISIONING_PROJECT_NUMBER@cloudservices.gserviceaccount.com --role roles/owner
Note: Both the PROJECT_ID and PROJECT_NUMBER are used in this code.
For Project
gcloud projects add-iam-policy-binding PROJECT_ID --member serviceAccount:PROJECT_NUMBER@cloudservices.gserviceaccount.com --role roles/owner
Note: Both the PROJECT_ID and PROJECT_NUMBER are used in this code.
Deploy the Pub/Sub Topic and Subscription in your Provisioning Project.
For Organization
gcloud deployment-manager deployments create red-canary-pub-sub-deployment --config rc_pubsub.yaml
Create an aggregated Log Sink.
For Organization
gcloud logging sinks create red-canary-log-sink pubsub.googleapis.com/projects/PROVISIONING_PROJECT_ID/topics/red-canary-log-export --organization=ORGANIZATION_ID --include-children --log-filter='protoPayload.serviceName!="k8s.io" AND logName!~"^projects/sys-.*"'
For Project
gcloud logging sinks create red-canary-log-sink pubsub.googleapis.com/projects/PROJECT_ID/topics/red-canary-log-export --log-filter='protoPayload.serviceName!="k8s.io" AND logName!~"^projects/sys-.*"'
Assign the Pub/Sub publisher role to your Organization’s Logging service account.
For Organization
gcloud projects add-iam-policy-binding PROVISIONING_PROJECT_ID --member='serviceAccount:service-org-ORGANIZATION_ID@gcp-sa-logging.iam.gserviceaccount.com' --role='roles/pubsub.publisher'
For Project
gcloud projects add-iam-policy-binding PROJECT_ID --member='serviceAccount:service-PROJECT_NUMBER@gcp-sa-logging.iam.gserviceaccount.com' --role='roles/pubsub.publisher'
Note: Both the PROJECT_ID and PROJECT_NUMBER are used in the code.
From Red Canary, select I’ve completed the provisioning steps.
Click Save.
Ingest Details
Red Canary ingests Google Cloud Platform (GCP) Audit logs from GCP environments. Additionally, Red Canary integrates with GCP to scan your environment regularly to discover new projects and resources.
Environments covered
Google Cloud Platform
Ingest details (GCP Cloud Audit Logs)
Red Canary ingests the following type of activity from GCP Cloud Audit Logs:
Admin Activity
Data Access
Policy Denied
System Events
Currently, Red Canary doesn’t collect Security Command Center alerts from GCP.
Finally, Red Canary integrates with GCP to scan your environment regularly to discover new projects and resources. This integration is established via the Cloud Asset API and the IAM Custom Role configured during onboarding, enabling Red Canary to read your GCP environment. The API method used is SearchAllResources, which requires the cloudasset.assets.searchAllResources permission, ensuring comprehensive visibility into your projects and resources.
Resource Discovery Tool
This section explains how to calculate billable resources inside a Google Cloud Platform (GCP) tenant so that a Red Canary sales representative can estimate for GCP Cloud Managed Detection and Response (MDR) services. These steps should be completed before implementation.
Prerequisites
You must have cloudasset.assets.searchAllResources permissions within a GCP Project or Organization name.
Bash
Red Canary uses Bash to accomplish this task, particularly the gcloud command line interface (CLI), which searches asset resources for a specified Project or Organization. GCP users will likely have at least one person skilled in Bash. Additionally, GCP CloudShell may be used to run the Bash script directly within the GCP Management Console.
Step 1: Run the script
Navigate to your GCP cloud console.
From the menu bar, click Activate Cloud Shell.
From the CloudShell Menu, click the extended menu and then click Upload.
Navigate to the location of the script, which you will have received from a Red Canary representative. You can also click here to download the script.
Select the script from the previous step (cont_billable_resources.sh).
At the prompt, enter ls -l.
The count_billable_resources.sh should appear in the listing.
At the prompt, enter chmod +x count_billable_resources.sh to assign the correct permissions so the script can be executed.
Once you’ve identified the Organization or Project the script should be run for, enter the following at the bash prompt:
./count_billable_resources.sh organizations/<ORGANIZATION_ID> or ./count_billable_resources.sh projects/<PROJECT_ID>
FAQ
What permissions are Red Canary asking for?
The ingest role is created with bindings to the following GCP predefined IAM roles:
Roles | Conditions | Why does Red Canary need this? |
---|---|---|
Attached to Red Canary Ingest: | ||
|
| Allows Red Canary to manage the Log Router Sink configuration for the Log Sink created for Telemetry Routing in Step 3 of the Integration Guide |
|
| Allows Red Canary to read from the subscription |
Attached to GCP Log Sink: | ||
| Allows the Log Sink to publish messages to the topic |
Additionally, the following permissions are necessary for Red Canary to perform specific actions, such as listing resources, getting sinks, monitoring the sink, getting subscriptions on a pub-sub topic, and listing things with the resource manager:
Permission | Why does Red Canary need this? |
---|---|
| Allows Red Canary to search information about your GCP Organization to enumerate projects and resources as they are added for monitoring, as well as License calculation purposes |
| Allows Red Canary to read filtered Cloud Monitoring measurements |
| Allows Red Canary to see information about the ingest topic |
| Allows Red Canary to validate that the permissions for managing the ingest topics is correctly configured |
| Allows Red Canary to see information about the subscription to the ingest topic for purposes of ingesting data |
| Allows Red Canary to read GCP Project information to enumerate resources as they are added for monitoring, as well as License calculation purposes |
| Allows Red Canary to list GCP Projects in the organization to enumerate resources as they are added for monitoring, as well as License calculation purposes |
For project scoped integrations only: | |
| Allows Red Canary to validate access to the required permissions across the approved projects |
For organization scoped integrations only: | |
| Allows Red Canary to read GCP Folder information to enumerate resources, and projects within an Organization organized by folders as they are added for monitoring, as well as License calculation purposes |
| Allows Red Canary to list GCP Folders in the organization to enumerate resources, and projects within an Organization organized by folders as they are added for monitoring, as well as License calculation purposes |
| Allows Red Canary to list GCP Organizations in your account to enumerate projects and resources as they are added for monitoring, as well as License Calculation purposes |
| Allows Red Canary to validate access to the required permissions across the integrated organziation |
Is the roles/logging.configWriter
required?
Yes, this role is required for Red Canary to be able to manage the configuration for the Log Sink that’s used to route telemetry. Red Canary is only able to manage the configuration for the Log Sink used in the integration.
How long does it take for the integration to show as Active after setup?
The integration will be created and set as “Provisioning”. Once it is sending telemetry to Red Canary, the status will automatically update to “Active”. This process may take around 20 minutes.
Why does Red Canary need access to my Log Sink resource in my GCP environment?
Red Canary requires this access to manage the inclusive and exclusive filtering of logs for the Integration’s Log Sink. Red Canary will then adjust the Log Sink’s filters to prioritize the routing of data that is useful for security analysis while balancing the amount of data egress from a customer’s environment.
What changes will Red Canary make to my GCP environment with this access?
Red Canary will only focus on making changes to the Integration’s Log Sink and its inclusive and exclusive filtering configuration to determine which logs are routed to Red Canary.
What is the scope of access that the IAM Role Binding grants Red Canary to my Integration?
The access granted to Red Canary is limited to the Log Sink used by the Integration to route logs to Red Canary. This access is configured using a conditional role binding that only grants Red Canary access to Log Sinks matching the expected resource naming pattern for the Integration.
Organization-level Integration
organizations/<YOUR_ORGANIZATION_ID>/sinks/red-canary-log-sinkProject-level Integration
projects/<YOUR_PROJECT_ID>/sinks/red-canary-log-sink
Why do I need to ensure my permissions are set to “Admin Read, Data Read, and Data Write”?
Data Read and Write capabilities are essential for effective security monitoring. These functions enable the detection of malicious activities, which are then communicated to our Detection Engineers (DEs) for investigation. Without Data Access logs enabled, we cannot capture the specific Read/Write actions generated by our auditing service, hindering our threat detection capabilities.
To emphasize the importance of Data Access logs, we have implemented a validation process that requires their enablement during Integration creation. This ensures that customers cannot proceed without activating this critical security measure.