Integrate Google Cloud Platform (GCP) with Red Canary
    • 30 Sep 2024
    • 11 Minutes to read
    • PDF

    Integrate Google Cloud Platform (GCP) with Red Canary

    • PDF

    Article summary

    Integrating Google Cloud Platform (GCP) with Red Canary provides enhanced threat detection and response capabilities for cloud environments. You can gain deeper visibility into cloud activities, identify potential threats faster, and mitigate risks more effectively. To integrate Google Cloud Platform (GCP) with Red Canary, follow the procedure below from beginning to end. Once all steps are completed successfully, the data should be ingested into Red Canary within ~20 minutes.

    Note: You can integrate GCP with Red Canary at an organization or project level. Steps are provided for both organizational and project-level integrations.

    Prerequisites

    • You must be subscribed to Red Canary’s Cloud Control Plane license 

    • You must have the following roles in GCP (which are encompassed by Editor and Owner in most cases):

      • Organization Administrator or Organization Role Administrator

      • Role Administrator or IAM Admin

      • Deployment Manager Editor

    Integration Resources

    Throughout this integration, you will create resources within your GCP environment. These resources enable Red Canary to acquire your logs and monitor your systems.

    • Pub/Sub topic. This topic, named red-canary-log-export, receives all published messages.

    • Pub/Sub topic subscription. A red-canary-log-ingest subscription is attached to the ‘red-canary-log-export’ topic and is consumed by Red Canary.

    • A Google Logging Sink named red-canary-log-sink receives all messages from the GCP Logging Service and publishes them on the red-canary-log-export topic. This Sink will receive messages from all the children’s projects at an organizational level. 

    • A new custom role named red_canary_ingestor provides the necessary permissions for the Red Canary Integration.

    • A new policy binding assigns the red_canary_ingestor role to Red Canary’s red-canary-gcp-log-ingest@red-canary-platform-production.iam.gserviceaccount.com service account for your organization or project.

    • A second binding policy assigns the built-in roles/pubsub.subscriber role to the Red Canary service account, enabling Red Canary to subscribe to the subscription.

    Step 1: Collect information required for integration

    To integrate your Organization with Red Canary, you’ll need to collect a few pieces of information:

    • Organization ID

    • Provisioning Project ID

    • Provisioning Project Number

    1. From your Red Canary homepage, click Integrations.

    2. From the Integrations section, locate and then click Google Cloud Platform.

    3. Enter a name for your new GCP integration. 

    4. Choose the scope of your integration by selecting Organization or Project.

      Note: If you select Project, click here to continue the Project integration process

    Organization

    1. If you selected Organization, enter your Organization ID

      1. To locate your Organization ID, navigate to your GCP cloud console and then select your Organization from the resource browser.

      2. Copy your Organization ID, and then paste it into Red Canary. 

    2. Enter your Provisioning Project ID.

      Note: The Provisioning Project can be any Project within your Organization; you can use an existing Project or create a new one. This setup provisions a Pub/Sub Topic and Subscription within this Project to use as the destination for log routing.

      1. To locate your Provisioning Project ID, navigate to your GCP cloud console and select your Project in the resource explorer.

      2. Copy your Project ID, and then paste it into Red Canary.

    3. From Red Canary, enter any Excluded Projects; this step is optional. 

      Note: Excluded Projects is a comma-separated list of project ID(s) that will be excluded from the Red Canary scan.

    4. From Red Canary, click Next.

    Project

    1. If you selected Project, enter your Project ID.

      1. To locate your Provisioning Project ID, navigate to your GCP cloud console and then select your Project in the resource explorer. 

      2. Copy your Project ID, and then paste it into Red Canary.

    2. From Red Canary, click Next.

    Step 2: Enable Data Access Logs

    Enabling Data Access logs ensures Red Canary can monitor activity in your GCP environment.

    1. From Red Canary, select I need to enable Data Access Logs.

      1. To complete this process, navigate to your GCP Audit Logs.

      2. Select your Organization or Project from the resource browser.

      3. Click Set Default Configuration.

      4. Select Admin Read, Data Read, and Data Write.

      5. Click Save.

    2. From Red Canary, select I’ve enabled Data Access logs.

      Note: To prevent unnecessary charges from GCP, ensure to opt out of data storage. GCP automatically directs logs to a default storage sink for storage. For our purposes, we require only the routing of these logs, not their storage.

    3. Click Next.

    Step 3: Configure Telemetry routing to Red Canary

    Red Canary provides setup files to provision resources in your GCP environment to enable the integration process. 

    The integration process can be completed in three different ways:

    From Red Canary, choose the scope of your integration by selecting Bash or Terraform.

    Bash

    1. Click setup files to download the files that Red Canary has provided.

    2. Unzip the files you downloaded in the previous step.

    3. Navigate to your GCP Cloud Shell, and then click Activate Cloud Shell.

    4. Upload the setup files to your GCP Cloud Shell.

    5. Use the following command to grant execution permissions to the setup file:

      1. For Organization

        chmod +x organization_provisioning.sh

      2. For Project

        chmod +x project_provisioning.sh

    6. Initiate the setup files with the following command to allocate the necessary resources for this integration:

      1. For Organization

        ./organization_provisioning.sh --organization-id ORGANIZATION_ID --provisioning-project-id PROVISIONING_PROJECT_ID

      2. For Project

        ./project_provisioning.sh --provisioning-project-id PROJECT_ID

    7. If you receive an Authorization confirmation pop-up, click  Authorize

    8. From Red Canary, select I’ve completed the provisioning steps.

    9. Click Save.  

    Terraform

    1. If you selected Terraform, click setup files to download the files that Red Canary has provided.

    2. Integrate the Terraform template downloaded in the previous step into your Infrastructure as Code (IaaC) pipeline. Follow your pipeline's standard process for deploying Terraform templates.

      1. It’s essential to edit the terraform files by replacing all instances of information contained in ‘<>’ with the necessary information. This step is crucial as it ensures the correct configuration for the organization or project-level integration.
        This process includes:
        - The Project ID of your provisioning Project
        - The region where resources will be created in your provisioning Project
        - Organization ID or Project ID
        - Project Number

      2. Add the Terraform template to your IAC.

      3. From your command line, enter the code below:
        terraform plan

        Note: When you plan your terraform, ensure that there are no unexpected errors, resources being created, or resources being deployed.

      4. From your command line, enter the code below:
        terraform apply

    3. From Red Canary, select I’ve completed the provisioning steps.

    4. Click Save.

    GCP Deployment Manager

    1. Click setup files to download the files that Red Canary has provided.

    2. Navigate to your GCP Cloud Shell.

    3. Upload the setup files to your GCP Cloud Shell.

    4. Upload the following three files:

      1. For Organization

        1. Pub/Sub template: rc_pubsub.jinja

        2. PubSub deployment: rc_pubsub.yaml

        3. Custom IAM role for Organization integration: organization_rc_ingest_custom_role.json

      2. For Project

        1. Pub/Sub template: rc_pubsub.jinja

        2. PubSub deployment: rc_pubsub.yaml

        3. Custom IAM role for Project integration: project_rc_ingest_custom_role.json

    5. Click Upload

    6. From your text editor, execute the following commands:

      Note: This step requires you to substitute the ORGANIZATION_ID, PROVISIONING_PROJECT_ID, and PROVISIONING_PROJECT_NUMBER using the information you collected in Step 1.

      1. Switch to your Provisioning Project.

        1. For Organization
          gcloud config set project PROVISIONING_PROJECT_ID

        2. For Project
          gcloud config set project PROJECT_ID

      2. Enable services required for this integration.

        1. For Organization & Project

          gcloud services enable deploymentmanager.googleapis.com pubsub.googleapis.com

      3. Create the Red Canary custom IAM role. 

        1. For Organization
          gcloud iam roles create red_canary_ingestor --organization=ORGANIZATION_ID --file=organization_rc_ingest_custom_role.json

        2. For Project
          gcloud iam roles create red_canary_ingestor --project=PROJECT_ID --file=project_rc_ingest_custom_role.json

      4. Assign the custom role to Red Canary’s service account.

        1. For Organization

          gcloud organizations add-iam-policy-binding ORGANIZATION_ID --member=serviceAccount:red-canary-gcp-log-ingest@red-canary-platform-production.iam.gserviceaccount.com --role=organizations/ORGANIZATION_ID/roles/red_canary_ingestor

        2. For Project

          gcloud projects add-iam-policy-binding PROJECT_ID --member=serviceAccount:red-canary-gcp-log-ingest@red-canary-platform-production.iam.gserviceaccount.com --role=projects/PROJECT_ID/roles/red_canary_ingestor

      5. Assign the Owner role to your Provisioning Project’s Cloud Services service account.

        1. For Organization

          gcloud projects add-iam-policy-binding PROVISIONING_PROJECT_ID --member serviceAccount:PROVISIONING_PROJECT_NUMBER@cloudservices.gserviceaccount.com --role roles/owner

          Note: Both the PROJECT_ID and PROJECT_NUMBER are used in this code.

        2. For Project

          gcloud projects add-iam-policy-binding PROJECT_ID --member serviceAccount:PROJECT_NUMBER@cloudservices.gserviceaccount.com --role roles/owner

          Note: Both the PROJECT_ID and PROJECT_NUMBER are used in this code.

      6. Deploy the Pub/Sub Topic and Subscription in your Provisioning Project.

        1. For Organization
          gcloud deployment-manager deployments create red-canary-pub-sub-deployment --config rc_pubsub.yaml

      7. Create an aggregated Log Sink.

        1. For Organization

          gcloud logging sinks create red-canary-log-sink pubsub.googleapis.com/projects/PROVISIONING_PROJECT_ID/topics/red-canary-log-export --organization=ORGANIZATION_ID --include-children --log-filter='protoPayload.serviceName!="k8s.io" AND logName!~"^projects/sys-.*"'

        2. For Project

          gcloud logging sinks create red-canary-log-sink pubsub.googleapis.com/projects/PROJECT_ID/topics/red-canary-log-export --log-filter='protoPayload.serviceName!="k8s.io" AND logName!~"^projects/sys-.*"'

      8. Assign the Pub/Sub publisher role to your Organization’s Logging service account.

        1. For Organization

          gcloud projects add-iam-policy-binding PROVISIONING_PROJECT_ID --member='serviceAccount:service-org-ORGANIZATION_ID@gcp-sa-logging.iam.gserviceaccount.com' --role='roles/pubsub.publisher'

        2. For Project
          gcloud projects add-iam-policy-binding PROJECT_ID --member='serviceAccount:service-PROJECT_NUMBER@gcp-sa-logging.iam.gserviceaccount.com' --role='roles/pubsub.publisher'

          Note: Both the PROJECT_ID and PROJECT_NUMBER are used in the code.

    7. From Red Canary, select I’ve completed the provisioning steps.

    8. Click Save

    Ingest Details

    Red Canary ingests Google Cloud Platform (GCP) Audit logs from GCP environments. Additionally, Red Canary integrates with GCP to scan your environment regularly to discover new projects and resources.

    Environments covered

    • Google Cloud Platform

    Ingest details (GCP Cloud Audit Logs)

    Red Canary ingests the following type of activity from GCP Cloud Audit Logs:

    • Admin Activity

    • Data Access

    • Policy Denied

    • System Events

    Currently, Red Canary doesn’t collect Security Command Center alerts from GCP.

    Finally, Red Canary integrates with GCP to scan your environment regularly to discover new projects and resources. This integration is established via the Cloud Asset API and the IAM Custom Role configured during onboarding, enabling Red Canary to read your GCP environment. The API method used is SearchAllResources, which requires the cloudasset.assets.searchAllResources permission, ensuring comprehensive visibility into your projects and resources.

    Resource Discovery Tool

    This section explains how to calculate billable resources inside a Google Cloud Platform (GCP) tenant so that a Red Canary sales representative can estimate for GCP Cloud Managed Detection and Response (MDR) services. These steps should be completed before implementation.

    Prerequisites 

    • You must have cloudasset.assets.searchAllResources permissions within a GCP Project or Organization name.

    Bash

    Red Canary uses Bash to accomplish this task, particularly the gcloud command line interface (CLI), which searches asset resources for a specified Project or Organization. GCP users will likely have at least one person skilled in Bash. Additionally, GCP CloudShell may be used to run the Bash script directly within the GCP Management Console.

    Step 1: Run the script

    1. Navigate to your GCP cloud console.

    2. From the menu bar, click Activate Cloud Shell.

    3. From the CloudShell Menu, click the extended menu and then click Upload.

    4. Navigate to the location of the script, which you will have received from a Red Canary representative. You can also click here to download the script.

    5. Select the script from the previous step (cont_billable_resources.sh).

    6. At the prompt, enter ls -l.

    7. The count_billable_resources.sh should appear in the listing.

    8. At the prompt, enter chmod +x count_billable_resources.sh to assign the correct permissions so the script can be executed.

    9. Once you’ve identified the Organization or Project the script should be run for, enter the following at the bash prompt:

      ./count_billable_resources.sh organizations/<ORGANIZATION_ID> or ./count_billable_resources.sh projects/<PROJECT_ID>

    FAQ

    What permissions are Red Canary asking for?

    The ingest role is created with bindings to the following GCP predefined IAM roles:

    Roles

    Conditions

    Why does Red Canary need this?

    Attached to Red Canary Ingest:

    roles/logging.configWriter

    resource.name == google_logging_sink.sink.id

    Allows Red Canary to manage the Log Router Sink configuration for the Log Sink created for Telemetry Routing in Step 3 of the Integration Guide

    roles/pubsub.subscriber

    subscription == pubsub_subsciption.id

    Allows Red Canary to read from the subscription

    Attached to GCP Log Sink:

    roles/pubsub.publisher

    Allows the Log Sink to publish messages to the topic

    Additionally, the following permissions are necessary for Red Canary to perform specific actions, such as listing resources, getting sinks, monitoring the sink, getting subscriptions on a pub-sub topic, and listing things with the resource manager:

    Permission

    Why does Red Canary need this?

    cloudasset.assets.searchAllResources

    Allows Red Canary to search information about your GCP Organization to enumerate projects and resources as they are added for monitoring, as well as License calculation purposes

    monitoring.timeSeries.list

    Allows Red Canary to read filtered Cloud Monitoring measurements

    pubsub.topics.get

    Allows Red Canary to see information about the ingest topic

    pubsub.topics.getIamPolicy

    Allows Red Canary to validate that the permissions for managing the ingest topics is correctly configured

    pubsub.subscriptions.get

    Allows Red Canary to see information about the subscription to the ingest topic for purposes of ingesting data

    resourcemanager.projects.get

    Allows Red Canary to read GCP Project information to enumerate resources as they are added for monitoring, as well as License calculation purposes

    resourcemanager.projects.list

    Allows Red Canary to list GCP Projects in the organization to enumerate resources as they are added for monitoring, as well as License calculation purposes

    For project scoped integrations only:

    resourcemanager.projects.getIamPolicy

    Allows Red Canary to validate access to the required permissions across the approved projects

    For organization scoped integrations only:

    resourcemanager.folders.get

    Allows Red Canary to read GCP Folder information to enumerate resources, and projects within an Organization organized by folders as they are added for monitoring, as well as License calculation purposes

    resourcemanager.folders.list

    Allows Red Canary to list GCP Folders in the organization to enumerate resources, and projects within an Organization organized by folders as they are added for monitoring, as well as License calculation purposes

    resourcemanager.organizations.list

    Allows Red Canary to list GCP Organizations in your account to enumerate projects and resources as they are added for monitoring, as well as License Calculation purposes

    resourcemanager.organizations.getIamPolicy

    Allows Red Canary to validate access to the required permissions across the integrated organziation

    Is the roles/logging.configWriter required?

    Yes, this role is required for Red Canary to be able to manage the configuration for the Log Sink that’s used to route telemetry. Red Canary is only able to manage the configuration for the Log Sink used in the integration.

    How long does it take for the integration to show as Active after setup?

    • The integration will be created and set as “Provisioning”. Once it is sending telemetry to Red Canary, the status will automatically update to “Active”. This process may take around 20 minutes. 

    Why does Red Canary need access to my Log Sink resource in my GCP environment?

    • Red Canary requires this access to manage the inclusive and exclusive filtering of logs for the Integration’s Log Sink. Red Canary will then adjust the Log Sink’s filters to prioritize the routing of data that is useful for security analysis while balancing the amount of data egress from a customer’s environment.

    What changes will Red Canary make to my GCP environment with this access?

    • Red Canary will only focus on making changes to the Integration’s Log Sink and its inclusive and exclusive filtering configuration to determine which logs are routed to Red Canary.

    What is the scope of access that the IAM Role Binding grants Red Canary to my Integration?

    • The access granted to Red Canary is limited to the Log Sink used by the Integration to route logs to Red Canary. This access is configured using a conditional role binding that only grants Red Canary access to Log Sinks matching the expected resource naming pattern for the Integration.

      • Organization-level Integration
        organizations/<YOUR_ORGANIZATION_ID>/sinks/red-canary-log-sink

      • Project-level Integration
        projects/<YOUR_PROJECT_ID>/sinks/red-canary-log-sink

    Why do I need to ensure my permissions are set to “Admin Read, Data Read, and Data Write”?

    Data Read and Write capabilities are essential for effective security monitoring. These functions enable the detection of malicious activities, which are then communicated to our Detection Engineers (DEs) for investigation. Without Data Access logs enabled, we cannot capture the specific Read/Write actions generated by our auditing service, hindering our threat detection capabilities.

    To emphasize the importance of Data Access logs, we have implemented a validation process that requires their enablement during Integration creation. This ensures that customers cannot proceed without activating this critical security measure.


    Was this article helpful?