Cannot use impersonate with domain wide delegation

This issue has been tracked since 2022-10-19.

TL;DR

Hello,

I setup a Federated identity between my GCP and a GitHub workflow.

The workload identity is set properly. I've tested it by installing gcloud SDK and running gcloud auth list. The later command outputs the correct SA being impersonated.

Also the google-github-actions/[email protected] without impersonalization, using token_format: 'access_token' passes correctly.

The impersonated SA has both Service Account Token Creator and Workload Identity User roles granted on the tools-iac-example GCP project.
Also, the SA has Domain Wide Delegation granted for all 4 specified scopes.

Am I missing something?

Thanks,
Damir Dezeljin

Expected behavior

The impersonation should work.

Observed behavior

The impersonalization fails with the following error:

##[debug]Evaluating condition for step: 'Authenticate to Google Cloud'
##[debug]Evaluating: success()
##[debug]Evaluating success:
##[debug]=> true
##[debug]Result: true
##[debug]Starting: Authenticate to Google Cloud
##[debug]Register post job cleanup for action: google-github-actions/[email protected]
##[debug]Loading inputs
##[debug]Loading env
Run google-github-actions/[email protected]
##[debug]Using workload identity provider "projects/73[1](https://github.com/example-io/tf-mgmt/actions/runs/3285540817/jobs/5412743142#step:3:1)01467[2](https://github.com/example-io/tf-mgmt/actions/runs/3285540817/jobs/5412743142#step:3:2)9[3](https://github.com/example-io/tf-mgmt/actions/runs/3285540817/jobs/5412743142#step:3:3)/locations/global/workloadIdentityPools/iac-gh-idpool/providers/iac-gh-idpool-provider"
##[debug]ID token url is https://pipelines.actions.githubusercontent.com/RoKEYgiuTaqbqfrcBb7sGLAs9VLI3QvF5bqVk65EcSqciIZUoT/00000000-0000-0000-0000-000000000000/_apis/distributedtask/hubs/Actions/plans/c22e10bb-b0d9-[4](https://github.com/example-io/tf-mgmt/actions/runs/3285540817/jobs/5412743142#step:3:4)918-9a71-014dd3[5](https://github.com/example-io/tf-mgmt/actions/runs/3285540817/jobs/5412743142#step:3:5)f8[6](https://github.com/example-io/tf-mgmt/actions/runs/3285540817/jobs/5412743142#step:3:6)01/jobs/5d0864fc-c269-5600-cd0[7](https://github.com/example-io/tf-mgmt/actions/runs/3285540817/jobs/5412743142#step:3:7)-7aaf32b0e293/idtoken?api-version=2.0&audience=https%3A%2F%2Fiam.googleapis.com%2Fprojects%2F23101461234%2Flocations%2Fglobal%2FworkloadIdentityPools%2Fiac-gh-idpool%2Fproviders%2Fiac-gh-idpool-provider
::add-mask::***
##[debug]Creating credentials file
Created credentials file at "/home/runner/work/tf-mgmt/tf-mgmt/gha-creds-baf7b4b41cd[8](https://github.com/example-io/tf-mgmt/actions/runs/3285540817/jobs/5412743142#step:3:8)aa52.json"
##[debug]Creating access token
Error: google-github-actions/auth failed with: retry function failed after 1 attempt: failed to sign JWT using [email protected]: (403) {
  "error": {
    "code": [40](https://github.com/example-io/tf-mgmt/actions/runs/3285540817/jobs/5412743142#step:3:41)3,
    "message": "The caller does not have permission",
    "status": "PERMISSION_DENIED"
  }
}
##[debug]Node Action run completed with exit code 1

Action YAML

steps:
  - uses: actions/[email protected]

  - name: Authenticate to Google Cloud
    id: "auth"
    uses: google-github-actions/[email protected]
    with:
      workload_identity_provider: projects/23101461234/locations/global/workloadIdentityPools/iac-gh-idpool/providers/iac-gh-idpool-provider
      service_account: [email protected]
      token_format: 'access_token'
      access_token_lifetime: 1800s
      access_token_scopes: https://www.googleapis.com/auth/cloud-platform, https://www.googleapis.com/auth/admin.directory.user, https://www.googleapis.com/auth/admin.directory.group, https://www.googleapis.com/auth/apps.groups.settings
      access_token_subject: [email protected]
      project_id: tools-iac-example

Log output

No response

Additional information

No response

sethvargo wrote this answer on 2022-10-20

Hi @damir-dezeljin

In order to support Domain-Wide Delegation via Workload Identity Federation, you must grant the external identity ("principalSet") roles/iam.serviceAccountTokenCreator in addition to roles/iam.workloadIdentityUser.

What are your current IAM permissions for both tf-management and the principalSet?

kbroughton wrote this answer on 2022-10-24

I got bit by an upper-cased github org name. Github doesn't distinguish case on the gh CLI and url paths, but the principalSet mapping is case sensitive.

damir-dezeljin wrote this answer on 2022-10-25

Thank you @sethvargo , I overlooked the point I had to add IAM roles to the principalSet.

Still, this alone didn't solve my problem, but this had to be done as well:

  • GitHub workflow GCP authentication step and related:
    ...
    env:
     GOOGLE_ADMIN_SCOPES: https://www.googleapis.com/auth/cloud-platform, https://www.googleapis.com/auth/admin.directory.user, https://www.googleapis.com/auth/admin.directory.group, https://www.googleapis.com/auth/apps.groups.settings
    ...
     permissions:
       contents: "read"
       id-token: "write"
    ...
       - name: Authenticate to Google Cloud
         id: "auth"
         uses: google-github-actions/[email protected]
         with:
           workload_identity_provider: ${{ env.GCP_WORKLOAD_IDPOOL_PROVIDER }}
           service_account: ${{ env.CICD_GCP_SA }}
           token_format: "access_token"
           access_token_lifetime: 1800s
           access_token_scopes: ${{ env.GOOGLE_ADMIN_SCOPES }}
           access_token_subject: ${{ env.GOOGLE_ADMIN_ACCOUNT }}
           create_credentials_file: true
    
  • TF Google Workspace provider had to be specified as follows:
    variable "googleworkspace_access_token" {
      type        = string
      description = "Oauth access token for workspace user"
    }
    ...
    provider "googleworkspace" {
      customer_id     = var.customer_id
      access_token    = var.googleworkspace_access_token
      service_account = var.super_admin_account
      oauth_scopes = [
        "https://www.googleapis.com/auth/admin.directory.user",
        "https://www.googleapis.com/auth/admin.directory.group",
        "https://www.googleapis.com/auth/apps.groups.settings"
      ]
    }
    
    ^^^
    Please note I had to specify this env var in the GitHub workflow for the terraform plan step to work:
    env:
      TF_VAR_googleworkspace_access_token: ${{ steps.auth.outputs.access_token }}
    
More Details About Repo
Owner Name google-github-actions
Repo Name auth
Full Name google-github-actions/auth
Language TypeScript
Created Date 2021-09-16
Updated Date 2023-03-24
Star Count 573
Watcher Count 16
Fork Count 116
Issue Count 3

YOU MAY BE INTERESTED

Issue Title Created Date Updated Date