GCP Secrets to K8s Secrets

Let us say that we have a secret in GCP, which is having multiple values in a single secret. For example, this is my secret

KEYCLOAK_USER=***
KEYCLOAK_PASSWORD=***
KC_DB=***
KC_DB_URL=***
KC_DB_USERNAME=***
KC_DB_PASSWORD=***
KC_PROXY=***
health-enabled=***
GCP_PROJECT_ID=***
GCP_PUBSUB_KEYCLOAK_EVENT_TOPIC_NAME=***

So, how can we pull this GCP’s secret into a Kubernetes Secret as environment variables.
I know few ways to get the job done.
Manually, we can pull the GCP’s secret and save it as a file and run the kubectl command to create secret from env file using the flag —from-env-file.

gcloud config set project [PROJECT_ID]

Set the project value to the required.

gcloud secrets versions access latest --secret=keycloak-secrets-test > text.txt #to retrieve the secret from GCP and save it as a file.

kubectl create secret generic secret-name --from-env-file=test.txt -n test

This is one way where it requires some manual efforts and it is not worthy when it comes to frequent changes in the GCP’s Secrets.

But if you want a way to sink the GCP’s secrets to K8s Secrets, then I suggest to use External Secrets.

Installation:

helm repo add external-secrets https://charts.external-secrets.io

helm install external-secrets \
   external-secrets/external-secrets \
   -n external-secrets \
   --create-namespace

## Verify the installation by using the command
kubectl get pods,crd -n external-secrets

To begin with create a service account in GCP that can access the secret’s in GCP.

gcloud iam service-accounts create external-secrets
gcloud secrets add-iam-policy-binding mysecret --member "serviceAccount:external-secrets@$project.iam.gserviceaccount.com" --role "roles/secretmanager.secretAccessor"

Create a service account in the required namespace in k8s.

Bind it with the Service Account created earlier in GCP.

gcloud iam service-accounts add-iam-policy-binding \
  external-secrets-sa@<YOUR-GCP-PROJECT>.iam.gserviceaccount.com \
  --role roles/iam.workloadIdentityUser \
  --member "serviceAccount:<YOUR-GCP-PROJECT>.svc.id.goog[external-secrets-namespace/external-secrets-ksa]"

Update the Kubernetes Service Account Annotation

kubectl annotate serviceaccount external-secrets-ksa \
  iam.gke.io/gcp-service-account=external-secrets-sa@<YOUR-GCP-PROJECT>.iam.gserviceaccount.com \
  -n external-secrets

Create a secret store with a manifest file

apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
  name: gcp-backend1
  namespace: test
spec:
  provider:
    gcpsm:
      projectID: ********  # Replace with your actual GCP project ID
      auth:
        workloadIdentity:
          clusterName: "******"  # Replace with your actual GKE cluster name
          clusterLocation: "asia-south1"  # Replace with the region (e.g., asia-south1)
          serviceAccountRef:
            name: external-secrets-ksa  # The Kubernetes Service Account (KSA) to use
            namespace: test  # Namespace where the KSA exists

Create a external secret using Manifest file:

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: external-secret-test2
  namespace: test
spec:
  refreshInterval: 1h           # rate SecretManager pulls GCPSM
  secretStoreRef:
    kind: SecretStore
    name: gcp-backend1           # name of the SecretStore (created in the above manifest)
  target:
    name: plain-text # name of the k8s Secret to be created in k8s
    creationPolicy: Owner
  dataFrom:
  - extract:
      key: keycloak-secrets-test  # name of the GCPSM secret

You can define the frequency to pull the GCP’s secrets here, so that the sync happens even there is a change.

Check the secrets in the k8s namespace.

kubectl get secrets -n namespace

kubectl describe secret secret-name -n namespace

This is the best way to store all the secrets in a single and making them a single k8s secrets as environment variable.