Skip to content

Instantly share code, notes, and snippets.

@dhduvall

dhduvall/TF pod output.txt

Created January 10, 2023 01:26
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save dhduvall/d89a1bb04d3d0dfd2fb5b2418cdd9529 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save dhduvall/d89a1bb04d3d0dfd2fb5b2418cdd9529 to your computer and use it in GitHub Desktop.
Download ZIP
Azure workload identity & terraform
Raw
azwi-quickstart-sh.yaml
id: /subscriptions/ssssssss-ssss-ssss-ssss-ssssssssssss/resourceGroups/azwi-quickstart-1798/providers/Microsoft.KeyVault/vaults/azwi-kv-f5f4
location: westus2
name: azwi-kv-f5f4
properties:
accessPolicies:
- applicationId: null
objectId: 07edf018-eb98-4ff8-a821-cc89b62a56aa
permissions:
certificates: null
keys: null
secrets:
- get
storage: null
tenantId: tttttttt-tttt-tttt-tttt-tttttttttttt
- applicationId: null
objectId: fcb69bc7-9cbd-47bb-b263-69800f850c02
permissions:
certificates:
- all
keys:
- all
secrets:
- all
storage:
- all
tenantId: tttttttt-tttt-tttt-tttt-tttttttttttt
createMode: null
enablePurgeProtection: null
enableRbacAuthorization: null
enableSoftDelete: true
enabledForDeployment: false
enabledForDiskEncryption: null
enabledForTemplateDeployment: null
hsmPoolResourceId: null
networkAcls: null
privateEndpointConnections: null
provisioningState: Succeeded
publicNetworkAccess: Enabled
sku:
family: A
name: standard
softDeleteRetentionInDays: 90
tenantId: tttttttt-tttt-tttt-tttt-tttttttttttt
vaultUri: https://azwi-kv-f5f4.vault.azure.net/
resourceGroup: azwi-quickstart-1798
systemData:
createdAt: '2023-01-10T00:49:50.209000+00:00'
createdBy: Duvall@coros.net
createdByType: User
lastModifiedAt: '2023-01-10T00:50:45.271000+00:00'
lastModifiedBy: Duvall@coros.net
lastModifiedByType: User
tags: {}
type: Microsoft.KeyVault/vaults
---
clientId: c8d74984-75d4-4ee5-84ce-6d2247b37094
id: /subscriptions/ssssssss-ssss-ssss-ssss-ssssssssssss/resourcegroups/azwi-quickstart-1798/providers/Microsoft.ManagedIdentity/userAssignedIdentities/azwi-ua-identity
location: westus2
name: azwi-ua-identity
principalId: 07edf018-eb98-4ff8-a821-cc89b62a56aa
resourceGroup: azwi-quickstart-1798
tags: {}
tenantId: tttttttt-tttt-tttt-tttt-tttttttttttt
type: Microsoft.ManagedIdentity/userAssignedIdentities
---
audiences:
- api://AzureADTokenExchange
id: /subscriptions/ssssssss-ssss-ssss-ssss-ssssssssssss/resourcegroups/azwi-quickstart-1798/providers/Microsoft.ManagedIdentity/userAssignedIdentities/azwi-ua-identity/federatedIdentityCredentials/kubernetes-federated-credential
issuer: https://eastus2.oic.prod-aks.azure.com/tttttttt-tttt-tttt-tttt-tttttttttttt/0716f870-c0aa-42d6-bd79-e64d1125bc6f/
name: kubernetes-federated-credential
resourceGroup: azwi-quickstart-1798
subject: system:serviceaccount:default:workload-identity-script-sa
type: Microsoft.ManagedIdentity/userAssignedIdentities/federatedIdentityCredentials
---
apiVersion: v1
kind: ServiceAccount
metadata:
annotations:
azure.workload.identity/client-id: c8d74984-75d4-4ee5-84ce-6d2247b37094
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"v1","kind":"ServiceAccount","metadata":{"annotations":{"azure.workload.identity/client-id":"c8d74984-75d4-4ee5-84ce-6d2247b37094"},"labels":{"azure.workload.identity/use":"true"},"name":"workload-identity-script-sa","namespace":"default"}}
creationTimestamp: "2023-01-10T00:50:46Z"
labels:
azure.workload.identity/use: "true"
name: workload-identity-script-sa
namespace: default
resourceVersion: "108042"
uid: 03ea031b-2b8f-4c16-8ec1-8eac52f97b6e
secrets:
- name: workload-identity-script-sa-token-r2p99
---
apiVersion: v1
kind: Pod
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"v1","kind":"Pod","metadata":{"annotations":{},"labels":{"azure.workload.identity/use":"true"},"name":"quick-start-script","namespace":"default"},"spec":{"containers":[{"env":[{"name":"KEYVAULT_URL","value":"https://azwi-kv-f5f4.vault.azure.net/"},{"name":"SECRET_NAME","value":"my-secret"}],"image":"ghcr.io/azure/azure-workload-identity/msal-go","name":"oidc"}],"nodeSelector":{"kubernetes.io/os":"linux"},"serviceAccountName":"workload-identity-script-sa"}}
creationTimestamp: "2023-01-10T00:50:49Z"
labels:
azure.workload.identity/use: "true"
name: quick-start-script
namespace: default
resourceVersion: "108071"
uid: 44e473b8-a315-4324-8f33-0b48c22b55e8
spec:
containers:
- env:
- name: KEYVAULT_URL
value: https://azwi-kv-f5f4.vault.azure.net/
- name: SECRET_NAME
value: my-secret
- name: AZURE_CLIENT_ID
value: c8d74984-75d4-4ee5-84ce-6d2247b37094
- name: AZURE_TENANT_ID
value: tttttttt-tttt-tttt-tttt-tttttttttttt
- name: AZURE_FEDERATED_TOKEN_FILE
value: /var/run/secrets/azure/tokens/azure-identity-token
- name: AZURE_AUTHORITY_HOST
value: https://login.microsoftonline.com/
image: ghcr.io/azure/azure-workload-identity/msal-go
imagePullPolicy: Always
name: oidc
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: kube-api-access-pn62x
readOnly: true
- mountPath: /var/run/secrets/azure/tokens
name: azure-identity-token
readOnly: true
dnsPolicy: ClusterFirst
enableServiceLinks: true
nodeName: aks-dc-34314468-vmss000001
nodeSelector:
kubernetes.io/os: linux
preemptionPolicy: PreemptLowerPriority
priority: 0
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
serviceAccount: workload-identity-script-sa
serviceAccountName: workload-identity-script-sa
terminationGracePeriodSeconds: 30
tolerations:
- effect: NoExecute
key: node.kubernetes.io/not-ready
operator: Exists
tolerationSeconds: 300
- effect: NoExecute
key: node.kubernetes.io/unreachable
operator: Exists
tolerationSeconds: 300
volumes:
- name: kube-api-access-pn62x
projected:
defaultMode: 420
sources:
- serviceAccountToken:
expirationSeconds: 3607
path: token
- configMap:
items:
- key: ca.crt
path: ca.crt
name: kube-root-ca.crt
- downwardAPI:
items:
- fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
path: namespace
- name: azure-identity-token
projected:
defaultMode: 420
sources:
- serviceAccountToken:
audience: api://AzureADTokenExchange
expirationSeconds: 3600
path: azure-identity-token
status:
conditions:
- lastProbeTime: null
lastTransitionTime: "2023-01-10T00:50:49Z"
status: "True"
type: Initialized
- lastProbeTime: null
lastTransitionTime: "2023-01-10T00:50:50Z"
status: "True"
type: Ready
- lastProbeTime: null
lastTransitionTime: "2023-01-10T00:50:50Z"
status: "True"
type: ContainersReady
- lastProbeTime: null
lastTransitionTime: "2023-01-10T00:50:49Z"
status: "True"
type: PodScheduled
containerStatuses:
- containerID: containerd://124f406bd3a8d3bcc51a971e749a2c3cc3c8dcc13425d315ebd04a82a2ef7673
image: ghcr.io/azure/azure-workload-identity/msal-go:latest
imageID: ghcr.io/azure/azure-workload-identity/msal-go@sha256:9aafef4dbe65385dedeaaed5491328406a17913043d62d38ec60283f79e532c7
lastState: {}
name: oidc
ready: true
restartCount: 0
started: true
state:
running:
startedAt: "2023-01-10T00:50:50Z"
hostIP: 10.224.0.7
phase: Running
podIP: 10.244.6.9
podIPs:
- ip: 10.244.6.9
qosClass: BestEffort
startTime: "2023-01-10T00:50:49Z"
Raw
azwi-quickstart-tf.yaml
id: /subscriptions/ssssssss-ssss-ssss-ssss-ssssssssssss/resourceGroups/azwi-quickstart/providers/Microsoft.KeyVault/vaults/azwi-quickstart-coros
location: westus2
name: azwi-quickstart-coros
properties:
accessPolicies:
- applicationId: null
objectId: 59e5fc59-98cd-4cfa-b461-a1fdf876b868
permissions:
certificates: []
keys: []
secrets:
- Get
storage: []
tenantId: tttttttt-tttt-tttt-tttt-tttttttttttt
- applicationId: null
objectId: fcb69bc7-9cbd-47bb-b263-69800f850c02
permissions:
certificates: []
keys: []
secrets:
- Backup
- Delete
- Get
- List
- Purge
- Recover
- Restore
- Set
storage: []
tenantId: tttttttt-tttt-tttt-tttt-tttttttttttt
createMode: null
enablePurgeProtection: null
enableRbacAuthorization: false
enableSoftDelete: true
enabledForDeployment: false
enabledForDiskEncryption: false
enabledForTemplateDeployment: false
hsmPoolResourceId: null
networkAcls: null
privateEndpointConnections: null
provisioningState: Succeeded
publicNetworkAccess: Enabled
sku:
family: A
name: standard
softDeleteRetentionInDays: null
tenantId: tttttttt-tttt-tttt-tttt-tttttttttttt
vaultUri: https://azwi-quickstart-coros.vault.azure.net/
resourceGroup: azwi-quickstart
systemData:
createdAt: '2023-01-09T21:47:51.032000+00:00'
createdBy: Duvall@coros.net
createdByType: User
lastModifiedAt: '2023-01-09T21:47:51.032000+00:00'
lastModifiedBy: Duvall@coros.net
lastModifiedByType: User
tags: {}
type: Microsoft.KeyVault/vaults
---
clientId: 59e5fc59-98cd-4cfa-b461-a1fdf876b868
id: /subscriptions/ssssssss-ssss-ssss-ssss-ssssssssssss/resourcegroups/azwi-quickstart/providers/Microsoft.ManagedIdentity/userAssignedIdentities/azwi-ua-identity
location: westus2
name: azwi-ua-identity
principalId: 9f3e7e0d-7145-4743-b4dd-67aa0ec930ae
resourceGroup: azwi-quickstart
tags: {}
tenantId: tttttttt-tttt-tttt-tttt-tttttttttttt
type: Microsoft.ManagedIdentity/userAssignedIdentities
---
audiences:
- api://AzureADTokenExchange
id: /subscriptions/ssssssss-ssss-ssss-ssss-ssssssssssss/resourcegroups/azwi-quickstart/providers/Microsoft.ManagedIdentity/userAssignedIdentities/azwi-ua-identity/federatedIdentityCredentials/kubernetes-federated-credential
issuer: https://eastus2.oic.prod-aks.azure.com/tttttttt-tttt-tttt-tttt-tttttttttttt/0716f870-c0aa-42d6-bd79-e64d1125bc6f/
name: kubernetes-federated-credential
resourceGroup: azwi-quickstart
subject: system:serviceaccount:default:workload-identity-sa
type: Microsoft.ManagedIdentity/userAssignedIdentities/federatedIdentityCredentials
---
apiVersion: v1
automountServiceAccountToken: true
kind: ServiceAccount
metadata:
annotations:
azure.workload.identity/client-id: 59e5fc59-98cd-4cfa-b461-a1fdf876b868
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"v1","kind":"ServiceAccount","metadata":{"annotations":{"azure.workload.identity/client-id":"fc4e1018-c0d9-498c-9ae0-84609c9865a2"},"labels":{"azure.workload.identity/use":"true"},"name":"workload-identity-sa","namespace":"default"}}
creationTimestamp: "2023-01-09T20:23:45Z"
labels:
azure.workload.identity/use: "true"
name: workload-identity-sa
namespace: default
resourceVersion: "50919"
uid: 67cec77f-fabc-4dbe-ace8-dca31b5ba2a9
secrets:
- name: workload-identity-sa-token-h8w7k
---
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: "2023-01-10T01:13:20Z"
labels:
azure.workload.identity/use: "true"
name: quick-start
namespace: default
resourceVersion: "115429"
uid: ed0889e0-ccd5-48b2-80e8-671345effb08
spec:
automountServiceAccountToken: true
containers:
- env:
- name: KEYVAULT_URL
value: https://azwi-quickstart-coros.vault.azure.net/
- name: SECRET_NAME
value: my-secret
- name: AZURE_CLIENT_ID
value: 59e5fc59-98cd-4cfa-b461-a1fdf876b868
- name: AZURE_TENANT_ID
value: tttttttt-tttt-tttt-tttt-tttttttttttt
- name: AZURE_FEDERATED_TOKEN_FILE
value: /var/run/secrets/azure/tokens/azure-identity-token
- name: AZURE_AUTHORITY_HOST
value: https://login.microsoftonline.com/
image: ghcr.io/azure/azure-workload-identity/msal-go
imagePullPolicy: Always
name: oidc
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: kube-api-access-x9xgk
readOnly: true
- mountPath: /var/run/secrets/azure/tokens
name: azure-identity-token
readOnly: true
dnsPolicy: ClusterFirst
enableServiceLinks: true
nodeName: aks-dc-34314468-vmss000000
preemptionPolicy: PreemptLowerPriority
priority: 0
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
serviceAccount: workload-identity-sa
serviceAccountName: workload-identity-sa
shareProcessNamespace: false
terminationGracePeriodSeconds: 30
tolerations:
- effect: NoExecute
key: node.kubernetes.io/not-ready
operator: Exists
tolerationSeconds: 300
- effect: NoExecute
key: node.kubernetes.io/unreachable
operator: Exists
tolerationSeconds: 300
volumes:
- name: kube-api-access-x9xgk
projected:
defaultMode: 420
sources:
- serviceAccountToken:
expirationSeconds: 3607
path: token
- configMap:
items:
- key: ca.crt
path: ca.crt
name: kube-root-ca.crt
- downwardAPI:
items:
- fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
path: namespace
- name: azure-identity-token
projected:
defaultMode: 420
sources:
- serviceAccountToken:
audience: api://AzureADTokenExchange
expirationSeconds: 3600
path: azure-identity-token
status:
conditions:
- lastProbeTime: null
lastTransitionTime: "2023-01-10T01:13:20Z"
status: "True"
type: Initialized
- lastProbeTime: null
lastTransitionTime: "2023-01-10T01:13:24Z"
message: 'containers with unready status: [oidc]'
reason: ContainersNotReady
status: "False"
type: Ready
- lastProbeTime: null
lastTransitionTime: "2023-01-10T01:13:24Z"
message: 'containers with unready status: [oidc]'
reason: ContainersNotReady
status: "False"
type: ContainersReady
- lastProbeTime: null
lastTransitionTime: "2023-01-10T01:13:20Z"
status: "True"
type: PodScheduled
containerStatuses:
- containerID: containerd://f815610edacec94020c3eaabc98e32eab35252dbaae1e53ce5aef54e177df5eb
image: ghcr.io/azure/azure-workload-identity/msal-go:latest
imageID: ghcr.io/azure/azure-workload-identity/msal-go@sha256:9aafef4dbe65385dedeaaed5491328406a17913043d62d38ec60283f79e532c7
lastState:
terminated:
containerID: containerd://f815610edacec94020c3eaabc98e32eab35252dbaae1e53ce5aef54e177df5eb
exitCode: 1
finishedAt: "2023-01-10T01:13:23Z"
reason: Error
startedAt: "2023-01-10T01:13:22Z"
name: oidc
ready: false
restartCount: 1
started: false
state:
waiting:
message: back-off 10s restarting failed container=oidc pod=quick-start_default(ed0889e0-ccd5-48b2-80e8-671345effb08)
reason: CrashLoopBackOff
hostIP: 10.224.0.6
phase: Running
podIP: 10.244.4.15
podIPs:
- ip: 10.244.4.15
qosClass: BestEffort
startTime: "2023-01-10T01:13:20Z"
Raw
azwi-quickstart.sh
#!/bin/bash
set -x
# 2
# environment variables for the Azure Key Vault resource
export KEYVAULT_NAME="azwi-kv-$(openssl rand -hex 2)"
export KEYVAULT_SECRET_NAME="my-secret"
export RESOURCE_GROUP="azwi-quickstart-$(openssl rand -hex 2)"
export LOCATION="westus2"
# environment variables for the AAD application
# [OPTIONAL] Only set this if you're using a Azure AD Application as part of this tutorial
# export APPLICATION_NAME="<your application name>"
# environment variables for the user-assigned managed identity
# [OPTIONAL] Only set this if you're using a user-assigned managed identity as part of this tutorial
export USER_ASSIGNED_IDENTITY_NAME="azwi-ua-identity"
# environment variables for the Kubernetes service account & federated identity credential
export SERVICE_ACCOUNT_NAMESPACE="default"
export SERVICE_ACCOUNT_NAME="workload-identity-script-sa"
export SERVICE_ACCOUNT_ISSUER=$(az aks show --resource-group danek-test-02 --name danek-test-02 --query "oidcIssuerProfile.issuerUrl" -otsv)
# 3
az group create --name "${RESOURCE_GROUP}" --location "${LOCATION}"
az keyvault create --resource-group "${RESOURCE_GROUP}" \
--location "${LOCATION}" \
--name "${KEYVAULT_NAME}"
az keyvault secret set --vault-name "${KEYVAULT_NAME}" \
--name "${KEYVAULT_SECRET_NAME}" \
--value "Hello\!"
# 4
# create a user-assigned managed identity if using user-assigned managed identity for this tutorial
az identity create --name "${USER_ASSIGNED_IDENTITY_NAME}" --resource-group "${RESOURCE_GROUP}"
export USER_ASSIGNED_IDENTITY_CLIENT_ID="$(az identity show --name "${USER_ASSIGNED_IDENTITY_NAME}" --resource-group "${RESOURCE_GROUP}" --query 'clientId' -otsv)"
# Identity isn't ready yet, so this will fail, complaining that it can't get the object ID from the principal name.
# The pod will start to work if this is repeated later once the identity is ready.
while ! az keyvault set-policy --name "${KEYVAULT_NAME}" \
--secret-permissions get \
--spn "${USER_ASSIGNED_IDENTITY_CLIENT_ID}"; do
sleep 10
i=$(( i + 1 ))
[[ $i -gt 10 ]] && exit 1
done
# 5
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: ServiceAccount
metadata:
annotations:
azure.workload.identity/client-id: ${APPLICATION_CLIENT_ID:-$USER_ASSIGNED_IDENTITY_CLIENT_ID}
labels:
azure.workload.identity/use: "true"
name: ${SERVICE_ACCOUNT_NAME}
namespace: ${SERVICE_ACCOUNT_NAMESPACE}
EOF
# 6
az identity federated-credential create \
--name "kubernetes-federated-credential" \
--identity-name "${USER_ASSIGNED_IDENTITY_NAME}" \
--resource-group "${RESOURCE_GROUP}" \
--issuer "${SERVICE_ACCOUNT_ISSUER}" \
--subject "system:serviceaccount:${SERVICE_ACCOUNT_NAMESPACE}:${SERVICE_ACCOUNT_NAME}"
# 7
export KEYVAULT_URL="$(az keyvault show -g ${RESOURCE_GROUP} -n ${KEYVAULT_NAME} --query properties.vaultUri -o tsv)"
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
name: quick-start-script
namespace: ${SERVICE_ACCOUNT_NAMESPACE}
labels:
azure.workload.identity/use: "true"
spec:
serviceAccountName: ${SERVICE_ACCOUNT_NAME}
containers:
- image: ghcr.io/azure/azure-workload-identity/msal-go
name: oidc
env:
- name: KEYVAULT_URL
value: ${KEYVAULT_URL}
- name: SECRET_NAME
value: ${KEYVAULT_SECRET_NAME}
nodeSelector:
kubernetes.io/os: linux
EOF
TENANT_ID=$(az account show -o tsv --query tenantId)
SUBSCRIPTION_ID=$(az account show -o tsv --query id)
(
az keyvault show -o yaml --name $KEYVAULT_NAME
printf "\n---\n\n"
az identity show --resource-group $RESOURCE_GROUP --name $USER_ASSIGNED_IDENTITY_NAME -o yaml
printf "\n---\n\n"
az identity federated-credential show -g $RESOURCE_GROUP --identity-name $USER_ASSIGNED_IDENTITY_NAME --name kubernetes-federated-credential -o yaml
printf "\n---\n\n"
kubectl -n $SERVICE_ACCOUNT_NAMESPACE get -o yaml sa $SERVICE_ACCOUNT_NAME
printf "\n---\n\n"
kubectl -n $SERVICE_ACCOUNT_NAMESPACE get -o yaml pod quick-start-script
) | sed -e "s/$TENANT_ID/tttttttt-tttt-tttt-tttt-tttttttttttt/g" -e "s/$SUBSCRIPTION_ID/ssssssss-ssss-ssss-ssss-ssssssssssss/" > azwi-quickstart-sh.yaml
Raw
azwi-quickstart.tf
terraform {
required_providers {
azuread = {
source = "hashicorp/azuread"
version = "=2.31.0"
}
azurerm = {
source = "hashicorp/azurerm"
version = "=3.38.0"
}
kubernetes = {
source = "hashicorp/kubernetes"
version = "=2.16.1"
}
null = {
source = "hashicorp/null"
version = "3.2.1"
}
}
}
provider "azuread" {}
provider "azurerm" {
features {
resource_group {
prevent_deletion_if_contains_resources = false
}
}
}
provider "kubernetes" {
config_path = "~/.kube/config"
config_context = "danek2"
}
data "azurerm_client_config" "current" {}
resource "azurerm_resource_group" "rg" {
name = "azwi-quickstart"
location = "westus2"
}
resource "azurerm_user_assigned_identity" "identity" {
location = azurerm_resource_group.rg.location
name = "azwi-ua-identity"
resource_group_name = azurerm_resource_group.rg.name
}
resource "azurerm_key_vault" "vault" {
name = "azwi-quickstart-coros"
location = azurerm_resource_group.rg.location
resource_group_name = azurerm_resource_group.rg.name
tenant_id = data.azurerm_client_config.current.tenant_id
sku_name = "standard"
access_policy {
tenant_id = data.azurerm_client_config.current.tenant_id
object_id = azurerm_user_assigned_identity.identity.client_id
secret_permissions = [
"Get",
]
}
access_policy {
tenant_id = data.azurerm_client_config.current.tenant_id
object_id = data.azurerm_client_config.current.object_id
# The CLI puts in "all", which is not accepted
secret_permissions = [
"Backup",
"Delete",
"Get",
"List",
"Purge",
"Recover",
"Restore",
"Set",
]
}
}
resource "azurerm_key_vault_secret" "example" {
name = "my-secret"
value = "Hello!"
key_vault_id = azurerm_key_vault.vault.id
}
data "azurerm_kubernetes_cluster" "cluster" {
name = "danek-test-02"
resource_group_name = "danek-test-02"
}
resource "azurerm_federated_identity_credential" "fedcred" {
parent_id = azurerm_user_assigned_identity.identity.id
name = "kubernetes-federated-credential"
resource_group_name = azurerm_resource_group.rg.name
audience = [ "api://AzureADTokenExchange" ]
issuer = data.azurerm_kubernetes_cluster.cluster.oidc_issuer_url
subject = "system:serviceaccount:${kubernetes_service_account.workload-identity-sa.metadata[0].namespace}:${kubernetes_service_account.workload-identity-sa.metadata[0].name}"
}
resource "kubernetes_service_account" "workload-identity-sa" {
metadata {
name = "workload-identity-sa"
namespace = "default"
annotations = {
"azure.workload.identity/client-id" = azurerm_user_assigned_identity.identity.client_id
}
labels = {
"azure.workload.identity/use" = "true"
}
}
}
resource "kubernetes_pod" "test" {
metadata {
name = "quick-start"
namespace = "default"
labels = {
"azure.workload.identity/use" = "true"
}
}
spec {
container {
image = "ghcr.io/azure/azure-workload-identity/msal-go"
name = "oidc"
env {
name = "KEYVAULT_URL"
value = azurerm_key_vault.vault.vault_uri
}
env {
name = "SECRET_NAME"
value = azurerm_key_vault_secret.example.name
}
}
service_account_name = kubernetes_service_account.workload-identity-sa.metadata[0].name
}
}
locals {
script = <<-EOT
(
az keyvault show -o yaml --name ${azurerm_key_vault.vault.name}
echo
echo "---"
echo
az identity show --resource-group ${azurerm_resource_group.rg.name} --name ${azurerm_user_assigned_identity.identity.name} -o yaml
echo
echo "---"
echo
az identity federated-credential show -g ${azurerm_resource_group.rg.name} --identity-name ${azurerm_user_assigned_identity.identity.name} --name ${azurerm_federated_identity_credential.fedcred.name} -o yaml
echo
echo "---"
echo
kubectl -n ${kubernetes_service_account.workload-identity-sa.metadata[0].namespace} get -o yaml sa ${kubernetes_service_account.workload-identity-sa.metadata[0].name}
echo
echo "---"
echo
kubectl -n ${kubernetes_service_account.workload-identity-sa.metadata[0].namespace} get -o yaml pod ${kubernetes_pod.test.metadata[0].name}
) | sed -e "s/${data.azurerm_client_config.current.tenant_id}/tttttttt-tttt-tttt-tttt-tttttttttttt/g" -e "s/${data.azurerm_client_config.current.subscription_id}/ssssssss-ssss-ssss-ssss-ssssssssssss/" > azwi-quickstart-tf.yaml
EOT
}
resource "null_resource" "output" {
triggers = {
script = local.script
}
provisioner "local-exec" {
command = "printf '%s\n' \"${local.script}\" | sh"
}
}
Raw
script xtrace excerpt.txt
+ az identity create --name azwi-ua-identity --resource-group azwi-quickstart-1798
ClientId Location Name PrincipalId ResourceGroup TenantId
------------------------------------ ---------- ---------------- ------------------------------------ -------------------- ------------------------------------
c8d74984-75d4-4ee5-84ce-6d2247b37094 westus2 azwi-ua-identity 07edf018-eb98-4ff8-a821-cc89b62a56aa azwi-quickstart-1798 tttttttt-tttt-tttt-tttt-tttttttttttt
++ az identity show --name azwi-ua-identity --resource-group azwi-quickstart-1798 --query clientId -otsv
+ export USER_ASSIGNED_IDENTITY_CLIENT_ID=c8d74984-75d4-4ee5-84ce-6d2247b37094
+ USER_ASSIGNED_IDENTITY_CLIENT_ID=c8d74984-75d4-4ee5-84ce-6d2247b37094
+ az keyvault set-policy --name azwi-kv-f5f4 --secret-permissions get --spn c8d74984-75d4-4ee5-84ce-6d2247b37094
Unable to find user with spn 'c8d74984-75d4-4ee5-84ce-6d2247b37094'
Unable to get object id from principal name.
+ sleep 10
+ i=1
+ [[ 1 -gt 10 ]]
+ az keyvault set-policy --name azwi-kv-f5f4 --secret-permissions get --spn c8d74984-75d4-4ee5-84ce-6d2247b37094
Location Name ResourceGroup
---------- ------------ --------------------
westus2 azwi-kv-f5f4 azwi-quickstart-1798
Raw
TF pod output.txt
(pts/2)dd0:~/coros/infrastructure-2/fresh_deployment% kubectl -n default logs quick-start
E0110 01:19:04.935479 1 main.go:60] "failed to get secret from keyvault" err=<
GET https://azwi-quickstart-coros.vault.azure.net/secrets/my-secret/
--------------------------------------------------------------------------------
RESPONSE 403: 403 Forbidden
ERROR CODE: Forbidden
--------------------------------------------------------------------------------
{
"error": {
"code": "Forbidden",
"message": "The user, group or application 'appid=59e5fc59-98cd-4cfa-b461-a1fdf876b868;oid=9f3e7e0d-7145-4743-b4dd-67aa0ec930ae;iss=https://sts.windows.net/8d24ddca-56ef-416b-aa0b-318257a21ae7/' does not have secrets get permission on key vault 'azwi-quickstart-coros;location=westus2'. For help resolving this issue, please see https://go.microsoft.com/fwlink/?linkid=2125287",
"innererror": {
"code": "AccessDenied"
}
}
}
--------------------------------------------------------------------------------
> keyvault="https://azwi-quickstart-coros.vault.azure.net/" secretName="my-secret"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment