Skip to content

Instantly share code, notes, and snippets.

@cpuguy83

cpuguy83/README.md

Last active November 13, 2019 03:59
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save cpuguy83/75f2f1c6556aa99118eef2830952c844 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save cpuguy83/75f2f1c6556aa99118eef2830952c844 to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. cpuguy83 revised this gist Nov 10, 2019. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion README.md
    View file
    Original file line number Diff line number Diff line change
    @@ -6,7 +6,7 @@ This *DOES NOT* detect a bad node/disk, only assists in cleaning it up.
    ### Usage

    ```terminal
    $ ./vmssfix.sh NODE_NAME PVC_NAME
    $ ./vmssfix.sh NODE_NAME PV_NAME
    ```
    This will trigger a detach of the pvc from the vmss instance in Azure.
    It will not perform any destructive action without user confirmation.
  2. cpuguy83 revised this gist Nov 9, 2019. 1 changed file with 3 additions and 4 deletions.
    7 changes: 3 additions & 4 deletions vmssfix.sh
    View file
    Original file line number Diff line number Diff line change
    @@ -55,18 +55,17 @@ for i in ${x[@]}; do

    lun="$(echo $i | awk '{ print $2}' )"


    echo az vmss disk detach --ids ${provider_id} --lun ${lun}
    read -r -p "Are you sure? [y/N] " response

    case "$response" in
    [yY][eE][sS]|[yY])
    az vmss disk detach --ids ${provider_id} --lun ${lun}
    ;;
    *)
    echo skipping
    continue
    ;;
    esac

    az vmss disk detach --ids ${provider_id} --lun ${lun}
    done

    done
  3. cpuguy83 revised this gist Nov 9, 2019. 1 changed file with 2 additions and 0 deletions.
    2 changes: 2 additions & 0 deletions README.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,6 +1,8 @@
    This script helps to clean up vmss nodes that may have Azure disks attached to it that should not be due to bugs in Kubernetes.
    The specific case for this is that a disk has been re-attached to by Kubernetes when it should not have been.

    This *DOES NOT* detect a bad node/disk, only assists in cleaning it up.

    ### Usage

    ```terminal
  4. cpuguy83 revised this gist Nov 9, 2019. 1 changed file with 10 additions and 0 deletions.
    10 changes: 10 additions & 0 deletions README.md
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,10 @@
    This script helps to clean up vmss nodes that may have Azure disks attached to it that should not be due to bugs in Kubernetes.
    The specific case for this is that a disk has been re-attached to by Kubernetes when it should not have been.

    ### Usage

    ```terminal
    $ ./vmssfix.sh NODE_NAME PVC_NAME
    ```
    This will trigger a detach of the pvc from the vmss instance in Azure.
    It will not perform any destructive action without user confirmation.
  5. cpuguy83 created this gist Nov 9, 2019.
    72 changes: 72 additions & 0 deletions vmssfix.sh
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,72 @@
    #!/bin/bash

    set -e -o pipefail

    node="$1"
    pvc="$2"

    usage() {
    echo $0 NODE PVC
    }

    if [ -z "${node}" ]; then
    echo "node must not be empty"
    usage
    exit 1
    fi

    if [ -z "${pvc}" ]; then
    echo "pvc must not be empty"
    usage
    exit 1
    fi

    provider_id="$(kubectl get node -o json ${node} | jq -r .spec.providerID | awk -F'azure://' '{ print $2 }')"

    instance_id="$(basename ${provider_id})"
    if [ -z "${instance_id}" ]; then
    echo "could not find vmss instance id, bailing out"
    exit 1
    fi

    rg="$(echo ${provider_id} | awk -F'/resourceGroups/' '{ print $2 }' | awk -F'/' '{ print $1 }')"
    if [ -z "${rg}" ]; then
    echo "could not find resource group name, bailing out"
    exit 1
    fi

    vmss_name="$(echo ${provider_id} | awk -F'/virtualMachineScaleSets/' '{ print $2 }' | awk -F '/' '{ print $1 }')"
    if [ -z "${vmss_name}" ]; then
    echo "could not find vmss name, bailing out"
    exit 1
    fi

    disks="$(az vmss show -g ${rg} -n ${vmss_name} --instance-id ${instance_id} -o json | jq -r '.storageProfile.dataDisks[] | "\(.managedDisk.id) \(.lun)"')"

    echo

    IFS=$'\n' x=($disks)
    for i in ${x[@]}; do
    if [[ ! "${i}" =~ "${pvc}" ]]; then
    continue
    fi



    lun="$(echo $i | awk '{ print $2}' )"


    echo az vmss disk detach --ids ${provider_id} --lun ${lun}
    read -r -p "Are you sure? [y/N] " response
    case "$response" in
    [yY][eE][sS]|[yY])
    ;;
    *)
    echo skipping
    continue
    ;;
    esac

    az vmss disk detach --ids ${provider_id} --lun ${lun}
    done