Skip to content

Instantly share code, notes, and snippets.

@tvon

tvon/rewrite-docker-tar

Last active January 31, 2020 02:32
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save tvon/a62214743b08204d1cfb5f15e4354efd to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save tvon/a62214743b08204d1cfb5f15e4354efd to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. tvon revised this gist Jan 31, 2020. 1 changed file with 2 additions and 2 deletions.
    4 changes: 2 additions & 2 deletions rewrite-docker-tar
    View file
    Original file line number Diff line number Diff line change
    @@ -4,8 +4,8 @@
    #
    # Exporting 'docker' tarballs with buildctl is creating manifests with no file
    # extensions (no .json or .tar.gz on the config or layers). This works with
    # dockerd because it never looks at the extensions and sniffs out what kidn of
    # file it is dealing with. Other tooling relies on extensions and checks for
    # dockerd because it never looks at the extensions just tries various things with
    # safe fallbacks. Other tooling relies on extensions and checks for
    # .json, .tar, and .tar.gz.
    #
    # This script will rebuild one of these containerd tarballs with file
  2. tvon created this gist Jan 31, 2020.
    76 changes: 76 additions & 0 deletions rewrite-docker-tar
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,76 @@
    #!/bin/bash
    #
    # rewrite-docker-tar ./image.tar ./output.tar
    #
    # Exporting 'docker' tarballs with buildctl is creating manifests with no file
    # extensions (no .json or .tar.gz on the config or layers). This works with
    # dockerd because it never looks at the extensions and sniffs out what kidn of
    # file it is dealing with. Other tooling relies on extensions and checks for
    # .json, .tar, and .tar.gz.
    #
    # This script will rebuild one of these containerd tarballs with file
    # extensions so most other tooling can process it.
    #
    # This is probably brittle as hell.

    set -euo pipefail

    image=${1:-''}
    result=${2:-''}

    if [[ -z "${image}" || -z "${result}" ]]; then
    echo "Usage: rewrite-docker-tar in.tar out.tar" && exit 1
    fi

    workdir=$(mktemp -d -t rewrite-docker-tar.XXXX)
    image_dir=${workdir}/image
    output_dir=${workdir}/output

    mkdir -p ${image_dir} ${output_dir}

    cleanup(){
    rc=${1:-0}
    find ${workdir}
    test -d ${workdir} && rm -rf ${workdir}
    exit ${rc}
    }

    trap cleanup EXIT

    contents=$(tar -tf ${image})
    tar -C ${image_dir} -xf ${image}

    manifest=${image_dir}/manifest.json

    # Extract the current manifest, I have only seen these fields in produced images.
    config_json=$(jq -r '.[].Config' ${manifest})

    layers=()
    for layer in $(jq -r '.[].Layers[]' ${manifest}); do
    layers+=(${layer})
    done

    # Rename files
    mv ${image_dir}/${config_json}{,.json}

    for layer in ${layers[@]}; do
    mv ${image_dir}/${layer}{,.tar.gz}
    done

    # Rewrite json
    #jq --arg manifest "${manifest}.json" '.[].Config = $manifest' ${image_dir}/manifest.json
    sed -i -e 's|'"${config_json}"'|'"${config_json}"'.json|' ${image_dir}/manifest.json
    for layer in ${layers[@]}; do
    sed -i -e 's|'"${layer}"'|'"${layer}"'.tar.gz|' ${image_dir}/manifest.json
    done

    # Build output tar, this could totally be incomplete if the format changes in containerd.
    #

    pushd ${image_dir} > /dev/null
    tar -cf ${output_dir}/out.tar -T /dev/null > /dev/null
    for file in *; do
    tar -rf ${output_dir}/out.tar ${file}
    done
    popd > /dev/null
    mv ${output_dir}/out.tar ${result}