Skip to content

Instantly share code, notes, and snippets.

@mcdonc

mcdonc/skopeo-copy-nix-to-docker-daemon.md

Created January 27, 2024 23:17
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save mcdonc/757f47737bd6e751bd47daf3ce58dae5 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save mcdonc/757f47737bd6e751bd47daf3ce58dae5 to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. mcdonc created this gist Jan 27, 2024.
    932 changes: 932 additions & 0 deletions skopeo-copy-nix-to-docker-daemon.md
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,932 @@
    I am using a project that uses skopeo under the hood. It generates a image json file that represents some Nix software
    that should be copied into a container image.

    I noticed a problem when I attempted to generate a particular container image. I debugged it to the point that
    I was able to know what command was "failing", and this turned out to be a `skopeo copy` command. When I run
    that `skopeo copy` command, it actually appears to succeed fine, but nothing winds up in the target container
    registry (the local `:docker-daemon`).

    I know skopeo is working in general on this system to generate images from Nix because I am totally able to generate a
    different container image within a different configuration, it works fine.

    Here's the output of `docker image ls` before I run the skopeo copy command:

    ```
    REPOSITORY TAG IMAGE ID CREATED SIZE
    eao_dash latest cc8c9f5f7473 N/A 2.58GB
    ```

    (that `eao_dash` image is an image generated using `skopeo-copy` from Nix on another project, which is how I know things
    are in-general working).

    Here is the command:

    ```
    /nix/store/ism9yqamyp6v4x82j2sdzsangkwj15v7-skopeo-1.13.3/bin/skopeo \
    --insecure-policy \
    copy \
    nix:/nix/store/23v8z9arf0q3zb5567b7a3plb9i9jxfh-image-climo2.json \
    docker-daemon:climo2:latest \
    --debug
    ```

    Here is its (truncated for brevity) output:

    ```
    time="2024-01-27T17:59:37-05:00" level=debug msg="Using blob info cache at /home/chrism/.local/share/containers/cache/blob-info-cache-v1.boltdb"
    time="2024-01-27T17:59:37-05:00" level=debug msg="IsRunningImageAllowed for image nix:"
    time="2024-01-27T17:59:37-05:00" level=debug msg=" Using default policy section"
    time="2024-01-27T17:59:37-05:00" level=debug msg=" Requirement 0: allowed"
    time="2024-01-27T17:59:37-05:00" level=debug msg="Overall: allowed"
    Getting image source signatures
    time="2024-01-27T17:59:37-05:00" level=debug msg="Manifest has MIME type application/vnd.oci.image.manifest.v1+json, ordered candidate list [application/vnd.docker.distribution.manifest.v2+json]"
    Copying blob sha256:1af7a14739dbe4ae5586bd203e36f221502e1fdcfec0615dc217d3a2e5248c9a
    time="2024-01-27T17:59:37-05:00" level=debug msg="Checking if we can reuse blob sha256:1af7a14739dbe4ae5586bd203e36f221502e1fdcfec0615dc217d3a2e5248c9a: general substitution = true, compression for MIME type \"application/vnd.oci.image.layer.v1.tar\" = true"
    time="2024-01-27T17:59:37-05:00" level=debug msg="Walking filesystem: /nix/store/s2f1sqfsdi4pmh23nfnrh42v17zsvi5y-libunistring-1.1"
    time="2024-01-27T17:59:37-05:00" level=debug msg="Walking filesystem: /nix/store/s2f1sqfsdi4pmh23nfnrh42v17zsvi5y-libunistring-1.1/lib"
    ... many other walking filesystem log messages --
    time="2024-01-27T17:59:39-05:00" level=debug msg="Walking filesystem: /nix/store/b5yhcx672hgzsi7dp922qhvqg81qfqr1-entrypoint"
    time="2024-01-27T17:59:39-05:00" level=debug msg="No compression detected"
    time="2024-01-27T17:59:39-05:00" level=debug msg="Using original blob without modification"
    time="2024-01-27T17:59:39-05:00" level=debug msg="docker tarfile: input with unknown size, streaming to disk first ..."
    time="2024-01-27T18:00:11-05:00" level=debug msg="... streaming done"
    time="2024-01-27T18:00:11-05:00" level=debug msg="Sending as tar file 1af7a14739dbe4ae5586bd203e36f221502e1fdcfec0615dc217d3a2e5248c9a.tar"
    Copying config sha256:dc3c287332a0f331b77e8a12b707e88a8d7afa3ca2b5357e0500ceb1d75b8feb
    time="2024-01-27T18:00:21-05:00" level=debug msg="No compression detected"
    time="2024-01-27T18:00:21-05:00" level=debug msg="Compression change for blob sha256:dc3c287332a0f331b77e8a12b707e88a8d7afa3ca2b5357e0500ceb1d75b8feb (\"application/vnd.docker.container.image.v1+json\") not supported"
    time="2024-01-27T18:00:21-05:00" level=debug msg="Using original blob without modification"
    time="2024-01-27T18:00:21-05:00" level=debug msg="Sending as tar file dc3c287332a0f331b77e8a12b707e88a8d7afa3ca2b5357e0500ceb1d75b8feb.json"
    Writing manifest to image destination
    time="2024-01-27T18:00:21-05:00" level=debug msg="Sending as tar link cb73c071de7f118d8a5f2d0b1d462d083a6434bd0cc63692422ce17d08324ef2/layer.tar -> ../1af7a14739dbe4ae5586bd203e36f221502e1fdcfec0615dc217d3a2e5248c9a.tar"
    time="2024-01-27T18:00:21-05:00" level=debug msg="Sending as tar file cb73c071de7f118d8a5f2d0b1d462d083a6434bd0cc63692422ce17d08324ef2/VERSION"
    time="2024-01-27T18:00:21-05:00" level=debug msg="Sending as tar file cb73c071de7f118d8a5f2d0b1d462d083a6434bd0cc63692422ce17d08324ef2/json"
    time="2024-01-27T18:00:21-05:00" level=debug msg="docker-daemon: Closing tar stream"
    time="2024-01-27T18:00:21-05:00" level=debug msg="Sending as tar file manifest.json"
    time="2024-01-27T18:00:21-05:00" level=debug msg="Sending as tar file repositories"
    time="2024-01-27T18:00:21-05:00" level=debug msg="docker-daemon: Waiting for status"
    time="2024-01-27T18:00:34-05:00" level=debug msg="docker-daemon: sending done, status <nil>"
    ```

    It exits with a success return code, however, it does not push anything into the local docker-daemon registry. The output
    of `docker image ls` remains unchanged.

    There is no useful `dockerd` output in `sudo journalctl -f`.

    The `dockerd` version is 24.0.5. I have also tried 24.08 and 23.08 with the same result.

    Finally, here is the (huge) json file it is using as input:

    ```json
    {
    "version": 1,
    "image-config": {
    "Env": [
    "DEVENV_DOTFILE='/.devenv'",
    "DEVENV_PROFILE='/nix/store/3j8p1zirxnw4v60fbab7b5qknfz83xy5-devenv-profile'",
    "DEVENV_ROOT='/'",
    "DEVENV_STATE='/.devenv/state'",
    "MSSQLTOOLSSERVICE_PATH='/nix/store/5fdjh005gwmn4acq10gl8y4vjm803lla-ms-sqltoolsservice-4.11.0.10/bin'",
    "ODBCINSTINI='ywgv8ian3q8mlkydnpqqa5xndvh05pwi-odbcini'",
    "ODBCSYSINI='/nix/store'",
    "PGDATA='/.devenv/state/postgres'",
    "PGHOST='/.devenv/state/postgres'",
    "PGPORT='5432'"
    ],
    "Entrypoint": [
    "/nix/store/b5yhcx672hgzsi7dp922qhvqg81qfqr1-entrypoint"
    ],
    "Cmd": [
    "/bin/bash\n"
    ]
    },
    "layers": [
    {
    "digest": "sha256:1af7a14739dbe4ae5586bd203e36f221502e1fdcfec0615dc217d3a2e5248c9a",
    "size": 1639057408,
    "diff_ids": "sha256:1af7a14739dbe4ae5586bd203e36f221502e1fdcfec0615dc217d3a2e5248c9a",
    "paths": [
    {
    "path": "/nix/store/s2f1sqfsdi4pmh23nfnrh42v17zsvi5y-libunistring-1.1"
    },
    {
    "path": "/nix/store/08n25j4vxyjidjf93fyc15icxwrxm2p8-libidn2-2.3.4"
    },
    {
    "path": "/nix/store/lmidwx4id2q87f4z9aj79xwb03gsmq5j-xgcc-12.3.0-libgcc"
    },
    {
    "path": "/nix/store/qn3ggz5sf3hkjs2c797xf7nan3amdxmp-glibc-2.38-27"
    },
    {
    "path": "/nix/store/7vq9zymw7zk2i0k1i4nfkl6hmlx3gxji-gcc-12.3.0-libgcc"
    },
    {
    "path": "/nix/store/myw67gkgayf3s2mniij7zwd79lxy8v0k-gcc-12.3.0-lib"
    },
    {
    "path": "/nix/store/8xgb8phqmfn9h971q7dg369h647i1aa0-zlib-1.3"
    },
    {
    "path": "/nix/store/q1c2flcykgr4wwg5a6h450hxbk4ch589-bash-5.2-p15"
    },
    {
    "path": "/nix/store/11xpmmwy95396nkhih3qc3814lqhqb8f-libunistring-0.9.10"
    },
    {
    "path": "/nix/store/1mpxs3109cjrbhmi3q1vmvc0djz102pl-libidn2-2.3.2"
    },
    {
    "path": "/nix/store/mij848h2x5wiqkwhg027byvmf9x3gx7y-glibc-2.33-50"
    },
    {
    "path": "/nix/store/k3d7ny5h682kixy91iw7sm653kxyqr8d-openssl-3.0.12"
    },
    {
    "path": "/nix/store/1ryrrnjsxnj48hic5bk03hgvcp38sbli-zstd-1.5.5"
    },
    {
    "path": "/nix/store/vbb1l2krfp75m5w3y2h52l0pbq6fydz8-attr-2.5.1"
    },
    {
    "path": "/nix/store/mmy4fc1arb2ig642chgprgms27vk9hgn-acl-2.3.1"
    },
    {
    "path": "/nix/store/wzqs107xkwdxls33skg2ma9rl7j1i68f-xz-5.4.4"
    },
    {
    "path": "/nix/store/chkjrz6qy7i9ma62j10ij1gks571g3jn-keyutils-1.6.3-lib"
    },
    {
    "path": "/nix/store/7d2qkskb5yg9p5lgvspapw15dkc3y3zx-libkrb5-1.20.2"
    },
    {
    "path": "/nix/store/p40jraqsma0yb56bblyk19xmvi4d3lkl-util-linux-minimal-2.39.2-lib"
    },
    {
    "path": "/nix/store/7b2iijic7casyjcr799iqn5i7q8ax72f-libgpg-error-1.47"
    },
    {
    "path": "/nix/store/5i51nfixxx3p3gshkfsjj4bzp7wajwxz-ncurses-6.4"
    },
    {
    "path": "/nix/store/injawg30ws0a6ydwbsvaaa1p0ky82pjr-gmp-with-cxx-6.3.0"
    },
    {
    "path": "/nix/store/155qsyx1mv11fsi48nz4dlc0vh1a3drx-bzip2-1.0.8"
    },
    {
    "path": "/nix/store/bblyj5b3ii8n6v4ra0nb37cmi3lf8rz9-coreutils-9.3"
    },
    {
    "path": "/nix/store/ksdrjk9519bx41hbam04xpgmgg4s6q5i-nghttp2-1.57.0-lib"
    },
    {
    "path": "/nix/store/7y0ng2kzq6zxxvjq10yi8xrda3f3yz8j-brotli-1.1.0-lib"
    },
    {
    "path": "/nix/store/4vqq3gb4qwr1s2vwgx09bl6slqxkh6v6-libssh2-1.11.0"
    },
    {
    "path": "/nix/store/aiv6dg0wz1k68l624x8m1vfqn9rmsdkl-curl-8.4.0"
    },
    {
    "path": "/nix/store/s3s7gv33p88kzbgki2bprg2a1nc7jnf8-gmp-with-cxx-6.3.0"
    },
    {
    "path": "/nix/store/4px7vzc531y10r7mmxps1s32cbshbjr2-pcre2-10.42"
    },
    {
    "path": "/nix/store/ixvka49i2mjnxzk8l8anykc1idilws1r-libxcrypt-4.4.36"
    },
    {
    "path": "/nix/store/aqmgny7rs6zyni27v3pxrqmpajdmdj1n-libmnl-1.0.5"
    },
    {
    "path": "/nix/store/jbdicrrzljvl9qnzp3rdzjbr94g6mnl7-libtasn1-4.19.0"
    },
    {
    "path": "/nix/store/8skq4a62klp1s3lldgf60ybq3h9z0m4r-libcap-2.69-lib"
    },
    {
    "path": "/nix/store/9kd4bc8fpclpvf1vdwlbila71svyb6w1-libffi-3.4.4"
    },
    {
    "path": "/nix/store/rx2wig5yhpbwhnqxdy4z7qivj9ln7fab-gnugrep-3.11"
    },
    {
    "path": "/nix/store/v0q6h046vc4ji6lxk0ivb3zg9cjcgh3k-libxml2-2.11.5"
    },
    {
    "path": "/nix/store/yrml8d394c72yffxqwpm4wl4ycfha4wr-readline-8.2p1"
    },
    {
    "path": "/nix/store/jzjqfff4cldlm2wpld313a2s1v6r3ycb-ncurses-6.2"
    },
    {
    "path": "/nix/store/c5giadpm71mvla6202mhz32x9criabna-zlib-1.2.11"
    },
    {
    "path": "/nix/store/44xy8mm4jbiyp41fg2g4dwjh6aqr69wm-perl5.38.0-TimeDate-2.33"
    },
    {
    "path": "/nix/store/xrwvr586g0i0rlw84k9dybijlkiz9ycl-perl5.38.0-URI-5.21"
    },
    {
    "path": "/nix/store/iz7vlhvsr1lll8lzwd36ffwkzccgqkjc-nettle-3.9.1"
    },
    {
    "path": "/nix/store/vwfzrh56pghvnmal8gmlxgyigndx6k9h-libnfnetlink-1.0.2"
    },
    {
    "path": "/nix/store/iq5yz5c09d6d3lbb67jrndipv2x8xhgj-systemd-minimal-libs-254.6"
    },
    {
    "path": "/nix/store/3rmfwcnhj1z3qya2lx0dlm7j129sfzfk-json-c-0.16"
    },
    {
    "path": "/nix/store/jcq3h77q118ahjz6424k90g4093ahzjm-p11-kit-0.25.0"
    },
    {
    "path": "/nix/store/1zf8d2qzpx32mbzfsyzd51xshaid6xqf-elfutils-0.190"
    },
    {
    "path": "/nix/store/1whrfrfzjbbjl7hfdh34qhbzlckhpalc-libgcrypt-1.10.2"
    },
    {
    "path": "/nix/store/r9yqs0v7af971b0rc7cmw7lsdjixwlj3-audit-3.1.2"
    },
    {
    "path": "/nix/store/wadmyilr414n7bimxysbny876i2vlm5r-bash-5.1-p8"
    },
    {
    "path": "/nix/store/rf6ys1x5xcyiia7wngr0k7kgsqv8rrjd-zstd-1.5.5-bin"
    },
    {
    "path": "/nix/store/874gxh0sgwkbwb7nfbp72sn7a058b7sa-lz4-1.9.4"
    },
    {
    "path": "/nix/store/jnkicl228rqax2r7kidg0dhwnmlfnmki-perl5.38.0-HTTP-Date-6.06"
    },
    {
    "path": "/nix/store/cx01qk0qyylvkgisbwc7d3pk8sliccgh-glibc-2.38-27-bin"
    },
    {
    "path": "/nix/store/xpidksbd07in3nd4sjx79ybwwy81b338-gnutar-1.35"
    },
    {
    "path": "/nix/store/7fsdrvpwxnfb8wrb9c6wa1hzw71gn4yz-gmp-6.3.0"
    },
    {
    "path": "/nix/store/n9s7hv5rw1fp1wfnc69iy3is12r26wfd-perl5.38.0-Clone-0.46"
    },
    {
    "path": "/nix/store/lmcd6kbhklpmcy65r65gk1ndxg233ail-perl5.38.0-Encode-Locale-1.05"
    },
    {
    "path": "/nix/store/fl6411hpgxys1807bj9h5w4f6idpf21r-perl5.38.0-IO-HTML-1.004"
    },
    {
    "path": "/nix/store/58x0bb5gzgsrgmirwkd7ly1q15slnlza-perl5.38.0-LWP-MediaTypes-6.04"
    },
    {
    "path": "/nix/store/fp3y35hsv1q080i99ll3wnyh5lmsasmi-perl5.38.0-HTTP-Message-6.45"
    },
    {
    "path": "/nix/store/vzg3i64357bfnhd9chnij1qldbl3nqz3-libXau-1.0.11"
    },
    {
    "path": "/nix/store/mp7kxwyb5jnndv5x53swr34c37yimljm-libXdmcp-1.1.4"
    },
    {
    "path": "/nix/store/a4rcsjklsqqrbpxcf46p6srp8x1bjpnf-libxcb-1.16"
    },
    {
    "path": "/nix/store/hmrg92skm94xhaippz5fqcrv992qq90b-libX11-1.8.7"
    },
    {
    "path": "/nix/store/202iqv4bd7lh6f7fpy48p7q4d96lqdp7-gzip-1.13"
    },
    {
    "path": "/nix/store/ik7jardq92dxw3fnz3vmlcgi9c8dwwdq-bzip2-1.0.8-bin"
    },
    {
    "path": "/nix/store/76z4cjs7jj45ixk12yy6k5z2q2djk2jb-xz-5.4.4-bin"
    },
    {
    "path": "/nix/store/gz6mkv6pfkkm7fb901kqvzwhh25s54c9-libossp-uuid-1.6.2"
    },
    {
    "path": "/nix/store/sr16hq3pdq96w5kbbksnngad2czxfvys-openssl-1.1.1l"
    },
    {
    "path": "/nix/store/sl7zsdx58kwgnf2dgj0qjm03i6d1223b-sqlite-3.36.0"
    },
    {
    "path": "/nix/store/pvkngxfr8gqx4absf8635gvydpvsxcb4-readline-6.3p08"
    },
    {
    "path": "/nix/store/f5d50ymyidsqkrwikkhrh4ci8sg1dvs4-expat-2.4.1"
    },
    {
    "path": "/nix/store/csxpmhlzyhdkjir2clakkfp66yyflfvg-bzip2-1.0.6.0.2"
    },
    {
    "path": "/nix/store/a3v9gckl2l893yn817ssfdkw1hb27ki9-xz-5.2.5"
    },
    {
    "path": "/nix/store/6wgljzd2648yv3a908ikdprzpnaga3x7-gdbm-1.20"
    },
    {
    "path": "/nix/store/6kgfmzx90c1a6afqnbkz6qprkzss476k-mime-types-9"
    },
    {
    "path": "/nix/store/2z8hacx9dphisbf7syd9hvkiw4578r23-libffi-3.4.2"
    },
    {
    "path": "/nix/store/3vx4ajfr2gy3zbl803frpabh4n85cacp-tzdata-2023c"
    },
    {
    "path": "/nix/store/kanzv0wxiscx6ax5xx81bzlpiy2qxwxd-python3-3.6.14"
    },
    {
    "path": "/nix/store/2gjifw9iylbkiv56jyvfi2ybyc0ivibi-icu4c-73.2"
    },
    {
    "path": "/nix/store/h72wbvjy0ijg1df58p50biv84j18823q-libevent-2.1.12"
    },
    {
    "path": "/nix/store/whd27ii6xl42nfzxgh6lx7iknmdfwdx7-libnl-3.7.0"
    },
    {
    "path": "/nix/store/y37fpsmfmcqvb12bydq2h4rw8s8i4wkd-unbound-1.18.0-lib"
    },
    {
    "path": "/nix/store/cx5hr3lcvplalqfqhawp4ishrn7ischx-dns-root-data-2019-01-11"
    },
    {
    "path": "/nix/store/xq5alk31bmx5gy5xlb56jbgkx0ac9f81-libnetfilter_conntrack-1.0.9"
    },
    {
    "path": "/nix/store/gifh4b7pqhkr3mn552r4jl9hj4yk7vn9-libnftnl-1.2.6"
    },
    {
    "path": "/nix/store/bzidh53vs7h7p1r8j73blir9xpi2650h-libpcap-1.10.4"
    },
    {
    "path": "/nix/store/h10gsyrw8z0h8cc4zg7g6v9c2mv97jxw-pcsclite-1.9.5"
    },
    {
    "path": "/nix/store/aqqw20z31b7sxrlkbdx953sff1cfghxj-libcbor-unstable-2023-01-29"
    },
    {
    "path": "/nix/store/4cjkx8v3z93v0l98hiwrv7960xshg9qb-libassuan-2.5.6"
    },
    {
    "path": "/nix/store/m4ikf53wndjg3dcwsnvicagn16kal1fy-db-4.8.30"
    },
    {
    "path": "/nix/store/14vj8gyjlp6nd6y54qqkic1ab3l9hf74-npth-1.6"
    },
    {
    "path": "/nix/store/n1gr11z8y0kpxcrm4mk0cshz2v15g4mh-lvm2-2.03.22-lib"
    },
    {
    "path": "/nix/store/jm1p5p48d6x6cl2c268g3dm0cxwa208p-libargon2-20190702"
    },
    {
    "path": "/nix/store/0fyxm3fyswhp0wbw1v8hki833qgprr37-glibc-2.38-27-getent"
    },
    {
    "path": "/nix/store/0bk337b0fpqdqjj4jsv7prmviw58fr08-gnutls-3.8.2"
    },
    {
    "path": "/nix/store/f0hfmvdm4xy7g03q4f8zbi2q4h6d5wss-ncurses-6.4-man"
    },
    {
    "path": "/nix/store/yjmdk8swmam79jkr4km6hjwdk4g22996-util-linux-minimal-2.39.2-login"
    },
    {
    "path": "/nix/store/x943z8xpk8jnmfd0czqwgyp3ghgdzw20-libseccomp-2.5.4-lib"
    },
    {
    "path": "/nix/store/x88ivkf7rmrhd5x3cvyv5vh3zqqdnhsk-bash-interactive-5.2-p15"
    },
    {
    "path": "/nix/store/wr5z7dghb487d4w7xa5y9a2nzfziyv82-tpm2-tss-4.0.1"
    },
    {
    "path": "/nix/store/w2lpzmvl3w613mz547zg1a99dfhxsj7z-util-linux-minimal-2.39.2-mount"
    },
    {
    "path": "/nix/store/rnxg089s7m17w48i1rvdxdg7jv194ayy-util-linux-minimal-2.39.2-swap"
    },
    {
    "path": "/nix/store/qnbwgvwprl13cw44p9sizyvlq7gr20z7-iptables-1.8.10"
    },
    {
    "path": "/nix/store/mdjq8j0lx2bdsri5zycpw597z78apqfa-libmicrohttpd-0.9.71"
    },
    {
    "path": "/nix/store/jakggpdjsardflc64id2jkvv2ba3bxbq-kexec-tools-2.0.26"
    },
    {
    "path": "/nix/store/j0ljmgj3p2sdx4wj9fgsl4c69yjrw9fx-gnupg-2.4.1"
    },
    {
    "path": "/nix/store/gwsk1v8pkyd8ds1gpjhv6wl95rcxfpwd-getent-glibc-2.38-27"
    },
    {
    "path": "/nix/store/cc5vgczy2ryqg7v8lsjypwq3n534rvnd-kbd-2.6.3"
    },
    {
    "path": "/nix/store/blimya0m8z5wfxh28nsddhlwgyrrvh4f-libfido2-1.13.0"
    },
    {
    "path": "/nix/store/awwmn6ksi2sqvyd6gx9g2dsy2hg0cmjm-libbpf-1.2.2"
    },
    {
    "path": "/nix/store/8hj2wknr00ihj7c0rd5wmfq8q9x9000w-kmod-31-lib"
    },
    {
    "path": "/nix/store/1vj56kk9k01g1spfcwj6d50aal9kvpam-kmod-31"
    },
    {
    "path": "/nix/store/18z7m0ikjz1hx3kb2z22y4017fx14v9l-linux-pam-1.5.2"
    },
    {
    "path": "/nix/store/0p54fqkhd1zgml2rksj9dykdihfl7k88-cryptsetup-2.6.1"
    },
    {
    "path": "/nix/store/035ar7f41f3plxl4q579ixzqgfyxrclq-libapparmor-3.1.6"
    },
    {
    "path": "/nix/store/hd191cw5fi2k194gg5kjqh7q47k44pq7-ncurses-6.4-dev"
    },
    {
    "path": "/nix/store/glmaxm53w5kfdbl8pl1yfyv8rmhszb7f-lz4-1.9.4-bin"
    },
    {
    "path": "/nix/store/yna5rl3z17hxjw1n3ppljn8h3gjbdi5m-libxml2-2.11.5-bin"
    },
    {
    "path": "/nix/store/9a4ivnm2hwggi6qjg0gcpk05f5hsw5r5-zlib-1.3-dev"
    },
    {
    "path": "/nix/store/7ngsbi9saqi6wy61f2dnsb8qw73xagsv-systemd-254.6"
    },
    {
    "path": "/nix/store/7j9lfm5nwwcgkdsir4m4pqs4sj0rwbq0-openssl-3.0.12-bin"
    },
    {
    "path": "/nix/store/3kzhxyviqpwv4ghm8pj3n325vs3mdk12-perl5.38.0-HTML-Tagset-3.20"
    },
    {
    "path": "/nix/store/7bj4kp0yf0camdasb88z4k4gy0dgm0ak-expat-2.5.0"
    },
    {
    "path": "/nix/store/am68gj0i5hdyfhagrilimq9zylf8wdwp-linux-headers-6.5"
    },
    {
    "path": "/nix/store/71fvzihav5227cvxisnb475vbal90457-sqlite-3.43.2"
    },
    {
    "path": "/nix/store/mrgib0s2ayr81xv1q84xsjg8ijybalq3-glibc-2.38-27-dev"
    },
    {
    "path": "/nix/store/bgjax7pyphpzfi7h5jdm3i4hc4q5aici-perl5.38.0-HTML-Parser-3.81"
    },
    {
    "path": "/nix/store/w09nf1k5cwkdcs13317nl3bbmdcxgz9m-mpfr-4.2.1"
    },
    {
    "path": "/nix/store/xpbvz28h8q08hwsfa4ghdmlgh6r87plw-postgresql-16.1-lib"
    },
    {
    "path": "/nix/store/vdzdipmq92zcxs47qdvm2fkahglmvgjq-mailcap-2.1.53"
    },
    {
    "path": "/nix/store/kjhznvfzdp095azz7blryagi76dhxmlr-libXext-1.3.5"
    },
    {
    "path": "/nix/store/sbij2dnwi3vmjpzpsr1al44vmqrrzy3m-libICE-1.1.1"
    },
    {
    "path": "/nix/store/ynik8rsd312mswcyjazjjsmq9d0xx8an-find-xml-catalogs-hook"
    },
    {
    "path": "/nix/store/62nz2ihx8g8m3s9lipmzl8ffk13vzbax-perl5.38.0-Net-SSLeay-1.92"
    },
    {
    "path": "/nix/store/zxk3x3ixryxixwyljz1sipxzqhg7cvvy-libkrb5-1.20.2-dev"
    },
    {
    "path": "/nix/store/r9vjd4py76amxqdrv1p3w36nrpvld304-systemd-254.6-dev"
    },
    {
    "path": "/nix/store/qsjbb26aqql7a0v8nhgs6fk6z3l9fn3p-readline-8.2p1-dev"
    },
    {
    "path": "/nix/store/n1v6jm64720xs8ziya9377g97g2jd7nd-lz4-1.9.4-dev"
    },
    {
    "path": "/nix/store/lhgj80qbyazybi5cb6kkxr8f9vj141xq-openssl-3.0.12-dev"
    },
    {
    "path": "/nix/store/hq1zn6xmgwkawp86bhdyfsmx89ma1923-zstd-1.5.5-dev"
    },
    {
    "path": "/nix/store/czjrzhjqdf9d6bfpw91qnji90yf5g0i9-libxml2-2.11.5-dev"
    },
    {
    "path": "/nix/store/cs1hycx3bnfyvxkq8jv5xr0pv5d7ksj7-icu4c-73.2-dev"
    },
    {
    "path": "/nix/store/vqs938kkhlq8x4yjyi0rg4wc31gqm1y3-unixODBC-2.3.12"
    },
    {
    "path": "/nix/store/ffsh04qav36c283cnxsknnb99k4c76jr-postgresql-16.1"
    },
    {
    "path": "/nix/store/w8f86kvpfp9fvazi5i4iimkrrsk5mcah-gdbm-1.23"
    },
    {
    "path": "/nix/store/limd2s0l2fwqx555gnw235xljh83900i-openssl-3.0.12"
    },
    {
    "path": "/nix/store/4dr83a4wn25zkb5pvsq2y3q3v586jzs9-expand-response-params"
    },
    {
    "path": "/nix/store/w8mwxc0q6qj1d8wdp8dkpbrmavwsfl6x-perl5.38.0-Try-Tiny-0.31"
    },
    {
    "path": "/nix/store/im29gnszjnfxyfw5pw8hva92d4y42mwb-python3.6-future-0.18.2"
    },
    {
    "path": "/nix/store/g04k0q0l1581vk60rqnssxdh1v232zj1-python3.6-wheel-0.36.2"
    },
    {
    "path": "/nix/store/6d7g7vlzvdxr66ppi0mcb5yp4l7hx8b0-perl5.38.0-Mozilla-CA-20230821"
    },
    {
    "path": "/nix/store/v1jvkzadd79k62agwgq16javfl9ammcl-perl5.38.0-IO-Socket-SSL-2.083"
    },
    {
    "path": "/nix/store/wx6hgc4gz4h18gn7jbg16kbca6zqsyq6-perl5.38.0-CGI-4.59"
    },
    {
    "path": "/nix/store/m5hvx64acckwyf99q9cykcpq95m21igk-perl5.38.0-FCGI-0.82"
    },
    {
    "path": "/nix/store/afz9z2iczjx13i66ymwwinwp2zvgc1ck-perl-5.38.0"
    },
    {
    "path": "/nix/store/01r2sdjs87b81pzgih4arb9qij1yx41n-perl5.38.0-Digest-HMAC-1.04"
    },
    {
    "path": "/nix/store/v1nmbk9bzlxdgfa4cjlsxrpxqabn0bjb-isl-0.20"
    },
    {
    "path": "/nix/store/3ysf58a4zshpi654ya4hsfbhvmnnjrqf-libmpc-1.3.1"
    },
    {
    "path": "/nix/store/mnlqpncq82lw07iy6yzbb7gxz78l10cy-binutils-2.40-lib"
    },
    {
    "path": "/nix/store/wmknncrif06fqxa16hpdldhixk95nds0-strip.sh"
    },
    {
    "path": "/nix/store/9c5qm297qnvwcf7j0gm01qrslbiqz8rs-gnused-4.9"
    },
    {
    "path": "/nix/store/hf2gy3km07d5m0p1lwmja0rg9wlnmyr7-gcc-12.3.0"
    },
    {
    "path": "/nix/store/f4ldf31rn0zyh4p47drs4462a1nmx9qx-sqlite-3.43.2-bin"
    },
    {
    "path": "/nix/store/1fn92b0783crypjcxvdv6ycmvi27by0j-binutils-2.40"
    },
    {
    "path": "/nix/store/kkcxvcj27x2lr1wkb6fnxlj3ssdx7vsm-pcre-8.45"
    },
    {
    "path": "/nix/store/xlyhjjnmwr11bzmbky4l779gy3nip7ga-libglvnd-1.7.0"
    },
    {
    "path": "/nix/store/6nds0a8xh53qz9adx271i17claw0qfrj-libselinux-3.3"
    },
    {
    "path": "/nix/store/ljwksjfdcrk0ac199pnhivxai8ksj89q-libGL-1.7.0"
    },
    {
    "path": "/nix/store/gd5sm7yrr7mfnz9gqyf9dxg1pmfs00vm-libXrender-0.9.11"
    },
    {
    "path": "/nix/store/g878xdyhrzlhbf80jnknnp14w82j919k-glib-2.78.1"
    },
    {
    "path": "/nix/store/85ndgar09amz954z8c4kvzfl1zk333k9-libSM-1.2.4"
    },
    {
    "path": "/nix/store/yhg3i1kd3lj1z3jmqgzsaapzr8kf51wp-manylinux2014"
    },
    {
    "path": "/nix/store/mci24w2n10pw4zsq9vj37dbggngr39q9-python3-3.11.7"
    },
    {
    "path": "/nix/store/9jvj17ydc2rl6jhxnqbdpl03k6pjrd9v-libkrb5-1.20.2"
    },
    {
    "path": "/nix/store/5pf6rhrrra110xr2xbnc3xz6fda6kjf3-openssl-1.1.1w"
    },
    {
    "path": "/nix/store/z9xlgxp9y0rp1fihhsv75grhv9lci3x3-git-2.42.0-doc"
    },
    {
    "path": "/nix/store/c54kyc3vn0jbq5hznimvx6wz15jxw8jq-gnu-config-2023-09-19"
    },
    {
    "path": "/nix/store/hvi3hx26haw91f1nd5l4hp92gkpv7rg1-msodbcsql17-17.7.1.1-1"
    },
    {
    "path": "/nix/store/dmfigbzp4b69m094wb76ky9cb0c94h6x-python3-3.11.7-env"
    },
    {
    "path": "/nix/store/1alqjnr40dsk7cl15l5sn5y2zdxidc1v-binutils-wrapper-2.40"
    },
    {
    "path": "/nix/store/mr1m7vvaqq522cgwm2fxglgkm1rn4irg-perl5.38.0-HTTP-Cookies-6.10"
    },
    {
    "path": "/nix/store/hw5cmhsd546gw4gjmra1cyaka08flx01-perl5.38.0-Test-RequiresInternet-0.05"
    },
    {
    "path": "/nix/store/b6gc735pb8fal9m883inhc7b6hqzkgp3-perl5.38.0-HTTP-CookieJar-0.014"
    },
    {
    "path": "/nix/store/9y36vjs6xsakhyn3p742f29g25a54h6q-perl5.38.0-File-Listing-6.16"
    },
    {
    "path": "/nix/store/9vvzj97qkxnlba148l2k0lhlgpb90rcj-perl5.38.0-Net-HTTP-6.23"
    },
    {
    "path": "/nix/store/lyw6rjqxj51ixvw214xgz2lzixm0bzyc-postgresql.conf"
    },
    {
    "path": "/nix/store/7274bwbvl9a7kcghb1kkbqdjqggm0wry-perl5.38.0-Test-Needs-0.002010"
    },
    {
    "path": "/nix/store/6najzl9j69ps3d9s3qgjyl3wqf5nwkbg-perl5.38.0-Test-Fatal-0.017"
    },
    {
    "path": "/nix/store/64399m6bbll8ylia5ysf451c3cj3n1kp-perl5.38.0-WWW-RobotRules-6.02"
    },
    {
    "path": "/nix/store/5rdhns8kf7r5085mnccbr53y47qqm01f-perl5.38.0-HTTP-Daemon-6.16"
    },
    {
    "path": "/nix/store/1sbvn9353xs871lavizyn2brv4rwhj84-perl5.38.0-HTTP-Negotiate-6.01"
    },
    {
    "path": "/nix/store/2y3vj12vrpkz1xr8asll41rs2mjnz8xr-python3.6-mssql-scripter-1.0.0a23"
    },
    {
    "path": "/nix/store/wxjk2v5vjxmpm7lkiy0mk9xcqcrhp007-gettext-0.21.1"
    },
    {
    "path": "/nix/store/qwb7dwq5g8dzb5v166mabqcvyjf8yjzq-perl5.38.0-Net-SMTP-SSL-1.04"
    },
    {
    "path": "/nix/store/ljn6af1x0wsfzirvq99pxrkmy8qnc39b-perl5.38.0-Authen-SASL-2.1700"
    },
    {
    "path": "/nix/store/jjbbjykz8q3h30qqknzsynp3vj8fvq73-perl5.38.0-TermReadKey-2.38"
    },
    {
    "path": "/nix/store/ialds7rp49iylmj11vj62anhjhfihzkm-perl5.38.0-libwww-perl-6.72"
    },
    {
    "path": "/nix/store/hfn73jilzfivws4mxsmmljm5jn3apqli-perl5.38.0-CGI-Fast-2.16"
    },
    {
    "path": "/nix/store/fmwq823zqsrwkv3kqlvh8p9yyglhg8rg-perl5.38.0-libnet-3.15"
    },
    {
    "path": "/nix/store/sk1nfklnmkiw61igpg6cag7idzbfl8b0-iana-etc-20230316"
    },
    {
    "path": "/nix/store/9z3v4n4lxhqwpi032g1j1a5adp4xxq49-pkg-config-0.29.2"
    },
    {
    "path": "/nix/store/7np6i72hk67bn8c0vd60pkl1vzx71dm3-setup-postgres"
    },
    {
    "path": "/nix/store/2ii0b9dq7gn6m54aix8kyr1kv8bqy9p9-ed-1.19"
    },
    {
    "path": "/nix/store/27vf1jzj26mxh9yfc4nm4vdj3m9q9gz0-python3-3.11.6"
    },
    {
    "path": "/nix/store/0yxmp3kyjfd3x4k9y13dxwbf5kdsvy39-perl5.38.0-FCGI-ProcManager-0.28"
    },
    {
    "path": "/nix/store/0y5vr6r9kmfp0km62l2mlb4w6ffrqdas-perl5.38.0-HTML-TagCloud-0.38"
    },
    {
    "path": "/nix/store/0cldli42g1whg3mgsj46ibalxc72hwc3-python3-3.6.14-env"
    },
    {
    "path": "/nix/store/xyff06pkhki3qy1ls77w10s0v79c9il0-reproducible-builds.sh"
    },
    {
    "path": "/nix/store/wzdsbnv2ba3nj91aql8jjdddfmkkdh7h-patch-shebangs.sh"
    },
    {
    "path": "/nix/store/wgrbkkaldkrlrni33ccvm3b6vbxzb656-make-symlinks-relative.sh"
    },
    {
    "path": "/nix/store/v4iswb5kwj33l46dyh2zqh0nkxxlr3mz-gnumake-4.4.1"
    },
    {
    "path": "/nix/store/qmfxld7qhk8qxlkx1cm4bkplg1gh6jgj-file-5.45"
    },
    {
    "path": "/nix/store/pag6l61paj1dc9sv15l7bm5c17xn5kyk-move-systemd-user-units.sh"
    },
    {
    "path": "/nix/store/ngg1cv31c8c7bcm2n8ww4g06nq7s4zhm-set-source-date-epoch-to-latest.sh"
    },
    {
    "path": "/nix/store/m54bmrhj6fqz8nds5zcj97w9s9bckc9v-compress-man-pages.sh"
    },
    {
    "path": "/nix/store/l974pi8a5yqjrjlzmg6apk0jwjv81yqw-findutils-4.9.0"
    },
    {
    "path": "/nix/store/kd4xwxjpjxi71jkm6ka0np72if9rm3y0-move-sbin.sh"
    },
    {
    "path": "/nix/store/jivxp510zxakaaic7qkrb7v1dd2rdbw9-multiple-outputs.sh"
    },
    {
    "path": "/nix/store/h9lc1dpi14z7is86ffhl3ld569138595-audit-tmpdir.sh"
    },
    {
    "path": "/nix/store/fyaryjvghbkpfnsyw97hb3lyb37s1pd6-move-lib64.sh"
    },
    {
    "path": "/nix/store/cickvswrvann041nqxb0rxilc46svw1n-prune-libtool-files.sh"
    },
    {
    "path": "/nix/store/cbj1ph7zi009m53hxs90idl1f5i9i941-patch-2.7.6"
    },
    {
    "path": "/nix/store/9vk5fqaik9zx14zpfxipg7lxkdk8c5g9-update-autotools-gnu-config-scripts-hook"
    },
    {
    "path": "/nix/store/90h6k8ylkgn81k10190v5c9ldyjpzgl9-gcc-wrapper-12.3.0"
    },
    {
    "path": "/nix/store/8q25nyfirzsng6p57yp8hsaldqqbc7dg-diffutils-3.10"
    },
    {
    "path": "/nix/store/85jldj870vzcl72yz03labc93bwvqayx-patchelf-0.15.0"
    },
    {
    "path": "/nix/store/7wfya2k95zib8jl0jk5hnbn856sqcgfk-gawk-5.2.2"
    },
    {
    "path": "/nix/store/5yzw0vhkyszf2d179m0qfkgxmp5wjjx4-move-docs.sh"
    },
    {
    "path": "/nix/store/yd58q40yb1mnn7f6p1c7sh6bwrkvsvs7-mssql-scripter"
    },
    {
    "path": "/nix/store/pyd8yylv6y0psnivp0rqjl8m5jlmm3n9-git-2.42.0"
    },
    {
    "path": "/nix/store/ljrkf0kwgckcy02kdlz8sjdnxb2pwjb4-pkg-config-wrapper-0.29.2"
    },
    {
    "path": "/nix/store/clfgxa0kck0d77wfvy64rhv4b5192089-process-compose-0.69.0"
    },
    {
    "path": "/nix/store/acsgd5wypjys1p7ssf3m8zg6ikwplmd5-start-postgres"
    },
    {
    "path": "/nix/store/3y5bc3l0s2sn20sp2ayvkww5wckw73ns-sqlite-3.43.2-dev"
    },
    {
    "path": "/nix/store/vvd3x6rbwjxw7rbnv7yrkd3jrk8270l6-pkg-config-0.29.2-man"
    },
    {
    "path": "/nix/store/wgqi79zvbj6wx4rdhjk0h8vlzxzrdl5j-numactl-2.0.16"
    },
    {
    "path": "/nix/store/pspa8i64ddmjf4h4vfkmagjlybyfblcf-pkg-config-0.29.2-doc"
    },
    {
    "path": "/nix/store/n2mbhy95l1bqc4y6n1dhbii5l1s0i0vm-lttng-ust-2.13.6"
    },
    {
    "path": "/nix/store/ymv18735ywf4jz24qmw0bsbj2i7g8fh5-pkg-config-wrapper-0.29.2-man"
    },
    {
    "path": "/nix/store/yj433506cgwsnp03dxmw5ms5n2zqaf2f-postgresql-16.1-man"
    },
    {
    "path": "/nix/store/wz3rmn84agcr0c8h5fzmi24kdhr8kz0b-gcc-12.3.0-man"
    },
    {
    "path": "/nix/store/qcvsvw7sjgrhhb24szl2bsywwp8h6dbl-postgresql-16.1-doc"
    },
    {
    "path": "/nix/store/ngs6rg35ml3n85l3c0p46ha5l0d83acg-gcc-12.3.0-checksum"
    },
    {
    "path": "/nix/store/m5ny1dfx2wn2w3i79i1cfmhspf0qiq1y-sqlite-3.43.2-debug"
    },
    {
    "path": "/nix/store/hccflxwxmdcxsywgjdnkc93a0yac1cv0-postgresql-16.1-debug"
    },
    {
    "path": "/nix/store/glrzci7hcif29nkni1124wzwgk2mqxkq-git-2.42.0-debug"
    },
    {
    "path": "/nix/store/js8ik0vndyb6b05hk4gl85xzdm4rc3s6-requirements.txt"
    },
    {
    "path": "/nix/store/2cqz81547zn9x76mh1vhs6p92arl74fs-pkg-config-wrapper-0.29.2-doc"
    },
    {
    "path": "/nix/store/04qa7dh7y6jn14pdhmz45vbn71vmj2s3-gcc-12.3.0-info"
    },
    {
    "path": "/nix/store/ywgv8ian3q8mlkydnpqqa5xndvh05pwi-odbcini"
    },
    {
    "path": "/nix/store/wr08yanv2bjrphhi5aai12hf2qz5kvic-stdenv-linux"
    },
    {
    "path": "/nix/store/k23afqb8da5s4283pai9zzvrnh4bvg9n-glibc-locales-2.38-27"
    },
    {
    "path": "/nix/store/a2008784g79p5rzk2hfpb68aib9fz7a1-devcontainer.json"
    },
    {
    "path": "/nix/store/5fdjh005gwmn4acq10gl8y4vjm803lla-ms-sqltoolsservice-4.11.0.10"
    },
    {
    "path": "/nix/store/3j8p1zirxnw4v60fbab7b5qknfz83xy5-devenv-profile"
    },
    {
    "path": "/nix/store/308y1ac0ah9qclgbm32wgkbyg521wjfc-init-venv.sh"
    },
    {
    "path": "/nix/store/l1sqg8bdg336r5hzyrqdzk7j91l9ckvl-devenv-shell-env"
    },
    {
    "path": "/nix/store/jmy11m3c935yyvs4njz3s52p9azgvg6f-coreutils-full-9.3"
    },
    {
    "path": "/nix/store/bdgs981rn1c2mpkrh49nsqiqbw4yvic0-source"
    },
    {
    "path": "/nix/store/jsphhmmrrvddmmpsayr0p6rkxifsvxvc-envsubst-1.4.2"
    },
    {
    "path": "/nix/store/6aw9inrgxrixnkbi31k9n9xf84zaxsxx-devenv-shell-env-bashrc"
    },
    {
    "path": "/nix/store/3w84ihqymrl2pizzkccp514s7286cfqx-bash-5.2-p15-man"
    },
    {
    "path": "/nix/store/ddxh87hs2c1k2xxh0ax7dgbjjaxfx9pf-create-paths",
    "options": {
    "rewrite": {
    "regex": "^/nix/store/ddxh87hs2c1k2xxh0ax7dgbjjaxfx9pf-create-paths",
    "repl": ""
    }
    }
    },
    {
    "path": "/nix/store/ci7rvk82kr0zfnapp5j9qgh1kskfrjqy-root",
    "options": {
    "rewrite": {
    "regex": "^/nix/store/ci7rvk82kr0zfnapp5j9qgh1kskfrjqy-root",
    "repl": ""
    }
    }
    },
    {
    "path": "/nix/store/b5yhcx672hgzsi7dp922qhvqg81qfqr1-entrypoint"
    }
    ],
    "mediatype": "application/vnd.oci.image.layer.v1.tar"
    }
    ],
    "arch": "amd64"
    }
    ```