Created
February 24, 2020 17:59
-
-
Save lava/62a6697abcaf6e5709939ae29e7bd4be to your computer and use it in GitHub Desktop.
Revisions
-
lava created this gist
Feb 24, 2020 .There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,41 @@ # Interactive Debugging on Github Actions Did you ever run into some issue where a job would behave slightly different in you CI environment than on your local machine? Did you ever wish you could run just a few commands in a shell on your build machine? These are, of course rhetorical questions. And if you're using Github Actions to run your CI jobs, you'll have noticed that this use case is not supported at all. Can't create a nice walled garden if *anyone* could just run a CI job, after all. There are some workarounds (e.g. https://github.com/nektos/act), but since they're not officially supported they can be a bit unstable. Also, even they usually don't reproduce the **exact** environment found on github's servers. Anyways, here's a cool technique to investigate your CI failures interactively. It's creating a reverse shell from the build machine, with strict TLS certificate pinning to prevent any random internet person to just look around your build. First, run this on any server connected to the internet (or at least connected to the build machine): ``` openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 365 -nodes openssl s_server -quiet -key key.pem -cert cert.pem -port 2222 ``` The `cert.pem` created from the first command here should be made available to the CI job. In your workflow definition, add the following step: ``` - name: Do regular CI stuff [...] - name: Spawn Reverse Shell on Failure if: failure() run: | sudo apt-get -qqy install openssl mkfifo /tmp/s; /bin/sh -i < /tmp/s 2>&1 | openssl s_client -quiet -strict -verify 1 -verify_return_error -CAfile cert.pem -connect $SERVER_IP:2222 > /tmp/s; rm /tmp/s ``` (In case you're wondering, yes, `-strict -verify` still has openssl ignoring the validity of the remote certificate) Of course, this technique is not unique to github, it can be used on any CI runner with network connectivity to the target host. Which these days is almost always the case. If the target port is changed from `2222` to `443`, it will even go through almost any firewalls.