Fixing "Permission denied (publickey)" when connecting to a Travis CI build container

I was attempting to debug a build with Travis CI's SSH connection option (sadly, I can't find any documentation about this feature -- it's the "Debug job" button).

After the Debug job starts up, it prints:

Use the following SSH command to access the interactive debugging environment: ssh <random-string>

But when I tried this, I got:

Permission denied (publickey).

It took me a while to figure out why this was happening. What public key should a temporary SSH connection use? After talking to my co-workers, who reported no problems connecting, I remembered I had set up a separate public key for GitHub. By specifying that key with -i I was able to connect:

ssh -i ~/.ssh/id_github_rsa <random-string>