Ok, got past the Sidekiq issue, I found this in the unicorn.stderr.log
WARN -- : attack prevented by Rack::Protection::IPSpoofing
which lead me to look at the IPSpoofing source code and figured out the problem was the HTTP_X_REAL_IP header I was forwarding… removed it and now I an access sidekiq.
Sidekiq appears to be running normally, but lightbox is still not working. Any other suggestions on where to look?
Edit (since I can’t post more than 3 consecutive replies!) :
Problem solved. I found the same error as this case
in our production logs, and he mentioned dns. DNS appeared to be working, however I had it as 1.1.1.1, 8.8.8.8 on the host, I changed that around to 8.8.8.8, 8.8.4.4, 1.1.1.1 and rebooted the host (restarting the container didn’t seem to help).
I have no idea why this change worked, as I could curl the image url in the container just fine, and dns appeared to be working.