I’ve been struggling with an issue for a while that doesn’t seem to be reported yet. I apologize for the odd number of moving parts, but I’ll attempt to describe it succinctly.
The TL;DR; is that when I paste a link into a message, the Ruby Gem that ultimately makes the HTTP GET request to that URL to look for embed data sends an HTTP request that is considered by some HTTP proxies to be invalid per the spec. This prevents previews from working in some cases:
The slightly longer version is this. We use a nice little service called Gitbook.io for our docs. Gitbook is a hosted solution and they use Cloudflare workers for internal redirects on their site. Part of their Cloudflare workers involve using the Node Fetch API to proxy HTTP requests. The Node fetch devs are VERY pedantic about following the spec and they will reject any GET request that has an HTTP body or even a
Content-Length header, even if that header is set to
And that is exactly what is happening. The Ruby gem that makes the HTTP request sends a
request header and this pisses off the node fetch proxy to no end and ends up in the request being rejected from the remote server. There has been much debate on different forums about whether a request body on a GET or even just a content length header is valid per the HTTP spec. I don’t have an issue with it, but that hasn’t stopped the Node fetch devs from closing every issue that’s ever been opened begging them to allow such a semantic.
I’m unfortunately stuck in the middle here.
- The Node Fetch project refuses to consider these HTTP requests as valid.
- Cloudflare support refuses to help me because I don’t have control over the Node-based workers in question
- Gitbook’s support refuses to help me because they agree with the Fetch developers (and I’m not sure they really care)
- And the HTTPrb library refuse to remove the header because they think it’s perfectly valid.
So that leaves me posting here asking if there is any way to control or change the HTTP GET requests made for link previews to include an acceptable set of HTTP headers such that proxies using incredibly pedantic libraries such as Node Fetch will not reject these requests?
If you want to try, here’s an example URL that’s hosted on Gitbook’s servers and uses their Node Fetch-powered Cloudflare worker.