Images from googleusercontent.com are not auto-crawled

pr-welcome

(Anton) #1

Here is an example:

Source code:

 [img]https://lh3.googleusercontent.com/yp81poko9nGRT_TvY7nEOGcwRL8dsMmS1ik4hctec54rsmVZki9473cKg2F3b6jbmwp4kU8kbKdl79iWHVBYhTkYycPI1KMMCVbZGjSrawFxROTDKG0BnUbwiYGjONQNuPkcOsRBbLOhCAC9lku2Pl4q2r7xNFd4DgGcbFalsN3yjhPOo-QXmfOIyNEQVQCUHS33QAGsC5Z-FNjCa1Q_sWqNN5M_f9d2H-04-GEn8acsc1mXdRs0PHL_OZ9ihet8otVlr4XWBRfmhlCYPsQSQPhNRZaMtRryzylIpu7_iiUykhZhUzDT7UwtG6Xu49Jpa7Wr_Fesb4woSXEtotUUu3wGmu5oodN_lIpasTmE6rWxop8eorgvtQekCFlcqLrT8rHDtLGNBNc4tQrnhM6o_HEB7ZRfyTITc1yROR8SDUFQmLqdy4gumJiO-65CEvg0Hq-fZ7_RwBk-SjGikDedGFlRCGkISokBQ1A2VnVYM039v6v_QYwEG_DVkd0CCnGwbS4DFgjdXrb3rgr1YTg6J1zdZvVrE5InjZzuI-JtWcohGjDpDPw7Wy5umb2BaKhu2uypczLTjweoxYxoriN9fEpVLyTfcKg=w400-h643-no[/img]

This image gets not auto-crawled. Is this a bug?


(Régis Hanol) #2

There’s no file extension. How are we supposed to know that the file is an image?


(Anton) #3

It’s inside an [img] tag, so we should suppose it is an image


(Jeff Atwood) #4

Not possible, all URLs would need to be retrieved and analyzed. Add query string to the end like ?x=.jpg


(Matt Palmer) #5

Why is it not possible to presume, a priori, that a URL in an [img] tag is, in fact, image data? Even if you just did a HEAD request on it and looked at the Content-Type of the response, looking for a media type of image/* (although given that you’ll end up doing a GET for the entire image 99.99% of the time straight after, that’s probably a premature optimisation). HTTP provides Content-Type for a very, very good reason: file extensions are an ugly hack. Worse, if we’re not parsing URLs and separating query string from path, I wouldn’t be surprised if we’re open to some sort of unsafe content injection vulnerabilities…


(Jeff Atwood) #6

Bbcode img tags are not used very often, and shouldn’t be. That is a red herring. So the question becomes:

Why can’t I paste in a random URL not ending in .png or .jpg and have it be detected as an image?


(Matt Palmer) #7

OK, we’ll go with that question. Making a single HEAD request to sniff the content type still doesn’t seem like a massive overhead.


(Sam Saffron) #8

Probably but this is going to have to wait on some oneboxing improvements we have slotted


(Anton) #9

One way or another, I’d like all images to be uploaded and not disappear
one day. Somehow users ended up inserting tens of such images and counting.

So, if not processed automatically, that will be hours for our mods to fix
manually. A fix would be really appreciated.


(Matt Palmer) #10

I think the “PR welcome” tag fits here quite nicely…