# Default robots.txt blocks Twitter Card images **URL:** \ **Category:** Bug\ **Created:** [March 3, 2021, 1:26pm UTC](https://meta.discourse.org/t/default-robots-txt-blocks-twitter-card-images/181762 "2021-03-03T13:26:22Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![HaPe](https://avatars.discourse-cdn.com/v4/letter/h/5e9695/32.png) [@HaPe](https://meta.discourse.org/u/HaPe)\ **Post date:** [March 3, 2021, 1:26pm UTC](https://meta.discourse.org/t/default-robots-txt-blocks-twitter-card-images/181762/1 "2021-03-03T13:26:23Z")
By default, the `robots.txt` file contains the following entry: ```plaintext User-agent: * # ... Disallow: /u ``` This configuration blocks the image for Twitter cards. Checking the card of various forums on [Twitter’s validator](https://cards-dev.twitter.com/validator) does not display the logo and shows the following warning: ![image](https://global.discourse-cdn.com/meta/original/3X/5/8/5807c34163c67cd6ebbbdb69f7c7bc89a063c515.png) > The image URL forum.[…].org/uploads/[…].png specified by the ‘twitter:image’ metatag may be restricted by the site’s robots.txt file, which will prevent Twitter from fetching it. The problem is that the path `/uploads` also starts with a `u` and is therefore matched by the `robots.txt` rule. I know that the `robots.txt` file can be overwritten but I think it would be good to change the default. Unfortunately, I don’t understand the code well enough to create a pull request. When overwriting, keep in mind that Twitter checks the `robots.txt` file less often than the actual html of the page, so it takes a few hours to get picked up. ---
**Author:** ![devporto](https://avatars.discourse-cdn.com/v4/letter/d/919ad9/32.png) [@devporto](https://meta.discourse.org/u/devporto)\ **Post date:** [March 3, 2021, 6:45pm UTC](https://meta.discourse.org/t/default-robots-txt-blocks-twitter-card-images/181762/2 "2021-03-03T18:45:45Z")
Encountered the same problem and posted here [https://meta.discourse.org/t/metatag-may-be-restricted-twitter-card-validator/181169](https://meta.discourse.org/t/metatag-may-be-restricted-twitter-card-validator/181169) Is there a quick fix to try? Thanks! ---
**Author:** ![HaPe](https://avatars.discourse-cdn.com/v4/letter/h/5e9695/32.png) [@HaPe](https://meta.discourse.org/u/HaPe)\ **Post date:** [March 3, 2021, 8:37pm UTC](https://meta.discourse.org/t/default-robots-txt-blocks-twitter-card-images/181762/3 "2021-03-03T20:37:16Z")
> [@devporto](#): > > Is there a quick fix to try? Yes, at least it makes the Twitter cards work. I hope that it does not have unintended side effects. You can open the setting “allow index in robots txt” and click “override robots.txt”. Then look up the line `Disallow: /u` and place a slash in the end, so it becomes `Disallow: /u/`. It will take Twitter a few hours to pick up the change. ---
**Author:** ![devporto](https://avatars.discourse-cdn.com/v4/letter/d/919ad9/32.png) [@devporto](https://meta.discourse.org/u/devporto)\ **Post date:** [March 4, 2021, 3:30am UTC](https://meta.discourse.org/t/default-robots-txt-blocks-twitter-card-images/181762/4 "2021-03-04T03:30:37Z")
> [@HaPe](#): > > You can open the setting “allow index in robots txt” and click “override robots.txt”. Then look up the line `Disallow: /u` and place a slash in the end, so it becomes `Disallow: /u/` . It will take Twitter a few hours to pick up the change. Not sure why the post I linked to was deleted. Will give it a try. Thanks for the tip! ---
**Author:** ![HaPe](https://avatars.discourse-cdn.com/v4/letter/h/5e9695/32.png) [@HaPe](https://meta.discourse.org/u/HaPe)\ **Post date:** [March 10, 2021, 11:21pm UTC](https://meta.discourse.org/t/default-robots-txt-blocks-twitter-card-images/181762/5 "2021-03-10T23:21:17Z")
Looks like this was a problem (see [this thread](https://meta.discourse.org/t/tweets-not-showing-images/84371/7)) in the past and then got re-introduced. In [this commit](https://github.com/discourse/discourse/commit/b52143feff8c32f21ed53033b6a0a65ee45dce0e#diff-0b4715952c840d37a89e4cf9933a695bfd7b592042bd097cf63f500af968ee27), the robots rules with slash got removed because it was thought they are no longer needed. Later, in [this commit](https://github.com/discourse/discourse/commit/758e1608626ce78566dd39004a4a6251d3bbfc60#diff-0b4715952c840d37a89e4cf9933a695bfd7b592042bd097cf63f500af968ee27) they were re-introduced but without the slash in the end. I put together a quick-and-dirty pull request that should fix the default robots.txt file: [FIX: Do not block `uploads` path in robots.txt by ByteHamster · Pull Request #12349 · discourse/discourse · GitHub](https://github.com/discourse/discourse/pull/12349) ---
**Author:** ![codinghorror](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/codinghorror/32/110067_2.png) [@codinghorror](https://meta.discourse.org/u/codinghorror)\ **Post date:** [March 10, 2021, 11:38pm UTC](https://meta.discourse.org/t/default-robots-txt-blocks-twitter-card-images/181762/6 "2021-03-10T23:38:16Z")
What do you think @sam? ---
**Author:** ![sam](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/sam/32/102149_2.png) [@sam](https://meta.discourse.org/u/sam)\ **Post date:** [March 10, 2021, 11:45pm UTC](https://meta.discourse.org/t/default-robots-txt-blocks-twitter-card-images/181762/7 "2021-03-10T23:45:01Z")
I can’t really look at the PR without a CLA, but anchoring and changing `/u` → `/u/` sounds reasonable and safe. ---
**Author:** ![HaPe](https://avatars.discourse-cdn.com/v4/letter/h/5e9695/32.png) [@HaPe](https://meta.discourse.org/u/HaPe)\ **Post date:** [March 11, 2021, 9:04am UTC](https://meta.discourse.org/t/default-robots-txt-blocks-twitter-card-images/181762/8 "2021-03-11T09:04:32Z")
Sorry - CLA is signed now 🙂 ---
**Author:** ![eviltrout](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/eviltrout/32/5275_2.png) [@eviltrout](https://meta.discourse.org/u/eviltrout)\ **Post date:** [March 11, 2021, 2:37pm UTC](https://meta.discourse.org/t/default-robots-txt-blocks-twitter-card-images/181762/9 "2021-03-11T14:37:01Z")
I’ve merged it. Let’s give it a shot. ---
**Author:** ![codinghorror](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/codinghorror/32/110067_2.png) [@codinghorror](https://meta.discourse.org/u/codinghorror)\ **Post date:** [March 18, 2021, 11:20pm UTC](https://meta.discourse.org/t/default-robots-txt-blocks-twitter-card-images/181762/10 "2021-03-18T23:20:56Z")