Discourse Meta
How to edit robot.txt for indexing user profiles?
תמיכה
Viki
(PethuVignesh)
7 באפריל, 2019, 4:04am
4
I am also want to index the Categories and Tags.
Is there a way to edit the robots.txt?
הצגת פוסט בנושא
נושאים קשורים
נושא
תגובות
צפיות
פעילות
Excluding user profiles in robots.txt (or allow edit of file)
Feature
4
2637
24 במאי, 2014
Indexing User Profiles
Product Support
2
202
4 בספטמבר, 2025
Pages listed in the robots.txt are crawled and indexed by Google
Product Support
18
3644
30 ביולי, 2019
Generic rules in "robots.txt" not picked up by Googlebot
Product Support
5
1131
3 במרץ, 2022
Why there are lots of Disallow rule in robots.txt?
Product Support
34
5035
22 בדצמבר, 2020