does discourse have as special user agent for crawling I need it when posting links from my site (angularJs) to serve SEO friendly content to it
Why not use the rss feeds, instead of just crawling the site?
I don’t have rss feeds on my website for now
here is what I get if I paste a url from my website because discourse bot is scraping my website
when I detect bots I have special prerended pages but here it’s going to the javascript version and it looks like the bot does not render it
1 „Gefällt mir“
Thanks, it would probably solve the issue !!
Hat das das Problem für dich gelöst?
Wie hast du die Änderung angewendet, nachdem du die .rb-Datei geändert hast?