After the passing of initial hysteria (which can overtake a person who finds that for the past nearly half year, google wasn’t paying any attention to his website and he didn’t even know about this!!), I’d try to list here only those causes, which are related to Discourse code, i.e. which are fixable neither by Google, nor by me (afa I can say), but by Discourse team. Blocked by Robots.txt : [image] Server Error 5xx [An email notification from Google Search Console about pages being indexed due to a server error. (Captioned by AI)] Content wider than Screen and Clickable Elements too close to click (on mobile screen) [A notification email alerting about new mobile usability issues detected for a website, with recommendations to fix the issues to enable the best experience and coverage in Google Search. (Captioned by AI)] Discussion Forum Structed Data issues: [image] Video Indexing Issues, but no other details: [A notification regarding video indexing issues on a website is shown in the image. (Captioned by AI)] When clicked on ‘Video Indexing Report’: [The image shows a section titled "Video indexing" and a note about video indexing status, indicating that if any video structured data items are found on the site, they will be reported in Enhancements > Videos. (Captioned by AI)] And when clicked on ‘Enhancements> Videos’ link: [A screenshot from a website showing a section about videos, explaining the validity of structured data items that define videos and providing a link to see indexed videos and reasons for not indexing. (Captioned by AI)] Interaction to Next Paint issue: [The image is a notification from Google Search Console indicating a Core Web Vitals INP issue with a mobile performance metric longer than 200ms, urging website owners to fix the issue for a better user experience and SEO. (Captioned by AI)] No video thumbnail found [The image shows a notification that there is no thumbnail URL provided for videos, indicating they are not indexed or served as video results on Google. (Captioned by AI)] [A screenshot of a Google search result for no thumbnail image provided for a video. (Captioned by AI)] Missing Field Url: [The image is a notification from Search Console about a structured data issue on a website, specifically a missing field 'url' in the Discussion forum section. (Captioned by AI)] Pls help.

Earlier when I happen to find the problem (all this can be skipped):: After several months of loosing users, today I found that my site has been blocked/forbiden to Google !!! [The image shows a webpage status indicating that the page is not indexed and blocked due to access forbidden with a status code 403. (Captioned by AI)] More Earlier, how I found this problem, all can be skipped: I did read almost all of this meta topic before asking. In Aug2024, I noted that traffic to my website has reduced up to 95%. But I ignored it thinking that may be I wasn’t posting enough. Bt today I found that whatever term I might search on google, restricting the search to my own site only: ***site:BathindaHelper.com jobs in bathinda*** , it was giving ZERO result (the single result it is showing from my site is actually just a suggestion that I create google ads to show this result from my site, which indicates that my site HAS indeed been indexed): [The image shows a paused Google ad campaign for "BathindaHelper.com jobs in Bathinda" on a search page, with a summary indicating that the ads are not showing due to the campaign being paused and an option to "Resume" the campaign. (Captioned by AI)] And finally I also checked Google Analytics (perpahs renamed to Google Events) and it s clearly showing that from 17 June, 2024 Google isn’t referring my site any more. [The image is a screenshot of a Google Analytics dashboard showing organic Google search query data for a website named BHcomwebsite-GA4, with a focus on organic search clicks over time. (Captioned by AI)]

When you made your site either using forced login or stopped to show topics TL0+? Google can use a site only if it is visible to world. Or you have blocked user agents of Google. Is this the same forum where you have DNS issues?

[图片] Jakke Flemming: Is this the same forum where you have DNS issues? Are you asking because of Site does not appear in google searches - #2 by Bathinda? I think the reply below is answering the question of the op.

I wrongly used the word ‘force’. (I was meaning to say that I was forcing Google search to produce search results from my own site BathindaHelper.com ) I didn’t create my site using any abnormal/force method. I didn’t deliberately tinkered anything related to TL0+ or related. For past half hour, I’ve found that (among some other one-two small issues) somehow my robots.txt file is the culprit, but I’ve not been able to find (yet) how to fix this. I don’t remember having DNS issues (are you talking about too long past?). My site is working ok, except that when I/admin hard refresh my browser, then sometimes it takes nearly 30 to 50 seconds to open, but after that it works ok. Thanks for replying. Edit: I’ve ‘deselected’ robots file option: [A screenshot from a web page showing options for indexing in robots.txt, with a warning about potential misconfiguration. (Captioned by AI)] but I can’t say if google search console is reporting that all is ok or not now: [The image is a screenshot of Google Search Console's indexed pages section, showing that the site has all its known pages indexed and available on Google. (Captioned by AI)]

Yeah, I missed totally order. And now we got a demonstration what can happen when answered old topics off topics an user doesn’t read topics :joy: Yes, my bad.

[图片] Bathinda Helper: somehow my robots.txt file is the culprit Check out this settings: allowed crawler user agents blocked crawler user agents But AFAIK Discourse hasn’t plain robots.txt per se as most of sites have, but it is done by some strange ruby-thingy, and there isn’t too many settings where an admin can adjust it. Except those two settings, and slowing down bots. [图片] Bathinda Helper: having DNS issues That was just me and my fast fingers :man_facepalming:

[图片] Bathinda Helper: Edit: I’ve ‘deselected’ robots file option: [A screenshot from a web page showing options for indexing in robots.txt, with a warning about potential misconfiguration. (Captioned by AI)] Did you disable that now or before indexing stopped? Specify in robots.txt that this site is allowed to be indexed by web search engines. If you do not permit search engines to index your site, it does not surprise me that they don’t.

[图片] Jakke Flemming: Check out this settings: Would do and report. [图片] Moin: Did you disable that now or before indexing stopped? I disabled this after opening this topic (say 30 min before now). While this problem has been there for 3 months. But I’ve not been independently able to verify has this ‘deselection’ been able to fix ‘Google Indexing’ fault or not. [图片] Moin: If you do not permit search engines to index your site, it does not surprise me that they don’t. I’m in doubt if I don’t disable/block sites by Robots.txt, then are ALL SITES ARE ALLOWED ? Or is it contrary, that if I don’t ENABLE sites by Robots.txt then all sites are BLOCKED from indexing?

[图片] Bathinda Helper: I’ve ‘deselected’ robots file option: I forgot totally that. You should select it. If you don’t use that then you must check and edit robots.txt manually to be sure it guides bots as you want. But you can take look if you find there something that would stop Google.

Ok. That means all discourse users (normally) would need to specify/give ‘Robots.txt’ file. And so, I’d read the topic about this (how and what should be there in this file) in detail tomorrow. 2nd, if its not too big to explain, can you tell any easy way with which I could tinker some settings in my Discourse Admin panel and at the same time check live/realtime if Google is now able to access (and then index) my site freely or is it still getting ‘Access Forbidden - 403’ error ?! Edit: Though I myself would try to find similar resources on google now/later.

我在Google Search Console中遇到关于错误的话语结构（或者是对我网站管理不当）的各种问题

产品支持

Moin 2024 年12 月 17 日 12:02 5

您是因为 Site does not appear in google searches - #2 by Bathinda 问的吗？我认为下面的回复正在回答发帖人的问题。

话题		回复	浏览量
Why isn't Google Indexing Discourse? SEO concerns Product Support seo	31	6472	2024 年6 月 1 日
Needing to edit robots.txt file - where is it? Product Support	40	8305	2019 年7 月 15 日
Google complaining – Indexed, though blocked by robots.txt Product Support	22	2851	2019 年12 月 13 日
Malformed robots.txt causing issues with indexing Product Support	7	2036	2019 年9 月 14 日
Sitelinks in Google disappearing Community Building	26	1899	2023 年1 月 27 日

我在Google Search Console中遇到关于错误的话语结构（或者是对我网站管理不当）的各种问题

相关话题