Shields.io unable to retrieve Discourse statistics API

haikalpribadi · 14 سبتمبر 2019، 1:25م

Hi everyone,

We’ve had a Discourse shield on our repository for a while now, and it recently stopped working.

If you go to https://shields.io/category/chat and select any Discourse shield, you can enter your discourse domain address and it will show you the shield with the correct statistics. You can try this with meta.discourse.org.

However, when we enter our discourse host address (https://discuss.grakn.ai), for any statistics and for both http/https, it always returns “invalid”.

When a host is not found, Shields.io would return “inaccessible”. Thus we assume “invalid” means it’s accessible but there are access rights issues or invalid responses.

Is it possible that a recent update/upgrade broke something on Discourse statistics API that Shields.io uses?

Thank you so much!

marianord · 15 سبتمبر 2019، 3:00ص

It’s working for me in my site, maybe you’re not setting correctly the protocol? Or the Grakn Discourse has any kind on modification that breaks that endpoint.

gerhard · 15 سبتمبر 2019، 10:35ص

You might want to ask Shields.io about that problem. It works with all other sites I tested, so this isn’t our bug.

haikalpribadi · 15 سبتمبر 2019، 7:20م

@marianord that’s exactly my question: where are the “protocols” you’re mentioning? How can they be configured? I’ve not changed any settings.

@gerhard given that shields.io is working for other Discourse sites, it does not seem likely to be an issue on their side. Unless they’re re not reading the output from our site statistics properly - but how can we find out about this? What is the endpoint from Discourse that is used to query the statistics? Perhaps we should start there?

marianord · 15 سبتمبر 2019، 7:27م

I’m mentioning http vs https.

max_grakn · 19 سبتمبر 2019، 11:05ص

This happened because our Discourse installation blocked Shields.io’s user agent (Shields.io). This setting is named whitelisted crawler user agents and can be edited at
<discourse_server>/admin/site_settings/category/all_results?filter=crawler

haikalpribadi · 19 سبتمبر 2019، 11:18ص

Interesting! Thank you @max_grakn! We did add Googlebot to the whitelist recently, I think that may be the cause.

@codinghorror are we meant to use Blacklist and Whitelist at the same time? As in, if you add things to whitelist, does that mean everything else is blacklisted (which therefore make the blacklist redundant)?

codinghorror · 19 سبتمبر 2019، 12:41م

No, the crawler whitelist is very dangerous, and should only be used carefully per the help text.

User agents of web crawlers that should be allowed to access the site. WARNING! SETTING THIS WILL DISALLOW ALL CRAWLERS NOT LISTED HERE!

barto_95 · 13 نوفمبر 2020، 10:16ص

Hi, I have the same problem when a test I received invalid data… do you have idea ?

It’s OK now, it’s necessary to activated anonymous statistic in :

Admin → setting → other → share anonymized statistics = Enabled

and now it’s ok for shields.io

spdegabrielle · 15 نوفمبر 2023، 6:22م

لدي نفس المشكلة، ولكن مشاركة الإحصائيات المجهولة ممكّنة بالفعل

[![Racket Discourse](upload://6fa5jbSn04vRLXdubAYmFJt5emf.svg)]

[]

Arkshine · 15 نوفمبر 2023، 7:41م

ذو صلة بـ:

github.com/discourse/discourse

DEV: Ability to collect stats without exposing them via API

main ← dev/ability-to-collect-stats-without-public-exposing

opened 06:37PM - 13 Oct 23 UTC

AndrewPrigorshnev

+228 -119

This adds the ability to collect stats without exposing them among other stats v…ia API. The most important thing I wanted to achieve is to provide an API where stats are not exposed by default, and a developer has to explicitly specify that they should be exposed (`expose_via_api: true`). Implementing an opposite solution would be simpler, but that's less safe in terms of potential security issues. When working on this, I had to refactor the current solution. I would go even further with the refactoring, but the next steps seem to be going too far in changing the solution we have, and that would also take more time. Two things that can be improved in the future: 1. Data structures for holding stats can be further improved 2. Core stats are hard-coded in the About template (it's hard to fix it without correcting data structures first, see point 1): https://github.com/discourse/discourse/blob/63a0700d45f755d0f432a9075ae7afbed9cd6ab0/app/views/about/index.html.erb#L61-L101 The most significant refactorings are: 1. Introducing the `Stat` model 2. Aligning the way the core and the plugin stats' are registered

السبب هو أنه تمت إعادة تسمية بعض الحقول في صيغتها الجمع.

topic_count → topics_count
post_count → posts_count
user_count → users_count
like_count → likes_count

سيحتاج شخص ما إلى تعديل الكود هنا للسماح بالرجوع إلى الصيغة الجمع:

github.com/badges/shields

services/discourse/discourse.service.js

master

import Joi from 'joi'
import { metric } from '../text-formatters.js'
import { nonNegativeInteger, url } from '../validators.js'
import { BaseJsonService, queryParams } from '../index.js'

const schemaSingular = Joi.object({
  topic_count: nonNegativeInteger,
  user_count: nonNegativeInteger,
  post_count: nonNegativeInteger,
  like_count: nonNegativeInteger,
}).required()

const schemaPlural = Joi.object({
  topics_count: nonNegativeInteger,
  users_count: nonNegativeInteger,
  posts_count: nonNegativeInteger,
  likes_count: nonNegativeInteger,
})

const schema = Joi.alternatives(schemaSingular, schemaPlural)

This file has been truncated. show original

spdegabrielle · 20 نوفمبر 2023، 3:52م

ما زال هذا معطلاً - هل يعمل لدى أي شخص آخر؟

![Racket Discourse](upload://7asTK98zfWLXRebm11uEN3KPM5N.svg)

BryanV · 6 ديسمبر 2023، 5:35م

تم فتح طلب سحب قبل أيام قليلة:

github.com/badges/shields

[Discourse] Update schema keys to use plural form (`topic_count` -> `topics_count`)

master ← joshuacwnewton:jn/9776-fix-discourse-js-schema

opened 08:16PM - 03 Dec 23 UTC

joshuacwnewton

+110 -71

This PR does a simple find and replace to update the Discourse schema to use the… current keys found in https://meta.discourse.org/site/statistics.json, as the older keys now return "Invalid response". > [!NOTE] > I was wondering, though, if the old keys should still be preserved, as to not break backwards compatibility with servers using the old names. > > (I tried finding the source of the change in Discourse's release notes/changelog, but there's surprisingly little information when querying e.g. "statistics.json" or "topic_count"/"topics_count". So, I'm not sure when this change actually took place, and thus it's hard to estimate the likelihood of there being servers out there using the old keys.) > > If we do want to support both sets of keys, then I'm not 100% sure what to do, as I've never written JavaScript before. (I tried reading the Joi docs, and got as far as using `.or()` in the schema definition for each pair of keys + using a `try`/`catch` when creating the `metricIntegrations` object, but I wasn't confident enough to commit that solution before discussing it first.) Fixes #9776.

ولكن بطبيعة الحال، هذا التغيير يضع عبئًا على shields.io لدعم كلا الإصدارين (إلى الأبد، على ما أعتقد) نظرًا لعدم وجود ضمان بأن أي مثيل Discourse معين قد تم تحديثه أم لا، لذا فهو ليس بسيطًا مثل التغيير إلى صيغة الجمع.

الموضوع		الردود	مرات العرض
Is there a nice GH badges (ala shields.io images) design to put in README to link from GitHub to our hosted discourse? Feature	17	5049	13 سبتمبر 2019
Discourse CDNs are blocked by privacy badger Site feedback	47	12786	15 أغسطس 2017
Can I use the Discourse API on an officially-hosted instance? Support	1	317	9 يوليو 2021
Access reports JSON data via Pipedream Dev rest-api	2	799	28 يوليو 2021
Daily Summary Feedback Site feedback feedback	235	6467	27 نوفمبر 2025

Shields.io unable to retrieve Discourse statistics API

الموضوعات ذات الصلة