שגיאת Onebox קוד 403

תמיכה

סיכום מבוסס בינה מלאכותית

SkyLin reports a 403 error when using the Onebox preview function for Udemy URLs, which previously worked fine. The issue began after a Discourse update, though the Onebox plugin version remained unchanged. Testing revealed that YouTube previews still work, and Udemy URLs generate previews successfully via Facebook’s debug tool, suggesting the problem is specific to Discourse’s Onebox handling. SkyLin compared Docker and Discourse repository updates before and after the error emerged, noting no relevant plugin changes. Further troubleshooting included restoring a pre-update VPS backup (which resulted in a 502 error), cloning the VPS to a new instance (which reproduced the 403 error), and analyzing HTTP headers via cURL. The headers show Udemy returning a 403 with Cloudflare protections, while Amazon URLs return 503, hinting at possible bot detection or rate-limiting. A similar historical issue involving Amazon Onebox was referenced here. The root cause remains unresolved, with suspicion falling on Discourse’s outbound request handling or Cloudflare’s response to Discourse’s server IP/user-agent.

1

The preview function of Onebox for udemy url got an error as below.

“Sorry, we were unable to generate a preview for this web page, because the web server returned an error code of 403.”

Not sure if it’s due to the discourse updating today or the site itself. Here comes the situation.

  1. The udemy site preview works fine before yesterday. Sample url:
    Udemy free coupon code 2021_0502-2 - udemy 優惠券 - StudyCamp 線上學習論壇

  2. The Onebox preview doesn’t work today. Sample url:
    Udemy free coupon code 2021_0503 - udemy 優惠券 - StudyCamp 線上學習論壇

I did some further testing.

  1. youtube Onebox preview works fine

  2. udemy preview on facebook debug tool works fine
    https://developers.facebook.com/tools/debug/

Sample udemy course url for testing: (Weird, the preview works fine here)
https://www.udemy.com/course/ultimate-devsecops-with-real-world-scenarios/?couponCode=AGER4U

Please advice. Thanks a lot!

2

The update of docker image before and after.
PS: the onebox plugin should be the same

3

Found a similar issue years ago.

4

The update info. before & after.
PS: After updating, the onebox failed with 403 error

https://github.com/discourse/docker_manager/compare/230dec2...93d7c8a

https://github.com/discourse/discourse/compare/54b7a0d0...f1e74c89

https://github.com/discourse/discourse-calendar/compare/082bb3d...6cc116d

https://github.com/discourse/discourse-chat-integration/compare/ec7cf1e...a1bd603

https://github.com/discourse/discourse-math/compare/c925f7e...afb8e9f

https://github.com/discourse/discourse-reactions/compare/75f9dd9...9238b8c

https://github.com/discourse/discourse-sitemap/compare/9f2e7de...e64f393
5

I did some further testing, none of them solve this bug.

Testing 1. Restore the backup* which before the bug occur. Reboot system then got a 502 error
PS: VPS disk image backup on 4/27, not discourse backup zip file

The new backup file can restore and reboot without problem, only old backup failed. It’s wierd. Theoretically the old backup should work fine as the other backup.

502
5021280×687 20.2 KB

Testing 2. Clone the VPS disk image backup to a new VPS, followed the steps below to setup a new domain. The new site works fine, but this bug still exist. It should be due to rebuilt will load new discourse image which is the same.

Testing 3. Follow the steps of web page below to see why 403.

The image here show the difference of onebox between meta(above) and my site(below).

Clipboard01
Clipboard011920×1080 125 KB

Testing urls for udemy and amazon. It seems one in 403 and one in 503.
https://www.udemy.com/course/power-bi-2021-proyectos-reales-para-volverte-un-master/

< HTTP/2 403
< date: Fri, 07 May 2021 10:18:28 GMT
< content-type: text/plain; charset=UTF-8
< content-length: 16
< x-frame-options: SAMEORIGIN
< cache-control: private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0
< expires: Thu, 01 Jan 1970 00:00:01 GMT
< set-cookie: __cfduid=dbbbf721820c12aac03e8116c362147d51620382708; expires=Sun, 06-Jun-21 10:18:28 GMT; path=/; domain=.udemy.com; HttpOnly; SameSite=Lax
< cf-request-id: 09e7ef0b3c00000b939211d000000001
< expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
< set-cookie: __cf_bm=f83cbaa075085ed2ae624e9bcfa5badf394f7ddd-1620382708-1800-Af1qbDD41LzYLWJX0XR39JAeFnjBdEmvhZO/0YloMwnBR8urb60m7+bcte18X+L59RfHKhzWeGPELN02cuiV9RA=; path=/; expires=Fri, 07-May-21 10:48:28 GMT; domain=.udemy.com; HttpOnly; Secure; SameSite=None
< x-content-type-options: nosniff
< server: cloudflare
< cf-ray: 64b9b4585d9b0b93-NRT
<
{ [16 bytes data]
100    16  100    16    0     0    290      0 --:--:-- --:--:-- --:--:--   296
* Connection #0 to host www.udemy.com left intact


< HTTP/2 503
< server: Server
< date: Fri, 07 May 2021 10:25:04 GMT
< content-type: text/html
< strict-transport-security: max-age=47474747; includeSubDomains; preload
< x-amz-rid: P43ZS4CBG7V92Z7W4VSC
< vary: Content-Type,Accept-Encoding,X-Amzn-CDN-Cache,X-Amzn-AX-Treatment,User-Agent
< last-modified: Wed, 30 Sep 2020 23:54:00 GMT
< etag: "a6f-5b0909d09d600"
< accept-ranges: bytes
< content-length: 2671
<
{ [1139 bytes data]
100  2671  100  2671    0     0  11871      0 --:--:-- --:--:-- --:--:-- 11871
* Connection #0 to host www.amazon.com left intact