AP processing dragging forum down?

Support
1

Hi @angus, I can't be certain about what is actually happening, which needs an admin and knowledgeable person with the ActivityPub plugin, but I observe that this forum becomes slow or even unreachable at times and it looks to be related somehow to posts that are being federated.

Notified about that in this post, saw it happening yesterday after @silverpill tooted this reply, and I got "503 Gateway timeout" from NGINX which I think may relate to this post.

3 Likes
2

I also noticed that SH becomes unresponsive when I publish a reply. This is because replies are delivered to many other servers, which start to load the reply tree, all at once.

Sorry about that

3 Likes
3

if this is the case, then it is basically a "thundering herd" issue similar to how link previews can cause issues. to which the mitigation is that softwares should stop auto-fetching resources that might not ever be seen, and to instead use on-demand deferred "lazy loading" strategies.

2 Likes
4

Hey guys, I’ll have a closer look at this soon, hopefully tomorrow.

3 Likes
5

I disable my plugin for that trouble I’ve received many request no stop from the same souce-instance or many others instances making the same type of request without none delay limit, sorry but this is weird all time requesting and flooding my server is too much.

6

angus:

Hey guys, I’ll have a closer look at this soon, hopefully tomorrow.

I don’t want to put pressure, and it may be inconvenient time now in the Summer vacation period, but if you have an opportunity to delve into the issue it would be great. Right now a single federated post may make the forum very slow for a period of time, and a single post + post edit can get the forum to a point where it shows nginx error screens to visitors.

7

Hey guys, if this is causing serious issues, I would note there a number of tools already in the plugin a community admin can use for this kind of scenario.

  1. There is comprehensive logging, including object logging, you can use to determine the source of any troublesome actors.

    • activity_pub_verbose_logging
    • activity_pub_object_logging

  2. The inbox and object retrieval has rate limiting and site settings you can use to control the rate at which requests are made.

    • activity_pub_rate_limit_post_to_inbox_per_minute
    • activity_pub_rate_limit_get_objects_per_minute

  3. There are site settings you can use for allowlist and blocklist of domains.

    • activity_pub_allowed_request_origins
    • activity_pub_blocked_request_origins

  4. There are admin actions you can use to “Reject” follows from problematic followers.

Furthermore, the amount of activity a server can handle is going to be relative to the resources of a server. Something which I won’t have visibility over.

@aschrijver I’d start with resources. What are the resources of the server? What exactly is being overloaded? I can’t even load socialhub.activitypub.rocks at all at the moment. I suspect there is a more fundamental issue with resource usage.

Then, my second question would be where the requests are coming from? You can figure this out with the logs. Then add any domains you’re getting a lot of requests from to activity_pub_blocked_request_origins.

If we’ve used those existing levers first, and we’re still seeing performance issues, then we can look at other potential causes.

2 Likes
8

Ok, SocialHub loaded, I fired up the logs and the culprit seems relatively clear.

Screenshot 2025-08-10 at 16.54.34
Screenshot 2025-08-10 at 16.54.343022Ă—182 25.4 KB

I’d suggest adding mastodon.sdf.org to the blocklist and sending a note to their admins that they’re spamming socialhub.

*edit It seems another issue was that signed requests had been disabled. I’ve done the following:

  1. Set Activity pub require signed requests to true.
  2. Set Activity pub blocked request origins to mastodon.sdf.org.

Let’s see if the issues persist.

9

Thanks for your quick reply, @angus. I am just a regular member of this forum, but this looks to be useful info for @how or other forum admins to look into.

10

No worries. I suspect simply enabling signed requests and adding mastodon.sdf.org to the blocklist is all that is needed.

1 Like
11

It should block everywhere. It acts as a DDOS-agent. Amount of requests it does is totally horrible.

12

Should I block mastodon.sdf.org? It seems to be a legit source, but maybe their configuration is problematic? Anyway if needed I should be able to do it.

13

angus:

I can’t even load socialhub.activitypub.rocks at all at the moment. I suspect there is a more fundamental issue with resource usage.

Yes the VPS is small. I could upgrade it to something stronger.