Getting deadlock errors in logs

Support
1 
Job exception: PG::TRDeadlockDetected: ERROR:  deadlock detected
DETAIL:  Process 29659 waits for ShareLock on transaction 892416679; blocked by process 29650.
Process 29650 waits for ShareLock on transaction 892416541; blocked by process 29659.
HINT:  See server log for query details.
CONTEXT:  while rechecking updated tuple (36923,3) in relation "discourse_activity_pub_actors"

CleanShot 2025-03-27 at 08.57.49@2x
CleanShot 2025-03-27 at 08.57.49@2x1920×500 96.5 KB

These are showing up in /logs @angus, I’m not sure if there are any user-facing issues. Stack trace points to a job handling deliveries:

rack-mini-profiler-3.3.1/lib/patches/db/pg.rb:69:in `exec_params'  
rack-mini-profiler-3.3.1/lib/patches/db/pg.rb:69:in `exec_params'  
(eval at /var/www/discourse/lib/method_profiler.rb:38):105:in `exec_params'
activerecord-7.2.2.1/lib/active_record/connection_adapters/postgresql_adapter.rb:894:in `block (2 levels) in exec_no_cache'  
activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract_adapter.rb:1004:in `block in with_raw_connection'  
activesupport-7.2.2.1/lib/active_support/concurrency/null_lock.rb:9:in `synchronize'  
activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract_adapter.rb:976:in `with_raw_connection'  
activerecord-7.2.2.1/lib/active_record/connection_adapters/postgresql_adapter.rb:893:in `block in exec_no_cache'  
activesupport-7.2.2.1/lib/active_support/notifications/instrumenter.rb:58:in `instrument'  
activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract_adapter.rb:1119:in `log'  
activerecord-7.2.2.1/lib/active_record/connection_adapters/postgresql_adapter.rb:892:in `exec_no_cache'  
activerecord-7.2.2.1/lib/active_record/connection_adapters/postgresql_adapter.rb:872:in `execute_and_clear'  
activerecord-7.2.2.1/lib/active_record/connection_adapters/postgresql/database_statements.rb:79:in `exec_delete'  
activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract/database_statements.rb:208:in `update'  
activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract/query_cache.rb:27:in `update'  
activerecord-7.2.2.1/lib/active_record/relation.rb:610:in `block in update_all'  
activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:421:in `with_connection'  
activerecord-7.2.2.1/lib/active_record/connection_handling.rb:296:in `with_connection'  
activerecord-7.2.2.1/lib/active_record/relation.rb:598:in `update_all'  
/var/www/discourse/plugins/discourse-activity-pub/lib/discourse_activity_pub/delivery_failure_tracker.rb:36:in `set_domain_actors_as_available'  
/var/www/discourse/plugins/discourse-activity-pub/lib/discourse_activity_pub/delivery_failure_tracker.rb:22:in `track_success'  
/var/www/discourse/plugins/discourse-activity-pub/app/jobs/discourse_activity_pub_deliver.rb:50:in `perform_request'  
/var/www/discourse/plugins/discourse-activity-pub/app/jobs/discourse_activity_pub_deliver.rb:15:in `execute'  
/var/www/discourse/app/jobs/base.rb:316:in `block (2 levels) in perform'
1 Like
3

This should address it

2 Likes
4

Hmm, that doesn’t seem to fix the issue unfortunately @angus. There’s a lot of actors, possibly, in the DB, and the batching is still insufficient.

On meta, for example, there are 90k+ rows in that model:

discourse(prod)> DiscourseActivityPubActor.where(domain: 'mastodon.social').size
=> 90264

Do we absolutely need to update their available column on every delivery?

5

I see. Hopefully this, together with the previous changes, will address it:

1 Like
6

This topic was automatically closed after 20 hours. New replies are no longer allowed.