User merge failed, corrupted database entries due to third party plugin

[riking (Kane York)]

PostReplyKey refers to the reply-to address in emails, it won’t interact with topic list rendering.

Checking the stacktrace, it’s failing inside TopicListSerializer dumping, somehow inside lib/instance/public.rb.

Try checking the topic featured users list for the old user ID (the set of avatars that shows up on the homepage).

[hellekin]

Hi @riking, thank you for responding… AFAIK apart from the PostReplyKey there’s no mention of this missing user id. But it might not be the actual old user id since I do not know that one. How do you think I could find it – or: what should I grep from the logs?

[riking (Kane York)]

If it is topic featured users, this query should find it:

SELECT id
FROM topics
WHERE featured_user1 = 1234
OR featured_user2 = 1234
OR featured_user3 = 1234
OR featured_user4 = 1234
OR last_post_user_id = 1234

[hellekin]

The query returned no result with the old userid, but one topic with the new user id, which is the topic that was already identified as problematic. I moved the topic to another category and indeed the https://socialhub.activitypub.rocks/c/meeting/apconf2020/53 category is now visible again.

Now I guess there’s a reference to the missing user_id 36 somewhere in the data related to topic 951, but I could not find it so far. Or is there another dependency I’m overlooking, wrt user 295?

[gerhard (Gerhard Schlager)]

Do you have a stacktrace for that error?

[hellekin]

Nothing more than what I posted in User merge failed, corrupted database entries, UI troubles and commented after:

@hellekin

I think the problem comes with ActiveRecord::RecordNotFound (Couldn't find all Users with 'id': (36, 50) (found 1 results, but was looking for 2).)

The ids match a user account that was merged, but the process seems to have broken…

[gerhard (Gerhard Schlager)]

Oh, I missed the stacktrace in User merge failed, corrupted database entries, UI troubles. So, it’s failing due to a plugin. I guess one of your plugins stored some kind of user information that hasn’t been merged. Could you give me a list of the installed plugins please?

[hellekin]

OK fetching that… I have connectivity issues. I’ll check with safe_mode to see if it works.
Safe-mode does not help…

hooks:
  after_code:
    - exec:
        cd: $home/plugins
        cmd:
          - git clone https://github.com/discourse/docker_manager.git
          - git clone https://github.com/discourse/discourse-assign.git
          - git clone https://github.com/discourse/discourse-bbb.git
          - git clone https://github.com/discourse/discourse-calendar.git
          - git clone https://github.com/discourse/discourse-canned-replies.git
          - git clone https://github.com/discourse/discourse-checklist.git
          - git clone https://github.com/discourse/discourse-code-review.git
          - git clone https://github.com/discourse/discourse-footnote.git
          - git clone https://github.com/discourse/discourse-github.git
          - git clone https://github.com/discourse/discourse-graphviz.git
          - git clone https://github.com/discourse/discourse-math.git
          - git clone https://github.com/discourse/discourse-policy.git
          - git clone https://github.com/discourse/discourse-prometheus.git
          - git clone https://github.com/discourse/discourse-prometheus-alert-receiver.git
          - git clone https://github.com/discourse/discourse-push-notifications.git
          - git clone https://github.com/discourse/discourse-saved-searches.git
          - git clone https://github.com/discourse/discourse-solved.git
          - git clone https://github.com/discourse/discourse-spoiler-alert.git
          - git clone https://github.com/discourse/discourse-voting.git
          - git clone https://github.com/discourse/discourse-yearly-review.git
          - git clone https://github.com/angusmcleod/discourse-events.git
          - git clone https://github.com/angusmcleod/discourse-locations.git
          - git clone https://github.com/davidtaylorhq/discourse-whos-online.git

I see already that I can upgrade the URL for the last ones. (but the rebuild can only happen tonight UTC like midnight)
Maybe ping @angus or @merefield for events and locations.

[gerhard (Gerhard Schlager)]

It might be the discourse-assign plugin as it seems to be the only plugin in your list that modifies the TopicQuery class. And I suspect that’s where the error occurs.

Can you verify my suspicion by disabling the plugin and checking if the category works? You can re-enable it afterwards. Also, it would be interesting to know if any of the topic’s custom fields references the merged user in some way or another. Can you check that out as well please?

./launcher enter app
rails c

id_of_problematic_topic = 123
Topic.find(id_of_problematic_topic).custom_fields

[hellekin]

After disabling discourse-assign:

• latest works
• top fails
• /t/951 still fails

Running the query… Ah! You nailed it!

 {"event_start"=>1601748000, "event_end"=>1601749800, "event_all_day"=>false, "event_timezone"=>"Etc/UTC", "event_rsvp"=>true, "event_going_max"=>100, "event_version"=>1, "event_going"=>[36, 50]}

Apparently the events plugin is skipped during merging. But that might be as well because it’s an older version we’re running. Tell me if I can help you test this case.

Thank you for solving this @gerhard!

[gerhard (Gerhard Schlager)]

You can update the custom field of that topic and remove the non-existent ID. @angus might want to add support for user merging to his plugin.

@gerhard

Plugin authors can hook into the merge process by handling the :merging_users event. I’ve already implemented it for the polls plugin, which merges the votes of the two merged users.

[hellekin]

For the record:

tcf = TopicCustomField.where(topic_id: problematic_topic_id)
c = tcf.last # in my case, YMMV
c.value = "[295,50]"
c.save

[angus (Angus McLeod)]

For posterity, in case that topic has multiple custom fields you should use:

tcf = TopicCustomField.where(plugin_name: 'discourse-events', topic_id: problematic_topic_id)
c = tcf.first
c.value = "[295,50]"
c.save

@hellekin I would migrate to 🗓 Discourse Event.

In fact, perhaps you could work with @fzngagan on a migration script from Events to Discourse Events? It’d be good to have a real case to work with there. Faiz, we should prioritise this, I’d like to formally deprecate Events.

[hellekin]

@angus, @fzngagan: do you think I can change the plugin line in the container configuration, rebuild, and then take care of running the script? Since I also need to change the git repositories for who’s who and wizards, I could do it at once. OTOH we’re in the middle of a hackathon and if the problem is solved I can as well wait for a week.

[fzngagan (Faizaan Gagan)]

@hellekin
I recently fixed a similar issue where deleting a user was causing problems.

I love the idea of migrating to 🗓 Discourse Event . The question then is, do you wish to retain expired events, and how many upcoming events do you have?

[angus (Angus McLeod)]

@fzngagan @hellekin You can move the convo here to figure out the details: Create a migration script from Events to Discourse Events - Open Source - Pavilion

[bletch (Danny Goodall)]

I’ll throw my hat in the ring for working on a migration script.

I’m a heavy user of Events. My Plugin for schools with children with behavioural difficulties creates 7 topics a day for 10+ children across multiple sites and has been running for over two years. Each topic represents a period of their school day and I need to be able to query by topic event date so I hope Discourse Events stores its data in a sympathetic place.

I’ll jump on the discussion you mentioned below.

[aschrijver (Arnold Schrijver)]

Great to see this wonderful cooperation. I love Discourse for that! Thanks everyone for all your efforts and involvement

[hellekin]

@fzngagan

do you wish to retain expired events, and how many upcoming events do you have?

Yes, we’d like to keep past events since they’re related to topics, especially the current ones related to the (still ongoing) ActivityPub conference 2020 – this week is the hackathon.
Apart from those we do not have many events, but will read about the migration – probably next week since this week is already more than full. Cheers! And what @aschrijver said: the Discourse community rocks!