neounix
(Dark Matter)
11 Febbraio 2021, 10:40am
1
We are currently running this recent release:
It’s still running in our production container (no issues).
I’m trying to bootstrap based on this release:
I’ve tried a number of times today, and each time it hangs here:
Checked the logs in the data and the standby app container (we are bootstrapping) and there are no errors, no issues.
Each time I bootstrap, it hangs on this line:
I, [2021-02-11T10:37:25.133098 #1] INFO -- : > cd /var/www/discourse && su discourse -c 'bundle exec rake db:migrate'
Any clues where to look to help debug this?
I’ve looked in all the logs I can think of (in all containers) and cannot find problems.
Thanks.
Additional Info:
Each time it hangs and I control C out of the bootstrap process, it seems to leave a postmaster process, which I can subsequently kill.
Here is an example:
Update:
Bootstrap rebuilds fine if I comment out the “db:migrate” line in the web template.
There is some strange issue with “rails db:migrate” hanging with zero errors I have never experienced before.
I wonder if there is an error (or some underlying issue) in this recent migration?
committed 02:35AM - 11 Feb 21 UTC
Follow up https://github.com/discourse/discourse/pull/11968
Dismiss all new t… opics using the same DismissTopicService. In addition, MessageBus receives exact topic ids which should be marked as `seen`.
See also:
# frozen_string_literal: true
class MoveNewSinceToNewTable < ActiveRecord::Migration[6.0]
def up
sql = <<~SQL
INSERT INTO dismissed_topic_users (user_id, topic_id, created_at)
SELECT users.id, topics.id, user_stats.new_since
FROM user_stats
JOIN users ON users.id = user_stats.user_id
JOIN user_options ON user_options.user_id = users.id
LEFT JOIN topics ON topics.created_at <= user_stats.new_since
LEFT JOIN topic_users ON topics.id = topic_users.topic_id AND users.id = topic_users.user_id
LEFT JOIN dismissed_topic_users ON dismissed_topic_users.topic_id = topics.id AND users.id = dismissed_topic_users.user_id
WHERE user_stats.new_since IS NOT NULL
AND topic_users.id IS NULL
AND dismissed_topic_users.id IS NULL
AND topics.archetype <> :private_message
AND topics.created_at >= GREATEST(CASE
WHEN COALESCE(user_options.new_topic_duration_minutes, :default_duration) = :always THEN users.created_at
WHEN COALESCE(user_options.new_topic_duration_minutes, :default_duration) = :last_visit THEN COALESCE(users.previous_visit_at,users.created_at)
This file has been truncated. show original
david
(David Taylor)
11 Febbraio 2021, 12:10pm
2
Hi @neounix , we’re aware of an issue with the migration you linked and we’re working on a fix right now
2 Mi Piace
neounix
(Dark Matter)
11 Febbraio 2021, 12:13pm
3
Hi @david
Thanks for the reply!
I was looking at the migration just now, and my first thought, as “not a Discourse DB migration expert”, was this might be the issue ?
add_index :dismissed_topic_users, %i(user_id topic_id), unique: true
Anyway, I’m not qualified to go much deeper into Discourse migration issues, so I’ll stand down and stand by.
Thanks again!
paresy
(Michael)
11 Febbraio 2021, 4:32pm
4
Can we safely abort the Migration without any data loss and go back a few commits?
It worked.
1 Mi Piace
Hey @neounix ,
sorry for including that heavy and broken migration into the codebase. Those changes were reverted so new deploy should be fine:
master ← KrisKotlarek:broken-migration
merged 09:50PM - 11 Feb 21 UTC
Revert of:
https://github.com/discourse/discourse/pull/12018
https://github.co… m/discourse/discourse/pull/12041
4 Mi Piace
neounix
(Dark Matter)
12 Febbraio 2021, 1:37am
6
Hi @kris.kotlarek
Thank you for the update:
I tend to agree that that this “heavy migration” more-than-likely requires additional review and stringent development testing on a test configuration with a large DB before merged into core. Lucky for our site, I always bootstrap in parallel to production so we did not suffer any downtime during the migration failure(s), so no worries on our end. Thanks for reverting so quickly.
FYI only (just being technically precise, sorry about that… ), I checked the DB after rebuilding just now and the “revert” process did not drop the table from DB, no big deal, just FYI only:
discourse=> \d dismissed_topic_users
Table "public.dismissed_topic_users"
Column | Type | Collation | Nullable | Default
------------+-----------------------------+-----------+----------+---------------------------------------------------
id | bigint | | not null | nextval('dismissed_topic_users_id_seq'::regclass)
user_id | integer | | |
topic_id | integer | | |
created_at | timestamp without time zone | | |
Indexes:
"dismissed_topic_users_pkey" PRIMARY KEY, btree (id)
"index_dismissed_topic_users_on_user_id_and_topic_id" UNIQUE, btree (user_id, topic_id)
Thank you again for your hard work on this and for quickly reverting the migration.
2 Mi Piace
neounix
(Dark Matter)
15 Febbraio 2021, 4:06am
7
This migration is working smoothly now, based on the latest commit:
master ← KrisKotlarek:new-approach-to-dismiss-new-migration
merged 09:50PM - 14 Feb 21 UTC
Original PR was reverted because of broken migration https://github.com/discours… e/discourse/pull/12058
I fixed it by adding this line
```
AND topics.id IN(SELECT id FROM topics ORDER BY created_at DESC LIMIT :max_new_topics)
```
This time it is left joining a limited amount of topics. I tested it on few databases and it worked quite smooth
4 Mi Piace
Falco
(Falco)
Chiuso
18 Febbraio 2021, 11:00am
8
This topic was automatically closed after 6 days. New replies are no longer allowed.