# Миграция базы данных зависает на огромном значении элемента "calendar-details" в таблице "post\_custom\_fields" **URL:** https://meta.discourse.org/t/database-migration-chokes-on-huge-value-of-a-calendar-details-item-in-table-post-custom-fields/172483 **Category:** Bug **Created:** [07.Декабрь.2020 11:54:54 UTC](https://meta.discourse.org/t/database-migration-chokes-on-huge-value-of-a-calendar-details-item-in-table-post-custom-fields/172483 "2020-12-07T11:54:54Z") **Posts on this page:** 14 **Page:** 1
### Author: ![amotl](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/amotl/32/109873_2.png) [@amotl](https://meta.discourse.org/u/amotl) #### Post date: [07.Декабрь.2020 11:54:54 UTC](https://meta.discourse.org/t/database-migration-chokes-on-huge-value-of-a-calendar-details-item-in-table-post-custom-fields/172483/1 "2020-12-07T11:54:54Z")
### TLDR; - В [посте #4](https://meta.discourse.org/t/problem-migrating-database-to-postgresql-12-with-discourse-calendar-installed/172483/4) и далее мы смогли подтвердить, что эта проблема была связана с [Discourse Calendar and Events](https://meta.discourse.org/t/discourse-calendar/97376), и миграция базы данных успешно завершилась после временного отключения этого плагина — спасибо @falco. - В [посте #9](https://meta.discourse.org/t/database-migration-chokes-on-huge-value-of-a-calendar-details-item-in-table-post-custom-fields/172483/9) и далее мы обнаружили, что `value` конкретного элемента `calendar-details` в одном сообщении содержал около 80 МБ данных. - После удаления проблемной строки, как описано в [посте #13](https://meta.discourse.org/t/database-migration-chokes-on-huge-value-of-a-calendar-details-item-in-table-post-custom-fields/172483/13), миграция базы данных полностью удалась, даже после повторного включения [Discourse Calendar and Events](https://meta.discourse.org/t/discourse-calendar/97376) — еще раз спасибо, @falco. - Контекст этой проблемы — миграция базы данных, применяемая с марта 2020 года, см. [DEV: Migrate from custom fields to tables - Pull Request #23 - discourse/discourse-calendar - GitHub](https://github.com/discourse/discourse-calendar/pull/23). Спасибо @nbianca, @eviltrout и @j.jaffeux. * * * Уважаемые разработчики Discourse, после некоторого перерыва, в течение которого мы не могли уделять внимание нашим экземплярам Discourse на [https://community.hiveeyes.org/](https://community.hiveeyes.org/) и [https://community.panodata.org/](https://community.panodata.org/), мы вчера вечером приступили к их обновлению. Обновление форума сообщества Hiveeyes прошло довольно гладко, как обычно. PostgreSQL был обновлен с версии 10 до версии 12, и наблюдать за процессом обновления было одно удовольствие. Отличная работа всей команды, вкладывающей душу в Discourse! Однако при попытке обновить форум сообщества Panodata мы столкнулись с некоторыми проблемами. На втором этапе выполнения команды `./launcher rebuild community.panodata.org` система испытывает экстремально высокую нагрузку и в настоящее время полностью загружает весь сервер. Я примерно определил, где начинается проблема, и поделюсь частью логов здесь. Похоже, что процесс миграции базы данных попадает в какой-то цикл при обращении к таблице `post_custom_fields`. Дайте знать, если вам потребуется дополнительная информация. Не создал ли я для нас серьезные проблемы? Я с нетерпением жду любых советов, которые помогут нам выйти из этой ситуации. Заранее спасибо и с наилучшими пожеланиями, Андреас. * * * ```plaintext I, [2020-12-07T11:33:28.554810 #1] INFO -- : > cd /var/www/discourse && su discourse -c 'bundle exec rake db:migrate' 2020-12-07 11:33:34.200 UTC [3888] discourse@discourse LOG: duration: 119.116 ms parse : SELECT t.oid, t.typname FROM pg_type as t WHERE t.typname IN ('int2', 'int4', 'int8', 'oid', 'float4', 'float8', 'bool', 'timestamp', 'timestamptz') 2020-12-07 11:33:34.413 UTC [3888] discourse@discourse LOG: duration: 213.398 ms bind : SELECT t.oid, t.typname FROM pg_type as t WHERE t.typname IN ('int2', 'int4', 'int8', 'oid', 'float4', 'float8', 'bool', 'timestamp', 'timestamptz') 2020-12-07 11:33:34.662 UTC [3888] discourse@discourse LOG: duration: 220.921 ms parse : SELECT t.oid, t.typname, t.typelem, t.typdelim, t.typinput, r.rngsubtype, t.typtype, t.typbasetype FROM pg_type as t LEFT JOIN pg_range as r ON oid = rngtypid WHERE t.typname IN ('int2', 'int4', 'int8', 'oid', 'float4', 'float8', 'text', 'varchar', 'char', 'name', 'bpchar', 'bool', 'bit', 'varbit', 'timestamptz', 'date', 'money', 'bytea', 'point', 'hstore', 'json', 'jsonb', 'cidr', 'inet', 'uuid', 'xml', 'tsvector', 'macaddr', 'citext', 'ltree', 'line', 'lseg', 'box', 'path', 'polygon', 'circle', 'interval', 'time', 'timestamp', 'numeric') OR t.typtype IN ('r', 'e', 'd') OR t.typinput = 'array_in(cstring,oid,integer)'::regprocedure OR t.typelem != 0 2020-12-07 11:33:34.820 UTC [3888] discourse@discourse LOG: duration: 157.985 ms bind : SELECT t.oid, t.typname, t.typelem, t.typdelim, t.typinput, r.rngsubtype, t.typtype, t.typbasetype FROM pg_type as t LEFT JOIN pg_range as r ON oid = rngtypid WHERE t.typname IN ('int2', 'int4', 'int8', 'oid', 'float4', 'float8', 'text', 'varchar', 'char', 'name', 'bpchar', 'bool', 'bit', 'varbit', 'timestamptz', 'date', 'money', 'bytea', 'point', 'hstore', 'json', 'jsonb', 'cidr', 'inet', 'uuid', 'xml', 'tsvector', 'macaddr', 'citext', 'ltree', 'line', 'lseg', 'box', 'path', 'polygon', 'circle', 'interval', 'time', 'timestamp', 'numeric') OR t.typtype IN ('r', 'e', 'd') OR t.typinput = 'array_in(cstring,oid,integer)'::regprocedure OR t.typelem != 0 2020-12-07 11:33:35.197 UTC [3888] discourse@discourse LOG: duration: 376.904 ms execute : SELECT t.oid, t.typname, t.typelem, t.typdelim, t.typinput, r.rngsubtype, t.typtype, t.typbasetype FROM pg_type as t LEFT JOIN pg_range as r ON oid = rngtypid WHERE t.typname IN ('int2', 'int4', 'int8', 'oid', 'float4', 'float8', 'text', 'varchar', 'char', 'name', 'bpchar', 'bool', 'bit', 'varbit', 'timestamptz', 'date', 'money', 'bytea', 'point', 'hstore', 'json', 'jsonb', 'cidr', 'inet', 'uuid', 'xml', 'tsvector', 'macaddr', 'citext', 'ltree', 'line', 'lseg', 'box', 'path', 'polygon', 'circle', 'interval', 'time', 'timestamp', 'numeric') OR t.typtype IN ('r', 'e', 'd') OR t.typinput = 'array_in(cstring,oid,integer)'::regprocedure OR t.typelem != 0 2020-12-07 11:33:35.388 UTC [3888] discourse@discourse LOG: duration: 165.044 ms statement: SELECT name, data_type, value FROM site_settings 2020-12-07 11:33:37.170 UTC [3888] discourse@discourse LOG: duration: 166.378 ms statement: SELECT a.attname, format_type(a.atttypid, a.atttypmod), pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod, c.collname, col_description(a.attrelid, a.attnum) AS comment FROM pg_attribute a LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum LEFT JOIN pg_type t ON a.atttypid = t.oid LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation WHERE a.attrelid = '"users"'::regclass AND a.attnum > 0 AND NOT a.attisdropped ORDER BY a.attnum 2020-12-07 11:33:40.578 UTC [3888] discourse@discourse ERROR: relation "user_api_key_scopes" does not exist at character 454 2020-12-07 11:33:40.578 UTC [3888] discourse@discourse STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod), pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod, c.collname, col_description(a.attrelid, a.attnum) AS comment FROM pg_attribute a LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum LEFT JOIN pg_type t ON a.atttypid = t.oid LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation WHERE a.attrelid = '"user_api_key_scopes"'::regclass AND a.attnum > 0 AND NOT a.attisdropped ORDER BY a.attnum 2020-12-07 11:33:41.954 UTC [3898] discourse@discourse LOG: duration: 232.874 ms statement: CREATE TABLE "calendar_events" ("id" bigserial primary key, "topic_id" integer NOT NULL, "post_id" integer, "post_number" integer, "user_id" integer, "username" character varying, "description" character varying, "start_date" timestamp NOT NULL, "end_date" timestamp, "recurrence" character varying, "region" character varying, "created_at" timestamp NOT NULL, "updated_at" timestamp NOT NULL) 2020-12-07 11:33:42.693 UTC [3898] discourse@discourse LOG: duration: 349.048 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:33:43.190 UTC [3898] discourse@discourse LOG: duration: 289.433 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:33:43.692 UTC [3898] discourse@discourse LOG: duration: 293.116 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:33:44.189 UTC [3898] discourse@discourse LOG: duration: 288.891 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:33:44.622 UTC [3898] discourse@discourse LOG: duration: 290.721 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:33:45.051 UTC [3898] discourse@discourse LOG: duration: 291.924 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:33:45.544 UTC [3898] discourse@discourse LOG: duration: 289.327 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:33:46.039 UTC [3898] discourse@discourse LOG: duration: 291.734 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:33:46.471 UTC [3898] discourse@discourse LOG: duration: 290.173 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:33:46.900 UTC [3898] discourse@discourse LOG: duration: 288.805 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:33:47.392 UTC [3898] discourse@discourse LOG: duration: 288.812 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:33:47.884 UTC [3898] discourse@discourse LOG: duration: 286.272 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:33:48.326 UTC [3898] discourse@discourse LOG: duration: 302.706 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:33:48.755 UTC [3898] discourse@discourse LOG: duration: 288.766 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:33:49.260 UTC [3898] discourse@discourse LOG: duration: 299.901 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:33:49.762 UTC [3898] discourse@discourse LOG: duration: 293.430 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:33:50.192 UTC [3898] discourse@discourse LOG: duration: 291.202 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:33:50.625 UTC [3898] discourse@discourse LOG: duration: 293.805 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:33:51.138 UTC [3898] discourse@discourse LOG: duration: 301.952 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:33:51.643 UTC [3898] discourse@discourse LOG: duration: 291.137 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:33:52.075 UTC [3898] discourse@discourse LOG: duration: 292.726 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:33:52.517 UTC [3898] discourse@discourse LOG: duration: 301.480 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:33:53.027 UTC [3898] discourse@discourse LOG: duration: 295.174 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:33:53.583 UTC [3898] discourse@discourse LOG: duration: 292.007 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:33:54.025 UTC [3898] discourse@discourse LOG: duration: 298.499 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:33:54.461 UTC [3898] discourse@discourse LOG: duration: 294.801 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:33:54.957 UTC [3898] discourse@discourse LOG: duration: 289.026 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:33:55.463 UTC [3898] discourse@discourse LOG: duration: 301.325 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:33:55.905 UTC [3898] discourse@discourse LOG: duration: 296.361 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:33:56.348 UTC [3898] discourse@discourse LOG: duration: 302.991 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:33:58.843 UTC [3898] discourse@discourse LOG: duration: 2289.322 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:33:59.376 UTC [3898] discourse@discourse LOG: duration: 322.300 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:34:00.445 UTC [3898] discourse@discourse LOG: duration: 921.013 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:34:00.749 UTC [3898] discourse@discourse LOG: duration: 111.277 ms bind : SELECT "calendar_events".* FROM "calendar_events" WHERE "calendar_events"."post_id" = 186 2020-12-07 11:34:02.055 UTC [3898] discourse@discourse LOG: duration: 1302.319 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:34:02.611 UTC [3898] discourse@discourse LOG: duration: 299.807 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:34:03.130 UTC [3898] discourse@discourse LOG: duration: 310.089 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:34:03.586 UTC [3898] discourse@discourse LOG: duration: 310.537 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:34:04.050 UTC [3898] discourse@discourse LOG: duration: 316.072 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:34:04.585 UTC [3898] discourse@discourse LOG: duration: 315.813 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:34:05.138 UTC [3898] discourse@discourse LOG: duration: 332.339 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:34:05.611 UTC [3898] discourse@discourse LOG: duration: 320.420 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:34:06.086 UTC [3898] discourse@discourse LOG: duration: 313.841 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:34:06.636 UTC [3898] discourse@discourse LOG: duration: 330.514 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:34:07.277 UTC [3898] discourse@discourse LOG: duration: 423.374 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:34:08.308 UTC [3898] discourse@discourse LOG: duration: 881.346 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:34:09.041 UTC [3898] discourse@discourse LOG: duration: 581.518 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:34:09.674 UTC [3898] discourse@discourse LOG: duration: 413.955 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:34:10.538 UTC [3898] discourse@discourse LOG: duration: 642.608 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:34:11.047 UTC [3898] discourse@discourse LOG: duration: 358.794 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:34:17.793 UTC [3898] discourse@discourse LOG: duration: 6599.545 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:34:33.146 UTC [3898] discourse@discourse LOG: duration: 15127.645 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:34:53.075 UTC [3898] discourse@discourse LOG: duration: 316.940 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:35:04.574 UTC [57] LOG: using stale statistics instead of current ones because stats collector is not responding 2020-12-07 11:35:19.396 UTC [3898] discourse@discourse LOG: duration: 26159.832 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:35:19.893 UTC [3898] discourse@discourse LOG: duration: 315.783 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:35:20.438 UTC [3898] discourse@discourse LOG: duration: 322.196 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:35:21.081 UTC [3898] discourse@discourse LOG: duration: 403.873 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:35:34.236 UTC [3898] discourse@discourse LOG: duration: 12995.895 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:35:55.785 UTC [3898] discourse@discourse LOG: duration: 20970.124 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 167:M 07 Dec 2020 11:36:31.074 * 10 changes in 300 seconds. Saving... 167:M 07 Dec 2020 11:36:31.267 * Background saving started by pid 3911 2020-12-07 11:36:40.770 UTC [3898] discourse@discourse LOG: duration: 44203.350 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 3911:C 07 Dec 2020 11:36:52.503 * DB saved on disk 3911:C 07 Dec 2020 11:36:52.518 * RDB: 1 MB of memory used by copy-on-write 167:M 07 Dec 2020 11:36:52.618 * Background saving terminated with success 2020-12-07 11:37:00.570 UTC [3898] discourse@discourse LOG: duration: 19089.557 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:37:09.917 UTC [3898] discourse@discourse LOG: duration: 9086.196 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:37:50.176 UTC [3898] discourse@discourse LOG: duration: 39768.704 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:38:06.180 UTC [3898] discourse@discourse LOG: duration: 15340.958 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:38:16.549 UTC [3898] discourse@discourse LOG: duration: 9526.000 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:38:45.231 UTC [3898] discourse@discourse LOG: duration: 28221.839 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:39:26.818 UTC [3898] discourse@discourse LOG: duration: 41289.489 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:39:56.099 UTC [3898] discourse@discourse LOG: duration: 28172.307 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:39:58.117 UTC [3898] discourse@discourse LOG: duration: 1384.587 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:39:59.305 UTC [3898] discourse@discourse LOG: duration: 758.125 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:41:07.711 UTC [3898] discourse@discourse LOG: duration: 67823.399 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:41:24.048 UTC [3898] discourse@discourse LOG: duration: 15546.933 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:41:35.033 UTC [3898] discourse@discourse LOG: duration: 10181.365 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:42:04.037 UTC [3898] discourse@discourse LOG: duration: 28545.266 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:43:41.573 UTC [3898] discourse@discourse LOG: duration: 92603.676 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:43:50.101 UTC [3898] discourse@discourse LOG: duration: 7813.096 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:44:03.377 UTC [3898] discourse@discourse LOG: duration: 12590.301 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:44:37.995 UTC [3898] discourse@discourse LOG: duration: 34268.080 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:44:54.399 UTC [3898] discourse@discourse LOG: duration: 15760.378 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:45:56.976 UTC [3898] discourse@discourse LOG: duration: 58722.471 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:46:46.091 UTC [3898] discourse@discourse LOG: duration: 48556.685 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:47:09.536 UTC [3898] discourse@discourse LOG: duration: 23130.829 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:47:33.762 UTC [3898] discourse@discourse LOG: duration: 23694.256 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:48:09.190 UTC [3898] discourse@discourse LOG: duration: 34922.442 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:48:09.539 UTC [3898] discourse@discourse LOG: duration: 127.286 ms parse : SELECT "calendar_events".* FROM "calendar_events" WHERE "calendar_events"."post_id" = 478 2020-12-07 11:49:05.314 UTC [3898] discourse@discourse LOG: duration: 55677.361 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:49:07.150 UTC [3898] discourse@discourse LOG: duration: 1647.675 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:49:19.072 UTC [3898] discourse@discourse LOG: duration: 11720.259 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:49:31.135 UTC [3898] discourse@discourse LOG: duration: 11582.021 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:49:44.365 UTC [3898] discourse@discourse LOG: duration: 12831.278 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:51:41.565 UTC [3898] discourse@discourse LOG: duration: 116599.870 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:52:22.060 UTC [3898] discourse@discourse LOG: duration: 39664.789 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:52:56.540 UTC [3898] discourse@discourse LOG: duration: 33136.236 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:52:57.214 UTC [3898] discourse@discourse LOG: duration: 139.491 ms parse : SELECT "calendar_events".* FROM "calendar_events" WHERE "calendar_events"."post_id" = 487 2020-12-07 11:52:57.472 UTC [3898] discourse@discourse LOG: duration: 258.338 ms bind : SELECT "calendar_events".* FROM "calendar_events" WHERE "calendar_events"."post_id" = 487 2020-12-07 11:53:08.567 UTC [3898] discourse@discourse LOG: duration: 10615.113 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:53:13.908 UTC [3898] discourse@discourse LOG: duration: 1035.837 ms parse : SELECT "calendar_events".* FROM "calendar_events" WHERE "calendar_events"."post_id" = 488 2020-12-07 11:53:14.443 UTC [3898] discourse@discourse LOG: duration: 475.898 ms bind : SELECT "calendar_events".* FROM "calendar_events" WHERE "calendar_events"."post_id" = 488 2020-12-07 11:53:21.659 UTC [3898] discourse@discourse LOG: could not send data to client: Broken pipe 2020-12-07 11:53:21.659 UTC [3898] discourse@discourse STATEMENT: SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc 2020-12-07 11:53:21.734 UTC [3898] discourse@discourse FATAL: connection to client lost 2020-12-07 11:53:21.734 UTC [3898] discourse@discourse STATEMENT: SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc Killed I, [2020-12-07T11:53:30.522597 #1] INFO -- : I, [2020-12-07T11:53:35.390781 #1] INFO -- : Terminating async processes I, [2020-12-07T11:53:35.815966 #1] INFO -- : Sending INT to HOME=/var/lib/postgresql USER=postgres exec chpst -u postgres:postgres:ssl-cert -U postgres:postgres:ssl-cert /usr/lib/postgresql/12/bin/postmaster -D /etc/postgresql/12/main pid: 50 2020-12-07 11:53:36.108 UTC [50] LOG: received fast shutdown request I, [2020-12-07T11:53:36.185340 #1] INFO -- : Sending TERM to exec chpst -u redis -U redis /usr/bin/redis-server /etc/redis/redis.conf pid: 167 167:signal-handler (1607342016) Received SIGTERM scheduling shutdown... 167:M 07 Dec 2020 11:53:36.211 # User requested shutdown... 167:M 07 Dec 2020 11:53:36.211 * Saving the final RDB snapshot before exiting. 2020-12-07 11:53:36.626 UTC [50] LOG: aborting any active transactions 2020-12-07 11:53:37.025 UTC [50] LOG: background worker "logical replication launcher" (PID 59) exited with exit code 1 2020-12-07 11:53:37.616 UTC [54] LOG: shutting down 167:M 07 Dec 2020 11:53:38.311 * DB saved on disk 167:M 07 Dec 2020 11:53:38.312 # Redis is now ready to exit, bye bye... 2020-12-07 11:53:40.331 UTC [50] LOG: database system is shut down FAILED -------------------- Pups::ExecError: cd /var/www/discourse && su discourse -c 'bundle exec rake db:migrate' failed with return # Location of failure: /pups/lib/pups/exec_command.rb:112:in `spawn' exec failed with the params {"cd"=>"$home", "hook"=>"db_migrate", "cmd"=>["su discourse -c 'bundle exec rake db:migrate'"]} a03fe2903df190bacab2f312cb2c8dca7707a9f5d9fcca0821ea5b7ce0e2d742 **FAILED TO BOOTSTRAP** please scroll up and look for earlier error messages, there may be more than one. ./discourse-doctor may help diagnose the problem. ``` ---
### Author: ![amotl](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/amotl/32/109873_2.png) [@amotl](https://meta.discourse.org/u/amotl) #### Post date: [07.Декабрь.2020 12:26:36 UTC](https://meta.discourse.org/t/database-migration-chokes-on-huge-value-of-a-calendar-details-item-in-table-post-custom-fields/172483/2 "2020-12-07T12:26:36Z")
Следуя ссылкам [Upgrade of postgres failed](https://meta.discourse.org/t/upgrade-of-postgres-failed/165413) и [PostgreSQL 12 update](https://meta.discourse.org/t/postgresql-12-update/151236), нам не удалось выявить ничего конкретного, что могло бы помочь нам в данной ситуации. Эта установка не является особенно большой: ```plaintext root@pulp:/srv/www/organizations/panodata/community.panodata.org# du -sch . 661M total ``` Попытка снова запустить приложение в порядке: > [@Falco](#): > > чтобы привести его в лучшее состояние. не удаётся, так как Rails-приложение не может подключиться к PostgreSQL: ```plaintext ==> shared/standalone/log/rails/unicorn.stderr.log <== I, [2020-12-07T12:20:58.960126 #459] INFO -- : Обновление списка Gem /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/pg-1.1.4/lib/pg.rb:56:in `initialize': не удалось подключиться к серверу: Нет такого файла или каталога (PG::ConnectionBad) Запущен ли сервер локально и принимает ли он подключения через Unix-доменный сокет "/var/run/postgresql/.s.PGSQL.5432"? из /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/pg-1.1.4/lib/pg.rb:56:in `new' из /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/pg-1.1.4/lib/pg.rb:56:in `connect' ``` ---
### Author: ![amotl](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/amotl/32/109873_2.png) [@amotl](https://meta.discourse.org/u/amotl) #### Post date: [07.Декабрь.2020 12:32:14 UTC](https://meta.discourse.org/t/database-migration-chokes-on-huge-value-of-a-calendar-details-item-in-table-post-custom-fields/172483/3 "2020-12-07T12:32:14Z")
```plaintext tail -n100 -F shared/standalone/log/rails/production.log ``` показывает: ```plaintext Создание области видимости :pending. Переопределение существующего метода DiscoursePostEvent::EventDate.pending. Создание области видимости :open. Переопределение существующего метода Poll.open. Миграция в AddSmtpAndImapToGroups (20181220115844) Миграция в AddImapFieldsToIncomingEmails (20190111183409) Миграция в AddSecureToUploads (20190716173854) Миграция в RsvpMigration (20191016154136) Миграция в AddLastSeenToCategoryUsers (20191107025041) Миграция в AddIndexToLastSeenAtOnCategoryUsers (20191107025140) Миграция в ChangeNotificationLevel (20191107032231) Миграция в RemoveSuppressFromLatestFromCategory (20191107190330) Миграция в AddTargetTagIdToTags (20191113193141) Миграция в AddGraphToPolls (20191114160613) Миграция в RenameSiteSettingAssignEmailer (20191119174425) Миграция в AddTimezoneToUserOptions (20191120015344) Миграция в AddUniqueIndexToDevelopers (20191128222140) Миграция в DropUnusedGoogleInstagramAuthTables (20191129144706) Миграция в AddFeaturedTopicIdToUserProfiles (20191202202212) Миграция в CreateStandaloneBookmarksTable (20191205100434) Миграция в AddGroupNameToPolls (20191206123012) Миграция в EnsuresUniqueAcceptedAnswerPostId (20191209095548) Миграция в AddIndexToInvites (20191211152404) Миграция в AddHashedApiKey (20191211170000) Миграция в PopulateTopicIdOnBookmarks (20191217035630) Миграция в RemoveKeyFromApiKeys (20191219112000) Миграция в AddDistinctBadgeCountToUserStats (20191220134101) Миграция в AddAccessControlColumnsToUpload (20191230055237) Миграция в AddFeaturedRankToUserBadges (20200107161405) Миграция в UpdateUserProfilesIndexes (20200109130028) Миграция в AddGroupToCustomEmojis (20200116092259) Миграция в RenameReplyIdColumn (20200116140132) Миграция в UpdatePostReplyIndexes (20200117141138) Миграция в AddTriggerToSyncPostReplies (20200117172135) Миграция в MakePostReplyIdColumnReadOnly (20200117174646) Миграция в DropUnusedColumns (20200120131338) Миграция в RemoveBounceScoreThresholdDeactivateSiteSetting (20200130115859) Миграция в MarkBookmarksTopicIdNotNull (20200203061927) Миграция в CreateCalendarEvents (20200226183018) ``` и (вероятно!) при второй попытке выполнения `./launcher rebuild community.panodata.org` было выведено: ```plaintext Создание области видимости :pending. Переопределение существующего метода DiscoursePostEvent::EventDate.pending. Создание области видимости :open. Переопределение существующего метода Poll.open. Миграция в CreateCalendarEvents (20200226183018) ``` ---
### Author: ![Falco](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/falco/32/179432_2.png) [@Falco](https://meta.discourse.org/u/Falco) #### Post date: [07.Декабрь.2020 12:34:04 UTC](https://meta.discourse.org/t/database-migration-chokes-on-huge-value-of-a-calendar-details-item-in-table-post-custom-fields/172483/4 "2020-12-07T12:34:04Z")
> [@amotl](#): > > `calendar_events` Я предлагаю удалить плагин календаря и попробовать снова. ---
### Author: ![amotl](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/amotl/32/109873_2.png) [@amotl](https://meta.discourse.org/u/amotl) #### Post date: [07.Декабрь.2020 12:35:50 UTC](https://meta.discourse.org/t/database-migration-chokes-on-huge-value-of-a-calendar-details-item-in-table-post-custom-fields/172483/5 "2020-12-07T12:35:50Z")
> [@Falco](#): > > Я предлагаю удалить плагин календаря и попробовать снова. Принято, спасибо! Я только что деактивировал оба этих плагина и снова собираю систему. ```plaintext # - git clone https://github.com/discourse/discourse-calendar.git # - git clone https://github.com/angusmcleod/discourse-events.git ``` ---
### Author: ![amotl](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/amotl/32/109873_2.png) [@amotl](https://meta.discourse.org/u/amotl) #### Post date: [07.Декабрь.2020 13:09:30 UTC](https://meta.discourse.org/t/database-migration-chokes-on-huge-value-of-a-calendar-details-item-in-table-post-custom-fields/172483/6 "2020-12-07T13:09:30Z")
Уважаемый Рафаэль, Я могу подтвердить, что плагин [Discourse Calendar (and Event)](https://meta.discourse.org/t/discourse-calendar/97376) действительно был причиной проблемы, поэтому я соответствующим образом изменю заголовок этого сообщения. Для информации, я также делюсь нашим текущим списком плагинов. С установленными этими плагинами (и отключённым `discourse-calendar`) задача миграции базы данных прошла успешно, и форум снова работает — [https://community.panodata.org/](https://community.panodata.org/). ```plaintext after_code: - exec: cd: $home/plugins cmd: - git clone https://github.com/discourse/docker_manager.git - git clone https://github.com/discourse/discourse-translator.git - git clone https://github.com/discourse/discourse-checklist.git - git clone https://github.com/discourse/discourse-assign.git - git clone https://github.com/discourse/discourse-solved.git - git clone https://github.com/discourse/discourse-math.git - git clone https://github.com/angusmcleod/discourse-tickets.git - git clone https://github.com/angusmcleod/discourse-knowledge-base.git - git clone https://github.com/angusmcleod/discourse-locations.git - git clone https://github.com/discourse/discourse-footnote.git - git clone https://github.com/discourse/discourse-graphviz.git - git clone https://github.com/angusmcleod/discourse-custom-wizard.git - git clone https://github.com/discourse/discourse-sitemap.git - git clone https://github.com/angusmcleod/discourse-events.git # - git clone https://github.com/discourse/discourse-calendar.git ``` Огромное спасибо, держите дух и с наилучшими пожеланиями, Андреас. P.S.: Как нам следует действовать при подаче отчёта об ошибке в `discourse-calendar`, если это уместно? ---
### Author: ![Falco](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/falco/32/179432_2.png) [@Falco](https://meta.discourse.org/u/Falco) #### Post date: [07.Декабрь.2020 13:22:38 UTC](https://meta.discourse.org/t/database-migration-chokes-on-huge-value-of-a-calendar-details-item-in-table-post-custom-fields/172483/7 "2020-12-07T13:22:38Z")
Теперь, когда вы успешно мигрировали на PG12, вы можете попробовать снова добавить плагин, чтобы проверить, пройдет ли миграция. ---
### Author: ![amotl](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/amotl/32/109873_2.png) [@amotl](https://meta.discourse.org/u/amotl) #### Post date: [07.Декабрь.2020 14:49:12 UTC](https://meta.discourse.org/t/database-migration-chokes-on-huge-value-of-a-calendar-details-item-in-table-post-custom-fields/172483/8 "2020-12-07T14:49:12Z")
> [@Falco](#): > > Попробуйте снова добавить плагин, чтобы проверить, сработает ли миграция. Я уже сделал это. Однако процесс миграции снова попадает в ту же ситуацию. > [@amotl](#): > > ```plaintext > 2020-12-07 11:33:41.954 UTC [3898] discourse@discourse LOG: duration: 232.874 ms statement: CREATE TABLE "calendar_events" ("id" bigserial primary key, "topic_id" integer NOT NULL, "post_id" integer, "post_number" integer, "user_id" integer, "username" character varying, "description" character varying, "start_date" timestamp NOT NULL, "end_date" timestamp, "recurrence" character varying, "region" character varying, "created_at" timestamp NOT NULL, "updated_at" timestamp NOT NULL) > 2020-12-07 11:33:42.693 UTC [3898] discourse@discourse LOG: duration: 349.048 ms execute : SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc > > ``` Как вы думаете, связано ли это с конкретными данными в нашем экземпляре в таблицах `calendar_events` и `post_custom_fields`? Похоже, процесс застревает на `post_id = 20`. Поэтому я попробовал следующее: ```plaintext ./launcher enter community.panodata.org rails console DB.query('SELECT "post_custom_fields"."name", "post_custom_fields"."value" FROM "post_custom_fields" WHERE "post_custom_fields"."post_id" = 20 ORDER BY id asc') ``` После этого система становится полностью неответной, а нагрузка на систему взлетает до небес. #### `top` ```plaintext top - 15:28:06 up 158 days, 14:05, 5 users, load average: 19.28, 12.96, 5.94 Tasks: 289 total, 1 running, 288 sleeping, 0 stopped, 0 zombie %Cpu(s): 2.3 us, 6.1 sy, 0.3 ni, 1.2 id, 90.0 wa, 0.0 hi, 0.0 si, 0.0 st MiB Mem : 7006.9 total, 40.1 free, 6177.4 used, 789.4 buff/cache MiB Swap: 4096.0 total, 2048.4 free, 2047.6 used. 659.2 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 12470 xxx 25 5 9229456 89424 4604 S 117.4 1.2 0:35.66 ruby 43 root 20 0 0 0 0 D 1.0 0.0 11:46.02 kswapd0 10035 xxx 20 0 7481716 4.2g 6020 D 1.0 61.3 0:12.60 bundle ``` #### `iotop` ```plaintext Total DISK READ: 1373.05 K/s | Total DISK WRITE: 29.79 K/s Current DISK READ: 1373.05 K/s | Current DISK WRITE: 3.17 M/s TID PRIO USER DISK READ DISK WRITE SWAPIN IO> COMMAND 27667 be/4 root 0.00 B/s 0.00 B/s 0.00 % 99.99 % sshd -D 10035 be/4 xxx 0.00 B/s 0.00 B/s 0.00 % 99.99 % pry 4863 be/4 xxx 0.00 B/s 0.00 B/s 0.00 % 99.99 % bash config/unicorn_launcher -E production -c config/unicorn.conf.rb 11145 be/4 xxx 0.00 B/s 0.00 B/s 0.00 % 99.99 % unicorn worker[0] -E production -c config/unicorn.conf.rb 11363 be/4 xxx 0.00 B/s 0.00 B/s 0.00 % 99.99 % sleep 1 11362 be/4 xxx 0.00 B/s 0.00 B/s 0.00 % 99.99 % sleep 1 43 be/4 root 0.00 B/s 0.00 B/s 0.00 % 99.99 % [kswapd0] 344 be/4 root 49.66 K/s 0.00 B/s 0.00 % 61.49 % [xfsaild/xvda4] 13189 be/4 xxx 0.00 B/s 9.93 K/s 0.00 % 56.78 % postgres: 12/main: walwriter 13357 be/5 xxx 0.00 B/s 0.00 B/s 0.00 % 20.45 % sidekiq 6.1.2 discourse [0 of 5 busy] [mini_scheduler*] 5497 be/5 xxx 4.97 K/s 0.00 B/s 2.82 % 0.00 % sidekiq 6.0.5 discourse [0 of 5 busy] [heartbeat] ``` Только после выхода из ```plaintext [3] pry(main)> ``` система медленно восстанавливается, и снова начинает отвечать. Похоже, происходит серьезная борьба за ресурсы. * * * ```plaintext DB.query_single('SELECT count(*) FROM "post_custom_fields"') => [2002] DB.query_single('SELECT count(*) FROM "post_custom_fields" WHERE post_id=20') => [2] ``` ```plaintext DB.query_single('SELECT name FROM "post_custom_fields" WHERE post_id=20') => ["calendar-details", "post_detected_lang"] ``` Однако, как только я выбираю столбец `value`, всё идёт наперекосяк. ---
### Author: ![amotl](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/amotl/32/109873_2.png) [@amotl](https://meta.discourse.org/u/amotl) #### Post date: [07.Декабрь.2020 14:49:38 UTC](https://meta.discourse.org/t/database-migration-chokes-on-huge-value-of-a-calendar-details-item-in-table-post-custom-fields/172483/9 "2020-12-07T14:49:38Z")
> [@amotl](#): > > Однако, как только я выбираю столбец `value`, всё идёт наперекосяк. Ха. Это вменяемо? ```plaintext DB.query_single('SELECT name, length(value) FROM "post_custom_fields" WHERE post_id=20') => ["calendar-details", 83361791, "post_detected_lang", 2] ``` Итак, столбец `value` этого конкретного элемента `calendar-details` содержит данные объёмом около 80 МБ. Думаю, мы на верном пути. Похоже, это единственный элемент, содержащий такую большую полезную нагрузку в своём значении `calendar-details`. ```plaintext print(DB.query("SELECT post_id, length(value) FROM post_custom_fields WHERE name='calendar-details' ORDER BY post_id").map { |r| r.post_id.to_s + ": " + r.length.to_s }.join("\n")) 20: 83361791 84: 113 133: 113 177: 58 223: 2 263: 113 379: 112 385: 130 439: 112 456: 112 457: 58 495: 117 552: 2 ``` ---
### Author: ![amotl](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/amotl/32/109873_2.png) [@amotl](https://meta.discourse.org/u/amotl) #### Post date: [07.Декабрь.2020 15:10:39 UTC](https://meta.discourse.org/t/database-migration-chokes-on-huge-value-of-a-calendar-details-item-in-table-post-custom-fields/172483/10 "2020-12-07T15:10:39Z")
Я продвигаюсь в уточнении этого вопроса. ```plaintext DB.query("SELECT topic_id, cooked FROM posts WHERE id=20") => [#<#:0x0000560b8b841880 @cooked= "

Если вы интересуетесь деятельностью в репозиториях Panodata GitHub, пожалуйста, следите за этой темой.

\n

Мы передаём уведомления о пушах из GitHub в Discourse, используя отличный GitHub - huw/github-to-discourse: Пересылает данные о коммитах из вебхука GitHub в тему Discourse.

", @topic_id=16>] ``` Итак, похоже, что это самый первый пост в теме [Logbook of code changes - The workbench - Panodata Community](https://community.panodata.org/t/logbook-of-code-changes/16). Насколько я могу доверять своему здравому смыслу, я никогда не назначал этому посту ничего, связанного с записью в календаре. На самом деле, это тема, куда автоматически публикуются сообщения с использованием [GitHub - huw/github-to-discourse: Forwards commit data from a GitHub webhook to a Discourse thread · GitHub](https://github.com/huw/github-to-discourse). Однако первый пост (`id=20`) был создан вручную. ---
### Author: ![Falco](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/falco/32/179432_2.png) [@Falco](https://meta.discourse.org/u/Falco) #### Post date: [07.Декабрь.2020 15:42:48 UTC](https://meta.discourse.org/t/database-migration-chokes-on-huge-value-of-a-calendar-details-item-in-table-post-custom-fields/172483/11 "2020-12-07T15:42:48Z")
> [@amotl](#): > > Я уже сделал это. Однако процесс миграции упирается в ту же ситуацию. Ну, значит, это никак не связано с обновлением PG12, так что это не моя вина 😛 > [@amotl](#): > > Итак, столбец `value` этого конкретного элемента `calendar-details` содержит примерно 80 МБ данных. Я думаю, мы на правильном пути. Да, как мы уже не раз говорили, использование плагина магазина или `post_custom_fields` для сложных задач не является устойчивым решением в долгосрочной перспективе. Именно поэтому плагин календаря выполняет [эту миграцию](https://github.com/discourse/discourse-calendar/blob/master/db/migrate/20200226183018_create_calendar_events.rb), чтобы перейти к правильным таблицам. К сожалению, похоже, она не была реализована достаточно рано для вас, и одно событие оказалось в некорректном состоянии. Удалите проблемную строку, и всё должно снова работать. ---
### Author: ![amotl](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/amotl/32/109873_2.png) [@amotl](https://meta.discourse.org/u/amotl) #### Post date: [07.Декабрь.2020 15:46:57 UTC](https://meta.discourse.org/t/database-migration-chokes-on-huge-value-of-a-calendar-details-item-in-table-post-custom-fields/172483/12 "2020-12-07T15:46:57Z")
> [@Falco](#): > > Что ж, значит, это не имеет ничего общего с обновлением PG12, так что это не моя вина. Совершенно верно! Я уже исправил заголовок темы, чтобы отразить текущий результат. Огромное спасибо за вашу помощь и обратную связь по этому вопросу. > [@Falco](#): > > Да, как мы уже не раз говорили, использование либо хранилища плагинов, либо post\_custom\_fields для сложных вещей неустойчиво в долгосрочной перспективе. Вот почему календарь выполняет [эту миграцию](https://github.com/discourse/discourse-calendar/blob/master/db/migrate/20200226183018_create_calendar_events.rb), чтобы перейти к правильным таблицам. Понятно. Еще раз спасибо за ваши разъяснения. > [@Falco](#): > > К сожалению, похоже, это произошло слишком поздно для вас, и событие оказалось в некорректном состоянии. Удалите проблемную строку, и всё должно снова работать. Хорошо, сделаю. Так что, я полагаю, нет смысла дальше расследовать первопричину, так как вы уже осведомлены о ней и улучшили ситуацию, обновив схему базы данных соответствующим образом через миграцию, на которую вы ссылаетесь здесь. Спасибо! ---
### Author: ![amotl](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/amotl/32/109873_2.png) [@amotl](https://meta.discourse.org/u/amotl) #### Post date: [07.Декабрь.2020 16:01:44 UTC](https://meta.discourse.org/t/database-migration-chokes-on-huge-value-of-a-calendar-details-item-in-table-post-custom-fields/172483/13 "2020-12-07T16:01:44Z")
Снова здравствуйте, > [@Falco](#): > > Удалите проблемную строку, и всё должно заработать снова. ```plaintext [70] pry(main)> DB.query("DELETE FROM post_custom_fields WHERE post_id=20 AND name='calendar-details'") => [] ``` После удаления проблемной строки мы снова активировали `discourse-calendar`. После ещё одной пересборки можем сообщить, что отсутствующие миграции успешно применены: - `20200226183018 CreateCalendarEvents: migrated (0.8584s)` - `20200409102640 CreatePostEventsTable: migrated (0.0381s)` - `20200409102643 RenameTablesToDiscoursePostEvent: migrated (0.1840s)` - `20201110225115 CreatePostEventDatesTable: migrated (0.0655s)` - `20201111005205 MoveDataToEventDates: migrated (0.0039s)` И теперь всё снова в порядке. Огромное спасибо за отличную работу, которую вы здесь выполняете. 💯 С наилучшими пожеланиями, Андреас. ---
### Author: ![eviltrout](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/eviltrout/32/5275_2.png) [@eviltrout](https://meta.discourse.org/u/eviltrout) #### Post date: [07.Декабрь.2020 18:42:55 UTC](https://meta.discourse.org/t/database-migration-chokes-on-huge-value-of-a-calendar-details-item-in-table-post-custom-fields/172483/14 "2020-12-07T18:42:55Z")