# Upgrade mislukt bij fout op dubbele sleutel ("index post hotlinked media op post id en url md5")

**URL:** https://meta.discourse.org/t/upgrade-fails-on-duplicate-key-error-index-post-hotlinked-media-on-post-id-and-url-md5/226002
**Category:** Bug
**Created:** [3 mei 2022 om 15:34 UTC](https://meta.discourse.org/t/upgrade-fails-on-duplicate-key-error-index-post-hotlinked-media-on-post-id-and-url-md5/226002 "2022-05-03T15:34:51Z")
**Posts on this page:** 11
**Page:** 1

<div class="post-metadata">

### Author: ![axfelix](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/axfelix/32/65166_2.png) [@axfelix](https://meta.discourse.org/u/axfelix)
#### Post date: [3 mei 2022 om 15:34 UTC](https://meta.discourse.org/t/upgrade-fails-on-duplicate-key-error-index-post-hotlinked-media-on-post-id-and-url-md5/226002/1 "2022-05-03T15:34:52Z")

</div>

Had an automatic rebuild fail, looks like variations on this issue:

```plaintext
2022-05-03 15:30:54.370 UTC [1262] discourse@discourse LOG: duration: 1707.909 ms statement: INSERT INTO post_hotlinked_media (post_id, url, status, upload_id, created_at, updated_at)
        SELECT
          post_id,
          obj.key AS url,
          'downloaded',
          obj.value::bigint AS upload_id,
          pcf.created_at,
          pcf.updated_at
        FROM post_custom_fields pcf
        JOIN json_each_text(pcf.value::json) obj ON true
        JOIN uploads ON obj.value::bigint = uploads.id
        WHERE name='downloaded_images'

```

`ERROR: duplicate key value violates unique constraint "index_post_hotlinked_media_on_post_id_and_url_md5"`

Had rebuilt less than a month ago with no errors so whatever introduced this seems to have been recent.

---

<div class="post-metadata">

### Author: ![axfelix](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/axfelix/32/65166_2.png) [@axfelix](https://meta.discourse.org/u/axfelix)
#### Post date: [3 mei 2022 om 16:09 UTC](https://meta.discourse.org/t/upgrade-fails-on-duplicate-key-error-index-post-hotlinked-media-on-post-id-and-url-md5/226002/3 "2022-05-03T16:09:04Z")

</div>

seems to be something to do with this recent commit?

> <https://github.com/discourse/discourse/blob/19d2d55011f5808bcbc177119765557a2900dea1/db/migrate/20220428094026_create_post_hotlinked_media.rb>

@david

---

<div class="post-metadata">

### Author: ![david](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/david/32/157490_2.png) [@david](https://meta.discourse.org/u/david)
#### Post date: [3 mei 2022 om 16:15 UTC](https://meta.discourse.org/t/upgrade-fails-on-duplicate-key-error-index-post-hotlinked-media-on-post-id-and-url-md5/226002/4 "2022-05-03T16:15:10Z")

</div>

Yep that’s it. I have a fix in the pipeline, should be merged in the next few minutes. Thanks for reporting @axfelix

[https://github.com/discourse/discourse/pull/16611](https://github.com/discourse/discourse/pull/16611)

---

<div class="post-metadata">

### Author: ![axfelix](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/axfelix/32/65166_2.png) [@axfelix](https://meta.discourse.org/u/axfelix)
#### Post date: [3 mei 2022 om 16:39 UTC](https://meta.discourse.org/t/upgrade-fails-on-duplicate-key-error-index-post-hotlinked-media-on-post-id-and-url-md5/226002/5 "2022-05-03T16:39:11Z")

</div>

Thanks, that was fast!

Do I need to do anything special to pull this down? I’m still getting `Caused by: ActiveRecord::RecordNotUnique: PG::UniqueViolation: ERROR: duplicate key value violates unique constraint "index_post_hotlinked_media_on_post_id_and_url_md5"` when running a launcher rebuild, even after doing a Docker prune.

---

<div class="post-metadata">

### Author: ![david](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/david/32/157490_2.png) [@david](https://meta.discourse.org/u/david)
#### Post date: [3 mei 2022 om 16:48 UTC](https://meta.discourse.org/t/upgrade-fails-on-duplicate-key-error-index-post-hotlinked-media-on-post-id-and-url-md5/226002/6 "2022-05-03T16:48:28Z")

</div>

The fix just hit tests-passed a few seconds ago. If you rebuild now, it should work much better. Let us know how it goes.

---

<div class="post-metadata">

### Author: ![david](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/david/32/157490_2.png) [@david](https://meta.discourse.org/u/david)
#### Post date: [3 mei 2022 om 18:07 UTC](https://meta.discourse.org/t/upgrade-fails-on-duplicate-key-error-index-post-hotlinked-media-on-post-id-and-url-md5/226002/7 "2022-05-03T18:07:13Z")

</div>

We found one more place which can cause this error. So if the first fix didn’t help, this one should 🤞

[https://github.com/discourse/discourse/commit/1c9d8b4999cd7ab885cc06d6c981511318008bb7](https://github.com/discourse/discourse/commit/1c9d8b4999cd7ab885cc06d6c981511318008bb7)

(now live in tests-passed)

---

<div class="post-metadata">

### Author: ![pfaffman](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/pfaffman/32/120154_2.png) [@pfaffman](https://meta.discourse.org/u/pfaffman)
#### Post date: [3 mei 2022 om 19:31 UTC](https://meta.discourse.org/t/upgrade-fails-on-duplicate-key-error-index-post-hotlinked-media-on-post-id-and-url-md5/226002/8 "2022-05-03T19:31:57Z")

</div>

On my development instance it does successfully migrate the database, but if I restore a backup from `2022-04-27-153103-v20220407195246.tar.gz` it fails:

```plaintext
                                                                                [361/9020]
Migrating the database... EXCEPTION: /home/pfaffman/src/discourse-repos/discourse/lib/discourse.rb:126:in `exec': Failed to migrat
e database.
Ignoring /home/pfaffman/src/discourse-repos/discourse/db/schema_cache.yml because it has expired. The cu
rrent schema version is 20220407195246, but the one in the schema cache file is 20220428094027.
rake aborted!
StandardError: An error has occurred, this and all later migrations canceled:

```

followed by some more of that with some of these:

```
ActiveRecord::StatementInvalid: PG::DuplicateObject: ERROR: type "hotlinked_media_status" already exist

```

---

<div class="post-metadata">

### Author: ![axfelix](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/axfelix/32/65166_2.png) [@axfelix](https://meta.discourse.org/u/axfelix)
#### Post date: [3 mei 2022 om 21:22 UTC](https://meta.discourse.org/t/upgrade-fails-on-duplicate-key-error-index-post-hotlinked-media-on-post-id-and-url-md5/226002/9 "2022-05-03T21:22:10Z")

</div>

Just chiming in to say that the first fix worked for us after it passed tests!

---

<div class="post-metadata">

### Author: ![david](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/david/32/157490_2.png) [@david](https://meta.discourse.org/u/david)
#### Post date: [3 mei 2022 om 21:30 UTC](https://meta.discourse.org/t/upgrade-fails-on-duplicate-key-error-index-post-hotlinked-media-on-post-id-and-url-md5/226002/10 "2022-05-03T21:30:53Z")

</div>

> [@axfelix](#):
>
> the first fix worked

Great, thanks for confirming!

> [@pfaffman](#):
>
> but if I restore a backup

I opened a PR with a fix for that one:

[https://github.com/discourse/discourse/pull/16624](https://github.com/discourse/discourse/pull/16624)

---

<div class="post-metadata">

### Author: ![pfaffman](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/pfaffman/32/120154_2.png) [@pfaffman](https://meta.discourse.org/u/pfaffman)
#### Post date: [4 mei 2022 om 14:05 UTC](https://meta.discourse.org/t/upgrade-fails-on-duplicate-key-error-index-post-hotlinked-media-on-post-id-and-url-md5/226002/11 "2022-05-04T14:05:43Z")

</div>

```plaintext
Finished!
[SUCCESS]
Restore done.

```

Thanks!

---

<div class="post-metadata">

### Author: ![david](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/david/32/157490_2.png) [@david](https://meta.discourse.org/u/david)
#### Post date: [4 mei 2022 om 17:03 UTC](https://meta.discourse.org/t/upgrade-fails-on-duplicate-key-error-index-post-hotlinked-media-on-post-id-and-url-md5/226002/12 "2022-05-04T17:03:07Z")

</div>


