Restore process cancelled at migrating uploads to S3 step

If you need any help with drilling down to the root cause of issues like this, please let me know. I’m happy to help.

5 Me gusta

So, tried to restore my 2.2GB backup and got this - the restore failed:

EXCEPTION: 44 posts are not remapped to new S3 upload URL. S3 migration failed for db ‘default’.

I am on the 2.6b6, so a fairly recent version. The site is fairly old, from 2016.

From time to time we run into backups were the automatic remapping and migration to S3 fails for various reasons.

This is an old topic, so any recommendation for corrective action? Having a backup I can’t actually use to restore makes me a little nervous.

Edit: I just realized I may be in the wrong thread, as this seems like the same issue

1 me gusta

@gerhard

Gave this another shot and I am still failing on the restore to the 44 posts are not remapped -error.

Temporary disabling of S3 uploads prior to the backup (proposed by @RGJ does not solve the issue, and currently I don’t have a working backup scheme, which is pretty damn serious.

Any suggestions on what to try?

Okay, this is the weirdest case I’ve seen so far. I stumpled upon this post.

https://meta.discourse.org/t/using-object-storage-for-uploads-s3-clones/148916/160?u=ljpp

After wasting hours on this already, I defined the DISCOURSE_CDN_URL just for kicks, a feature my site does not use. The backup was restored successfully.

What is the significance of this parameter, in this scenario? @Falco @pfaffman

EDIT:

I take that back. After success with the test, I took a fresh backup from production and once again started to migrate to a new server. It failed again, but the error message changed.

EXCEPTION: rake posts:missing_uploads identified 3 issues. S3 migration failed for db ‘default’.

EDIT2:

So I kept drilling the database and found these 3 posts.

  • 2 were very old images, that were first published but then moderator replaced the images with a modified version. These were from 2016 and 2018.
  • But the last 1 was odd. It was a very recent post just a couple of weeks ago by myself and it contained a oneboxed https:// link to a development server, that no longer exists. So a broken link breaks the restore process.

I simply deleted these posts manually.

Now, again, a test run shows that a backup might actually succeed. We’ll see.

2 Me gusta

@ljpp I seem to be in a similar situation with a restore failing with

EXCEPTION: 1 posts are not remapped to new S3 upload URL. S3 migration failed for db ‘default’.

Can you elaborate on how you found and deleted the offending posts? What commands are used?

1 me gusta

He deleted the posts in question.

But you wiped your install so you can’t do that, right?

1 me gusta

Tengo este problema al restaurar mi base de datos. ¿Cómo encontraste las publicaciones ofensivas?

Creo que el código que realiza la comprobación está aquí:

Así que creo que esto:

prefix = @migrate_to_multisite ? "uploads/#{@current_db}/original/" : "original/"
base_url = File.join(SiteSetting.Upload.s3_base_url, prefix)
bad = Upload.by_users.where("url NOT LIKE '#{base_url}%'")

Y para mayor seguridad

good = Upload.by_users.where("url LIKE '#{base_url}%'")

Avísame si esto te funciona y veré si puedo crear un tema.

1 me gusta

discourse(prod) > prefix = @migrate_to_multisite ? “uploads/#{RailsMultisite::ConnectionManagement.current_db}/original/” : “original/”
discourse(prod) > base_url = File.join(SiteSetting.Upload.s3_base_url, prefix)
discourse(prod) > bad_uploads = Upload.by_users.where(“url NOT LIKE ‘#{base_url}%’”)
discourse(prod) > bad_uploads.count
=> 0

Esta comprobación me dio 0 errores

¿Hiciste eso en el servidor original? ¿O necesitas hacerlo después de que la base de datos se haya restaurado antes de que realice esa comprobación, usando la opción --pause?

Lo hice en el servidor antiguo.
Al restaurar obtuve el mismo error

[2026-01-16 13:45:52] EXCEPTION: 3 publicaciones no se remapean a la nueva URL de carga de S3. La migración de S3 falló para la base de datos ‘default’.
[2026-01-16 13:45:52] /var/www/discourse/lib/file_store/to_s3_migration.rb:132:in `raise_or_log’

Bueno. Vaya. No estoy seguro de qué más decirte.

¿El good encontró tus subidas?

Probablemente usaría la opción --pause y detendría la restauración antes de que hiciera esa comprobación.

Haría lo que dijo Richard.

1 me gusta

No estoy seguro de su situación exacta, pero es posible que simplemente desee desactivar las cargas a S3, realizar la copia de seguridad, restaurar y volver a activarlas. Esto me ha salvado muchas veces.

2 Me gusta

Entonces, tengo que desactivar estas opciones, ¿correcto?

image

Correcto.

  • desmarcar “habilitar subidas a s3”
  • copia de seguridad / descarga
  • subida / restauración
  • marcar “habilitar subidas a s3”
3 Me gusta

Mientras marcaba “habilitar subidas a s3” obtuve un error

No hablo italiano si no es sobre comida… tal vez nos hagas un favor a todos y lo copies/pegues en Google Translate :slight_smile:

Lo siento

Habilitar subidas a S3

Almacena las subidas en el espacio de disco de Amazon S3. IMPORTANTE: requiere que se proporcionen credenciales de S3 válidas (tanto la clave de acceso como el secreto de clave de acceso).

No puedes habilitar las subidas a S3 porque ya están habilitadas a nivel global, y habilitar las subidas a S3 a nivel de sitio podría causar problemas críticos con las subidas.

1 me gusta

Supongo que ya lo hiciste en app.yml.
¿Las (nuevas) subidas funcionan correctamente? Si es así, entonces no hay ningún problema.

En app.yml tengo configurado s3, correcto.

Entonces, ¿qué tengo que hacer para hacer una copia de seguridad y restaurar de forma segura sin comprobar las subidas?

Simplemente desmarcar “habilitar subidas s3” no funciona