Discourse not respecting the number of S3 backups to keep

[ljpp]

Backups are duplicating and not respecting number to keep on disk

@Wingtip Our Discourse instance recently has started to misbehave with regards to backups. Among other things, it’s taking backups twice a day when it is set to take them every other day and it is not respecting the setting we have in place to only maintain a single backup on disk. This is causing the disk to fill up with large backups, leading to subsequent backup failures. Here are our settings: [22%20AM] Here’s some of the recent backup activity: [58%20AM] This results in inadequate sp…

Similar issue has re-occurred for us. Just noticed I have dozens of local backups (1.6GB each), while the setting is restricted to value 3. This has worked for years, but I do recall one incident a long time ago when a similar issue happened.

• Backups complete successfully
• Uploads to S3 are successful
• Running the stable branch. Just upgraded to 2.3.7 and rebooted.

Need to dig in deeper at a better time.

Ping @rizka

Update:

The issue has started on 11th of September. That date does not match to our service breaks or anything we would have done for the site.

Update:

This is not an isolated case, as my other small sandbox instance has the same issue, it seems. This is in completely different infrastructure, hosted at Digital Ocean. Here the backups have not been deleted since September 16th. Uploads are successful here as well.

[ljpp]

Okay…now this is starting to look like a stupid user error – I have completely missed the change in how the backup management is currently working. So Discourse now manages and deletes S3 backups directly, without the need to purge the old backups with a bucket rule? Now increasing the value to 30, as backups should not eat up local disk space.

The number of backups that were stored in the S3 buckets did not match the setting 3 though.

[codinghorror (Jeff Atwood)]

I have no issue with AWS S3 backups on my self hosted instance:

[ljpp]

I have now reconfigured backups and bucket rules to sane values, matching the current Discourse behavior. Like said, I had the number of backups set to 3, based on the old logic of backups. Now it is set at 30.

Please keep the thread open, and I’ll report back in 30 days to verify that Discourse now respects the re-defined new value.

[ljpp]

Forgot about this issue, but now had a look. I have 97 backups stored in S3 while the setting is at 30.

Still running the 2.3 branch, preparing to update to 2.4 soon.

[JammyDodger]

Por curiosidad, ¿lo resolviste al final?

[ljpp]

Ah, esto es historia antigua. Tengo la sensación de que se resolvió de alguna manera, pero hemos estado en CDCK SaaS durante algunos años y ya no recuerdo claramente este problema.

¿Supongo que lo informaría algún otro autohospedador, si este siguiera siendo un problema?

[pfaffman (Jay Pfaffman)]

Tuve exactamente el mismo problema y en los últimos 10 minutos descubrí que la razón era que de alguna manera había activado s3_disable_cleanup. Creo que solo tenía que ver con las cargas de s3, no con las copias de seguridad. Pero eso fue un error.