We recently had our download_remote_images_to_local disabled by system as s ‘silent’ change. As a site owner, it would have been useful to receive a DM about this. I’m not sure if there are more changes that can be updated by system like this though.
9 Likes
I’ve seen that happen when the disk is full. Was it that or something else?
2 Likes
It wasn’t full, just going below the threshold we set for downloading remote images.
3 Likes
I think the suggestion is reasonable, unfortunately I do not think we will have time to work on this for quite a while.
Placing a pr-welcome on it in case anyone in the community wants to investigate this. (A PM to admins when we fiddle with the setting)
6 Likes
The “system” broke our site by changing the default trust level to “0” when we had set it to be “1”.
It is very surprising that the system can make ad hoc changes to our production site without any kind of approval or notice and we need to prevent this from happening again in the future.
Is there any way to disable these automatic system changes?
This should not happen, plenty of sites have default of TL1. We need more context, can you open a new topic about this.
3 Likes
So it turns out that this happened due to bootstrap mode on a new server/app that we were setting up.
But my message was less about this specific system change and more about how any system change should ideally require approval so things don’t break. If approval isnt possible then at minimum an email notifying all admins about the change (which I think is more or less what the OP was asking after) would be great.
Thanks for all the work on Discourse!
1 Like
Hi @Earnie_Baird, this is great feedback.
I guess one suggestion would be to have more clarity on bootstrap mode, maybe when you see that on the header and click on it it should very clearly spell out what happened.
2 Likes
Unless there were changes I wasn’t aware of (and I don’t think there were), I strongly approve this feature request.
I’ve encountered this issue several times when temporarily lacking disk space.
Every single time, I noticed the setting was disabled only by chance. I didn’t think of looking at setting changes when I reach a disk space threshold, even after experiencing his multiple times.
I’m pretty sure there are many instances in the wild that have this setting disabled without the admin even knowing it, just because they ran out of disk space one year ago.
The setting change is logged in /admin/logs/staff_action_logs?filters=%7B"subject"%3A"download_remote_images_to_local"%7D, but I don’t remember ever getting any notification when it triggers.
Ideally, I’d like either at least a warning in the dashboard, a notification in the user menu, or an email.
The context of the following quote was quite specific (and old), but it also applies here.
An absence of any kind of notification when a setting is changed by @system can be detrimental.
When I notice download_remote_images_to_local has been disabled at some point, I run one of (or both, once) these rails scripts to trigger the download of remote files:
Rebake all posts from a given date
i = 0
Post.where('created_at >= ?', Date.new(2023, 5, 1)).where('user_id > 0').find_each do |post|
post.rebake!
puts "Post #{post.id}, Created at #{post.created_at}"
i += 1
end
puts "Total number of posts rebaked: #{i}"
Rebake all posts between two given dates
i = 0
Post.where('created_at >= ? AND created_at < ?', Date.new(2021, 12, 1), Date.new(2022, 3, 1)).where('user_id > 0').find_each do |post|
post.rebake!
puts "Post #{post.id}, Created at #{post.created_at}"
i += 1
end
puts "Total number of posts rebaked: #{i}"
2 Likes
Reading Get admin notification from logs? - #4 by JammyDodger made me think of this topic again. I think you could also use the “Schedule a PM with Data Explorer results” or “Schedule a post in a topic with Data Explorer results” automation script and a data explorer query to receive notifications on changes on site settings by system.
Something like
SELECT subject, previous_value, new_value, updated_at
FROM user_histories uh
where uh.action = 3
AND uh.acting_user_id = -1
AND uh.updated_at > CURRENT_TIMESTAMP - INTERVAL '60 minutes'
order by updated_at desc
Then you set up the automation with a recurrence matching the interval in the query, and skip if there are no results to avoid noise.
You could also improve the query to filter for the exact subject, but I thought maybe other automatic changes are interesting as well.
3 Likes
Hi @Moin
Thank your this, it sounds like a very helpful idea!
I ran this query but it doesn’t show changes such as the automatic disabling of the Narrative..
That’s right! There is no entry in the logs for this. The automation only notifies you about changes that are logged in the staff action logs. That’s why I suggested that an entry should always be created for such changes.
This query deliberately only queries a short period of time. When I comment out the time restriction, I see at least some changes, but there are no log entries for most changes.
- AND uh.updated_at > CURRENT_TIMESTAMP - INTERVAL '60 minutes'
+ --AND uh.updated_at > CURRENT_TIMESTAMP - INTERVAL '60 minutes'
However, to prevent the automation script from reporting those old changes, I have added this time restriction.
2 Likes
This is a helpful query for any site to have for troubleshooting, when some behavior has unexpectedly changed. It might even be you or a fellow admin who changed something and it had an unexpected effect.
I created an automation here on meta to run this query weekly, to show all logged changes in the last week. We have alot of cooks in this kitchen and it can be hard sometimes to maintain an overview.
Seems to me the feature request here would be to log all staff actions, more comprehensively.
The query I shared is limited to changes on site settings and limited to actions by system. So it helps noticing when settings are changed by system either because of an automation, like disabling image download because of a full disk or the setting changes when bootstrap mode ends. In combination with a post or PM by the automation plugin this can help in cases like the one described in the OP.
But the query doesn’t really help to find unexpected changes like the change that disabled the welcome message of discobot @gassim would have liked to know about. Most setting changes that happen during a Discourse update aren’t logged and that’s what makes them so hard to track down.
I modified your query to show me all logged changes in the last week.
I agree with you that a gap needs to be filled. Not enough actions are being logged.