I’ve got a site where it seems that a bunch of uploads have been removed from the uploads table in the database, but still exist in the filesystem. This leaves links to these broken. I’ve fixed a few by re-uploading files into a new topic to create the record in the DB (at the time I thought that the files were missing too, but @angus graciously pointed out why I wasn’t finding them–One day I’ll ask: why do we have both sha1 and base62 names for all of these assets?) and then re-baking the posts that include those uploads, like this:
def rebake_posts_with_uploads(topic_id, post_number)
p = Post.find_by(topic_id: topic_id, post_number: post_number)
exit unless p
re = /upload:\/\/(.+?)\)/
shas= p.raw.scan(re)
shas.each do |sha|
posts = Post.where("raw like '%#{sha[0]}%'")
next unless posts
puts "Found #{posts.count - 1} #{sha[0]}"
posts.each do |post|
next if post.id == p.id
puts "rebake #{post.id}--#{BASE_URL}/t/-/#{post.topic_id}/#{post.post_number}"
post.rebake!
end
end
end
My new plan, since it seems that the files exist in the file system but not in the database, is to do something like this:
for (all files in /shared/uploads/default/original/1x) do |file|
unless file is in uploads table
create upload record
for each post that includes that upload record
rebake
Does that seem right? I’m looking at uploads.rake and don’t see anything that seems to do this already. This is sort of the opposite of
but instead of FileUtils.rm(file_path) I’d instead do an Upload.create, I think.
If this seems really stupid or there is a much better solution, I’d love to hear it before I go down this little rabbit hole.
Thanks.
I don’t know how this happened. I was hoping to pin the blame on a custom plugin, but I’m afraid that’s not the case. It may be related to another discussion, in which someone said:
I just solved a simulator problem by creating a post and into it inserting a link to all of the images and then rebaking all of the posts that contain transparent.gif.
This looks like a somewhat more elegant solution.
I would say make a database-only backup and give it a try. I’m sort of on vacation, but if you have a budget I can see what I can do.
And I too have an important client on ams3; I’ve not yet moved them to aws, but I think that’s happening soon. I have a friend who worked for digital ocean who recommended ams3 because it was their best data center and was largely underutilized (this was now long ago). That didn’t work out as I’d hoped.
dir = Rails.root.join("public", "uploads", "default", "original")
created = 0
Dir.glob(dir.join("**", "*")).select { |f| File.file?(f) }.each do |path|
sha = File.basename(path, File.extname(path))
next if Upload.find_by(sha1: sha)
ext = File.extname(path).delete(".")
relative = path.sub("#{Rails.root}/public", "")
u = Upload.new
u.sha1 = sha
u.url = relative
u.original_filename = File.basename(path)
u.filesize = File.size(path)
u.extension = ext
u.user_id = -1
u.save!(validate: false)
created += 1
puts "Created upload #{u.id}: #{sha}"
end
puts "Total created: #{created}"
重新烘焙引用了已恢复上传的帖子：
fixed_posts = 0
Upload.where(user_id: -1).find_each do |u|
short = u.short_url
next unless short
Post.where("raw LIKE '%upload://%'").find_each do |p|
urls = p.raw.scan(/upload:\/\/^[^\s\]\)]+/)
urls.each do |url|
decoded = Upload.sha1_from_short_url(url)
if decoded == u.sha1
p.rebake!
fixed_posts += 1
puts "Rebaked post #{p.id}"
break
end
end
end
end
puts "Total rebaked: #{fixed_posts}"
