איך אני מוחק את קובץ אווטאר המשתמש?

[snakeninny]

Hi team,

My website is hosted in China and as you know all Chinese websites need to register for a license from the government, hence all Chinese websites are under supervision.

Someone recently reported to my CDN service provider, Qiniu, which is also a CN company, that some images on my website are illegal. My CDN provider informed me that I should delete these images and ‘refresh URL caches’, or they can suspend my account according to law. But I found these images were user avatars, which I don’t know how to delete.

According to How to Delete Uploaded Files? - #3 by codinghorror and How to Delete Uploaded Files? - #25 by Falco uploaded images w/o references will be auto deleted in 48 hours. I have replaced these user avatars with the default letter ones, but nothing seems to be happening to the old user avatars after 72 hours, maybe it’s because user avatars are not considered uploaded images?

Any ideas on how can I delete the user avatar files completely on my server?

Thanks in advance,
Yinglu

[pfaffman (Jay Pfaffman)]

I think it still works like this.

From the URL of the file you want to delete, get the filename.

cd /var/discourse/shared/uploads
find . -name =FILENAME= | xargs exec rm

And then you’d do whatever to purge it from your CDN.

[snakeninny]

Thanks for helping me, I tried but didn’t work

The file is https://iosre.com/user_avatar/iosre.com/lincwee/135/5414_2.png so I guess the filename is 5414_2.png, correct?

And the /uploads is under /shared/standalone rather than /shared
And there’s no such a file under /uploads, as you can see in the shell:

root@iosre:/var/discourse/shared# ls
standalone
root@iosre:/var/discourse/shared# cd standalone/
root@iosre:/var/discourse/shared/standalone# ls
backups  postgres_backup  postgres_run  state  uploads
log      postgres_data    redis_data    tmp
root@iosre:/var/discourse/shared/standalone# find . -name 5414_2.png
root@iosre:/var/discourse/shared/standalone# cd uploads/
root@iosre:/var/discourse/shared/standalone/uploads# find . -name 5414_2.png
root@iosre:/var/discourse/shared/standalone/uploads#

Any more ideas?

[pfaffman (Jay Pfaffman)]

Oh sorry. Avatars work differently. I would have to look at the code to understand where to look. If this is an emergency and you have a budget please contact me directly. I likely won’t have free time to investigate otherwise. Perhaps someone else knows.

[snakeninny]

Sorry this is a personal website and I don’t have any budget for now. Thanks for your reply anyway!

[riking (Kane York)]

To prevent users from uploading future avatars, you can disable allow uploaded avatars and enable the selectable avatars site setting with a large selection.

@snakeninny

I have replaced these user avatars with the default letter ones, but nothing seems to be happening to the old user avatars after 72 hours, maybe it’s because user avatars are not considered uploaded images?

I think you may need to upload a replacement placeholder in order for automatic deletion - the user record is still keeping the uploaded avatar image alive even though it’s not in use.

---

Upload files are named based on the SHA1 of the image content - that may help you to find the backing file?

[pfaffman (Jay Pfaffman)]

Something like

a=UserAvatar.where(user_id: 1234)
u=Upload.find(a.custom_upload_id)
upload_url=u.url

Then

cd /var/discourse/shared/standalone
rm UPLOAD_URL_FROM_ABOVE

[snakeninny]

@riking

To prevent users from uploading future avatars, you can disable allow uploaded avatars and enable the selectable avatars site setting with a large selection.

I have over 7k registered users and only less than 5 users use illegal avatars w/o knowing it, so I don’t want to make it a 0 or 1 problem; I want to solve it case by case rather than stopping all users from using custom avatars.

@riking

I think you may need to upload a replacement placeholder in order for automatic deletion - the user record is still keeping the uploaded avatar image alive even though it’s not in use.

How do I do this? I have no idea at all

[snakeninny]

@riking

Upload files are named based on the SHA1 of the image content - that may help you to find the backing file?

For example this image https://iosre.com/user_avatar/iosre.com/lincwee/135/5414_2.png

snakeninny@bogon ~ % shasum /Users/snakeninny/Desktop/5414_2.png.jpeg 
c8d561c5484a1f197abd32995411caaa25e53bd6  /Users/snakeninny/Desktop/5414_2.png.jpeg

root@iosre:~# cd /var/discourse/shared/standalone
root@iosre:/var/discourse/shared/standalone# find ./ -name *c8d561c5484a1f197abd32995411caaa25e53bd6*
root@iosre:/var/discourse/shared/standalone# 

No luck. Is this what you mean? And any ideas?

[snakeninny]

@pfaffman

a=UserAvatar.where(user_id: 1234)
u=Upload.find(a.custom_upload_id)
upload_url=u.url

Is this a piece of code? Where and how do I execute it? I used to be an iOS developer and am not familiar with front-end programming

[pfaffman (Jay Pfaffman)]

It’s for the rails console. You’d get there with

cd /var/discourse 
./launcher enter app 
rails c

You can use exit to quit. And inside the container you cd to /shared rather than the full path above.

[snakeninny]

For user lincwee in https://iosre.com/user_avatar/iosre.com/lincwee/135/5414_2.png
UserAvatar.where(user_id: 'lincwee') returns an empty array. Any ideas?

root@iosre:~# cd /var/discourse/shared/
root@iosre:/var/discourse/shared# /var/discourse/launcher enter app 
WARNING: Docker version 17.05.0-ce deprecated, recommend upgrade to 17.06.2 or newer.
root@iosre-app:/var/www/discourse# rails c
[1] pry(main)> a=UserAvatar.where(user_id: 1234)
=> []
[2] pry(main)> a=UserAvatar.where(user_id: lincwee)
NameError: undefined local variable or method `lincwee' for main:Object
from (pry):2:in `__pry__'
[3] pry(main)> a=UserAvatar.where(user_id: 'lincwee')
=> []
[4] pry(main)> 

[pfaffman (Jay Pfaffman)]

You’ll need the ID, not username.

u=User.find_by(username: "lincwee")

You can then see the id or access it with u.id

[snakeninny]

Thanks it worked to some extent. I’ve run

cd /var/discourse/shared/
/var/discourse/launcher enter app
rails c

In the terminal and then executed the code

uid = User.find_by(username: "user_name").id
user_avatars = UserAvatar.where(user_id: uid)
user_avatar = user_avatars[0]
upload_url = Upload.find(user_avatar.custom_upload_id).url

to list all avatar URLs and deleted them from my server.
But I guess I still need to further refresh the system/cache? How should I do this?

[snakeninny]

I got another email from my CDN provider and they have frozen my CDN account. So I have to update this thread for further help.

The new email said that the following links were illegal.

https://cdn.iosre.com/user_avatar/iosre.com/baal998/135/2210_2.png
https://cdn.iosre.com/user_avatar/iosre.com/baal998/64/2210_2.png
https://cdn.iosre.com/user_avatar/iosre.com/baal998/75/2210_2.png
https://cdn.iosre.com/user_avatar/iosre.com/baal998/75/2210_2.png
https://cdn.iosre.com/user_avatar/iosre.com/baal998/75/2210_2.png
https://cdn.iosre.com/user_avatar/iosre.com/lincwee/135/5414_2.png
https://cdn.iosre.com/user_avatar/iosre.com/baal998/75/2210_2.png
https://cdn.iosre.com/user_avatar/iosre.com/lincwee/75/5414_2.png
https://cdn.iosre.com/user_avatar/iosre.com/baal998/96/2210_2.png
https://cdn.iosre.com/user_avatar/iosre.com/lincwee/90/5414_2.png

As you can see these are 2 user avatars, which are memes of a former Chinese president. Again, the CDN provider asked me to delete these files on the server and refresh CDN.

I could see from the link that the users were “baal998” and “lincwee”. According to the code snippet in my last post, the uploaded avatar URL of “lincwee” was /uploads/default/original/2X/5/55512211b1c8969c8038b79840464952cd3eb089.jpeg while for “baal998” it was /uploads/default/original/2X/c/cb2188eaeecc3a648f021fa00da4734bd60ca183.jpg. I then run find /var/discourse/shared/ -name *55512211b1c8969c8038b79840464952cd3eb089* and deleted all files found, it worked for “lincwee”. But no luck on “baal998”, I could still access https://iosre.com/user_avatar/iosre.com/baal998/135/2210_2.png even though there were no more files with the name *cb2188eaeecc3a648f021fa00da4734bd60ca183*.

So where the hell is https://iosre.com/user_avatar/iosre.com/baal998/135/2210_2.png stored on my server?

[codinghorror (Jeff Atwood)]

Hmm, that is a difficult situation. Maybe @falco or someone else who worked on avatars recently could advise you?

[snakeninny]

Or to make it simpler: How does Discourse parse a visit to https://iosre.com/user_avatar/iosre.com/baal998/135/2210_2.png? If someone familiar with the source code can tell me the source file or the class/function, that would be really helpful. Thanks!

[pfaffman (Jay Pfaffman)]

Look in the Uploads model, I think.

Also, start with pulling from your server, not the CDN. You want to be sure that you are solving the problem in the right place. Once you’re sure that the server isn’t sending the images you can work with the CDN.

[snakeninny]

@pfaffman

start with pulling from your server, not the CDN

Since https://cdn.iosre.com/user_avatar/iosre.com/baal998/135/2210_2.png is from CDN,
https://iosre.com/user_avatar/iosre.com/baal998/135/2210_2.png is supposed to be from my server, right?

[pfaffman (Jay Pfaffman)]

Yes and you see that it is getting sent by your server. So that’s the link to be trying to fix. After you make that stop sending the images you can clear the CDN