discourse-azure-blob-storage
I have created the Azure Blob Storage plugin as part of my Outreachy internship with Discourse. It allows storing uploads into the Azure Blob storage.
Azure account
- If you don’t have an Azure account yet, create one for free
- In your Microsoft Azure portal, go to Storage account and add a new account - make sure to select Blob storage under Account kind
- Go over to your newly created storage account and find Containers in the settings. Add a container into which uploads should be stored and set the public access level to Blob.
- Find Access keys in the storage account settings to fill in later when configuring Discourse.
Discourse configuration
- Install the plugin with the help of instructions here: Install Plugins in Discourse
- Go to Admin Settings → Plugins, fill in the storage account name, access key and container name (cdn is optional) and enable the plugin
- That’s it! Uploads should be stored into Azure Blob from now on.
Note: The plugin has yet to be thoroughly tested, I’d be happy if you give it a go and report any potential problems.
Bulk moving existing uploads
After activation of this plugin, all new uploads will start going to Azure. However, existing uploads will stay in the default /uploads/default/
directory on the local hard disk. To move them to Azure Blob Storage, follow these steps:
1. Download
Use your favorite file transfer tool to log into the VPS and download the entire directory tree of /var/discourse/shared/standalone/uploads/default
.
Note: Notice that we do not upload the files to Azure Blob Storage yet. If you do, and you don’t change the post URL’s quickly enough, then the background scrubber may run and many of your uploaded files may end up in the tombstoned
folder instead! If that happens, all you can do is to move those files back to the container.
2. Backup database
Go to Admin->Backups
and take a snapshot backup of the database.
3. Change upload url’s in the database
SSH into the VPS and do the following:
cd /var/discourse
sudo ./launcher enter app
su -c 'psql discourse' postgres
psql#
update uploads
set url = replace(url,'/uploads/default/', '//xxxxxxxx.blob.core.windows.net/yyyyyyyyy/')
where url like '/uploads/default/%';
psql#
update optimized_images
set url = replace(url,'/uploads/default/', '//xxxxxxxx.blob.core.windows.net/yyyyyyyyy/')
where url like '/uploads/default/%';
psql# \q
where xxxxxx
is the name of the Azure Storage Account and yyyyyy
is the name of the container.
4. Remap URL’s in posts
rake posts:remap["/uploads/default/","//xxxxxxxx.blob.core.windows.net/yyyyyyy/"]
where xxxxxx
is the name of the Azure Storage Account and yyyyyy
is the name of the container.
Beware, Azure Blob Storage uses HTTPS for access mostly. If the original URL is HTTP (i.e. http://
) then you have to manually fix these to use https://
.
5. Rebake all the posts
rake posts:rebake
6. Upload
Use your favorite tool to log into the Azure Blob Storage (e.g. Microsoft Azure Storage Explorer which is free, or CloudBerry Explorer for Azure Blob Storage, also free). Upload the entire tree into the target container (there should be two folders, original
and optimized
).