# Azure Blob Storage Plugin

**URL:** https://meta.discourse.org/t/azure-blob-storage-plugin/79392
**Category:** Plugin
**Tags:** broken
**Created:** [30. Januar 2018 um 21:28 UTC](https://meta.discourse.org/t/azure-blob-storage-plugin/79392 "2018-01-30T21:28:29Z")
**Posts on this page:** 20
**Page:** 3

<div class="post-metadata">

### Author: ![schungx](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/schungx/32/70989_2.png) [@schungx](https://meta.discourse.org/u/schungx)
#### Post date: [5. Juli 2018 um 14:47 UTC](https://meta.discourse.org/t/azure-blob-storage-plugin/79392/46 "2018-07-05T14:47:32Z")

</div>

After updating to the latest, it seems to be working now. The correct `post_uploads` record seems to be created.

🎉

---

<div class="post-metadata">

### Author: ![schungx](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/schungx/32/70989_2.png) [@schungx](https://meta.discourse.org/u/schungx)
#### Post date: [17. Juli 2018 um 03:10 UTC](https://meta.discourse.org/t/azure-blob-storage-plugin/79392/47 "2018-07-17T03:10:47Z")

</div>

Now that Azure Blob seems to be working fine, any chance to upload backups to it?

Am I too greedy?

---

<div class="post-metadata">

### Author: ![sam](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/sam/32/102149_2.png) [@sam](https://meta.discourse.org/u/sam)
#### Post date: [17. Juli 2018 um 03:56 UTC](https://meta.discourse.org/t/azure-blob-storage-plugin/79392/48 "2018-07-17T03:56:26Z")

</div>

Our first priority is sorting out asset hosting like we have for s3, we will eventually get to backups

---

<div class="post-metadata">

### Author: ![schungx](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/schungx/32/70989_2.png) [@schungx](https://meta.discourse.org/u/schungx)
#### Post date: [28. Juli 2018 um 04:06 UTC](https://meta.discourse.org/t/azure-blob-storage-plugin/79392/49 "2018-07-28T04:06:27Z")

</div>

The update to S3 seems to break Azure Blob. I’m getting `Oops` after the recent upgrade.

```plaintext
ActionView::Template::Error (undefined method `split' for nil:NilClass) /var/www/discourse/app/models/site_setting.rb:144:in `s3_base_url'

```

This commit:

[https://github.com/discourse/discourse/commit/875008522de225ddcd7b14638b9f054a0acad7b5#diff-b2f320bf18d0c321021eb4a8f6704169](https://github.com/discourse/discourse/commit/875008522de225ddcd7b14638b9f054a0acad7b5#diff-b2f320bf18d0c321021eb4a8f6704169)

changes `s3_base_url`:

> <https://github.com/discourse/discourse/blob/main/app/models/site_setting.rb#L143-L146>

`s3_upload_bucket` appears to be null when using Azure Blob.

I believe the Azure Blob plugin needs to override `s3_base_url` or `s3_upload_bucket`.

---

<div class="post-metadata">

### Author: ![schungx](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/schungx/32/70989_2.png) [@schungx](https://meta.discourse.org/u/schungx)
#### Post date: [28. Juli 2018 um 04:19 UTC](https://meta.discourse.org/t/azure-blob-storage-plugin/79392/50 "2018-07-28T04:19:34Z")

</div>

But it is strange because according to this:

> <https://github.com/discourse/discourse-azure-blob-storage/blob/main/plugin.rb#L42-L45>

`s3_base_url` should return the azure URL instead of falling into the base implementation – which it does by getting the error for `split`.

---

<div class="post-metadata">

### Author: ![maja](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/maja/32/129533_2.png) [@maja](https://meta.discourse.org/u/maja)
#### Post date: [28. Juli 2018 um 07:15 UTC](https://meta.discourse.org/t/azure-blob-storage-plugin/79392/51 "2018-07-28T07:15:13Z")

</div>

Yes, with recent changes this issue should be fixed. Can you make sure you’re using the plugin from this repository [GitHub - discourse/discourse-azure-blob-storage: A Discourse plugin for storing images and files in azure blob storage · GitHub](https://github.com/discourse/discourse-azure-blob-storage) and the last version of it?

---

<div class="post-metadata">

### Author: ![schungx](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/schungx/32/70989_2.png) [@schungx](https://meta.discourse.org/u/schungx)
#### Post date: [28. Juli 2018 um 08:29 UTC](https://meta.discourse.org/t/azure-blob-storage-plugin/79392/52 "2018-07-28T08:29:36Z")

</div>

OK, I find that I was not using the official one.

I’ll switch to the official one and try again.

EDIT: The official plugin works fine! Thanks!

---

<div class="post-metadata">

### Author: ![schungx](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/schungx/32/70989_2.png) [@schungx](https://meta.discourse.org/u/schungx)
#### Post date: [30. Juli 2018 um 07:01 UTC](https://meta.discourse.org/t/azure-blob-storage-plugin/79392/53 "2018-07-30T07:01:28Z")

</div>

Just realize that this plugin keeps generating some strange error messages in the logs, but doesn’t seem to be causing any ill effects:

```plaintext
Job exception: BlobNotFound (404): The specified blob does not exist.
RequestId:deb4165d-401e-0133-49b4-27e5ab000000
Time:2018-07-30T03:19:56.6174455Z

/var/www/discourse/plugins/discourse-azure-blob-storage/gems/2.5.1/gems/azure-core-0.1.13/lib/azure/core/http/http_request.rb:153:in `call' 
/var/www/discourse/plugins/discourse-azure-blob-storage/gems/2.5.1/gems/azure-core-0.1.13/lib/azure/core/http/signer_filter.rb:28:in `call' 
/var/www/discourse/plugins/discourse-azure-blob-storage/gems/2.5.1/gems/azure-core-0.1.13/lib/azure/core/http/signer_filter.rb:28:in `call' 
/var/www/discourse/plugins/discourse-azure-blob-storage/gems/2.5.1/gems/azure-core-0.1.13/lib/azure/core/http/http_request.rb:110:in `block in with_filter' 
/var/www/discourse/plugins/discourse-azure-blob-storage/gems/2.5.1/gems/azure-core-0.1.13/lib/azure/core/service.rb:36:in `call' 
/var/www/discourse/plugins/discourse-azure-blob-storage/gems/2.5.1/gems/azure-core-0.1.13/lib/azure/core/filtered_service.rb:34:in `call' 
/var/www/discourse/plugins/discourse-azure-blob-storage/gems/2.5.1/gems/azure-core-0.1.13/lib/azure/core/signed_service.rb:41:in `call' 
/var/www/discourse/plugins/discourse-azure-blob-storage/gems/2.5.1/gems/azure-storage-common-1.0.1/lib/azure/storage/common/service/storage_service.rb:60:in `call' /var/www/discourse/plugins/discourse-azure-blob-storage/gems/2.5.1/gems/azure-storage-blob-1.0.1/lib/azure/storage/blob/blob_service.rb:176:in `call' 
/var/www/discourse/plugins/discourse-azure-blob-storage/gems/2.5.1/gems/azure-storage-blob-1.0.1/lib/azure/storage/blob/blob.rb:761:in `copy_blob_from_uri' 
/var/www/discourse/plugins/discourse-azure-blob-storage/gems/2.5.1/gems/azure-storage-blob-1.0.1/lib/azure/storage/blob/blob.rb:831:in `copy_blob' 
/var/www/discourse/plugins/discourse-azure-blob-storage/lib/azure_blob_store.rb:30:in `remove_file' 
/var/www/discourse/lib/file_store/base_store.rb:20:in `remove_upload' 
/var/www/discourse/app/models/upload.rb:53:in `block in destroy' 
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/activerecord-5.2.0/lib/active_record/connection_adapters/abstract/database_statements.rb:254:in `block in transaction' 
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/activerecord-5.2.0/lib/active_record/connection_adapters/abstract/transaction.rb:230:in `block in within_new_transaction' /usr/local/lib/ruby/2.5.0/monitor.rb:226:in `mon_synchronize' 
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/activerecord-5.2.0/lib/active_record/connection_adapters/abstract/transaction.rb:227:in `within_new_transaction' 
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/activerecord-5.2.0/lib/active_record/connection_adapters/abstract/database_statements.rb:254:in `transaction' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/activerecord-5.2.0/lib/active_record/transactions.rb:212:in `transaction' 
/var/www/discourse/app/models/upload.rb:52:in `destroy' 
/var/www/discourse/app/jobs/scheduled/clean_up_uploads.rb:68:in `block in execute' 
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/activerecord-5.2.0/lib/active_record/relation/batches.rb:70:in `block (2 levels) in find_each' 
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/activerecord-5.2.0/lib/active_record/relation/batches.rb:70:in `each' 
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/activerecord-5.2.0/lib/active_record/relation/batches.rb:70:in `block in find_each' 
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/activerecord-5.2.0/lib/active_record/relation/batches.rb:136:in `block in find_in_batches' 
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/activerecord-5.2.0/lib/active_record/relation/batches.rb:238:in `block in in_batches' 
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/activerecord-5.2.0/lib/active_record/relation/batches.rb:222:in `loop' 
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/activerecord-5.2.0/lib/active_record/relation/batches.rb:222:in `in_batches' 
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/activerecord-5.2.0/lib/active_record/relation/batches.rb:135:in `find_in_batches' 
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/activerecord-5.2.0/lib/active_record/relation/batches.rb:69:in `find_each' 
/var/www/discourse/app/jobs/scheduled/clean_up_uploads.rb:63:in `execute' 
/var/www/discourse/app/jobs/base.rb:134:in `block (2 levels) in perform' 
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/rails_multisite-2.0.5/lib/rails_multisite/connection_management.rb:63:in `with_connection' 
/var/www/discourse/app/jobs/base.rb:129:in `block in perform' 
/var/www/discourse/app/jobs/base.rb:125:in `each' 
/var/www/discourse/app/jobs/base.rb:125:in `perform' 
/var/www/discourse/app/jobs/base.rb:180:in `perform' 
/var/www/discourse/lib/scheduler/manager.rb:91:in `process_queue' 
/var/www/discourse/lib/scheduler/manager.rb:37:in `block in initialize'

```

It seems to be trying to tombstone files that do not exist in Azure Blob Storage.

I can’t seem to get a handle on what those files are yet… There are a bunch of posts that come in via `email in` that are put in queue and that I rejected. Perhaps the attached images inside those images are not yet copied into Azure? Just a thought…

EDIT: Tracing thru the code a bit, it also can be a single upload that exists in `uploads` but no on the Azure Blob Storage. Therefore the task can be constantly trying to move it to `tombstone`, but constantly failing at the `remove_file` call, without actually deleting the `uploads` record.

So in other words, it can be caused by a single missing upload file over and over again… but how can I find out WHICH upload it is?

---

<div class="post-metadata">

### Author: ![schungx](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/schungx/32/70989_2.png) [@schungx](https://meta.discourse.org/u/schungx)
#### Post date: [1. August 2018 um 03:27 UTC](https://meta.discourse.org/t/azure-blob-storage-plugin/79392/54 "2018-08-01T03:27:42Z")

</div>

I am wondering, is there a way to trap the error and output more meaningful error messages?

For example:

> <https://github.com/discourse/discourse-azure-blob-storage/blob/main/lib/azure_blob_store.rb#L30-L36>

These calls on `blob_service` will throw if the blob doesn’t exist. However, the exception only returns a 403 (BlobNotFound) error, without any indication on what the non-existing path is.

Would it be a good idea to put try/catch around all `blob_service` calls so at least we get the context of the call in the error messages?

Alternatively, wouldn’t it be a _better_ idea to _skip_ the tombstoning if the blob no longer exists instead of failing?

---

<div class="post-metadata">

### Author: ![maja](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/maja/32/129533_2.png) [@maja](https://meta.discourse.org/u/maja)
#### Post date: [1. August 2018 um 08:30 UTC](https://meta.discourse.org/t/azure-blob-storage-plugin/79392/55 "2018-08-01T08:30:38Z")

</div>

I’m adding improvements to the plugin this week, I’ll fix the behavior for this case as well, thanks for reporting!

---

<div class="post-metadata">

### Author: ![Heidi\_V](https://avatars.discourse-cdn.com/v4/letter/h/e9c0ed/32.png) [@Heidi\_V](https://meta.discourse.org/u/Heidi_V)
#### Post date: [14. August 2018 um 21:22 UTC](https://meta.discourse.org/t/azure-blob-storage-plugin/79392/56 "2018-08-14T21:22:05Z")

</div>

Hey, I wanted to update one of the instructions:  
in

Azure account

1. If you don’t have an Azure account yet, [create one for free](https://azure.microsoft.com/en-us/free/)
2. In your Microsoft Azure portal, go to **Storage account** and add a new account - make sure to select **Blob storage** under Account kind
3. Go over to your newly created storage account and scroll down to find **BLOB SERVICE**. Click on **Blobs** , and on the upper left corner, click on **+ Container**. Add a container into which uploads should be stored and set the public access level to **Blob**.
4. Find **Access keys** in the storage account settings to fill in later when configuring Discourse.

---

<div class="post-metadata">

### Author: ![ckshen](https://avatars.discourse-cdn.com/v4/letter/c/ad7895/32.png) [@ckshen](https://meta.discourse.org/u/ckshen)
#### Post date: [31. August 2018 um 05:08 UTC](https://meta.discourse.org/t/azure-blob-storage-plugin/79392/57 "2018-08-31T05:08:35Z")

</div>

> [@maja](#):
>
> Note: Notice that we **do not** upload the files to Azure Blob Storage yet. If you do, and you don’t change the post URL’s quickly enough, then the background scrubber may run and many of your uploaded files may end up in the `tombstoned` folder instead! If that happens, all you can do is to move those files back to the container.

What happens when you rebake without uploading the files yet? Will the posts rebake any differently compared to when the files are there?

---

<div class="post-metadata">

### Author: ![schungx](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/schungx/32/70989_2.png) [@schungx](https://meta.discourse.org/u/schungx)
#### Post date: [11. Oktober 2018 um 00:28 UTC](https://meta.discourse.org/t/azure-blob-storage-plugin/79392/58 "2018-10-11T00:28:30Z")

</div>

Recently I notice that all uploads are failing with the following error:

```plaintext
undefined method `is_image?’ for FileHelper:Class

```

All uploads are failing at this point. Any ideas?

This started happening after a recent rebuild so must be certain recent changes in core.

---

<div class="post-metadata">

### Author: ![maja](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/maja/32/129533_2.png) [@maja](https://meta.discourse.org/u/maja)
#### Post date: [11. Oktober 2018 um 08:13 UTC](https://meta.discourse.org/t/azure-blob-storage-plugin/79392/59 "2018-10-11T08:13:12Z")

</div>

I have updated the plugin to apply the change from the core. Can you please try to rebuild again?

---

<div class="post-metadata">

### Author: ![schungx](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/schungx/32/70989_2.png) [@schungx](https://meta.discourse.org/u/schungx)
#### Post date: [12. Oktober 2018 um 04:03 UTC](https://meta.discourse.org/t/azure-blob-storage-plugin/79392/60 "2018-10-12T04:03:48Z")

</div>

Seems fixed now. Thanks!

---

<div class="post-metadata">

### Author: ![joshyates1980](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/joshyates1980/32/109818_2.png) [@joshyates1980](https://meta.discourse.org/u/joshyates1980)
#### Post date: [19. Oktober 2018 um 01:47 UTC](https://meta.discourse.org/t/azure-blob-storage-plugin/79392/61 "2018-10-19T01:47:49Z")

</div>

great plugin, thanks @maja

fresh install of my hobby project with your plugin:

[https://philosophia.chat/](https://philosophia.chat/)

---

<div class="post-metadata">

### Author: ![schungx](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/schungx/32/70989_2.png) [@schungx](https://meta.discourse.org/u/schungx)
#### Post date: [5. März 2019 um 06:55 UTC](https://meta.discourse.org/t/azure-blob-storage-plugin/79392/62 "2019-03-05T06:55:30Z")

</div>

There is an annoying issue I would like to suggest a change for.

As I review my error logs, I keep getting a whole bunch of

```plaintext
Job exception: BlobNotFound (404): The specified blob does not exist.
RequestId:891fdac2-801e-0080-20b2-d25979000000
Time:2019-03-04T17:47:07.8968878Z

```

It is usually from here:

```plaintext
/var/www/discourse/plugins/discourse-azure-blob-storage/lib/azure_blob_store.rb:30:in `remove_file'
/var/www/discourse/lib/file_store/base_store.rb:20:in `remove_upload'
/var/www/discourse/app/models/upload.rb:105:in `block in destroy'

```

I believe that there is a blob that doesn’t exist (a mismatch between database records and the actual blob store).

However, there is no way to know _which_ blob is causing the problem. The `RequestId` looks like the unique id for Sidekiq.

I suggest trapping errors in Azure blob operations then appending the blob path to the error messages to facilitate trouble-shooting.

---

<div class="post-metadata">

### Author: ![sam](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/sam/32/102149_2.png) [@sam](https://meta.discourse.org/u/sam)
#### Post date: [5. März 2019 um 23:24 UTC](https://meta.discourse.org/t/azure-blob-storage-plugin/79392/63 "2019-03-05T23:24:43Z")

</div>

Sure sounds like a good change, care to try a PR. ATM we have no customers using the Azure storage plugin so it is hard to justify shifting resources to it.

---

<div class="post-metadata">

### Author: ![schungx](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/schungx/32/70989_2.png) [@schungx](https://meta.discourse.org/u/schungx)
#### Post date: [6. März 2019 um 10:25 UTC](https://meta.discourse.org/t/azure-blob-storage-plugin/79392/64 "2019-03-06T10:25:35Z")

</div>

OK. Let me give it a shot.

---

<div class="post-metadata">

### Author: ![schungx](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/schungx/32/70989_2.png) [@schungx](https://meta.discourse.org/u/schungx)
#### Post date: [7. März 2019 um 12:08 UTC](https://meta.discourse.org/t/azure-blob-storage-plugin/79392/65 "2019-03-07T12:08:08Z")

</div>

Did one. Hopefully I get Ruby right…

[https://github.com/discourse/discourse-azure-blob-storage/pull/10](https://github.com/discourse/discourse-azure-blob-storage/pull/10)

[Vorherige Seite](https://meta.discourse.org/t/azure-blob-storage-plugin/79392.md?page=2)

[Nächste Seite](https://meta.discourse.org/t/azure-blob-storage-plugin/79392.md?page=4)
