Azure Blob Storage Plugin

Got an error upgrading Discourse due to a faraday conflict:


I, [2020-05-22T05:46:54.081847 #1]  INFO -- : > cd /var/www/discourse && su discourse -c 'bundle exec rake db:migrate'
rake aborted!
Gem::ConflictError: Unable to activate faraday_middleware-0.11.0, because faraday-1.0.1 conflicts with faraday (>= 0.7.4, < 1.0)
/var/www/discourse/lib/plugin_gem.rb:20:in `load'
/var/www/discourse/lib/plugin/instance.rb:635:in `gem'
/var/www/discourse/plugins/discourse-azure-blob-storage/plugin.rb:10:in `activate!'
/var/www/discourse/lib/plugin/instance.rb:541:in `instance_eval'
/var/www/discourse/lib/plugin/instance.rb:541:in `activate!'
lib/discourse.rb:224:in `block in activate_plugins!'
lib/discourse.rb:221:in `each'
lib/discourse.rb:221:in `activate_plugins!'
/var/www/discourse/config/application.rb:297:in `block in <class:Application>'
/var/www/discourse/lib/plugin_initialization_guard.rb:5:in `plugin_initialization_guard'
/var/www/discourse/config/application.rb:296:in `<class:Application>'
/var/www/discourse/config/application.rb:65:in `<module:Discourse>'
/var/www/discourse/config/application.rb:64:in `<top (required)>'
/var/www/discourse/Rakefile:7:in `require'
/var/www/discourse/Rakefile:7:in `<top (required)>'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/exe/rake:27:in `<top (required)>'
/usr/local/bin/bundle:23:in `load'
/usr/local/bin/bundle:23:in `<main>'

Does discourse-azure-blob-storage use faraday?

Yes, indeed it does
https://github.com/discourse/discourse-azure-blob-storage/blob/67d13a285c17b6c4ca2d9154a8bed0a4f0fd5733/plugin.rb#L10

Great.

https://github.com/discourse/discourse-azure-blob-storage/pull/14

This fixes it.

3 Likes

Is this plugin still broken?

Anyone currently using this plugin for uploads to Azure Blob Storage? If yes, is it stable? Please share your experience.

I just installed it yesterday. I am still playing around with it but it seems generally functional.

Hello

I try to use your plugin… Some feature are OK. For example I can add my website logo and it go my azure blob storage.

But

When I upload picture on post, the image is not show. I have this :

Citation ![quete-viking-coin-master|362x500](upload://tBpGuefbyQNcVbDgJzOyv32uxFX.jpeg)

Do you have an idea ? Is your plugin is always working ?

Thanks for anwser

I just tried installing this plugin on v2.7.0 beta1 and got this error, which looks similar to the faraday issue above but with nokogiri:

I, [2021-01-13T22:25:07.678431 #1]  INFO -- : > cd /var/www/discourse && su discourse -c 'bundle     exec rake db:migrate'
rake aborted!
Gem::ConflictError: Unable to activate azure-storage-common-2.0.1, because nokogiri-1.11.1-x86_64-linux conflicts with nokogiri (~> 1.10.4)
/var/www/discourse/lib/plugin_gem.rb:22:in `load'
/var/www/discourse/lib/plugin/instance.rb:705:in `gem'
/var/www/discourse/plugins/discourse-azure-blob-storage/plugin.rb:12:in `activate!'
/var/www/discourse/lib/plugin/instance.rb:611:in `instance_eval'
/var/www/discourse/lib/plugin/instance.rb:611:in `activate!'
lib/discourse.rb:228:in `block in activate_plugins!'
lib/discourse.rb:225:in `each'
lib/discourse.rb:225:in `activate_plugins!'
/var/www/discourse/config/application.rb:305:in `block in <class:Application>'
/var/www/discourse/lib/plugin_initialization_guard.rb:5:in `plugin_initialization_guard'
/var/www/discourse/config/application.rb:304:in `<class:Application>'
/var/www/discourse/config/application.rb:71:in `<module:Discourse>'
/var/www/discourse/config/application.rb:70:in `<top (required)>'
/var/www/discourse/Rakefile:7:in `require'
/var/www/discourse/Rakefile:7:in `<top (required)>'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.3/exe/rake:27:in `<top (required)>'
/usr/local/bin/bundle:23:in `load'
/usr/local/bin/bundle:23:in `<main>'

EDIT: Works fine with version: d6121249d3856a40382637ed12db50fc43c1fb1d (2.6.0)

1 Like

also receiving this error after upgrade to v2.7.0beta1. CAn this plugin be updated to run on the newest release?

I, [2021-01-19T19:52:36.859512 #1]  INFO -- : > cd /var/www/discourse && su discourse -c 'bundle exec rake plugin:pull_compatible_all'
rake aborted!
Gem::ConflictError: Unable to activate azure-storage-common-2.0.1, because nokogiri-1.11.1-x86_64-linux conflicts with nokogiri (~> 1.10.4)
/var/www/discourse/lib/plugin_gem.rb:22:in `load'
/var/www/discourse/lib/plugin/instance.rb:705:in `gem'
/var/www/discourse/plugins/discourse-azure-blob-storage/plugin.rb:12:in `activate!'
/var/www/discourse/lib/plugin/instance.rb:611:in `instance_eval'
/var/www/discourse/lib/plugin/instance.rb:611:in `activate!'
lib/discourse.rb:228:in `block in activate_plugins!'
lib/discourse.rb:225:in `each'
lib/discourse.rb:225:in `activate_plugins!'
/var/www/discourse/config/application.rb:305:in `block in <class:Application>'
/var/www/discourse/lib/plugin_initialization_guard.rb:5:in `plugin_initialization_guard'
/var/www/discourse/config/application.rb:304:in `<class:Application>'
/var/www/discourse/config/application.rb:71:in `<module:Discourse>'
/var/www/discourse/config/application.rb:70:in `<top (required)>'
/var/www/discourse/Rakefile:7:in `require'
/var/www/discourse/Rakefile:7:in `<top (required)>'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.3/exe/rake:27:in `<top (required)>'
/usr/local/bin/bundle:23:in `load'
/usr/local/bin/bundle:23:in `<main>'

It is categorized on the #plugin:broken-plugin category, so it’s not working at the moment, and there appear to be no one interested in get it back in shape.

2 Likes

That is a sad story. :frowning:

Only got this error when I tried to rebuild today.

https://github.com/discourse/discourse-azure-blob-storage/pull/15

Essentially, the azure-storage-common gem needs to bump a version because one of its dependencies now conflict with Discourse.

For those who can’t wait (I understand your feeling because if Azure Blob suddenly doesn’t work, you site is not coming back up or it loses all its contents), pull from my fork instead:

https://github.com/schungx/discourse-azure-blob-storage

I don’t monitor meta as much as I used to, so I may not see these messages until months later. For those in need, raise an issue at my GitHub repo and I should respond.

2 Likes

Hopefully the PR will be merged in soon and the broken-plugin tag can be removed.

When this happens, it is usually errors during post-processing.

Check your sidekiq logs to see what’s blocking you from uploading to Azure.

Got a Unicorn error today when I rebuilt my app:

E, [2021-03-29T01:34:52.651460 #756] ERROR -- : app error: uninitialized constant IPAddr::PRIVATE_RANGES (NameError)
E, [2021-03-29T01:34:52.651585 #756] ERROR -- : /var/www/discourse/plugins/discourse-azure-blob-storage/gems/2.7.2/gems/azure-storage-common-2.0.2/lib/azure/storage/common/core/utility.rb:218:in `private?'
E, [2021-03-29T01:34:52.651600 #756] ERROR -- : /var/www/discourse/lib/middleware/request_tracker.rb:220:in `is_private_ip?'
E, [2021-03-29T01:34:52.651612 #756] ERROR -- : /var/www/discourse/lib/middleware/request_tracker.rb:233:in `rate_limit'
E, [2021-03-29T01:34:52.651640 #756] ERROR -- : /var/www/discourse/lib/middleware/request_tracker.rb:166:in `call'
E, [2021-03-29T01:34:52.651650 #756] ERROR -- : /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/railties-6.0.3.5/lib/rails/engine.rb:527:in `call'
E, [2021-03-29T01:34:52.651660 #756] ERROR -- : /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/railties-6.0.3.5/lib/rails/railtie.rb:190:in `public_send'
E, [2021-03-29T01:34:52.651670 #756] ERROR -- : /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/railties-6.0.3.5/lib/rails/railtie.rb:190:in `method_missing'
E, [2021-03-29T01:34:52.651681 #756] ERROR -- : /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rack-2.2.3/lib/rack/urlmap.rb:74:in `block in call'
E, [2021-03-29T01:34:52.651691 #756] ERROR -- : /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rack-2.2.3/lib/rack/urlmap.rb:58:in `each'
E, [2021-03-29T01:34:52.651753 #756] ERROR -- : /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rack-2.2.3/lib/rack/urlmap.rb:58:in `call'
E, [2021-03-29T01:34:52.651768 #756] ERROR -- : /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/unicorn-6.0.0/lib/unicorn/http_server.rb:634:in `process_client'
E, [2021-03-29T01:34:52.651779 #756] ERROR -- : /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/unicorn-6.0.0/lib/unicorn/http_server.rb:732:in `worker_loop'
E, [2021-03-29T01:34:52.651789 #756] ERROR -- : /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/unicorn-6.0.0/lib/unicorn/http_server.rb:547:in `spawn_missing_workers'
E, [2021-03-29T01:34:52.651802 #756] ERROR -- : /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/unicorn-6.0.0/lib/unicorn/http_server.rb:143:in `start'
E, [2021-03-29T01:34:52.651816 #756] ERROR -- : /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/unicorn-6.0.0/bin/unicorn:128:in `<top (required)>'
E, [2021-03-29T01:34:52.651830 #756] ERROR -- : /var/www/discourse/vendor/bundle/ruby/2.7.0/bin/unicorn:23:in `load'
E, [2021-03-29T01:34:52.651843 #756] ERROR -- : /var/www/discourse/vendor/bundle/ruby/2.7.0/bin/unicorn:23:in `<main>'

I’ll look at this more tomorrow but the update above doesn’t seem to fix it.

@schungx

The error seems to complain that you’re using a private IP range?

1 Like

Maybe I’m wrong but that suggests the constant PRIVATE_RANGES isn’t initialized. That constant should be defined here:

https://github.com/Azure/azure-storage-ruby/blob/master/common/lib/azure/core/utility.rb#L197

I’m not sure if this is a name conflict between the IPAddr class defined in the above utility.rb and that one defined in discourse itself but that would be my initial concern. I’m just not sure how I’d proceed in fixing that. My first thought would be submitting a PR that places the IPAddr class in discourse in a module. Any further thoughts?

https://github.com/Azure/azure-storage-ruby/blob/master/common/lib/azure/core/utility.rb#L221

It seems the private? call is on line 221, instead of line 218 as in the error message.

So perhaps it is the wrong version?

The last version where line 218 makes sense is 1.1.0 where you can see is exactly the error position:

https://github.com/Azure/azure-storage-ruby/blob/v1.1.0-common/common/lib/azure/storage/common/core/utility.rb#L218

OK, I’m really baffled here. Turns out that the plugin is still not correct.

Whenever it tries to store a file, it comes back with:

Blob can not be stored: missing dependency for Faraday::Adapter::NetHttpPersistent: cannot load such file -- net/http/persistent
Url: //chenhsongweb.blob.core.windows.net/support/original/2X/e/e5224c77df15646017fb5e4747576206c33904f6.jpeg

So obviously somewhere I’m missing a require 'net/http/persistent', but I tried adding it to everywhere and it still gives this error.

For example, I’m adding it to plugin.rb and lib/azure_blob_store.rb… still this error.

The error came from a call to cache_file:

/var/www/discourse/plugins/discourse-azure-blob-storage/lib/azure_blob_store.rb:27:in `rescue in store_file'
/var/www/discourse/plugins/discourse-azure-blob-storage/lib/azure_blob_store.rb:17:in `store_file'
/var/www/discourse/plugins/discourse-azure-blob-storage/lib/azure_blob_store.rb:9:in `store_upload'

Can some kind soul teach me how to get rid of the missing dependency error?

Ok. Could be needed in the rake task. I’ll add it and see…

EDIT: Nope. That’s not it.

It is failing at upload_creator

/var/www/discourse/plugins/discourse-azure-blob-storage/lib/azure_blob_store.rb:25:in `rescue in store_file'
/var/www/discourse/plugins/discourse-azure-blob-storage/lib/azure_blob_store.rb:15:in `store_file'
/var/www/discourse/plugins/discourse-azure-blob-storage/lib/azure_blob_store.rb:7:in `store_upload'
/var/www/discourse/lib/upload_creator.rb:160:in `block (2 levels) in create_for'
/var/www/discourse/lib/upload_creator.rb:159:in `open'
/var/www/discourse/lib/upload_creator.rb:159:in `block in create_for'

Now I’m totally baffled. How do I get rid of that missing dependency error?

EDIT 2:

For some strange reason, when I click on the little arrow at the end of the stack trace, it brings me to discourse/discourse-azure-blob-storage instead of my own fork. I am quite sure I’m using my own fork in app.config.

EDIT 3:

I tried moving back a version to azure-storage-common 2.0.1 which does not use net-http-persistent, but then it uses nokogiri version 1.10 which conflicts with something that requires version 1.11… Arrrrhhhhhgggghhhh…

Anybody knows how to resolve nokogiri version conflicts?

@pfaffman do you have any ideas? :pray:

EDIT 4:

@maja not sure if you’re still following up on this plugin. As the original author you may have an idea what’s going on.

I have been baffled at an error for some time.

Recently it seems Discourse starts requiring nokogiri version 1.11, which mandates updating to azure-storage-common version 2.0.1. However, that version uses net-http-persistent to make HTTP calls (previous versions didn’t).

Every time an azure call is made, it fails with a missing dependency: missing dependency for Faraday::Adapter::NetHttpPersistent: cannot load such file -- net/http/persistent.

The strange thing is, even if I put require 'net/http/persistent' each time before the call, it still fails with the exact same error. Even though require returns true meaning that the module is loaded.

It seems to start happening during the second time calling azure functions. For certain uploads, the first time will succeed, and the second_ time on will start giving missing dependency.

It almost feels like something is overwriting the require cache wiping out net/http/persistent.

EDIT 5:

Tried cloning azure-storage-blob and modifying to remove the net-http-persistent but can’t find a way to get Discourse to load from an alternative git repo… it seems always to be loading the official gem.

EDIT 6:

Update: There is a fix which works reasonably. However, it is still undergoing some further testing as it seems to conflict with other places.

The main error came from the fact that Faraday has its own dependency management system which conflicts with the Bundler.

The current solution basically just ignores the error from Faraday and loads the dependency statically. However, some kind soul may also point out a way to do it more elegantly…

1 Like

Alright, this plugin is so broken that there isn’t a point to continue with it.

I recommend everyone to uninstall this plugin and use Flexify.IO instead.

2 Likes