Azure Blob Storage Plugin

bsawicki · August 15, 2020, 11:52am

I just installed it yesterday. I am still playing around with it but it seems generally functional.

Ludovik_Dopierala · September 9, 2020, 11:08pm

Hello

I try to use your plugin… Some feature are OK. For example I can add my website logo and it go my azure blob storage.

But

When I upload picture on post, the image is not show. I have this :

Citation ![quete-viking-coin-master|362x500](upload://tBpGuefbyQNcVbDgJzOyv32uxFX.jpeg)

Do you have an idea ? Is your plugin is always working ?

Thanks for anwser

Karl_Romanowski · January 14, 2021, 6:34pm

I just tried installing this plugin on v2.7.0 beta1 and got this error, which looks similar to the faraday issue above but with nokogiri:

I, [2021-01-13T22:25:07.678431 #1]  INFO -- : > cd /var/www/discourse && su discourse -c 'bundle     exec rake db:migrate'
rake aborted!
Gem::ConflictError: Unable to activate azure-storage-common-2.0.1, because nokogiri-1.11.1-x86_64-linux conflicts with nokogiri (~> 1.10.4)
/var/www/discourse/lib/plugin_gem.rb:22:in `load'
/var/www/discourse/lib/plugin/instance.rb:705:in `gem'
/var/www/discourse/plugins/discourse-azure-blob-storage/plugin.rb:12:in `activate!'
/var/www/discourse/lib/plugin/instance.rb:611:in `instance_eval'
/var/www/discourse/lib/plugin/instance.rb:611:in `activate!'
lib/discourse.rb:228:in `block in activate_plugins!'
lib/discourse.rb:225:in `each'
lib/discourse.rb:225:in `activate_plugins!'
/var/www/discourse/config/application.rb:305:in `block in <class:Application>'
/var/www/discourse/lib/plugin_initialization_guard.rb:5:in `plugin_initialization_guard'
/var/www/discourse/config/application.rb:304:in `<class:Application>'
/var/www/discourse/config/application.rb:71:in `<module:Discourse>'
/var/www/discourse/config/application.rb:70:in `<top (required)>'
/var/www/discourse/Rakefile:7:in `require'
/var/www/discourse/Rakefile:7:in `<top (required)>'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.3/exe/rake:27:in `<top (required)>'
/usr/local/bin/bundle:23:in `load'
/usr/local/bin/bundle:23:in `<main>'

EDIT: Works fine with version: d6121249d3856a40382637ed12db50fc43c1fb1d (2.6.0)

cgreco · January 20, 2021, 1:27am

also receiving this error after upgrade to v2.7.0beta1. CAn this plugin be updated to run on the newest release?

I, [2021-01-19T19:52:36.859512 #1]  INFO -- : > cd /var/www/discourse && su discourse -c 'bundle exec rake plugin:pull_compatible_all'
rake aborted!
Gem::ConflictError: Unable to activate azure-storage-common-2.0.1, because nokogiri-1.11.1-x86_64-linux conflicts with nokogiri (~> 1.10.4)
/var/www/discourse/lib/plugin_gem.rb:22:in `load'
/var/www/discourse/lib/plugin/instance.rb:705:in `gem'
/var/www/discourse/plugins/discourse-azure-blob-storage/plugin.rb:12:in `activate!'
/var/www/discourse/lib/plugin/instance.rb:611:in `instance_eval'
/var/www/discourse/lib/plugin/instance.rb:611:in `activate!'
lib/discourse.rb:228:in `block in activate_plugins!'
lib/discourse.rb:225:in `each'
lib/discourse.rb:225:in `activate_plugins!'
/var/www/discourse/config/application.rb:305:in `block in <class:Application>'
/var/www/discourse/lib/plugin_initialization_guard.rb:5:in `plugin_initialization_guard'
/var/www/discourse/config/application.rb:304:in `<class:Application>'
/var/www/discourse/config/application.rb:71:in `<module:Discourse>'
/var/www/discourse/config/application.rb:70:in `<top (required)>'
/var/www/discourse/Rakefile:7:in `require'
/var/www/discourse/Rakefile:7:in `<top (required)>'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.3/exe/rake:27:in `<top (required)>'
/usr/local/bin/bundle:23:in `load'
/usr/local/bin/bundle:23:in `<main>'

Falco · January 20, 2021, 2:14am

It is categorized on the #plugin:broken-plugin category, so it’s not working at the moment, and there appear to be no one interested in get it back in shape.

andrueastman · January 22, 2021, 10:30am

That is a sad story.

schungx · February 20, 2021, 3:17pm

Only got this error when I tried to rebuild today.

https://github.com/discourse/discourse-azure-blob-storage/pull/15

Essentially, the azure-storage-common gem needs to bump a version because one of its dependencies now conflict with Discourse.

For those who can’t wait (I understand your feeling because if Azure Blob suddenly doesn’t work, you site is not coming back up or it loses all its contents), pull from my fork instead:

https://github.com/schungx/discourse-azure-blob-storage

I don’t monitor meta as much as I used to, so I may not see these messages until months later. For those in need, raise an issue at my GitHub repo and I should respond.

schungx · February 20, 2021, 3:25pm

Hopefully the PR will be merged in soon and the broken-plugin tag can be removed.

schungx · February 20, 2021, 3:27pm

When this happens, it is usually errors during post-processing.

Check your sidekiq logs to see what’s blocking you from uploading to Azure.

bsawicki · March 29, 2021, 4:58am

Got a Unicorn error today when I rebuilt my app:

E, [2021-03-29T01:34:52.651460 #756] ERROR -- : app error: uninitialized constant IPAddr::PRIVATE_RANGES (NameError)
E, [2021-03-29T01:34:52.651585 #756] ERROR -- : /var/www/discourse/plugins/discourse-azure-blob-storage/gems/2.7.2/gems/azure-storage-common-2.0.2/lib/azure/storage/common/core/utility.rb:218:in `private?'
E, [2021-03-29T01:34:52.651600 #756] ERROR -- : /var/www/discourse/lib/middleware/request_tracker.rb:220:in `is_private_ip?'
E, [2021-03-29T01:34:52.651612 #756] ERROR -- : /var/www/discourse/lib/middleware/request_tracker.rb:233:in `rate_limit'
E, [2021-03-29T01:34:52.651640 #756] ERROR -- : /var/www/discourse/lib/middleware/request_tracker.rb:166:in `call'
E, [2021-03-29T01:34:52.651650 #756] ERROR -- : /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/railties-6.0.3.5/lib/rails/engine.rb:527:in `call'
E, [2021-03-29T01:34:52.651660 #756] ERROR -- : /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/railties-6.0.3.5/lib/rails/railtie.rb:190:in `public_send'
E, [2021-03-29T01:34:52.651670 #756] ERROR -- : /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/railties-6.0.3.5/lib/rails/railtie.rb:190:in `method_missing'
E, [2021-03-29T01:34:52.651681 #756] ERROR -- : /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rack-2.2.3/lib/rack/urlmap.rb:74:in `block in call'
E, [2021-03-29T01:34:52.651691 #756] ERROR -- : /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rack-2.2.3/lib/rack/urlmap.rb:58:in `each'
E, [2021-03-29T01:34:52.651753 #756] ERROR -- : /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rack-2.2.3/lib/rack/urlmap.rb:58:in `call'
E, [2021-03-29T01:34:52.651768 #756] ERROR -- : /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/unicorn-6.0.0/lib/unicorn/http_server.rb:634:in `process_client'
E, [2021-03-29T01:34:52.651779 #756] ERROR -- : /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/unicorn-6.0.0/lib/unicorn/http_server.rb:732:in `worker_loop'
E, [2021-03-29T01:34:52.651789 #756] ERROR -- : /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/unicorn-6.0.0/lib/unicorn/http_server.rb:547:in `spawn_missing_workers'
E, [2021-03-29T01:34:52.651802 #756] ERROR -- : /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/unicorn-6.0.0/lib/unicorn/http_server.rb:143:in `start'
E, [2021-03-29T01:34:52.651816 #756] ERROR -- : /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/unicorn-6.0.0/bin/unicorn:128:in `<top (required)>'
E, [2021-03-29T01:34:52.651830 #756] ERROR -- : /var/www/discourse/vendor/bundle/ruby/2.7.0/bin/unicorn:23:in `load'
E, [2021-03-29T01:34:52.651843 #756] ERROR -- : /var/www/discourse/vendor/bundle/ruby/2.7.0/bin/unicorn:23:in `<main>'

I’ll look at this more tomorrow but the update above doesn’t seem to fix it.

@schungx

schungx · April 2, 2021, 7:29am

The error seems to complain that you’re using a private IP range?

bsawicki · April 2, 2021, 8:41am

Maybe I’m wrong but that suggests the constant PRIVATE_RANGES isn’t initialized. That constant should be defined here:

github.com

Azure/azure-storage-ruby/blob/master/common/lib/azure/core/utility.rb#L197


      
              white: 37,
            }.each do |key, value|
              define_method key do
                "\e[#{value}m" + self + "\e[0m"
              end
            end
          end
          
          # Code validate private/public IP acceptable ranges.
          class IPAddr
            PRIVATE_RANGES = [
              IPAddr.new('10.0.0.0/8'),
              IPAddr.new('172.16.0.0/12'),
              IPAddr.new('192.168.0.0/16')
            ]
          
            def private?
              return false unless self.ipv4?
              PRIVATE_RANGES.each do |ipr|
                return true if ipr.include?(self)
              end

I’m not sure if this is a name conflict between the IPAddr class defined in the above utility.rb and that one defined in discourse itself but that would be my initial concern. I’m just not sure how I’d proceed in fixing that. My first thought would be submitting a PR that places the IPAddr class in discourse in a module. Any further thoughts?

schungx · April 4, 2021, 9:02am

https://github.com/Azure/azure-storage-ruby/blob/master/common/lib/azure/core/utility.rb#L221

It seems the private? call is on line 221, instead of line 218 as in the error message.

So perhaps it is the wrong version?

The last version where line 218 makes sense is 1.1.0 where you can see is exactly the error position:

https://github.com/Azure/azure-storage-ruby/blob/v1.1.0-common/common/lib/azure/storage/common/core/utility.rb#L218

schungx · April 26, 2021, 6:32am

OK, I’m really baffled here. Turns out that the plugin is still not correct.

Whenever it tries to store a file, it comes back with:

Blob can not be stored: missing dependency for Faraday::Adapter::NetHttpPersistent: cannot load such file -- net/http/persistent
Url: //chenhsongweb.blob.core.windows.net/support/original/2X/e/e5224c77df15646017fb5e4747576206c33904f6.jpeg

So obviously somewhere I’m missing a require 'net/http/persistent', but I tried adding it to everywhere and it still gives this error.

For example, I’m adding it to plugin.rb and lib/azure_blob_store.rb… still this error.

The error came from a call to cache_file:

/var/www/discourse/plugins/discourse-azure-blob-storage/lib/azure_blob_store.rb:27:in `rescue in store_file'
/var/www/discourse/plugins/discourse-azure-blob-storage/lib/azure_blob_store.rb:17:in `store_file'
/var/www/discourse/plugins/discourse-azure-blob-storage/lib/azure_blob_store.rb:9:in `store_upload'

Can some kind soul teach me how to get rid of the missing dependency error?

schungx · April 26, 2021, 8:33am

Ok. Could be needed in the rake task. I’ll add it and see…

EDIT: Nope. That’s not it.

It is failing at upload_creator…

/var/www/discourse/plugins/discourse-azure-blob-storage/lib/azure_blob_store.rb:25:in `rescue in store_file'
/var/www/discourse/plugins/discourse-azure-blob-storage/lib/azure_blob_store.rb:15:in `store_file'
/var/www/discourse/plugins/discourse-azure-blob-storage/lib/azure_blob_store.rb:7:in `store_upload'
/var/www/discourse/lib/upload_creator.rb:160:in `block (2 levels) in create_for'
/var/www/discourse/lib/upload_creator.rb:159:in `open'
/var/www/discourse/lib/upload_creator.rb:159:in `block in create_for'

Now I’m totally baffled. How do I get rid of that missing dependency error?

EDIT 2:

For some strange reason, when I click on the little arrow at the end of the stack trace, it brings me to discourse/discourse-azure-blob-storage instead of my own fork. I am quite sure I’m using my own fork in app.config.

EDIT 3:

I tried moving back a version to azure-storage-common 2.0.1 which does not use net-http-persistent, but then it uses nokogiri version 1.10 which conflicts with something that requires version 1.11… Arrrrhhhhhgggghhhh…

Anybody knows how to resolve nokogiri version conflicts?

@pfaffman do you have any ideas?

EDIT 4:

@maja not sure if you’re still following up on this plugin. As the original author you may have an idea what’s going on.

I have been baffled at an error for some time.

Recently it seems Discourse starts requiring nokogiri version 1.11, which mandates updating to azure-storage-common version 2.0.1. However, that version uses net-http-persistent to make HTTP calls (previous versions didn’t).

Every time an azure call is made, it fails with a missing dependency: missing dependency for Faraday::Adapter::NetHttpPersistent: cannot load such file -- net/http/persistent.

The strange thing is, even if I put require 'net/http/persistent' each time before the call, it still fails with the exact same error. Even though require returns true meaning that the module is loaded.

It seems to start happening during the second time calling azure functions. For certain uploads, the first time will succeed, and the second_ time on will start giving missing dependency.

It almost feels like something is overwriting the require cache wiping out net/http/persistent.

EDIT 5:

Tried cloning azure-storage-blob and modifying to remove the net-http-persistent but can’t find a way to get Discourse to load from an alternative git repo… it seems always to be loading the official gem.

EDIT 6:

Update: There is a fix which works reasonably. However, it is still undergoing some further testing as it seems to conflict with other places.

The main error came from the fact that Faraday has its own dependency management system which conflicts with the Bundler.

The current solution basically just ignores the error from Faraday and loads the dependency statically. However, some kind soul may also point out a way to do it more elegantly…

schungx · August 19, 2021, 2:44am

Alright, this plugin is so broken that there isn’t a point to continue with it.

I recommend everyone to uninstall this plugin and use Flexify.IO instead.

Topic		Replies	Views
Plugin to store uploads into Azure blob storage Feature	14	1915	February 23, 2018
Huge amount of storage transactions Installation	13	1581	June 24, 2020
Configure an S3 compatible object storage provider for uploads Self-Hosting cdn , configuring , how-to , reference	230	38492	October 24, 2024
Secure Uploads Announcements secure-uploads	44	15835	October 25, 2024
Discourse OAuth2 Basic Plugin official , oauth2 , auth-plugins	27	60826	November 9, 2024

Azure Blob Storage Plugin

Related topics