Plugin to store uploads into Azure blob storage


(Stephen Chung) #1

I’m wondering if there is a plugin to save uploaded files into Azure blob storage. Probably not…

If so, any pointers as to what plugin template I can study to write something like this?

Amazon S3 support is baked into the core instead of being a plugin, so I’m not sure if it will be easy to do in a plugin.


Azure storage for upload files
Azure Blob Storage Plugin
(Sam Saffron) #4

@maja can you update the topic here with your progress?


(Maja) #6

I have created a repository with what I’ve done in previous week: GitHub - majakomel/discourse-azure-blob-storage. Basic functionality is there, apart from purge_tombstone, also there seems to be an issue with avatars that I need to fix.


(Maja) #7

Plugin now works with Azure CDN as well. It would be great to get some feedback, if someone feels like testing it. Plugin is activated as soon as it’s installed (enable setting doesn’t do anything yet). It can be configured under Admin -> Settings -> Plugins, account name, access key and container name are required.


(Sam Saffron) #8

@schungx / @SimonWu I was wondering if you would have a bit of time to test out the new plugin by @maja.

We really want to get azure blob storage working well and this is the first time we have a plugin :cake:

@maja we should pull out a dedicated topic for this in the #plugin category that documents how to use, concerns etc.


(Simon Wu) #9

Sure, I will give a try.


(Stephen Chung) #10

Love to. I’ll install it when I get to sit at my desk.


(Simon Wu) #11

It throws error when I try to upload a picture in post.

NameError (undefined local variable or method `locked_by_id' for #<Post:0x00007f7684893e40>
Did you mean?  locked?)
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activemodel-5.1.4/lib/active_model/attribute_methods.rb:432:in `method_missing'

Stacktrace

/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activemodel-5.1.4/lib/active_model/attribute_methods.rb:432:in `method_missing'
/var/www/discourse/app/models/post.rb:739:in `locked?'
/var/www/discourse/app/serializers/post_serializer.rb:364:in `include_locked?'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/active_model_serializers-0.8.3/lib/active_model/serializer.rb:374:in `include?'
(eval):64:in `_fast_attributes'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/active_model_serializers-0.8.3/lib/active_model/serializer.rb:467:in `rescue in attributes'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/active_model_serializers-0.8.3/lib/active_model/serializer.rb:455:in `attributes'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/active_model_serializers-0.8.3/lib/active_model/serializer.rb:479:in `_serializable_hash'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/active_model_serializers-0.8.3/lib/active_model/serializer.rb:361:in `serializable_hash'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/active_model_serializers-0.8.3/lib/active_model/serializer.rb:349:in `as_json'
/var/www/discourse/app/serializers/post_stream_serializer_mixin.rb:31:in `block in posts'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.4/lib/active_record/relation/delegation.rb:39:in `each'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.4/lib/active_record/relation/delegation.rb:39:in `each'
/var/www/discourse/app/serializers/post_stream_serializer_mixin.rb:24:in `map'
/var/www/discourse/app/serializers/post_stream_serializer_mixin.rb:24:in `posts'
/var/www/discourse/app/serializers/post_stream_serializer_mixin.rb:13:in `post_stream'
(eval):3:in `_fast_attributes'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/active_model_serializers-0.8.3/lib/active_model/serializer.rb:467:in `rescue in attributes'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/active_model_serializers-0.8.3/lib/active_model/serializer.rb:455:in `attributes'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/active_model_serializers-0.8.3/lib/active_model/serializer.rb:479:in `_serializable_hash'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/active_model_serializers-0.8.3/lib/active_model/serializer.rb:361:in `serializable_hash'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/active_model_serializers-0.8.3/lib/active_model/serializer.rb:349:in `as_json'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.4/lib/active_support/json/encoding.rb:33:in `encode'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.4/lib/active_support/json/encoding.rb:20:in `encode'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.4/lib/active_support/core_ext/object/json.rb:39:in `to_json'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/active_model_serializers-0.8.3/lib/active_model/serializer.rb:333:in `to_json'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/multi_json-1.12.1/lib/multi_json/adapters/oj.rb:21:in `dump'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/multi_json-1.12.1/lib/multi_json/adapters/oj.rb:21:in `dump'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/multi_json-1.12.1/lib/multi_json/adapter.rb:25:in `dump'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/multi_json-1.12.1/lib/multi_json.rb:139:in `dump'
/var/www/discourse/app/controllers/application_controller.rb:365:in `render_json_dump'
/var/www/discourse/app/controllers/topics_controller.rb:747:in `block (2 levels) in perform_show_response'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_controller/metal/mime_responds.rb:201:in `respond_to'
/var/www/discourse/app/controllers/topics_controller.rb:739:in `perform_show_response'
/var/www/discourse/app/controllers/topics_controller.rb:116:in `show'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_controller/metal/basic_implicit_render.rb:4:in `send_action'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/abstract_controller/base.rb:186:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_controller/metal/rendering.rb:30:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/abstract_controller/callbacks.rb:20:in `block in process_action'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.4/lib/active_support/callbacks.rb:131:in `run_callbacks'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/abstract_controller/callbacks.rb:19:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_controller/metal/rescue.rb:20:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_controller/metal/instrumentation.rb:32:in `block in process_action'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.4/lib/active_support/notifications.rb:166:in `block in instrument'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.4/lib/active_support/notifications/instrumenter.rb:21:in `instrument'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.4/lib/active_support/notifications.rb:166:in `instrument'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_controller/metal/instrumentation.rb:30:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_controller/metal/params_wrapper.rb:252:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.4/lib/active_record/railties/controller_runtime.rb:22:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/abstract_controller/base.rb:124:in `process'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionview-5.1.4/lib/action_view/rendering.rb:30:in `process'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-mini-profiler-0.10.7/lib/mini_profiler/profiling_methods.rb:76:in `block in profile_method'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_controller/metal.rb:189:in `dispatch'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_controller/metal.rb:253:in `dispatch'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/routing/route_set.rb:49:in `dispatch'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/routing/route_set.rb:31:in `serve'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/journey/router.rb:50:in `block in serve'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/journey/router.rb:33:in `each'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/journey/router.rb:33:in `serve'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/routing/route_set.rb:834:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-protection-2.0.0/lib/rack/protection/frame_options.rb:31:in `call'
/var/www/discourse/lib/middleware/omniauth_bypass_middleware.rb:24:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-2.0.3/lib/rack/conditional_get.rb:25:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-2.0.3/lib/rack/head.rb:12:in `call'
/var/www/discourse/lib/middleware/anonymous_cache.rb:147:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-2.0.3/lib/rack/session/abstract/id.rb:232:in `context'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-2.0.3/lib/rack/session/abstract/id.rb:226:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/middleware/cookies.rb:613:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/middleware/callbacks.rb:26:in `block in call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.4/lib/active_support/callbacks.rb:97:in `run_callbacks'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/middleware/callbacks.rb:24:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/middleware/debug_exceptions.rb:59:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/middleware/show_exceptions.rb:31:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/logster-1.2.9/lib/logster/middleware/reporter.rb:31:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/railties-5.1.4/lib/rails/rack/logger.rb:36:in `call_app'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/railties-5.1.4/lib/rails/rack/logger.rb:26:in `call'
/var/www/discourse/config/initializers/100-quiet_logger.rb:16:in `call'
/var/www/discourse/config/initializers/100-silence_logger.rb:29:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/middleware/remote_ip.rb:79:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/middleware/request_id.rb:25:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-2.0.3/lib/rack/method_override.rb:22:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/middleware/executor.rb:12:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-2.0.3/lib/rack/sendfile.rb:111:in `call'
/var/www/discourse/plugins/discourse-prometheus/lib/middleware/metrics.rb:17:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-mini-profiler-0.10.7/lib/mini_profiler/profiler.rb:171:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/message_bus-2.1.2/lib/message_bus/rack/middleware.rb:63:in `call'
/var/www/discourse/lib/middleware/request_tracker.rb:138:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/railties-5.1.4/lib/rails/engine.rb:522:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/railties-5.1.4/lib/rails/railtie.rb:185:in `public_send'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/railties-5.1.4/lib/rails/railtie.rb:185:in `method_missing'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-2.0.3/lib/rack/urlmap.rb:68:in `block in call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-2.0.3/lib/rack/urlmap.rb:53:in `each'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-2.0.3/lib/rack/urlmap.rb:53:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/unicorn-5.4.0/lib/unicorn/http_server.rb:606:in `process_client'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/unicorn-5.4.0/lib/unicorn/http_server.rb:701:in `worker_loop'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/unicorn-5.4.0/lib/unicorn/http_server.rb:549:in `spawn_missing_workers'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/unicorn-5.4.0/lib/unicorn/http_server.rb:142:in `start'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/unicorn-5.4.0/bin/unicorn:126:in `<top (required)>'
/var/www/discourse/vendor/bundle/ruby/2.4.0/bin/unicorn:23:in `load'
/var/www/discourse/vendor/bundle/ruby/2.4.0/bin/unicorn:23:in `<main>'

@maja, could you help?


(Jeff Atwood) #12

Looks like your code is out of date, do you have latest Discourse version?


(Simon Wu) #13

image
I just re-provision the image for installing plugin. I think the version is latest.


(Jeff Atwood) #14

The reference in the error is to a new feature @eviltrout added today.


(Simon Wu) #15

Lucky me :slight_smile:


(Jeff Atwood) #16

I still suspect your DB or code is out of date unless you can repro this on try.discourse.org


(Simon Wu) #17

The issue get resolved after update the DB.
@maja The Azure blob storage plugin works fine for me now. Thanks a lot.


(Erlend Sogge Heggen) #18