Plugin to store uploads into Azure blob storage

I’m wondering if there is a plugin to save uploaded files into Azure blob storage. Probably not…

If so, any pointers as to what plugin template I can study to write something like this?

Amazon S3 support is baked into the core instead of being a plugin, so I’m not sure if it will be easy to do in a plugin.

@maja can you update the topic here with your progress?

1 Like

I have created a repository with what I’ve done in previous week: https://github.com/majakomel/discourse-azure-blob-storage. Basic functionality is there, apart from purge_tombstone, also there seems to be an issue with avatars that I need to fix.

6 Likes

Plugin now works with Azure CDN as well. It would be great to get some feedback, if someone feels like testing it. Plugin is activated as soon as it’s installed (enable setting doesn’t do anything yet). It can be configured under Admin -> Settings -> Plugins, account name, access key and container name are required.

5 Likes

@schungx / @SimonWu I was wondering if you would have a bit of time to test out the new plugin by @maja.

We really want to get azure blob storage working well and this is the first time we have a plugin :cake:

@maja we should pull out a dedicated topic for this in the #plugin category that documents how to use, concerns etc.

4 Likes

Sure, I will give a try.

2 Likes

Love to. I’ll install it when I get to sit at my desk.

2 Likes

It throws error when I try to upload a picture in post.

NameError (undefined local variable or method `locked_by_id' for #<Post:0x00007f7684893e40>
Did you mean?  locked?)
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activemodel-5.1.4/lib/active_model/attribute_methods.rb:432:in `method_missing'

Stacktrace

/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activemodel-5.1.4/lib/active_model/attribute_methods.rb:432:in `method_missing'
/var/www/discourse/app/models/post.rb:739:in `locked?'
/var/www/discourse/app/serializers/post_serializer.rb:364:in `include_locked?'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/active_model_serializers-0.8.3/lib/active_model/serializer.rb:374:in `include?'
(eval):64:in `_fast_attributes'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/active_model_serializers-0.8.3/lib/active_model/serializer.rb:467:in `rescue in attributes'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/active_model_serializers-0.8.3/lib/active_model/serializer.rb:455:in `attributes'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/active_model_serializers-0.8.3/lib/active_model/serializer.rb:479:in `_serializable_hash'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/active_model_serializers-0.8.3/lib/active_model/serializer.rb:361:in `serializable_hash'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/active_model_serializers-0.8.3/lib/active_model/serializer.rb:349:in `as_json'
/var/www/discourse/app/serializers/post_stream_serializer_mixin.rb:31:in `block in posts'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.4/lib/active_record/relation/delegation.rb:39:in `each'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.4/lib/active_record/relation/delegation.rb:39:in `each'
/var/www/discourse/app/serializers/post_stream_serializer_mixin.rb:24:in `map'
/var/www/discourse/app/serializers/post_stream_serializer_mixin.rb:24:in `posts'
/var/www/discourse/app/serializers/post_stream_serializer_mixin.rb:13:in `post_stream'
(eval):3:in `_fast_attributes'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/active_model_serializers-0.8.3/lib/active_model/serializer.rb:467:in `rescue in attributes'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/active_model_serializers-0.8.3/lib/active_model/serializer.rb:455:in `attributes'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/active_model_serializers-0.8.3/lib/active_model/serializer.rb:479:in `_serializable_hash'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/active_model_serializers-0.8.3/lib/active_model/serializer.rb:361:in `serializable_hash'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/active_model_serializers-0.8.3/lib/active_model/serializer.rb:349:in `as_json'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.4/lib/active_support/json/encoding.rb:33:in `encode'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.4/lib/active_support/json/encoding.rb:20:in `encode'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.4/lib/active_support/core_ext/object/json.rb:39:in `to_json'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/active_model_serializers-0.8.3/lib/active_model/serializer.rb:333:in `to_json'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/multi_json-1.12.1/lib/multi_json/adapters/oj.rb:21:in `dump'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/multi_json-1.12.1/lib/multi_json/adapters/oj.rb:21:in `dump'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/multi_json-1.12.1/lib/multi_json/adapter.rb:25:in `dump'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/multi_json-1.12.1/lib/multi_json.rb:139:in `dump'
/var/www/discourse/app/controllers/application_controller.rb:365:in `render_json_dump'
/var/www/discourse/app/controllers/topics_controller.rb:747:in `block (2 levels) in perform_show_response'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_controller/metal/mime_responds.rb:201:in `respond_to'
/var/www/discourse/app/controllers/topics_controller.rb:739:in `perform_show_response'
/var/www/discourse/app/controllers/topics_controller.rb:116:in `show'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_controller/metal/basic_implicit_render.rb:4:in `send_action'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/abstract_controller/base.rb:186:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_controller/metal/rendering.rb:30:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/abstract_controller/callbacks.rb:20:in `block in process_action'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.4/lib/active_support/callbacks.rb:131:in `run_callbacks'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/abstract_controller/callbacks.rb:19:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_controller/metal/rescue.rb:20:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_controller/metal/instrumentation.rb:32:in `block in process_action'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.4/lib/active_support/notifications.rb:166:in `block in instrument'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.4/lib/active_support/notifications/instrumenter.rb:21:in `instrument'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.4/lib/active_support/notifications.rb:166:in `instrument'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_controller/metal/instrumentation.rb:30:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_controller/metal/params_wrapper.rb:252:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.4/lib/active_record/railties/controller_runtime.rb:22:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/abstract_controller/base.rb:124:in `process'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionview-5.1.4/lib/action_view/rendering.rb:30:in `process'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-mini-profiler-0.10.7/lib/mini_profiler/profiling_methods.rb:76:in `block in profile_method'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_controller/metal.rb:189:in `dispatch'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_controller/metal.rb:253:in `dispatch'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/routing/route_set.rb:49:in `dispatch'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/routing/route_set.rb:31:in `serve'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/journey/router.rb:50:in `block in serve'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/journey/router.rb:33:in `each'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/journey/router.rb:33:in `serve'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/routing/route_set.rb:834:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-protection-2.0.0/lib/rack/protection/frame_options.rb:31:in `call'
/var/www/discourse/lib/middleware/omniauth_bypass_middleware.rb:24:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-2.0.3/lib/rack/conditional_get.rb:25:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-2.0.3/lib/rack/head.rb:12:in `call'
/var/www/discourse/lib/middleware/anonymous_cache.rb:147:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-2.0.3/lib/rack/session/abstract/id.rb:232:in `context'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-2.0.3/lib/rack/session/abstract/id.rb:226:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/middleware/cookies.rb:613:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/middleware/callbacks.rb:26:in `block in call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.4/lib/active_support/callbacks.rb:97:in `run_callbacks'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/middleware/callbacks.rb:24:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/middleware/debug_exceptions.rb:59:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/middleware/show_exceptions.rb:31:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/logster-1.2.9/lib/logster/middleware/reporter.rb:31:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/railties-5.1.4/lib/rails/rack/logger.rb:36:in `call_app'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/railties-5.1.4/lib/rails/rack/logger.rb:26:in `call'
/var/www/discourse/config/initializers/100-quiet_logger.rb:16:in `call'
/var/www/discourse/config/initializers/100-silence_logger.rb:29:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/middleware/remote_ip.rb:79:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/middleware/request_id.rb:25:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-2.0.3/lib/rack/method_override.rb:22:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/middleware/executor.rb:12:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-2.0.3/lib/rack/sendfile.rb:111:in `call'
/var/www/discourse/plugins/discourse-prometheus/lib/middleware/metrics.rb:17:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-mini-profiler-0.10.7/lib/mini_profiler/profiler.rb:171:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/message_bus-2.1.2/lib/message_bus/rack/middleware.rb:63:in `call'
/var/www/discourse/lib/middleware/request_tracker.rb:138:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/railties-5.1.4/lib/rails/engine.rb:522:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/railties-5.1.4/lib/rails/railtie.rb:185:in `public_send'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/railties-5.1.4/lib/rails/railtie.rb:185:in `method_missing'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-2.0.3/lib/rack/urlmap.rb:68:in `block in call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-2.0.3/lib/rack/urlmap.rb:53:in `each'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-2.0.3/lib/rack/urlmap.rb:53:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/unicorn-5.4.0/lib/unicorn/http_server.rb:606:in `process_client'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/unicorn-5.4.0/lib/unicorn/http_server.rb:701:in `worker_loop'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/unicorn-5.4.0/lib/unicorn/http_server.rb:549:in `spawn_missing_workers'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/unicorn-5.4.0/lib/unicorn/http_server.rb:142:in `start'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/unicorn-5.4.0/bin/unicorn:126:in `<top (required)>'
/var/www/discourse/vendor/bundle/ruby/2.4.0/bin/unicorn:23:in `load'
/var/www/discourse/vendor/bundle/ruby/2.4.0/bin/unicorn:23:in `<main>'

@maja, could you help?

Looks like your code is out of date, do you have latest Discourse version?

1 Like

image
I just re-provision the image for installing plugin. I think the version is latest.

The reference in the error is to a new feature @eviltrout added today.

Lucky me :slight_smile:

I still suspect your DB or code is out of date unless you can repro this on try.discourse.org

The issue get resolved after update the DB.
@maja The Azure blob storage plugin works fine for me now. Thanks a lot.

4 Likes