Failed to bootstrap (related to the `tombstone` bucket)


(Camille Roux) #1

Hi,

After an update (I use a standard Docker install on DigitalOcean), I couldn’t open any subject. Because I didn’t know what to do, I decided to rebuild my install but it doesn’t work. The rebuild fails. Here is the weird parts of the logs:

I, [2014-05-29T08:58:36.297430 #34]  INFO -- : 
I, [2014-05-29T08:58:36.351808 #34]  INFO -- : > cd /var/www/discourse && sudo -E -u discourse bundle exec rake db:migrate
rake aborted!
Excon::Errors::MovedPermanently: Expected(200) <=> Actual(301 Moved Permanently)
  response => #<Excon::Response:0x007f74ee1d8b60 @data={:body=>"<Error><Code>PermanentRedirect</Code><Message>The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.</Message><RequestId>6FCF245726458865</RequestId><Bucket>tombstone</Bucket><HostId>...</HostId><Endpoint>s3.amazonaws.com</Endpoint></Error>", :headers=>{"x-amz-request-id"=>"6FCF245726458865", "x-amz-id-2"=>"...", "Content-Type"=>"application/xml", "Transfer-Encoding"=>"chunked", "Date"=>"Thu, 29 May 2014 08:58:56 GMT", "Server"=>"AmazonS3"}, :status=>301, :remote_ip=>"54.239.36.18"}, @body="<Error><Code>PermanentRedirect</Code><Message>The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.</Message><RequestId>6FCF245726458865</RequestId><Bucket>tombstone</Bucket><HostId>...L</HostId><Endpoint>s3.amazonaws.com</Endpoint></Error>", @headers={"x-amz-request-id"=>"6FCF245726458865", "x-amz-id-2"=>"...", "Content-Type"=>"application/xml", "Transfer-Encoding"=>"chunked", "Date"=>"Thu, 29 May 2014 08:58:56 GMT", "Server"=>"AmazonS3"}, @status=301, @remote_ip="54.239.36.18">
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/seed-fu-discourse-2.2.1/lib/seed-fu/runner.rb:46:in `eval'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/excon-0.28.0/lib/excon/middlewares/response_parser.rb:8:in `response_call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/excon-0.28.0/lib/excon/connection.rb:361:in `response'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/excon-0.28.0/lib/excon/connection.rb:254:in `request'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/fog-1.18.0/lib/fog/xml/sax_parser_connection.rb:36:in `request'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/fog-1.18.0/lib/fog/core/deprecated/connection.rb:18:in `request'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/fog-1.18.0/lib/fog/aws/storage.rb:526:in `request'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/fog-1.18.0/lib/fog/aws/requests/storage/copy_object.rb:33:in `copy_object'
/var/www/discourse/lib/file_store/s3_store.rb:145:in `remove'
/var/www/discourse/lib/file_store/s3_store.rb:92:in `remove_file'
/var/www/discourse/lib/file_store/s3_store.rb:25:in `remove_upload'
/var/www/discourse/app/models/upload.rb:40:in `block in destroy'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.1/lib/active_record/connection_adapters/abstract/database_statements.rb:209:in `transaction'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.1/lib/active_record/transactions.rb:208:in `transaction'
/var/www/discourse/app/models/upload.rb:39:in `destroy'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.1/lib/active_record/persistence.rb:167:in `destroy!'
/var/www/discourse/app/models/user_avatar.rb:21:in `update_system_avatar!'
/var/www/discourse/app/models/user.rb:631:in `refresh_avatar'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:424:in `block in make_lambda'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:221:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:221:in `block in halting_and_conditional'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:215:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:215:in `block in halting_and_conditional'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:86:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:86:in `run_callbacks'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.1/lib/active_record/callbacks.rb:302:in `create_or_update'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.1/lib/active_record/persistence.rb:103:in `save'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.1/lib/active_record/validations.rb:51:in `save'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.1/lib/active_record/attribute_methods/dirty.rb:21:in `save'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.1/lib/active_record/transactions.rb:268:in `block (2 levels) in save'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.1/lib/active_record/transactions.rb:329:in `block in with_transaction_returning_status'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.1/lib/active_record/connection_adapters/abstract/database_statements.rb:209:in `transaction'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.1/lib/active_record/transactions.rb:208:in `transaction'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.1/lib/active_record/transactions.rb:326:in `with_transaction_returning_status'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.1/lib/active_record/transactions.rb:268:in `block in save'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.1/lib/active_record/transactions.rb:283:in `rollback_active_record_state!'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.1/lib/active_record/transactions.rb:267:in `save'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/seed-fu-discourse-2.2.1/lib/seed-fu/seeder.rb:74:in `seed_record'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/seed-fu-discourse-2.2.1/lib/seed-fu/seeder.rb:36:in `block (2 levels) in seed'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/seed-fu-discourse-2.2.1/lib/seed-fu/seeder.rb:36:in `map'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/seed-fu-discourse-2.2.1/lib/seed-fu/seeder.rb:36:in `block in seed'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.1/lib/active_record/connection_adapters/abstract/database_statements.rb:209:in `transaction'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.1/lib/active_record/transactions.rb:208:in `transaction'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/seed-fu-discourse-2.2.1/lib/seed-fu/seeder.rb:35:in `seed'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/seed-fu-discourse-2.2.1/lib/seed-fu/active_record_extension.rb:32:in `seed'
(eval):9:in `block (2 levels) in run_file'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/seed-fu-discourse-2.2.1/lib/seed-fu/runner.rb:46:in `eval'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/seed-fu-discourse-2.2.1/lib/seed-fu/runner.rb:46:in `block (2 levels) in run_file'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/seed-fu-discourse-2.2.1/lib/seed-fu/runner.rb:58:in `block in open'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/seed-fu-discourse-2.2.1/lib/seed-fu/runner.rb:57:in `open'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/seed-fu-discourse-2.2.1/lib/seed-fu/runner.rb:57:in `open'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/seed-fu-discourse-2.2.1/lib/seed-fu/runner.rb:36:in `block in run_file'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.1/lib/active_record/connection_adapters/abstract/database_statements.rb:211:in `block in transaction'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.1/lib/active_record/connection_adapters/abstract/database_statements.rb:219:in `within_new_transaction'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.1/lib/active_record/connection_adapters/abstract/database_statements.rb:211:in `transaction'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.1/lib/active_record/transactions.rb:208:in `transaction'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/seed-fu-discourse-2.2.1/lib/seed-fu/runner.rb:35:in `run_file'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/seed-fu-discourse-2.2.1/lib/seed-fu/runner.rb:26:in `block in run'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/seed-fu-discourse-2.2.1/lib/seed-fu/runner.rb:25:in `each'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/seed-fu-discourse-2.2.1/lib/seed-fu/runner.rb:25:in `run'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/seed-fu-discourse-2.2.1/lib/seed-fu.rb:29:in `seed'
/var/www/discourse/lib/tasks/db.rake:3:in `block in <top (required)>'

/pups/lib/pups/exec_command.rb:85:in `spawn': cd /var/www/discourse && sudo -E -u discourse bundle exec rake db:migrate failed with return #<Process::Status: pid 271 exit 1> (RuntimeError)
	from /pups/lib/pups/exec_command.rb:55:in `block in run'
	from /pups/lib/pups/exec_command.rb:53:in `each'
	from /pups/lib/pups/exec_command.rb:53:in `run'
	from /pups/lib/pups/command.rb:6:in `run'
	from /pups/lib/pups/config.rb:85:in `block (2 levels) in run_commands'
	from /pups/lib/pups/config.rb:76:in `each'
	from /pups/lib/pups/config.rb:76:in `block in run_commands'
	from /pups/lib/pups/config.rb:75:in `each'
	from /pups/lib/pups/config.rb:75:in `run_commands'
	from /pups/lib/pups/config.rb:71:in `run'
	from /pups/lib/pups/cli.rb:31:in `run'
	from /pups/bin/pups:8:in `<main>'
d2d1668c3a9cd28f651ce2c80a3161f981728d4b93a2b72cda61abe357d2b8e1
FAILED TO BOOTSTRAP

Have you got any idea of what is happening? thanks


(Jeff Atwood) #2

Does your Discourse config use Amazon S3 for anything?

If you try to do this, does it work?

cd /var/docker
git pull
./launcher rebuild app

(Camille Roux) #3

Hi,

I’m using Amazon S3 for uploads but the name of my bucket is not tombstone as mentioned in logs.
I don’t see any changes using git pull.


(Camille Roux) #4

I used ./launcher start app and the app is “working” again but some 500 errors appear some time, like when I try to post a new subject. This seems to be the same error than before:

2014-05-29T09:42:48Z 34 TID-oxc87mpek WARN: {:current_db=>"default", :current_hostname=>"forum.pragmaticentrepreneurs.com"}
2014-05-29T09:42:48Z 34 TID-oxc87mpek WARN: Expected(200) <=> Actual(301 Moved Permanently)
  response => #<Excon::Response:0x007fe8e6e967a0 @data={:body=>"<Error><Code>PermanentRedirect</Code><Message>The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.</Message><RequestId>534E29CBFB6310B0</RequestId><Bucket>tombstone</Bucket><HostId>...</HostId><Endpoint>s3.amazonaws.com</Endpoint></Error>", :headers=>{"x-amz-request-id"=>"534E29CBFB6310B0", "x-amz-id-2"=>"...", "Content-Type"=>"application/xml", "Transfer-Encoding"=>"chunked", "Date"=>"Thu, 29 May 2014 09:42:47 GMT", "Server"=>"AmazonS3"}, :status=>301, :remote_ip=>"54.239.36.18"}, @body="<Error><Code>PermanentRedirect</Code><Message>The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.</Message><RequestId>534E29CBFB6310B0</RequestId><Bucket>tombstone</Bucket><HostId>...</HostId><Endpoint>s3.amazonaws.com</Endpoint></Error>", @headers={"x-amz-request-id"=>"534E29CBFB6310B0", "x-amz-id-2"=>"...", "Content-Type"=>"application/xml", "Transfer-Encoding"=>"chunked", "Date"=>"Thu, 29 May 2014 09:42:47 GMT", "Server"=>"AmazonS3"}, @status=301, @remote_ip="54.239.36.18">
2014-05-29T09:42:48Z 34 TID-oxc87mpek WARN: /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/excon-0.28.0/lib/excon/middlewares/expects.rb:6:in `response_call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/excon-0.28.0/lib/excon/middlewares/response_parser.rb:8:in `response_call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/excon-0.28.0/lib/excon/connection.rb:361:in `response'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/excon-0.28.0/lib/excon/connection.rb:254:in `request'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/fog-1.18.0/lib/fog/xml/sax_parser_connection.rb:36:in `request'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/fog-1.18.0/lib/fog/core/deprecated/connection.rb:18:in `request'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/fog-1.18.0/lib/fog/aws/storage.rb:526:in `request'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/fog-1.18.0/lib/fog/aws/requests/storage/copy_object.rb:33:in `copy_object'
/var/www/discourse/lib/file_store/s3_store.rb:145:in `remove'
/var/www/discourse/lib/file_store/s3_store.rb:92:in `remove_file'
/var/www/discourse/lib/file_store/s3_store.rb:25:in `remove_upload'
/var/www/discourse/app/models/upload.rb:40:in `block in destroy'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.1/lib/active_record/connection_adapters/abstract/database_statements.rb:211:in `block in transaction'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.1/lib/active_record/connection_adapters/abstract/database_statements.rb:219:in `within_new_transaction'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.1/lib/active_record/connection_adapters/abstract/database_statements.rb:211:in `transaction'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.1/lib/active_record/transactions.rb:208:in `transaction'
/var/www/discourse/app/models/upload.rb:39:in `destroy'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.1/lib/active_record/persistence.rb:167:in `destroy!'
/var/www/discourse/app/models/user_avatar.rb:21:in `update_system_avatar!'
/var/www/discourse/app/jobs/scheduled/create_missing_avatars.rb:9:in `block in execute'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.1/lib/active_record/relation/batches.rb:52:in `block (2 levels) in find_each'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.1/lib/active_record/relation/batches.rb:52:in `each'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.1/lib/active_record/relation/batches.rb:52:in `block in find_each'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.1/lib/active_record/relation/batches.rb:125:in `find_in_batches'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.1/lib/active_record/relation/batches.rb:51:in `find_each'
/var/www/discourse/app/jobs/scheduled/create_missing_avatars.rb:7:in `execute'
/var/www/discourse/app/jobs/base.rb:131:in `block (2 levels) in perform'

(Sam Saffron) #5

I saw this before, I kind of want to rip out support for amazon s3 image uploads (a topic for another topic)

I believe @eviltrout had a workaround for this


(Kane York) #6

Will the automatic S3 backups be kept? That seems like a useful feature regardless of difficulties with managing uploads, simply because backups generally contain all the uploads and are really big.


(Sam Saffron) #7

yes, fine with that, but I want all images to either come from local or a CDN that does origin pull like say … CloudFront (another Amazon product)


(Robin Ward) #8

The workaround should be in master. Try pulling the very latest version and trying again :smile:


(Camille Roux) #9

I updated the app using /admin/docker and nothing changed. What’s the name of your commit?


(Sam Saffron) #10

Try again, with my latest commit, that code path is no longer triggered.


(Camille Roux) #11

You rocks! Thanks :slight_smile:


(Jeff Atwood) #12

This topic was automatically closed after 24 hours. New replies are no longer allowed.