Embed poll_feed fail with UTF8 title

Bug
1

If you configure the embed with feed and the title has special characters, as á or é, the system fail with this error:

 Job exception: incompatible character encodings: ASCII-8BIT and UTF-8

And the backtrace

 /opt/bitnami/apps/discourse/htdocs/app/models/search_observer.rb:39:in `update_topics_index'
/opt/bitnami/apps/discourse/htdocs/app/models/search_observer.rb:63:in `index'
/opt/bitnami/apps/discourse/htdocs/app/models/search_observer.rb:89:in `after_save'
/opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.0.0/gems/rails-observers-0.1.2/lib/rails/observers/active_model/observing.rb:352:in `update'
/opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.0.0/gems/rails-observers-0.1.2/lib/rails/observers/activerecord/observer.rb:118:in `block (2 levels) in define_callbacks'
/opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:424:in `block in make_lambda'
/opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:221:in `call'
/opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:221:in `block in halting_and_conditional'
/opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:215:in `call'
/opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:215:in `block in halting_and_conditional'
/opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:86:in `call'
/opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:86:in `run_callbacks'
/opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.8/lib/active_record/callbacks.rb:302:in `create_or_update'
/opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.8/lib/active_record/persistence.rb:103:in `save'
/opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.8/lib/active_record/validations.rb:51:in `save'
/opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.8/lib/active_record/attribute_methods/dirty.rb:21:in `save'
/opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.8/lib/active_record/transactions.rb:268:in `block (2 levels) in save'
/opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.8/lib/active_record/transactions.rb:329:in `block in with_transaction_returning_status'
/opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.8/lib/active_record/connection_adapters/abstract/database_statements.rb:199:in `transaction'
/opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.8/lib/active_record/transactions.rb:208:in `transaction'
/opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.8/lib/active_record/transactions.rb:326:in `with_transaction_returning_status'
/opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.8/lib/active_record/transactions.rb:268:in `block in save'
/opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.8/lib/active_record/transactions.rb:283:in `rollback_active_record_state!'
/opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.8/lib/active_record/transactions.rb:267:in `save'
/opt/bitnami/apps/discourse/htdocs/lib/post_creator.rb:251:in `save_post'
/opt/bitnami/apps/discourse/htdocs/lib/post_creator.rb:74:in `block in create'
/opt/bitnami/apps/discourse/htdocs/lib/post_creator.rb:133:in `call'
/opt/bitnami/apps/discourse/htdocs/lib/post_creator.rb:133:in `block in transaction'
/opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.8/lib/active_record/connection_adapters/abstract/database_statements.rb:199:in `transaction'
/opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.8/lib/active_record/transactions.rb:208:in `transaction'
/opt/bitnami/apps/discourse/htdocs/lib/post_creator.rb:131:in `transaction'
/opt/bitnami/apps/discourse/htdocs/lib/post_creator.rb:69:in `create'
/opt/bitnami/apps/discourse/htdocs/app/models/topic_embed.rb:40:in `block in import'
/opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.8/lib/active_record/connection_adapters/abstract/database_statements.rb:201:in `block in transaction'
/opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.8/lib/active_record/connection_adapters/abstract/database_statements.rb:209:in `within_new_transaction'
/opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.8/lib/active_record/connection_adapters/abstract/database_statements.rb:201:in `transaction'
/opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.8/lib/active_record/transactions.rb:208:in `transaction'
/opt/bitnami/apps/discourse/htdocs/app/models/topic_embed.rb:33:in `import'
/opt/bitnami/apps/discourse/htdocs/app/jobs/scheduled/poll_feed.rb:39:in `import_topic'
/opt/bitnami/apps/discourse/htdocs/app/jobs/scheduled/poll_feed.rb:33:in `block in import_topics'
/opt/bitnami/apps/discourse/htdocs/app/jobs/scheduled/poll_feed.rb:32:in `each'
/opt/bitnami/apps/discourse/htdocs/app/jobs/scheduled/poll_feed.rb:32:in `import_topics'
/opt/bitnami/apps/discourse/htdocs/app/jobs/scheduled/poll_feed.rb:26:in `poll_feed'
/opt/bitnami/apps/discourse/htdocs/app/jobs/scheduled/poll_feed.rb:16:in `execute'
/opt/bitnami/apps/discourse/htdocs/app/jobs/base.rb:154:in `block (2 levels) in perform'

I change the line 39 of app/models/search_observer.rb from

search_data = title.dup << " " << scrub_html_for_search(cooked)[0...Topic::MAX_SIMILAR_BODY_LENGTH]

to

search_data = title.force_encoding(Encoding::UTF_8).dup << " " << scrub_html_for_search(cooked)[0...Topic::MAX_SIMILAR_BODY_LENGTH]

and works for me.

2 Likes
2

Go ahead and submit a PR for that. You can probably do it through the Github web UI.

3

https://github.com/discourse/discourse/commit/ea31edbf74ebc84f002dbda80bcdf94f5e8b280a

This topic is now closed. New replies are no longer allowed.

1 Like
5

I think this PR caused problems with bootstrap so I am reverting it. Comments on the PR.

1 Like
6

I have a new instalation with docker. The first weeks I din’t have the problem, but after update to 1.3.0beta6+11 the problem return again.

How can I modify the file app/models/search_observer.rb in Docker installation? I don’t find it.

7

pretty sure this is resolved, open a new topic if still an issue