URLs do not get Onebox-ed on conversion/import


#1

Continuing the discussion from phpBB conversion and general (Linux / Docker / Ruby / GEMs) help:

On my current phpBB forum I use custom BBCode to post YouTube, SoundCloud, Facebook and Twitter links ‘inline’, similar to Onebox on Discourse.
I’m using a slightly customized version of the phpbb3.rb phpBB import script where I strip the custom BBCode tags to keep the bare URLs.
After the import the URLs are shown as clickable links in the converted posts (without any Markdown or BBCode) but they are not converted into Onebox format as I had hoped.
I tried the Rebuild HTML option on some posts and even did a complete rebake of the whole database but that does not show the URLs as Onebox.
Only if I manually edit the posts (without making any changes) the links will be converted to Onebox.

At what point should the Onebox conversion take place? Only on post creation or post edit in the user edit window?

(all the required domain names like youtube.com etc where enabled in Discourse before running the import script)


(Dean Taylor) #2

Check your /sidekiq page as admin, see if there is perhaps a back log of queued items there.


#3

I have 219646 enqueued items like this:
Jobs::ProcessPost [{"post_id"=>68071, "bypass_bump"=>true, "current_site_id"=>"default"}]

I only have 69278 posts though.

What does this mean? The import took about 3 hours and seems complete. The rebake took about an hour.


(Dean Taylor) #4

Part of the baking process is handled in the background.

The truth is it can take a long time, several days for a large forum import.

The easiest way is to just wait for the backlog to clear, stop rebaking etc.

Here are a few threads / posts I created when handling this:


(Kane York) #5

… for a large forum import. Once you’re all running, it’s fine :wink:


#6

This number has not decreased since yesterday. Is this something to worry about?


#7

I’m running my test installation following this guide:

Apparently this does not include automatically starting Sidekiq.
Is there a config file that I can edit to include Sidekiq in the startup procedure of Discourse (I now start with bundle exec rails server)


#8

Okay, the Sidekiq queue has now processed and ‘bare’ YouTube URLs do now show as Onebox. Twitter, Facebook and YouTube in the ‘youtu.be’ form are not shown as Onebox.
SoundCloud shows an empty SoundCloud window, a manual ‘Rebuld HTML’ on the post fixes that.

Well, good enough for me for now. Thank you DeanMarkTaylor for pointing me in the right direction :+1: