Does Onebox engine cache content locally?

robmc · March 23, 2017, 11:48am

Question: when Onebox pulls in content from a site, including title, image, description and (in some cases) price, is that information taken as a snapshot at the time it is posted, or is that information updated each time the thread is loaded?

I am interested in whether certain data fields that can change over time, such as price of a product on Amazon, would be fixed in a conversation, or whether that element would be updated as the original page is updated.

This is particularly relevant if, for example, a product is removed from that site, and the page no longer exists. Would the onebox in the discussion continue to show the original data, or will it return an error?

I know that images can be fetched but is this the case for the other fields as well?

david · March 23, 2017, 12:02pm

Most onebox implementations take a “snapshot” when the post is created. That snapshot will only be updated if you “rebuild HTML” on the post, or when the automatic ‘rebake’ happens periodically (see the rebake old posts count site setting). This is the case for the amazon onebox.

Some oneboxes make use of an iFrame, so they should always show the up-to-date information from the site (e.g. youtube).

robmc · March 23, 2017, 12:48pm

Thanks @david

So, in the case of standard implementation, a user posts a link (to Amazon) and a snapshot of that product is captured and placed on the site. This data remains until the author edits the post, or a rebake happens.

Looking at the site settings, however, it seems that the default site setting is to rebake the latest ‘x’ posts every 15 minutes, so does that mean that Amazon is polled every 15 minutes for a change to the data?

rebake old posts count [250]:
Number of old posts to be rebaked every 15 minutes.

Presumably, if this is an iFrame (such as Youtube) this is bespoke and shows live data each time.

I am asking as I am developing a new Discourse site and I am considering how to implement our own site as onebox (probably as a plugin). However, when posting a link to an amazon product from a previous conversation on this site, I got the following:

I cannot get this to update (and as you see it was 2 hours ago).

If you click on that title, you are taken to the correct page:

david · March 23, 2017, 12:53pm

Yep, Discourse also caches any onebox requests for 24 hours, to prevent repeated requests to third party sites like amazon.

github.com

discourse/discourse/blob/49a0f16ce488817bba89d57b0701a85e93b10752/lib/oneboxer.rb#L143


      
          
          def self.blank_onebox
            { preview: "", onebox: "" }
          end
          
          def self.onebox_cache_key(url)
            "onebox__#{url}"
          end
          
          def self.onebox_raw(url)
            Rails.cache.fetch(onebox_cache_key(url), expires_in: 1.day) do
              uri = URI(url) rescue nil
              return blank_onebox if uri.blank? || SiteSetting.onebox_domains_blacklist.include?(uri.hostname)
              options = { cache: {}, max_width: 695, sanitize_config: Sanitize::Config::DISCOURSE_ONEBOX }
              r = Onebox.preview(url, options)
              { onebox: r.to_s, preview: r.try(:placeholder_html).to_s }
            end
          rescue => e
            # no point warning here, just cause we have an issue oneboxing a url
            # we can later hunt for failed oneboxes by searching logs if needed
            Rails.logger.info("Failed to onebox #{url} #{e} #{e.backtrace}")

If you’re in a development environment and want to purge the cache, there are some suggestions in this thread. Do not purge the redis cache on a production site - it will mess with lots of things:

Stelmsind · July 17, 2017, 11:11am

Hi,

Would you accept a PR that creates an additional setting for whitebox hosts that shouldn’t be cached?

24 hours gives us some grief, but waiting for the 15 minute rebake would be fine.

system · July 30, 2019, 9:10am

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Why are oneboxes cached into posts? Feature	9	1309	December 20, 2020
How to avoid potential issues when rebaking all posts? Support onebox	6	943	January 3, 2023
Fetching onebox images locally Dev	3	464	December 5, 2020
Clear Onebox Cache Dev	6	2244	January 18, 2021
Amazon onebox fails on my site, but works here Bug	19	3755	March 31, 2016

Does Onebox engine cache content locally?

Related topics