Discourse theme components now support Wasm 🎊

sam · April 7, 2022, 6:05am

WebAssembly (Wasm) is a technology that ships in all modern browsers that lets developers ship portable binary programs.

This means that developers can use almost any programming language and target the web.

In the context of Discourse this opens up the door to shipping a fairly rich set of extensions that were only available to plugin creators in the past.

Examples could be:

Image watermarking / resizing / cropping
Graph generation using graphviz or svgbob
Programming sandboxes (eg: a posts that embed a Ruby runtime)
and much more

In the past due to the Discourse Content Security Policy access to Wasm was shut down except for installations with both local uploads and no CDN.

New interfaces were added to client side to support shipping JavaScript assets in a theme that are unconditionally reachable via local domain.

This enables theme developers to cleanly host Wasm, the flow is:

component → local web worker → Wasm hosted on CDN

Discourse svgbob is an end-to-end example of the new patterns, the 2 key changes are:

All .js assets are also made available off CDN on local server:

{
...
  "assets": {
    "worker": "assets/will-be-avilable-on-local.js"
  }
}

The URL for the local asset is accessible via settings.theme_uploads_local

So in the example above:

settings.theme_uploads_local.worker; // local asset
settings.theme_uploads.worker; // cdn asset

merefield · April 16, 2022, 11:16am

Not specifically wasm related, but I found something else interesting about this code, is it new?

In your example, loadScript is no longer required? (I believe the import is redundant?):

github.com

discourse/discourse-svgbob/blob/6d7818035fb1eab755ba4facb4b837e738aa0b51/javascripts/discourse/api-initializers/discourse-svgbob.js#L1

      
        
            import loadScript from "discourse/lib/load-script";
            import { apiInitializer } from "discourse/lib/api";
            import { later } from "@ember/runloop";
            import { Promise } from "rsvp";
            
            
let wasm = undefined;
            
            
const webWorkerUrl = settings.theme_uploads_local.worker;
            let webWorker;
            
            
const wasmUrl = settings.theme_uploads.wasm;

and instead worker script is loaded JIT when the worker is found not to exist?

github.com

discourse/discourse-svgbob/blob/6d7818035fb1eab755ba4facb4b837e738aa0b51/javascripts/discourse/api-initializers/discourse-svgbob.js#L71

      
        
              });
            }
            
            
let messageSeq = 0;
            let resolvers = {};
            
            
async function cookSvgBob(text) {
              let seq = messageSeq++;
            
            
  if (!webWorker) {
                webWorker = new Worker(webWorkerUrl);
                webWorker.postMessage(['wasmUrl', wasmUrl]);
                webWorker.onmessage = function(e) {
                  let incomingSeq = e.data[0];
                  let converted = e.data[1];
            
            
      resolvers[incomingSeq](converted);
                  delete resolvers[incomingSeq];
                }
              }

using the URL built from the asset reference.

That’s a nice pattern and very helpful!

However, I have a question about workers.

If I used a third party worker script, and that contains importScripts() statements to include additional scripts into the worker’s global scope, how do I include those scripts for import inside the theme component?

I might be asking: how do we require a script from url within a Discourse Theme Component?

Falco · April 16, 2022, 2:28pm

What I did in this scenario is using a postMessage from the main JS script with the URLs to be imported. This is received o the message handler on the worker that can importScript the URLs received.

merefield · April 16, 2022, 2:30pm

Thank you Rafael!. Do you have an open source example to reference?

Falco · April 16, 2022, 2:34pm

I used this same pattern in core

github.com

discourse/discourse/blob/c81d4597a779ae730571b8eaa3bd61c6acb51b0c/app/assets/javascripts/discourse/app/services/media-optimization-worker.js#L112

      
        
              return this.install();
            }
            
            
async install() {
              this.installPromise = new Promise((resolve, reject) => {
                this.afterInstalled = resolve;
                this.failedInstall = reject;
                this.logIfDebug("Installing worker.");
                this.startWorker();
                this.registerMessageHandler();
                this.worker.postMessage({
                  type: "install",
                  settings: {
                    mozjpeg_script: getURLWithCDN("/javascripts/squoosh/mozjpeg_enc.js"),
                    mozjpeg_wasm: getURLWithCDN("/javascripts/squoosh/mozjpeg_enc.wasm"),
                    resize_script: getURLWithCDN(
                      "/javascripts/squoosh/squoosh_resize.js"
                    ),
                    resize_wasm: getURLWithCDN(
                      "/javascripts/squoosh/squoosh_resize_bg.wasm"
                    ),

github.com

discourse/discourse/blob/c81d4597a779ae730571b8eaa3bd61c6acb51b0c/public/javascripts/media-optimization-worker.js#L166

      
        
                  }
                  break;
                default:
                  logIfDebug(`Sorry, we are out of ${e}.`);
              }
            };
            
            
async function loadLibs(settings) {
              if (self.codecs) return;
            
            
  importScripts(settings.mozjpeg_script);
              importScripts(settings.resize_script);
            
            
  let encoderModuleOverrides = {
                locateFile: function (path, prefix) {
                  // if it's a mem init file, use a custom dir
                  if (path.endsWith(".wasm")) return settings.mozjpeg_wasm;
                  // otherwise, use the default, the prefix (JS file's dir) + the path
                  return prefix + path;
                },
                onRuntimeInitialized: function () {

Heinenen · March 20, 2024, 5:28pm

I am using the theme creator to create a new theme component that is supposed to use wasm. As a starting point, I tried uploading svgbob as a theme component. However, I am not allowed to do that, because it contains a wasm file.

Is that an intended limitation of the theme creator? Or can one not just install svgbob as-is?

sam · March 20, 2024, 9:50pm

I suspect we just have file type limitation on theme creator that need lifting.

david · March 20, 2024, 9:59pm

I reset themecreator to use the default ‘theme authorized extensions’, which includes WASM. So it should work now @Heinenen.

(not sure why it was on a non-default setting… maybe in the past one of the common extensions was missing )

Heinenen · March 20, 2024, 10:16pm

Yes, very cool, I can confirm that it works now. Thank you!

Heinenen · March 20, 2024, 10:24pm

Okay, it seems I have to paddle back a bit, I don’t think it’s working perfectly quite yet.
I tried uploading svgbob again and that worked.

After now also trying to upload my own wasm-file, taken from MDN, I get the error:
An error occurred: You supplied invalid parameters to the request: string contains null byte

On my dev container, this is not a problem and I can upload the same file without a problem.

hellekin · March 21, 2024, 7:05pm

Isn’t running a binary from the website a security exposure that should be chosen carefully? That may be a reason why it was not default. What do you think @Roman_Rizzi?

david · March 21, 2024, 7:09pm

It is allowed in themes by default. It’s just the site setting on discourse.theme-creator.io which had been changed. (that’s a regular discourse site, with a plugin that allows anyone to upload and share themes).

WASM is still sandboxed by the browser, so the security exposure is equivalent to a .js file.

Topic		Replies	Views
Defer loading JavaScript in Themes and Components dev	4	1766	April 7, 2022
Discourse-webpack: A boilerplate for developing JS-heavy Discourse components dev	3	1757	January 13, 2022
Is anyone working on a Discourse Wiki? feature	41	15449	May 15, 2020
Beginner's guide to using Discourse Themes admins reference	22	68692	February 22, 2023
Faster (and smaller) uploads in Discourse with Rust, WebAssembly and MozJPEG \| Blog blog	35	4070	February 9, 2023

Discourse theme components now support Wasm 🎊

Related Topics