Upgrading Mathjax to version 4

[mcmcclur (Mark McClure)]

Here, by the way, is an example of mathematical proofreading via AI in this development version in action:

The user can optionally highlight a specific mathematical expression in the markdown editor and request that it be proofread. It might be nice to have that as an option in the new math editor.

[mcmcclur (Mark McClure)]

@sam It turns out that KaTeX isn’t working quite right in the new plugin. If you take a look at it, it’s quite clearly a missing font problem and, indeed, the console indicates that there a bunch of font (Woff-v2) files that are not found.

The fonts actually exist in the correct location, which should be in a folder named ‘fonts’ adjacent to the style sheet. Both of these files exist on my development version, for example:
http://localhost:4200/plugins/discourse-math/katex/katex.min.css
and
http://localhost:4200/plugins/discourse-math/katex/fonts/KaTeX_Size2-Regular.woff2

When we take a look at the Network tab of the console, it appears that it’s trying to download something like
http://localhost:4200/t/my-topic/fonts/KaTeX_Size2-Regular.woff2

Thus, the browser is looking in a ‘fonts’ folder adjacent to the topic URL, which is clearly wrong.

I suppose this all has something to do with the loadKaTeX function defined in the katex-bundle, which I guess loads KaTeX in a non-standard way leading to KaTeX not really knowing where the CSS file lives. I don’t know how to fix it, though.

[sam (Sam Saffron)]

Thanks so much Mark, I went ahead and merged your commits, will have a look at the Katex thing toda

@mcmcclur

Here, by the way, is an example of mathematical proofreading via AI in this development version in action:

Is this just via a custom prompt for the proofreader persona?

@Ethsim2

Before doing so, I wanted to ask what the recommended environment is at this stage - whether early testing in a production-like setup l is appropriate, or whether you’d prefer this to be done using a development environment until the first production release.

I would not worry about it, Mark and I have this under control.

[mcmcclur (Mark McClure)]

@sam

Thanks so much Mark, I went ahead and merged your commits, will have a look at the Katex thing toda

Awesome on both counts!

@sam

Is this just via a custom prompt for the proofreader persona?

That’s exactly right - yes. It’s obviously kinda fortunate that “Proofread” is a close enough description to what I want it to do.

[sam (Sam Saffron)]

I just pushed a fix for Katex, let me know if it works, I need to run it by @david to confirm it is safe, our pattern for loading CSS is not compatible with relative links inside the CSS file cause we are embedding a stylesheet

[sam (Sam Saffron)]

Note, I moved this to a new dedicate topic, cause it was getting a bit lost and noisy on the main math topic. Really looking forward to this upgrade, I feel we are very close.

[mcmcclur (Mark McClure)]

@sam

I just pushed a fix for Katex, let me know if it works,

KaTeX appears to be working fine to me with your latest commit!

It looks to me like some tests have not passed but I agree that the functionality is all there.

I’m pumped, too!

[mcmcclur (Mark McClure)]

If anyone is just itching to try out this new version or if (like me) they can’t wait since they’ve got classes that will use it starting in a few days, then you can install this standalone version here:

GitHub - mcmcclur/discourse-math: A standalone repo for discourse-math

I based it off of this commit, I believe, changed it to draw MathJax from a CDN, rather than internally.

You can see it in action in this post on my Discourse site.

[mcmcclur (Mark McClure)]

I think it’s worth pointing out that this upgrade will be significantly more than an upgrade of MathJax. It will also

• Relax parsing in a way that will work better with LLMs, as requested here and
• Fix issues with the new rich editor, as reported here.

[mcmcclur (Mark McClure)]

@sam and @david

I see that this pull request has been merged and I also notice that you both put in a fair amount of work to push this across the finish line - so thank you very much for that!

I’m not planning on installing this version just yet, since I’ve already got it working sufficiently on my Discourse site; I do have a couple of questions, though.

1. About the release schedule: The code already appears in main on GitHub. I suppose that if one installed a brand new Discourse instance today, then they would automatically have this brand new version of the the math plugin. Is that correct?
2. About the loaders: As far as I can tell from the code, it appears that the idea of defining loadMathjax and loadKatex functions in the core was abandoned. The latest version appears to revert to loadScript, though it’s not clear to me where the script is coming from. Is that correct? If so, why is that?

Thanks again!

[Ethsim2 (Ethan)]

I’m currently on latest-release +17 on a fairly math-heavy production instance.

Really nice to see [grid] now working properly on Meta - that’s a big quality-of-life improvement for math-dense posts. I’m happy to compromise on [center] where needed.

For context, I previously ran into some edge-cases around math inside [grid] / [details], which I documented here (in case it’s still useful):

Inline KaTeX/MathJax $...$ inside [grid] can leak BBCode when used with [details]

i thought i should at least conculde mathematically accurate information, not ChatGPT waffle \frac{1}{r^2 \sin\theta} \left| \begin{matrix} \textcolor{orange}{\hat{r}} & r\hat{\theta} & r\sin\theta\,\hat{\phi} \\ \partial_r & \boxed{\partial_\theta} & \boxed{\partial_\phi} \\ A_r & \boxed{r A_\theta} & \boxed{r\sin\theta\, A_\phi} \end{matrix} \right| \frac{1}{r^2 \sin\theta} \left| \begin{matrix} \textcolor{orange}{\hat{r}} & r\hat{\theta} & r\sin\theta\,\hat{\phi} \\ \p…

Thanks again to everyone working on the MathJax 4 upgrade - it’s great to see this level of polish landing.

[Ethsim2 (Ethan)]

I’m seeing MathJax’s “Erroneous nesting of equation structures” when using \begin{align}...\end{align} inside $$...$$. This appears to be MathJax enforcing the “don’t nest equation environments” rule more strictly during the v4 upgrade work.

In practice, the reliable Discourse pattern is to use aligned inside the $$ wrapper:

$$
{\large
\begin{aligned}
a &= b \\
  &= c
\end{aligned}
}
$$

This avoids nesting a display equation structure (align) inside another display wrapper ($$...$$), which is what triggers the error.

[mcmcclur (Mark McClure)]

I agree with your analysis @Ethsim2 .

This behavior agrees with MathJaxV4, as well as pure LaTeX, so I suppose there’s no “bug” here. Definitely worth pointing out that use of aligned over align is the way to go, though!