It seems simple static file requests like the discourse logo already take a very long time.
https://endownload.com/uploads/default/optimized/1X/_129430568242d1b7f853bb13ebea28b3f6af4e7_2_180x180.png → 14kb and 700ms
The 342kb font file is served at a mere 83 kB/s, taking over 4 seconds. For comparison, downloading that same file from here on meta takes 0.01 second.
Pages do serve quickly so my guess is that it’s your disk that is slow.
Quick fix: put Cloudflare in front
Better fix: get another VPS provider