Rebuild fails with yarn errors

I’m getting an error when upgrading my instance with ./launcher rebuild app on Ubuntu 22.04. As best I can tell, the problems start here:

error /var/www/discourse/app/assets/javascripts/node_modules/discourse: Command failed.
Exit code: 1
Command: yarn --silent --cwd .. patch-package
Arguments: 
Directory: /var/www/discourse/app/assets/javascripts/node_modules/discourse
Output:
error Command "patch-package" not found.
error Command failed with exit code 1.

There are a few “_ has unmet peer dependency” warnings above, but this is the first error (apart from the “_ already exists” when setting up the DB, but I understand that to be expected behavior).

Unfortunately, the upgrade where this first started happening was the day before I went on a family road trip, so I was a bit rushed and didn’t have time to troubleshoot in the moment. But I’m getting this consistently. The host is fully-updated and I don’t recall having done anything particularly unique with the setup. I did put /var/discourse on a separate Digital Ocean volume ~3 months ago (which I know is Not Recommended™ for performance reasons, but for a forum with ones of users, I figured it was worth the tradeoff) but it worked fine after that change as far as I can tell.

1 Like

I think those are all just warnings.

If the site didn’t come back up you can try

./launcher start app

I ran ./launcher start app and get an HTTP 502 now. From ./launcher logs app, I see

[Wed 19 Jul 2023 10:18:34 PM UTC] Run reload cmd: sv reload nginx
fail: nginx: runsv not running
[Wed 19 Jul 2023 10:18:34 PM UTC] Reload error for :
Started runsvdir, PID is 530
ok: run: redis: (pid 542) 0s
ok: run: postgres: (pid 543) 0s
supervisor pid: 538 unicorn pid: 569
Could not find rtlcss-0.2.1, mini_racer-0.8.0, puma-6.3.0, msgpack-1.7.1, libv8-node-18.16.0.0-x86_64-linux in locally installed gems
Run `bundle install` to install missing gems.
config/unicorn_launcher: line 71: kill: (569) - No such process
config/unicorn_launcher: line 15: kill: (569) - No such process
(538) exiting

This repeats quite a bit.

Maybe run bundle install?

To do that, I have to enter the container. I did that, and when I start the container afterward, the web interface gives

Oops

The software powering this discussion forum encountered an unexpected problem. We apologize for the inconvenience.

Detailed information about the error was logged, and an automatic notification generated. We'll take a look at it.

No further action is necessary. However, if the error condition persists, you can provide additional detail, including steps to reproduce the error, by posting a discussion topic in the site's feedback category.

./launcher logs app gives

[Wed 19 Jul 2023 11:52:05 PM UTC] Domains not changed.
[Wed 19 Jul 2023 11:52:05 PM UTC] Skip, Next renewal time is: 2023-08-21T00:34:21Z
[Wed 19 Jul 2023 11:52:05 PM UTC] Add '--force' to force to renew.
[Wed 19 Jul 2023 11:52:05 PM UTC] Installing key to: /shared/ssl/distroleaders.club_ecc.key
[Wed 19 Jul 2023 11:52:05 PM UTC] Installing full chain to: /shared/ssl/distroleaders.club_ecc.cer
[Wed 19 Jul 2023 11:52:05 PM UTC] Run reload cmd: sv reload nginx
fail: nginx: runsv not running
[Wed 19 Jul 2023 11:52:05 PM UTC] Reload error for :
Started runsvdir, PID is 530
ok: run: redis: (pid 544) 0s
ok: run: postgres: (pid 543) 0s
supervisor pid: 538 unicorn pid: 575

If I enter the container again, I can successfully run sv reload nginx, but that doesn’t change the behavior.

Also, if I re-run ./launcher rebuild app, then I have to re-enter the container to run bundle install again.

There’s another error there somewhere. Include the whole log.

Do you have any non standard plugins?

Not that I recall (unless @mattdm snuck some in when I wasn’t looking :smile: ). discourse-doctor says “No non-official plugins detected” as well.

1 Like

I meant the full log from a rebuild.

My apologies. Here’s the output of a rebuild. Is that what you were looking for?

The second line tells the whole story

fatal: not a git repository (or any parent up to mount point /var)

Your /var/discourse directory is not a git repository, which breaks our auto-update of launcher and the tools we use to bootstrap Discourse, which in turn means you are stuck in an old docker image.

You need to undo the changes you did to break that.

2 Likes

When you copied the files to your volume you failed to copy . git, so you’ll need to clone it and copy the other stuff into there.

You should probably make have your volume have just the uploads, which I assume is what is taking up the bulk of the space and you can then have just the uploads and backups on the volume.

1 Like

Oh my goodness. I bet I can guess exactly what I did and it involves the fact that * doesn’t match .git in an rsync command. Wouldn’t be the first time that I used rsync to make my life more difficult. Thanks for catching that!

Is there a technical reason that the rebuild can’t exit at that point? It would have made troubleshooting easier (although starting at the top instead of the bottom would have, too).

2 Likes

Thankfully, I still have the pre-migration tarball because I haven’t lost all of my sysadmin skills over the years. :smile:

This is on the smallest Digital Ocean droplet, so it’s mostly the system that is taking up space. Uploads and backups together are ~100MB. I think what I’ll probably end up doing, if I make a change, is to move to a larger droplet that has more disk space.

Thanks for the help, I really appreciate it!

2 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.