Upgrade failed spectacularily

Betas after testing eventually get rolled out as a stable release. Depending on how big of change s to core may require special steps to be performed via command line

A VPS(Virtual Private Server) is what you have paid for your Discourse install to run off of. Unless your running on your home PC; which typically will likely be an unsupported install.

I was using Upcloud as my VPS provider. I am now using Cobtabo

All betas after enough testing becomes what ppl call Stable releases.

In the Link Moin provided you. You have:

  • Beta still not far in testing (not recommended for production)
  • Tests Passed (Passed numerous tests and considered ready for production) Team Recommended branch for security.
  • Stable if your running your own custom plugins & Theme/theme components. Great as changes are very slow. But you are at more if security risks as updates are slow.

Hosted Discourse Plans typically use Tests-Passed branch to ensure best security and good stability.

Don’t take this the wrong way. But with you not being familiar with terms like VPS. Suggests you are very new to this. Did you install Discourse yourself or maybe inherit a discourse forum? We all were new at one time to discourse setup/maintaining

Not an opinion. It is a fact if the software you are choosing to use. Which is on par with most open-source and even closed source software you use I did write vps. Android capitalized the V & P.

However fair enough I imagine you will just have to figure things out for yourself; if you do not really want help learning.

Good luck! :beers::sunglasses::+1::sparkles:

I notice @anon55243134 has deleted almost all their posts. I really think there are lessons to be learned here for the team and for maintenance of the update scripts and the messaging around updating.

@anon55243134 is someone who has been running a self-hosted discourse for years and has now got a damaged and non-functioning installation - just by following the prompts to upgrade.

If that happened to me I’d be very annoyed and distressed at potentially losing my forum contents. Having opted for self-hosting I might not be ready or able to pay big money to get it fixed, if that’s even possible.

I think there are insufficient warnings and checks

  • has the user taken a recent backup (not a hosting services snapshot!)
  • has the user downloaded it
  • is the user told that the web-based update might fail and require a command line update
  • is the user asked to check if their OS is very old
  • is the user told that migrating to a new up to date server might end up being the best approach
  • is the user warned that major updates (such as a database update) can be perilous and if inexperienced waiting a week might be a good idea, for problems to be found and fixed

More worrying still, in one of the deleted posts I see some pretty dramatic failures which were not trapped and the script continued:

cat: /shared/postgres_data/PG_VERSION: No such file or directory
...
E: Unable to locate package postgresql--pgvector
cp: cannot stat '/etc/postgresql//main/*': No such file or directory
sh: 1: /usr/lib/postgresql/bin/postgres: not found
...
Finding the real data directory for the source cluster      
could not get data directory using "/usr/lib/postgresql/bin/postgres" -D "/shared/postgres_data" -C data_directory: No such file or directory
Failure, exiting

I haven’t checked the scripts, but I would expect things not existing is an indication that trouble lies ahead, and it’s time to stop.

5 Likes

Sorry to open an existing can of worms! In my case I upgraded Ubuntu and Postgres and then re-ran the sudo ./luancher rebuild app in the /var/discourse dir and everything seems to have rebuilt correctly and the site is back up.

Thanks to everyone who assisted me with this effort. I appreciate the help and do not know where I would be without this community.

Thanks!

5 Likes

There is definitely opportunitues here to improve Discourse. Having a Stable instance running now for 7 to 8 years there has always been times where I have needed to Upgrade via the server command line. This is even covered in the documentation with a recommended frequency.

However the Documentation is not as easy as it could be to access. The Document Categories plugin is definitely an improvement. But still imho not as good as it could be.

My recommendations to improve this would be links directly in the Admin web interface. Maybe with say a (?) link to bring up a popul with some info and a link to a Topic here in Meta with more detailed info.

With the Update panel it would be handy to also have extra info in a similar fashion with even say with Core & Docker to grey out the button with a required message to do from server command line with a link to that particular release notes with first section detailing requirements. Ie Docker Version X and Ubuntu LTS version X(or equivalent Linux officially supported Distros). The linked topic should imho also include some server command line copy paste items

With the scripts not sure how easy this might be to do. But have the initial script do a check for some base requirements. If required dependency not present exit with a message and maybe a link to needed base info on how to.

The failed upgrade message does need to be more intuitive. While it says to scroll up for earlier errors. I have found there are some errors that seem common and do not affect rebuilding. So exporting key errors to log file that caused the rebuild to fail would be much better. However these proposed changes will likely take a fair bit of work and time.

With Documentation > Self-Hosting really need a more thorough getting started with an introduction of what one should know before self hosting. Like decent knowledge of the OS like Ubuntu LTS with some basic info on maintaining and the Distro upgrade. Best practises on backups and direct how tos. These live could maybe even be added as a topic with tags in Staff category with links to Meta.

Bloomberg iirc made a good topic in what happened in this topic. For my part U do apologize to @anon55243134 . However they also need to own their own part. If you’re coming for support one needs to be willing to listen to what is said and provide info requested so all that are able can help guide them to possible solutions.

We all may have ideas in our opinions how the design etc could be better. But in lieu of changes we would like to have. We have to accept how it is at present.

I know how distressing it is to have damaging downtime. Awhile back I had a problem with the client I volunteer admin for. I petitioned them for over a month when I could not rebuild the app due to server being too small and the free up space how tos here were unable to resolve, they ignored my counsel and ultimately as I warned the server suffered a major crash. They ended up paying a member here to fix the issue that ended up involving a new server being deployed with ample space. The site was down for over 2werks due to their negligence. Later on they were not maintaining the mail server and while the site was not down. The lack of notification emails has caused a lot of damage. I could go in with more. But not a Discourse issue. That is a self hosting issue.

I did long long ago now had a rebuild issue that was caused by a template file. The log gave me enough to explore a guess if commenting out the template file. It worked to resolve my issue. When it came up on here I posted what I did which helped to identify the issue for the team.

On all sides we need to strive to improve. Take time to read and listen to those with the experience & skills to assist resolving issues. That is how I have gained my awareness with things I am able to do. For things (which especially with Discourse’s complexity) I am not experienced with I research as best I can and ask for help and take the counsel of all here that are indeed more advanced in understanding this fantastic software…

@anon55243134 if you’re willing to give things a chance maybe we can all help to get you back online. We just need to avoid during this process deviating to “how we think it should be” and accept for the moment “how it is”. Once fixed up we can venture on lessons learned and start a good discussion with recommendations on how to potentially improve things and accept if the team is receptive (they generally are) that this will take a fair amount of time with other projects being worked on. On our end we can work on ideas and those that have real know how can if they have time work on some of the needed info for how tos, best practices etc…

United Together Everyone Achieves More. Decided we accomplish very little to nothing at all.

5 Likes