Migration trouble related to DiscourseJsProcessor?

Hey @david Having trouble with a bootstrap:

I, [2023-08-24T16:50:36.568059 #1]  INFO -- : > cd /var/www/discourse && su discourse -c 'bundle exec rake db:migrate'                                          
rake aborted!                                                                                                                                                                                 
Errno::ENOENT: No such file or directory @ rb_sysopen - tmp/js-processor.js                                                                                                                   
/var/www/discourse/lib/discourse_js_processor.rb:140:in `read'                                                                                                                                
/var/www/discourse/lib/discourse_js_processor.rb:140:in `create_new_context'                                                                                                                  
/var/www/discourse/lib/discourse_js_processor.rb:156:in `block in v8'                                                                                                                         
/var/www/discourse/lib/discourse_js_processor.rb:154:in `synchronize'                                                                                                                         
/var/www/discourse/lib/discourse_js_processor.rb:154:in `v8'                                                                                                                                  
/var/www/discourse/lib/discourse_js_processor.rb:169:in `block in v8_call'                                                                                                                    
/var/www/discourse/lib/discourse_js_processor.rb:168:in `synchronize'                                                                                                                         
/var/www/discourse/lib/discourse_js_processor.rb:168:in `v8_call'                                                                                                                             
/var/www/discourse/lib/discourse_js_processor.rb:193:in `perform'                                  

Is this related to DEV: Use esbuild to make DiscourseJsProcessor by CvX · Pull Request #23223 · discourse/discourse · GitHub

1 Like

Is there any more backtrace available? Or can you see which migration is triggering it?

2 Likes

Actually don’t worry, @cvx and I have found the problem - we’ll have a fix soon.

1 Like

It just worked!

I thought it was going to get fixed by a new base image a few minutes back, but it failed once after that. . .

Anyway, seems good now.

1 Like

Ok great, glad to hear it’s working! We have a bit of a chicken/egg situation which we need to resolve. Following the commit you linked, assets:precompile needs to be run at least once before running db:migrate. But the reverse is also true - assets:precompile needs an up-to-date db schema.

Out of interest, what was your process here? Were you doing a docker_manager-based upgrade via the UI? Or was this a ./launcher rebuild app? (or something else entirely?)

1 Like

I spoke too soon.

It just failed again, but . … . oh. But the last time the database had already been migrated?

When it worked I ran a ./launcher bootstrap x from the command line.

Then I ran it with Ansible, which does

      git pull && git checkout main && ./launcher bootstrap  {{ discourse_yml }} {{ launcher_args | default("")}}

(I guess I need to remove git checkout main–not sure why that was there)

Aha. But Ansible is dropping the database first (this is for a site I use for migrations, so starting anew is a frequent occurrence). So that’s consistent with your eggs. Sigh. But then I turned off the drop_database and ran it twice from ansible, and it failed, and then bootstrapped again from the command line, and it still failed. There are no clues on the migrations:

Installing mysql2 0.5.5 with native extensions
Bundle complete! 137 Gemfile dependencies, 173 gems now installed.
Gems in the groups 'test' and 'development' were not installed.
Bundled gems are installed into `./vendor/bundle`

I, [2023-08-24T17:24:31.403199 #1]  INFO -- : Replacing types { with set_real_ip_from 192.168.1.0/24;
set_real_ip_from 172.19.0.0/24;
set_real_ip_from 172.18.0.0/24;
set_real_ip_from 172.17.0.0/24;
set_real_ip_from 38.242.7.193/28;
real_ip_recursive on;
real_ip_header X-Forwarded-For;
types {
 in /etc/nginx/conf.d/discourse.conf
I, [2023-08-24T17:24:31.403687 #1]  INFO -- : > cd /var/www/discourse && su discourse -c 'LOAD_PLUGINS=0 bundle exec rake plugin:pull_compatible_all'
I, [2023-08-24T17:24:33.084210 #1]  INFO -- : discourse-microsoft-auth is already at latest compatible version

I, [2023-08-24T17:24:33.084593 #1]  INFO -- : > cd /var/www/discourse && su discourse -c 'bundle exec rake db:migrate'
rake aborted!
Errno::ENOENT: No such file or directory @ rb_sysopen - tmp/js-processor.js
/var/www/discourse/lib/discourse_js_processor.rb:140:in `read'
/var/www/discourse/lib/discourse_js_processor.rb:140:in `create_new_context'
/var/www/discourse/lib/discourse_js_processor.rb:156:in `block in v8'
/var/www/discourse/lib/discourse_js_processor.rb:154:in `synchronize'
/var/www/discourse/lib/discourse_js_processor.rb:154:in `v8'
/var/www/discourse/lib/discourse_js_processor.rb:169:in `block in v8_call'
/var/www/discourse/lib/discourse_js_processor.rb:168:in `synchronize'
/var/www/discourse/lib/discourse_js_processor.rb:168:in `v8_call'
/var/www/discourse/lib/discourse_js_processor.rb:193:in `perform'
/var/www/discourse/lib/pretty_text.rb:54:in `apply_es6_file'

But also could 'templates/enable-ruby-yjit.yml' be the problem? EDIT: That wasn’t it. And then I removed the mysql and import templates. Still no joy.

Has this problem been around for a while? I upgraded another site recently that’s on ECS and it seemed like there was something strange going on with the migration and then the assets were broken. It’s a huge database, though, so I thought I might just be impatient, and also I did some of that process by hand, so I thought I’d just been careless.

1 Like

@cvx just merged this which should solve the inter-dependency problem (once it hits tests-passed):

That would make sense :+1:.

We think the error is being triggered if any database migrations include a call to the markdown engine (PrettyText). In the vast majority of existing sites, that’s rare. But on a brand new site, we seed new topics into the database, which involves cooking some markdown.

No, just the last few hours (since DEV: Use esbuild to make DiscourseJsProcessor by CvX · Pull Request #23223 · discourse/discourse · GitHub)

1 Like

Well, I’m seeing that commit when I git pull in my dev environment, but it’s still failing, both from ansible and when run locally (and I’m not dropping the database).

It hasn’t quite hit tests-passed yet, so I’d expect it to still be failing for a standard install.

Is this a local ‘production’ cluster? Or a development environment?

1 Like

It’s production. It’s got traefiks as a reverse proxy, but it’s pretty much a standard install other than that (and a database built from the standard postgres13 image, but it does have pgvector).

I mentioned my dev environment because I did a git pull there trying to see if the commit had hit tests-passed; I do see FIX: Compile js-processor before db:migrate (#23229) in git log there).

I’ll go back to testing what I was testing before and try again in a bit.

I guess your development environment is running on main, so that’s why the commit shows up. We hit a snag on our internal CI for tests-passed, but hopefully that should be resolved in the next 30 minutes or so :crossed_fingers:

1 Like

Ok should be there now. Please can you give it another try and let us know how you get on @pfaffman

1 Like

I ran one bootstrap/deploy without dropping the database: It worked!

I ran another bootstrap/deploy that did drop the database: It worked!

1 Like

Am I the only one who has a blank screen when updating?

You likely need to do a command line rebuild

This topic was automatically closed after 10 hours. New replies are no longer allowed.