All right, I'm stupid - pups syntax?


(Lee_Ars) #1

I give up. I am a simple journalist. Regexes have defeated me.

Goal: Add a location to Discourse’s nginx.conf file such that when the container is rebuilt, the location gets added.

    location ~ ^/uploads/default/customizations/ {
        try_files $uri =404;
        proxy_set_header Host $http_host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $thescheme;
        proxy_set_header X-Sendfile-Type X-Accel-Redirect;
        proxy_set_header X-Accel-Mapping $public/=/downloads/;
        expires 1y;
        add_header Cache-Control public;
    }

How to do this: edit my app.yml file and use @sam’s pups bootstrapper to stuff it in there.

Problem: Since I can’t specify an arbitrary line number to insert at, I have to instead use the pups replace function. I need to add this new location below an existing location in discourse.conf that already contains a regex (location ~ /_?emoji.*\.(png|gif|jpg|jpeg)$/). Since pups uses regex matching to find its replace string in the first place…this is blowing up like a nuclear bomb and I totally do not understand how the hell to proceed.

I hate to be the dumbass who begs for someone else to write his code…but please someone help me. I have been chewing on this for a few days on and off and I just can’t wrap my brain around how to proceed. I tried scanning through the other templates in the template directory and I couldn’t find anywhere in any of them with representative syntax for me top copy.

Sorry :frowning:


(Régis Hanol) #2

Have you tried only matching enough of the regex to be sure you’re replacing the right spot? It’s much easier to match location ~ /_?emoji than the whole line.


(Lee_Ars) #3

I…had not tried that. This is why I’m not a developer.

Giving that a shot—my addition to app.yml looks like this:

run:                                                                             
  - replace:                                                                     
      filename: "/etc/nginx/conf.d/discourse.conf"                               
      from: location ~ /_?emoji                                                  
      to: "location ~ /_?emoji.*\.(png|gif|jpg|jpeg)$/ {                         
      expires 1y;                                                                
      add_header Cache-Control public;                                           
    }                                                                            
                                                                                 
    location ~ ^/uploads/default/customizations/ {                               
        try_files $uri =404;                                                     
        proxy_set_header Host $http_host;                                        
        proxy_set_header X-Real-IP $remote_addr;                                 
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;             
        proxy_set_header X-Forwarded-Proto $thescheme;                           
        proxy_set_header X-Sendfile-Type X-Accel-Redirect;                       
        proxy_set_header X-Accel-Mapping $public/=/downloads/;                   
        expires 1y;                                                              
        add_header Cache-Control public;                                         
    }"         

However, on running ./launcher rebuild app, the build aborts with this error when it gets to the app.yml processing:

(<unknown>): found unknown escape character while parsing a quoted scalar at line 59 column 11 -e LANG=en_US.UTF-8
YAML syntax error. Please check your /var/discourse/containers/*.yml config files.

Line 59 is the to: "location ~ /_?emoji.*\.(png|gif|jpg|jpeg)$/ line.

Plus, I feel a little funky having to replace the emoji location because if you guys ever need to change it, I’m clobbering your changes. The whole “you have to find something to replace if you want to do a straight addition” is what’s really confusing me here.


(Régis Hanol) #4

I believe you can’t use "foo" for multiline strings in yaml. Here’s what I would do

run:
  - replace:
      filename: "/etc/nginx/conf.d/discourse.conf"
      from: location ~ /_?emoji
      to: |
        location ~ /_?emoji.*\.(png|gif|jpg|jpeg)$/ {
          expires 1y;
          add_header Cache-Control public;
        }

        location ~ ^/uploads/default/customizations/ {
            try_files $uri =404;
            proxy_set_header Host $http_host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $thescheme;
            proxy_set_header X-Sendfile-Type X-Accel-Redirect;
            proxy_set_header X-Accel-Mapping $public/=/downloads/;
            expires 1y;
            add_header Cache-Control public;
        }

(Lee_Ars) #5

Ugh, this is getting complicated.

That works, but the resulting discourse.conf file contains this at the replace location:

    # cache emojis
    location ~ /_?emoji.*\.(png|gif|jpg|jpeg)$/ {
  expires 1y;
  add_header Cache-Control public;
}

location ~ ^/uploads/default/customizations/ {
    try_files $uri =404;
    proxy_set_header Host $http_host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $thescheme;
    proxy_set_header X-Sendfile-Type X-Accel-Redirect;
    proxy_set_header X-Accel-Mapping $public/=/downloads/;
    expires 1y;
    add_header Cache-Control public;
}.*\.(png|gif|jpg|jpeg)$/ {
      expires 1y;
      add_header Cache-Control public;
    }

…like it’s doing an insert?

I am really sorry to be so dumb about this.


(Régis Hanol) #6

I looked at the code and when the from argument is a string, it will simply replace its content with the content of the to argument.

Here’s what I would do

replace:
  filename: "/etc/nginx/conf.d/discourse.conf"
  from: "location ~ ^/plugins/ {"
  to: |
    location ~ ^/uploads/default/customizations/ {
      ...
    }
    location ~ ^/plugins/ {

I’m using location ~ ^/plugins/ { as a marker and adding your customization before it.


(Lee_Ars) #7

Oh yeah, that’s a lot better—that way I’m not clobbering anything. Lemme give this a shot. If it works you’ll be my new best friend!

edit - It works! Screw @codinghorror, I don’t like him anymore. @zogstrip is the best now.


(cpradio) #8

So not to hijack this conversation, but I have a question semi-related. Is there a way from a plugin perspective to do a replace in the discourse.conf file? I might have a reason for doing it, but only under specific scenarios.

If not, I’ll just make it part of my README and those who need to do it, can.


(Régis Hanol) #9

I don’t think it’s possible for the plugins to use that syntax since it’s only used during the bootstrapping of the container and the plugins are only “executed” once the container starts.

(cc @sam for confirmation)


(cpradio) #10

Yeah, that was my line of thinking too, it seemed the plugin would be too late. Not a big deal. I could probably include a template file and the user can move that into their appropriate location and make the app.yml change accordingly.


(Lee_Ars) #11

I don’t have any context here, so apologizes for jumping in, but what’s your plugin doing that it needs to make changes to the Nginx configuration? That ought to be outside the scope of anything the Discourse application is doing.


(cpradio) #12

The service it interacts with doesn’t support HTTPS yet, so I’m looking at setting up a reverse proxy, I’m still working my way through the best way to do this, whether it can be inside the Discourse container or must be outside it, etc. Right now I’m trying to do it inside to see what happens (it may horribly break), in which this was all for naught and my directions will have to be entirely different.

So part of the question is, I don’t have the answer yet, but I’m trying to fumble my way through it.


(Lee_Ars) #13

What’s your plugin? I’d be curious to test. I’m using Discourse with mandatory SSL provided outside the container (the stack is HAProxy for SSL termination > Varnish > Nginx > Discourse (which itself is nginx > unicorn). Could provide you another testing data point if you’d like. (Forum is here.)

Edit - we should break this out into another thread, too—or make one over on the bigdino forum and we can talk there, your choice :slight_smile:

Edit^2 - Ah, replygif, I see it. Giving it a shot to see what happens…


(cpradio) #14

Yes, that is the one. I just recently setup an SSL server to see what kind of solutions I can come up with. :smile: This is completely outside of my realm of knowledge though, so it is purely trial and error so far.