Has anyone yet tried this behind cloudflare without opening metrics to the whole public?
Problem is that the original IP of prometheus is not at the usual spot in HTTP headers when behind cloudflare (or basically any proxy).
Is there any way to make this accessible via http only or add an additional (sub)domain to discourse to create an SSL cert for?
having it accessible via IP or an additional subdomain that could be switched to transparent mode in Cloudflare would be two option. The other would probably be to modify the plugin to check for a different http header. Should be X-Forwarded-For or CF-Connecting- IP I think. I’m just the kind of guy that is able to do so. The latter would probably be the smartest solution.
Any help is appreciated.
EDIT: I just realised that cloudflare itself is very well using the right IP behind cloudflare. It’s either the plugin doing something different than cloudflare itself or…
That’s exaclty what I don’t want to do. I’d rather not have a lot of additional stuff happening on the production server as that is quite loaded with forum traffic already. Also if I need to do a reverse proxy then I probably wouldn’t need to run grafana and prometheus locally as i could easily just reverse proxy /metrics out on IP and non https - but that all sounds hacky. I thought there might be a better way.
It’s not a lot of stuff. It’s pretty low overhead.
Oh, wait. But if you have stuff set up so that the correct remote IP gets to Discourse, then you can just set the remote IP to wherever you’re running prometheus and it’ll work. That’s what I’ve done with some sites that I’ve monitored. I’m pretty sure that’s why DISCOURSE_PROMETHEUS_TRUSTED_IP_ALLOWLIST_REGEX got added.
Are you using the cloudflare template? (The recommended way to do things is use a real CDN and let Discourse decide which stuff to proxy, but you don’t want to do that either.)
I am using a test machine that is not proxied by Cloudflare. I setup the prometheus exporter there together with the DISCOURSE_PROMETHEUS_TRUSTED_IP_ALLOWLIST_REGEX set to my monitoring server and things just work well.
On the production machine the same thing doesn’t.
Yes, I’m using the cloudflare template
templates:
- "templates/postgres.template.yml"
- "templates/redis.template.yml"
- "templates/web.template.yml"
- "templates/web.ratelimited.template.yml"
## Uncomment these two lines if you wish to add Lets Encrypt (https)
- "templates/web.ssl.template.yml"
- "templates/web.letsencrypt.ssl.template.yml"
- "templates/cloudflare.template.yml"
and IP addresses on users admin are looking good.
I’m very happy with how cloudflare works so far and it helped a lot already with DDOS protection so I’d rather not switch to some different CDN solution - at least I don’t see a reason.
Regarding the prometheus exporter I’m not sure why but it doesn’t seem to do the IP check the same way discourse generally does it. I’m not fit enough in ruby to get this assumption validated in the code but i’m very curious about it.
If your community is one that attracts DDOS attacks then Cloudflare is a great solution.
Oh, it runs on its own port, so it bypasses NGINX, which is what that template changes, so that’s why it’s different. So if you want it to have https then you’d need an external nginx to do that, I’d think. Or just access it directly via http and trust that the IP-level restriction is good enough (and trust the intervening networks).
Not sure if we’re talking about the same thing. I was talking about the exporter plugin (this thread is about) not about prometheus. The exporter doesn’t seem to handle IP check the same way discourse itself does it - or otherwise the problem shouldn’t exist, right?
I got it working for now. Seeing @sam 's example dashboard I wonder how I would get redis and postrgres data in. Likely with two separate exporters. Just wonder how I would have the connect to postgres and redis within Discourse
