# Grok Pattern for Nginx logs

**URL:** https://meta.discourse.org/t/grok-pattern-for-nginx-logs/59372
**Category:** Development
**Created:** [March 17, 2017, 7:18pm UTC](https://meta.discourse.org/t/grok-pattern-for-nginx-logs/59372 "2017-03-17T19:18:04Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![jackzampolin](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/jackzampolin/32/121017_2.png) [@jackzampolin](https://meta.discourse.org/u/jackzampolin)
#### Post date: [March 17, 2017, 7:18pm UTC](https://meta.discourse.org/t/grok-pattern-for-nginx-logs/59372/1 "2017-03-17T19:18:04Z")

</div>

It looks like discourse spits out [custom Nginx logs](https://github.com/discourse/discourse/blob/master/config/nginx.sample.conf#L28). There wouldn’t happen to be a [`grok`](https://github.com/logstash-plugins/logstash-patterns-core/blob/master/patterns/grok-patterns) pattern for those logs anywhere? I looked and was unable to find anything.

---

<div class="post-metadata">

### Author: ![mpalmer](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/mpalmer/32/45740_2.png) [@mpalmer](https://meta.discourse.org/u/mpalmer)
#### Post date: [March 17, 2017, 8:20pm UTC](https://meta.discourse.org/t/grok-pattern-for-nginx-logs/59372/2 "2017-03-17T20:20:35Z")

</div>

I’m not aware of one. We haven’t written one ourselves because we collect all access log data through haproxy.

---

<div class="post-metadata">

### Author: ![sam](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/sam/32/102149_2.png) [@sam](https://meta.discourse.org/u/sam)
#### Post date: [March 17, 2017, 8:32pm UTC](https://meta.discourse.org/t/grok-pattern-for-nginx-logs/59372/3 "2017-03-17T20:32:25Z")

</div>

We have this 🙂

[https://github.com/discourse/discourse/blob/master/plugins/discourse-nginx-performance-report/lib/log\_analyzer.rb#L11-L15](https://github.com/discourse/discourse/blob/master/plugins/discourse-nginx-performance-report/lib/log_analyzer.rb#L11-L15)

It is in ruby and probably easily portable to an elastic grok rule

---

<div class="post-metadata">

### Author: ![jackzampolin](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/jackzampolin/32/121017_2.png) [@jackzampolin](https://meta.discourse.org/u/jackzampolin)
#### Post date: [March 21, 2017, 7:38pm UTC](https://meta.discourse.org/t/grok-pattern-for-nginx-logs/59372/4 "2017-03-21T19:38:20Z")

</div>

Thanks for the pointer! I’ll drop this here for anyone looking:

I’m parsing the discourse logs with the [`telegraf`](https://github.com/influxdata/telegraf) [`logparser`](https://github.com/influxdata/telegraf/blob/master/plugins/inputs/logparser/README.md) plugin. The following config works to parse this:

```plaintext
[[inputs.logparser]]
  files = [
    "/var/discourse/shared/standalone/log/var-log/nginx/access.log",
    "/var/discourse/shared/standalone/log/var-log/nginx/access.log.1"
  ]
  from_beginning = true
  [inputs.logparser.grok]
    measurement="nginx_access"
    patterns=[
      '''\[%{CUSTOM_TIMESTAMP:timestamp:ts-httpd}\] %{IP:client_ip} "%{NOTSPACE:method:tag} %{NOTSPACE:request_path} %{NOTSPACE:http_version}" "%{DATA:user_agent}" "%{DATA:discourse_route:tag}" %{NUMBER:response_code:tag} %{NUMBER:resp_bytes:int} "%{DATA:x_referrer}" %{NUMBER} %{NUMBER:resp_time:float} "%{DATA:username:tag}"'''
    ]
    custom_patterns = '''
      CUSTOM_TIMESTAMP %{MONTHDAY}/%{MONTH}/%{YEAR}:%{HOUR}:%{MINUTE}:%{SECOND} %{CUSTOM_TZ}
      CUSTOM_TZ [+-][0-9]{4}
    '''

```

The data is as follows in InfluxDB line protocol:

```plaintext
{
  measurement: nginx_access,
  tags: [
    method,
    discourse_route,
    response_code,
    username
  ],
  fields: {
    client_ip: string,
    request_path: string,
    http_version: string,
    user_agent: string,
    resp_bytes: int,
    x_referer: string,
   resp_time: float,
  }
}

```
