Purging old IP logs?

Is there a way to purge IP logs stored by Discourse, at least the old ones? If not, do you plan to implement such functionality?

Discourse is used by dissident communities in authoritative countries (Russia, Iran etc.) and one of the complains is that storing IPs indefinitely poses a grave risk for community members if an administrative account gets compromised by authorities.

2 Likes

Can you be more specific about which logs and where?

I believe we do rotate old nginx automatically :thinking:

3 Likes

Well let’s see. Outside the Discourse Docker container we have

/var/discourse/shared/standalone/log

within that

/var/discourse/shared/standalone/log/rails

       0 Jan 24 23:37 production_errors.log
      20 Jan 17 23:36 production_errors.log-20190119.gz
      20 Jan 18 23:49 production_errors.log-20190120.gz
      20 Jan 19 23:36 production_errors.log-20190121.gz
      20 Jan 20 23:42 production_errors.log-20190122.gz
      20 Jan 22 00:05 production_errors.log-20190123.gz
      20 Jan 22 23:43 production_errors.log-20190124.gz
       0 Jan 23 23:53 production_errors.log-20190125
 8752920 Jan 25 17:06 production.log
  926575 Jan 18 23:48 production.log-20190119.gz
  728846 Jan 19 23:36 production.log-20190120.gz
  972063 Jan 20 23:42 production.log-20190121.gz
 1114974 Jan 22 22:09 production.log-20190122.gz
 1264281 Jan 22 23:43 production.log-20190123.gz
 1058870 Jan 23 23:53 production.log-20190124.gz
11916058 Jan 24 23:37 production.log-20190125
     383 Jan 24 23:37 unicorn.stderr.log
     999 Jan 18 23:49 unicorn.stderr.log-20190119.gz
     195 Jan 19 23:36 unicorn.stderr.log-20190120.gz
   46254 Jan 20 23:42 unicorn.stderr.log-20190121.gz
   49660 Jan 22 00:05 unicorn.stderr.log-20190122.gz
     200 Jan 22 23:43 unicorn.stderr.log-20190123.gz
     381 Jan 23 23:53 unicorn.stderr.log-20190124.gz
     756 Jan 24 23:37 unicorn.stderr.log-20190125
       0 Jan 24 23:37 unicorn.stdout.log
      93 Jan 18 12:22 unicorn.stdout.log-20190119.gz
      20 Jan 18 23:49 unicorn.stdout.log-20190120.gz
      20 Jan 19 23:36 unicorn.stdout.log-20190121.gz
      92 Jan 21 13:39 unicorn.stdout.log-20190122.gz
      20 Jan 22 00:05 unicorn.stdout.log-20190123.gz
      20 Jan 22 23:43 unicorn.stdout.log-20190124.gz
       0 Jan 23 23:53 unicorn.stdout.log-20190125

and

/var/discourse/shared/standalone/log/var-log

  3471 Jan 23 23:53 alternatives.log
  2066 Dec 14 17:02 alternatives.log.1
   693 Feb 10  2018 alternatives.log.10.gz
   162 Jan 31  2018 alternatives.log.11.gz
   135 Jan 22  2018 alternatives.log.12.gz
   251 Nov 29 23:57 alternatives.log.2.gz
   119 Oct 31 00:39 alternatives.log.3.gz
   148 Oct  2 00:36 alternatives.log.4.gz
   117 Sep 19 01:03 alternatives.log.5.gz
   243 Jun 16  2018 alternatives.log.6.gz
   251 May 24  2018 alternatives.log.7.gz
   787 May  1  2018 alternatives.log.8.gz
   665 Mar  6  2018 alternatives.log.9.gz
     0 Jan 22 00:05 auth.log
147724 Jan 25 17:05 auth.log.1
  9709 Jan 21 13:17 auth.log.2.gz
 11130 Jan 15 19:05 auth.log.3.gz
  9023 Jan  9 03:25 auth.log.4.gz
218238 Jan 23 23:53 dpkg.log
101445 Dec 31 23:59 dpkg.log.1
  1495 Mar 29  2018 dpkg.log.10.gz
  2061 Feb 26  2018 dpkg.log.11.gz
  2932 Jan 31  2018 dpkg.log.12.gz
  6633 Nov 29 23:57 dpkg.log.2.gz
  1720 Nov  1 00:43 dpkg.log.3.gz
  1629 Sep 28 00:48 dpkg.log.4.gz
  1563 Aug 31 00:52 dpkg.log.5.gz
  2326 Jul 19  2018 dpkg.log.6.gz
  4031 Jun 28  2018 dpkg.log.7.gz
  2386 May 24  2018 dpkg.log.8.gz
  1339 May  1  2018 dpkg.log.9.gz
      0 Jan 22 00:05 kern.log
    336 Jan 21 13:39 kern.log.1
    119 Jan 15 19:11 kern.log.2.gz
    131 Jan  9 03:35 kern.log.3.gz
    119 Jan  3 18:38 kern.log.4.gz
      0 Oct  2 00:36 monit.log
      0 Jan 22 00:05 syslog
  84788 Jan 25 17:05 syslog.1
   8903 Jan 21 13:17 syslog.2.gz
  10027 Jan 15 19:08 syslog.3.gz
   8447 Jan  9 03:30 syslog.4.gz
  27373 Jan  3 18:35 syslog.5.gz
   1075 Dec 14 19:59 syslog.6.gz
  12831 Dec 14 16:33 syslog.7.gz
   4096 Dec 31 23:59 unattended-upgrades

within that there are the apt, nginx, postgres, redis, and unattended-upgrades folders. Here’s the nginx one:

/var/discourse/shared/standalone/log/var-log/nginx

       0 Jan 24 23:37 access.letsencrypt.log
       0 Jan 23 23:53 access.letsencrypt.log.1
      20 Jan 22 23:43 access.letsencrypt.log.2.gz
      20 Jan 22 00:05 access.letsencrypt.log.3.gz
      20 Jan 20 23:42 access.letsencrypt.log.4.gz
      20 Jan 19 23:36 access.letsencrypt.log.5.gz
      20 Jan 18 23:49 access.letsencrypt.log.6.gz
      20 Jan 17 23:36 access.letsencrypt.log.7.gz
15337523 Jan 25 17:13 access.log
21559109 Jan 24 23:37 access.log.1
 2164577 Apr 24  2018 access.log.10.gz
 1595100 Apr 23  2018 access.log.11.gz
 1395371 Apr 22  2018 access.log.12.gz
 1975848 Apr 21  2018 access.log.13.gz
 2424946 Apr 20  2018 access.log.14.gz
 1742255 Jan 23 23:53 access.log.2.gz
 1907890 Jan 22 23:43 access.log.3.gz
 1708260 Jan 22 00:05 access.log.4.gz
 1264066 Jan 20 23:42 access.log.5.gz
 1165242 Jan 19 23:36 access.log.6.gz
 1489283 Jan 18 23:49 access.log.7.gz
 2626302 Apr 25  2018 access.log.9.gz
       0 Jan 24 23:37 error.letsencrypt.log
       0 Jan 23 23:53 error.letsencrypt.log.1
      20 Jan 22 23:43 error.letsencrypt.log.2.gz
      20 Jan 22 00:05 error.letsencrypt.log.3.gz
      20 Jan 20 23:42 error.letsencrypt.log.4.gz
      20 Jan 19 23:36 error.letsencrypt.log.5.gz
      20 Jan 18 23:49 error.letsencrypt.log.6.gz
      20 Jan 17 23:36 error.letsencrypt.log.7.gz
     400 Jan 25 12:06 error.log
    1213 Jan 24 10:24 error.log.1
     487 Apr 19  2018 error.log.10.gz
     488 Apr 18  2018 error.log.11.gz
     536 Apr 16  2018 error.log.12.gz
     329 Apr 15  2018 error.log.13.gz
     349 Apr 14  2018 error.log.14.gz
     264 Jan 23 00:34 error.log.2.gz
     407 Jan 22 16:53 error.log.3.gz
    2010 Jan 21 21:34 error.log.4.gz
     501 Jan 20 09:49 error.log.5.gz
     354 Jan 19 05:03 error.log.6.gz
     718 Jan 18 18:49 error.log.7.gz
     320 Apr 20  2018 error.log.9.gz

per ncdu the nginx logs are definitely the largest:

   55.8 MiB [##########] /nginx
   25.8 MiB [#####     ] /rails 
   10.2 MiB [#         ] /postgres
    9.7 MiB [#         ] /redis

Just over 100 MB of logs on this Discourse instance.

Does that look good to you @falco?

4 Likes