User export failed in multisite


(Rodrigo Farcas) #1

Trying to export my 160.000 userbase, and just got a message saying “user export failed, contact staff member”

Could not find anything related to this on logs.

Any ideas?


(Rodrigo Farcas) #2

Any ideas or things to try here?

Thanks


(Jeff Atwood) #3

Any ideas for places to look here @techapj?


(Arpit Jalan) #4

If the export failed there should be an error log at /logs.


(Rodrigo Farcas) #5

@techapj I don’t see anything related to that issue on logs.Is there something I should try to find there?


(Arpit Jalan) #6

Try exporting users again with /logs open in another tab. I am sure if the export fails again, you will see the new error log.


(Rodrigo Farcas) #7

@techAPJ did that, but nothing got into logs. I do noticed CPU use was very high and for about 15 mins forums were blank.


(Arpit Jalan) #8

Are you on latest version? Are you hosted on DO? How much memory?

I just tried exporting users here on meta and it worked flawlessly.


(Robbo) #9

We are hosted on linode. 8GB memory with 2GB swap. I never executed the export but I assume that we run out of memory. Our last update was a few days ago.


(Jeff Atwood) #10

Are you using a Docker based install? We do not support any other kind of install. Only the official Docker based install is supported.


(Robbo) #11

Yes it is docker with multisite.


(Jeff Atwood) #12

OK, so my guess is multisite is causing the logging problem somehow and possibly the export error as well.


(Robbo) #13

In around 24 hours our linodes will be going down for maintenance, at the same time we will be upgrading to the next plan for 16GB memory. Right after I will run the user export while monitoring /log. Should I keep and eye on /sidekiq too and do errors get logged to files regardless of logster working or not?

If it fails again I will just export manually with navicat but no harm in trying first after an upgrade.


(Rodrigo Farcas) #14

With our 16gb memory updated Linode, the issue stills, we can’t export this userlist.
There is no error message on logs.

Any ideas on anything else to try?


(Rodrigo Farcas) #15

New update: it delayed a lot, but I got the export this time, so the reason was discourse going out of ram with a big userbase.


(Jeff Atwood) #16

Something to add to your list @techapj. If large user CSV exports blow up memory, that is bad.


(Rodrigo Farcas) #17

Hey,
Allowing partial downloads, from UID X to Y would be really helpful as well, specially for incremental user list exports


(Mittineague) #18

We recently asked our co-Admin for a User Export.

It took a while and he was starting to think it wasn’t working, but eventually he did get a 219,953 line behemoth.

I agree it would be nice to batch result sets somehow.


(Arpit Jalan) #19

I will be working on this issue today.


(Arpit Jalan) #20

So I’ve been working on this issue (intermittently) since past 4 days, and the solution/fix was more complex than I expected!

Initially I thought that the CSV file creation code was consuming the better part of memory, but after close observation (on my local instance) I noticed that the queries fetching user info were the culprit while the CSV file creation code was lightning fast!

So I optimized the queries for fetching user info and the results after optimization are so much better! Here is the quick comparison of CSV export time before and after optmization:

Total Users Before optmization After optmization
37K 7 minutes 1 minute
188K Infinite loop under 6 minutes

I learned a lot about optimizing queries and memory usage while working on this issue and I will try to optimize more export queries.


https://github.com/discourse/discourse/commit/700c005c26473a6963ba9273b7ebd9622b2c591f