I have a list of emails in a CSV file that I am awarding a badge to. I expected a group to not be matched on our forum (I am using the list of all attendees of an event, and not all of them have a forum account). It seemed to work, many were matched just fine – but I noticed that someone who should have gotten one had not.
I tested it again with just his email address in a CSV, and it still popped up as not matching an existing user.
These warnings are unrelated to your issue. Everyone who did not configure a MaxMind licence key sees them. Since people are wasting time investigating these warnings, when they face other issues, I wanted to mute them. However, my commit was reverted, since it broke some other CI tests and dev environments. Needs to be redone in another way.
Regarding your issue, here are the relevant code blocks:
I do not see any obvious bug, but I am also no Ruby programmer. I tested it on our instance with a single and two email addresses in a simple comma-separated CSV, and it did work. It did also work, regardless whether I use LF or CRLF file endings, just to rule out some common file parser issues. How do you create the CSV file, and does it look fine when you open it in a simple text editor, like notepad, nano, vi, whatever?
Awesome, thanks for your help. That’s what I was thinking about the MaxMind warning, but I only had time to read briefly before I logged off yesterday.
The CSV file is generated from Salesforce and it has one email address on each line, no commas or spaces. When I paste it into a basic text editor, it looks totally fine; no extra characters or anything. I also tried with a new CSV with a single email address and had the same error.
Strictly seen, it is no CSV then, which stands for “comma-separated values” . But I know the file extension is widely used also with other separators, and more importantly, line breaks are common as data set separator, so one email per line should work, and … it does work here.
Actually, I did a wrong test yesterday, since I added my valid email first and the invalid 2nd. I thought it just skips non-existing emails silently, but actually it parses the first entry of each line only, hence does not allow multiple email addresses on each line. The code actually makes this quite clear:
line = CSV.parse_line(line).first&.strip
And then it adds the whole line value as one email or username entry to match.
When I add multiple emails into one line each, mixing valid and invalid ones, it shows the correct number of successfully applied batches, and lists all emails which could not be matched. Again tested with Windows and UNIX line endings:
Multi-line or multi-value is however anyway not your issue, if it fails on the very first email despite it is correct . Not sure how to easily debug without rebuilding the Discourse instance. You use the official Docker image installer?
EDIT: Lol, the image caption was added by some KI, in German language, at least here in my German locale browser. Pretty detailed, and very correct .
I will double check with our IT team, they’re the ones that set it up so I’m not exactly sure if it’s on Docker.
Also, a somewhat strange update… I think I solved it, but no idea why or how.
Here’s my current chain of events:
The original file was an .xlsx file downloaded from Salesforce, which I saved as a CSV – this didn’t award my test user, but did work with others.
I tested with just the one email in a fresh CSV – that didn’t work either.
I tried a few things to clean up the data – this is when I started getting the 502 error and no emails were going through. I think I may have somehow added extra empty fields or something weird here that made the file too large and unable to be parsed, thus the (new) 502 error.
Pasting the emails from that same CSV into a fresh, new CSV seems to have solved it – my test user was awarded the badge!
… and possibly the strangest part, he’s the only one that was missing the badge?! I don’t know if I fully believe this, because it seems unlikely that this issue would only impact his email and nobody else’s. If it makes a difference, his was the very first email in the file.
Great that you solved it. Probably some spacial character found its way inside, like “test” and "test" here on discourse resulting in different double quote characters, despite I entered the same.
There is a 50,000 entries limit in Discourse code for this CSV, probably it can be overwritten with config. However, I guess there were no 50,000 lines? Probaly the parser itself, or even the webserver, have other lower size/character limits.
Btw, you can just pass a new CSV with all emails (and this particular one corrected) to bulk badge and it will show you results as if everything was newly assigned. So you can check and assure whether really all others got it.
. But I know the file extension is widely used also with other separators, and more importantly, line breaks are common as data set separator, so one email per line should work, and … it does work here.
.