We have a strange problem with our Discourse server. For several weeks the email polling from our internal POP3 server worked without any problems. On February 16th the reception of email suddenly stopped. Strangely, we see activity from Discourse in the log files of our POP3 server:
Feb 25 13:54:49 mailout popa3d[12458]: Authentication passed for discourse
Feb 25 13:54:49 mailout popa3d[12458]: 1545 messages (12060237 bytes) loaded
Feb 25 13:54:49 mailout popa3d[12458]: 0 (0) deleted, 1545 (12060237) left
However, no new emails are processed.
We are using the newest Discourse version 2.7.0.beta4 (since today 2.7.0.beta3 before). These are our email settings:
Go into the email server, delete all the emails there. Obviously, there needs to be some way to keep the ones that you want to keep…
Then wait for Discourse to POP3 again.
This now works for me and emails start showing up again.
I have a suspicion that some internal email ID counter is messed up during the latest version update. @codinghorror Maybe it is set to a very large number so all emails get treated as “already read” or something…
If I don’t delete all existing emails, new emails just pile up.
That alone shouldn’t stop Discourse from polling, but it’s possible that an incoming email raises an exception. There’s no error handling in the new code…
@jzedlitz Are you still seeing the problem and want to help get to the bottom of this? Are there any related errors in /logs? Do you see an error in /sidekiq/scheduler when you search for Jobs::PollMailbox? Do you see an error when you manually poll in the rails console?
./launcher enter app
rails c
Jobs::PollMailbox.new.execute({})
def mail_too_old?(mail_string)
mail = Mail.new(mail_string)
date_header = mail.header['Date']
return false if date_header.blank?
date = Time.parse(date_header.to_s)
date < 1.week.ago
end
So I guess maybe there could be a silent error if Time.parse fails? Since we don’t even attempt to do anything if the Date header is missing.
Sorry for not responding sooner. We have more or less solved the problem by emptying the POP3 mailbox for Discourse. Immediately Discourse was able to pick up messages again. I transferd some of the newer emails into the now-empty POP3 mailbox. These could also be processed. However, as soon as I put some of the 1,400 older email into the mailbox Discourse won’t process any messages.
I’ve just stumbled on the same issue. I noticed something was off when emails should have created messages. The POP3 mailbox had a few emails in it (I run a small instance).
Putting messages one by one seems to have done the trick, I think it works better now but I have to wait a bit to confirm this.
Unfortunately, the problem has occurred again. Using formail I have split the mbox into the single messages and fed them one by one into the POP3 account. Some (even older) messages were processed by Discourse, other not.
Is there a way to trigger more log messages from Jobs::PollMailbox.new.execute({}) to see why a message is ignored?
Which logfile exactly? I tried tail -f /var/discourse/shared/standalone/log/rails/*.log but no messages show up while Discourse is trying to fetch the emails.
Could you apply the following changes to app/jobs/scheduled/poll_mailbox.rb inside the docker container before running Jobs::PollMailbox.new.execute({}) in the rails console?
pop3.start(SiteSetting.pop3_polling_username, SiteSetting.pop3_polling_password) do |pop|
+ mail_string = nil
pop.each_mail do |p|
+ mail_string = nil
mail_string = p.pop
break if mail_too_old?(mail_string)
process_popmail(mail_string)
p.delete if SiteSetting.pop3_polling_delete_from_server?
+ rescue => e
+ puts e.message
+ puts e.backtrace.join("\n\t")
+ puts "", "MESSAGE:", mail_string if mail_string
end
end
Now that I know which file to look in, I was able to do some experimenting myself. The problem is the mail_too_old check. After I removed it the stuck messages were processed.
Yeah, I suspected that, but it would be interesting to know which email prevented mail_too_old? from working. Was it really too old? Did it fail to parse the email? Something else? You can PM me the email that was stopping POP3 from working if you want.
