POP3 polling time and debugging


(James Hayward) #1

HI

We are currently testing an installation of Discourse to replace our Google Group, which is used my a large percentage as just a mailing list. The email-in and reply facilities of Discourse make it a viable alternative, and I’ve just invited the hardcore mailing list people onto the testing site to see what they think.

Reaction so far is promising, but they have been a couple of related things that I hope the community can help with?

Firstly, we have some emails not getting through. We can see them in the inbox, and I’ve searched and found out that sidekiq is responsible for polling. Something has failed on there, but I can’t figure out what. I could do with some help debugging this. Other emails are getting through, even from the same person.

Secondly, there is a bit of a delay. I have reduced the imposed delay on outgoing messages to one minute for testing, but I may well raise this again. It seems that the biggest issue is the 5 minute polling delay - is there a way of decreasing this?

I hope someone is kind enough to help!

James


(Robin Ward) #2

Are there any errors in the sidekiq console? /sidekiq or logster? /logs? If there is one you could paste to us that would be really informative.

Unfortunately not. We had issues in the past with polling too frequently from gmail (every 1 minute) and getting blocked.

One thing that could work, although it would require a little development, is to use an incoming mailer API such as mailgun to forward mails to discourse via a HTTP POST. Then we could process them very quickly. Unfortunately that’s not on our roadmap right now as we’re focused on shipping v1.0 of discourse.


(James Hayward) #3

There’s lots of errors in the /logs (thanks for that, I didn’t realise that was there!), but nothing related to the email. Looking into it more, I think it is to do with the minimum character limit - does that fail silently on email-in?

That’s a shame. I understand that 1.0 is the focus at the moment, but is there a chance that this will become a configurable item in the future? Is it something I can change in the code for now?(dangerous, I know!)

The HTTP POST sounds like a good plan. I wish I had the Ruby knowledge to implement it!


(Robin Ward) #4

Ah yes that must be it. If validation fails it goes into a black hole. I wonder what we should do about that? Send an email back saying so?

It will put you on a DIY maintance path but the file is app/jobs/scheduled/poll_mailbox.rb. You can change every 5.minutes to be every 1.minutes.


(Michael Downey) #5

Yes please. It seems that all failure modes (including e-mail address mismatch) are currently black holes so no one knows their new post or reply didn’t make it onto the site, unless they happen to sign in and check. (Or if I sign in to the mailbox to check up on things and see replies that didn’t get appended to topics.)


(Robin Ward) #6

@riking I know this code is quite different from the Ember error handling you’ve been working on, but it’s the same ball of wax. I wonder how it fits into your schedule and could you look into it?

I’d rather not have people be so confused by this behaviour.


(James Hayward) #7

Thank you. I understand the risks (not happy about them!) and seeing as this is just our testing version of the forum at the moment I can just about accept them!

How do I reload the scheduled jobs when I change that, or does the system reload them automagically?

It would be good if the emails didn’t fail silently!


(Robin Ward) #8

Actually it looks like we accepted a PR to make this a site setting over the weekend.

https://github.com/discourse/discourse/commit/6aa44fd41266812ff13fc1da3ce13753f5640cb6

You can now customize it in the admin section. It requires a restart though but will no longer require a patch.


(James Hayward) #9

Awesome! I’ll stop trying to get vagrant working on my machine then :smile:

I was going to try and make this change, so I’ll look at the PR to see if I
was thinking of the right lines!