Ok, we found the culprit. It was a single mail, as soon as I moved that mail into a separate folder, the rest of the mails were processed correctly.
An IMAP client can see that mail just fine. I’d send that mail’s raw contents (incl. headers and all) in a private message, if you want, so you can check and debug the reason for hanging the process.