The way i eventually solved this problem is to turn on post approval for incoming emails. This is done by setting the approval level limit to 1 instead of 0.
So all posts from email gets added to the pending list. Then you can delete those users and blacklist their ip.
Cons of this is that a post will be created for each. This will inflate your db.