Occasionally very high CPU load

I have indeed no swap.

Might there anything be wrong with my install that is using too much memory?

Increasing the RAM seems to have fixed it for now.