I’m very interested by this topic but I don’t have the knowledge to answer most of your questions.
However,
I think Discourse wants to keep posts clean and easy to read.
By removing extra line breaks and trimming spaces at the beginning of lines, it ensures posts look consistent.
This helps especially when users, sometimes even without realizing (I know some!), add random spaces or lines which could make their content harder to read.
It may look weird to force this, especially when you actually want a gap in your list, but my guess is that it’s to keep numbered lists as simple as possible, with no need to re-number every element when you add a new one to your post (or in a wiki post).