Is pagination impossible or just hard?

It’s not entirely impossible. I made a proper pagination plugin a few years ago (sorry, it won’t be open sourced). If anyone is attempting it, perhaps just as a learning project, I’d be happy to look at your code and tell you what direction to go in. But I wouldn’t recommend it for a production site, especially one of any size.

*edit looking at my old plugin a bit more, I don’t think I’d recommend it as a learning project either, unless you’re really keen. It’s hard.