Vimeo just actually introduced this exact feature pretty recently:
Their player is a lot more customizable in general but note it looks like this is only an option with paid accounts, and of course only works with your videos. But seems like a good bet.
Otherwise for YouTube videos I think a manually created list / table right below the video might be better than nothing, even if more just for reference vs interactive clickable thing.