IIRC there are problems if you clone directly (I think this), that’s why using the tokens to clone are the recommended way.
But I think you may want to give this a shot, if you want to use SSH keys: