If you want your site to only have external authentication, disable the enable local logins site setting. With only a single authentication method, users will be taken directly to that external method.
No need for CSS to hide the buttons, either!
Should you need to get in with local logins disabled, remember the /u/admin-login route for admins to be sent an email login link.