Hello,
Yeah the previous one is for the header signup button. 
I think in this case the easiest way is to override the core template to add icon. We can do it with plugin outlet too but I think on this page it doesn’t really matter because if we use plugin outlet then we have to create new buttons and hide the default buttons section.
You can delete the previous version component what you created because that is not relevant and won’t solve your problem.
I made a simple theme component what you can install. I added some settings where you can change these buttons icons (Sign Up and Log In).