The Heart is a Font Awesome. If you want Thumbs up you need to add Fontawesome icon to site icons and then use that in the Reactions plugin.
The icon is not emoji.
fa-thumbs-up I think is the icon your looking for.
The Theme-component also uses the same method to change the icon to another Fontawesome5 icon