You can hide the like button with CSS, but that won’t prevent users from liking posts (a plugin would be required to ensure that), if they figure out how to un-hide the button for example.
I suppose the easiest way would be using CSS Classes for Current User's Groups which adds the current user groups to the body classes, then use CSS such as:
body.category-your-category-slug:not(.group-group_name) {
button.like {
display: none;
}
}
Let me know if you have trouble understanding
Here’s a related topic which, alas, didn’t have any reply. I cross-link so helping on one topic will help solving the other.