Thanks @Jagster.
Yes, I noticed it that there is no access after leaving the group.
But still the category appearing in user side. And latest posts also appearing in latest tab.
I just want to mute the related category to not show it there.
I am looking for a automated process for this scenario.