If you site works on the fringes of the legal or is otherwise likely to anger people who are willing to spend thousands of dollars (or at least hundreds) to hurt you I’d say that you’ve already spent too much time on this.
If you’re not going to lose your life or a lot of money every minute that you’re site is down, I’d ignore them.
The ip address of nearly every web site you’ve used (including this one) is public.