Any downsides to proxy via port instead of unix domain socket?

As long as you block people from connecting directly to port 8008 using your preferred method, no problem.