Also as an idea I have would be maybe to implement something like http://instantclick.io here? It starts the ajax call as soon as the user hovers the avatar and shows intention to click on it.
There is a test at Test your click speed - InstantClick that shows that you can easily save about 400ms of loading time with this. Even on mobile.