Right, by default for self-installs it sets the letter avatar urls to /letter_avatar_proxy which nginx proxies to avatars.discourse.org so that the server IP is revealed instead of all your visitors’. (Though note that the IPs are not stored or logged anyways.)
You can switch by changing the url format to replace /letter_avatar_proxy/ with https://avatars.discourse.org/ .