Is the ‘force https’ setting enabled on your Discourse forum? If it is, Discourse should be returning an avatar_template with an https URL. The URL provided by the Discourse avatar template is used by the WordPress plugin to create the link.
I had to deactivate the force https on my discourse install because it caused a problem (infinite redirect) with another app I’m using for the SSO login
If you can fix that, the avatar links will start using HTTPS. The links will not be updated until new comment data is fetched from Discourse though.
I don’t think that customizing the comment_html template will work for solving this problem. I will add a filter to the WP Discourse function that is used for setting the avatar_url. If you are unable to get ‘force https’ working on your forum, you will be able to hook into this filter to rewrite the avatar URLs. I’ll let you know when it’s been added.