Can the email template for new chat messages be customized

Yes, you are correct those email templates are not in the usual section.

I think the only thing you can do is search for the text of the email template, and replace that via Admin > Customize > Site texts