With my setup my issue was not with the letsencrypt certificate, but rather with the way that my DNS records were setup. That’s what makes troubleshooting hard, because your DNS, your letencrypt and your app.yml all need to be aligned to make this work.
My understanding is that to “make www. work” there are two options:
(1) is to make sure that your letsencrypt works for the subdomain so that people can access your site through https://www.yoursite.com - this is probably ideal, but I never got this working. See responses from @pfaffman and @dionbeukes and ignore mine if this is what you are after.
(2) is more simple which is to setup the DNS for your site so that anyone typing in the www.yousite.com simply gets taken to the https://yoursite.com automatically–they won’t get any security warnings.
You do this through your DNS (e.g., in your host provider) by making sure that you have a CNAME record for yoursite.com but not www.yoursite.com