Just a quick update -
The ext_mime_db above that we use follows IANA Media Types defined here - https://www.iana.org/assignments/media-types/media-types.xhtml. Unfortunately, this means that mp4 is correctly application/mp4 
Though, looking into our S3 uploader implementation further, I do see that we are adding Content-Disposition header "attachment" for pretty much every upload that is not an image, but it feels like it was only intended to be added for svgs. Using "attachment" would result in the content to get downloaded rather than open in a new tab. For videos, it is a mixture of application/video and attachment that is causing this.
We can at least remove this header, I think.