I am guessing you didn’t set up Twitter API for twitter login? That’s required to get proper Twitter oneboxing in my experience, even if you don’t plan to let users use their Twitter accounts to log in, although that is a nice benefit.
It really doesn’t matter if the image is 500k in today’s world, to be honest. And with retina devices being so common, you want a 2x sized image anyhow in most cases.