I use an EX40 dedicated server from http://hetzner.de which for 49,- EUR per month is quite affordable.
Since the server is running OpenVZ (from my pre-Docker eras) to enable easy hosting of different applications I first tried to run Docker inside an OpenVZ container. This works but uses much disk space because Docker inside an OpenVZ container only supports the VFS file system. VFS does not “sym link” files used by more than one image layer. AUFS does - see Supported Filesystems — Project Atomic for background information. Besides that Docker itself and Discourse were running well except some small problems - see Launcher rebuild breaks because the database system is starting up
This setup was using way to much disk space though and not being supported by Discource. So I created a KVM based virtual machine using the recommended Ubuntu 14.04 + Docker. This works very well - I’m currently happy with this setup. Running OpenVZ and KVM on CentOS 6.6 side by side seems not to be a problem at all. I use official software packages - there is no fancy hacking involved.