What you want to achieve can be done by overriding the existing page templates. There is an example of a column removal here: