Disable post creating via the website

You could hide buttons using CSS.

There’s no way to do this with permissions out of the box AFAIK. If users can log into the site using the same identity as the app, then they will have the same access rights to create and reply to topics.