It looks like I can add replace sections directly to app.yml instead of using templates as mentioned here:
replace
app.yml