If you check the discourse postgresql db, you will see that there is a table for site settings and that most site settings are persisted (saved) in the db.
Since you did not specific the exact site settings you are interested in, it’s hard to make an exact reply; but in general, once these site settings are set up in the db, there is little reason to set them on each rebuild in the yml file.
Of course, this does not apply for any custom site settings which are not persisted in the db.
Hope this helps.
a_server:~# docker exec -it data bash
a_server-data:/# su postgres
postgres@a_server-data:/$ psql discourse
psql (12.2 (Debian 12.2-2.pgdg100+1))
Type "help" for help.
discourse=# \dt *site*
List of relations
Schema | Name | Type | Owner
public | site_settings | table | discourse
public | sitemaps | table | discourse
discourse=# select count(id) from site_settings;
If you compare your site setting directives in your yml file with the db; you can have a pretty good idea of which of your yml site settings directives are redundant.