Everything here is going to depend on the model. I can see a lot of general interest here, but nobody has suggested which model to use and nobody proved that the model can do anything remotely useful.
Even getting good stuff out of OPT is hard and Facebook added a fair bit of params. My general concern here (also to the industry at wide) is that in the open space there is nothing even remotely close to GPT 3 devinci, and GPT 4 is on the horizon and will make it even harder to compete.