We’ve just recently added a periodic, every 6 hours, test of models to see if they are working well and warn admins about LLMs with expired API keys or models who have been deprecated.
That said, we have identified a few problems with the check and are working to fix those.