An idea maybe worth of exploring in your thinking: I feel there could be a strong emphasis on the synchronous/asynchronous aspect of both systems with an integrated easy “switching function” from one to the other.
The forum could detect if the other user is online or not and push towards the appropriate one. For example: “xyz is currently online, do you want to initiate a chat instead?” when starting a private message, or the opposite: “xyz doesn’t seem to be available right now, do you want to initiate a private message instead?”.
If the last posts in a chat session go unanswered for more than x minutes and/or the user is disconnected since more than x minutes, move automatically the chat to a private message. Maybe with some kind of “Continuing the discussion from:” like when creating a linked topic and ideally with some quote/context.
In an ongoing private message, show a notice on the side when the interlocutor is online with “xyz is online now, do you want to continue this conversation in a chat session?” and do the opposite.
This sounds nice to me as a coexisting mechanism and implies immediately the differences and benefits of both. Direct (chat) will also tend to be quicker and shorter messages, but that idea should come naturally to mind.