And I finally got this setup and working correctly. Ultimately it had to do with getting IAM setup correctly, being in the same region, and then making sure the instances were on the same account in AWS. Once that was done things started working just fine.
For reference, this guide was extremely helpful:
I marked the last reply by @sam as the solution as it pointed me down at least the right rabbit hole.