Hi @Alex Kline,
Welcome to Microsoft Q&A Platform. Thank you for reaching out & hope you are doing well.
As an original poster cannot accept their own answer, I am reposting it so that you can accept it an answer. Accepted answer will help other community members navigate to the appropriate solutions.
Issue: Azure to on-premises connection issue
Solution: The problem was a missing policy with the on-premises firewall allowing traffic from Azure to on-premises. Once the policy was created, OP could connect and ping from Azure to on-Prem resources.
Please remember to "Accept Answer" and "Upvote it" so that others in the community who are experiencing similar challenges can easily find a solution.
Your contribution is greatly appreciated.
Thanks,
Sai Prasanna.