Welcome to Microsoft Q&A, thank you for posting your question here!
To allow users in your organization to access organizational YouTube account using Azure AD is absolutely possible.
To simplify the steps you can easily do this:
- If you have a developer, you can follow the steps BUT start with STEP (4) four in this LINK. To register the app.
- You can use this step-by-step from keivingpapp.com to Shared YouTube with Azure AD and install the MyApps Secure Sign-In Extension.
Have a good success in your configuration.
Kindly let me know if the above helps or you need further assistance or explanation on this issue.
Best Regards,
Sina