Plugin using shared runtime gets loaded in the ribbon, but doesn't react to button presses

We are seeing a strange problem with some of our documents:

A published Excel add-in gets for some documents (totally random) stops showing the sidebar when the user clicks on a button in the ribbon. We have added console logs to the script entry point to see if the code gets initialized, but no logs are written. So apparently it's not even getting loaded. The ribbon gets created though, just the buttons don't react to clicks.

It works fine for some documents, but for some – just stops reacting to buttons.

Has anyone seen this behavior? Is there a know issue for that?