You are right, the Issue is not guessing the URI, but somebody “loosing” it. If you secure the API even a known URL is no Issue because you still need to authenticate.
My first Idea was to add an additional endpoint to the AssetContentController that needs Asset.Read permission and to have a flag for each asset marking it “secure” or “not secure”. Secure assets can only be fetched from the secure endpoint and the unsecure endpoint should no longer deliver them.
This would make the secure asset optional and downwards compatible.