[IMPLEMENTED] robots.txt and social sharing of content with images


Context : cloud edition.

The robots.txt on https://cloud.squidex.io (https://cloud.squidex.io/robots.txt) :

User-agent: *
Disallow: /

appears to block agents to index images, which is a problem for sharing strategies (twittrer, facebook…).

Twitter cards validator (https://cards-dev.twitter.com/validator) :
INFO: Page fetched successfully
INFO: 51 metatags were found
INFO: twitter:card = summary_large_image tag found
INFO: Card loaded successfully
WARN: The image URL https://cloud.squidex.io/api/assets/ad2aa2c2-ad8e-4ef2-9635-2110ee4a5e4d?width=355 specified by the ‘twitter:image’ metatag may be restricted by the site’s robots.txt file, which will prevent Twitter from fetching it.

Any thoughts about that ?

Could you allow robots on the folder /api/assets/ ?
User-agent: *
Allow: /api/assets/*


I can add it, no problem.

Done: https://cloud.squidex.io/robots.txt

1 Like