CafuneBot

What it is

CafuneBot is Cafuné's crawler. It keeps our coffee catalogue current by reading the public product pages of UK specialty roasters: names, origins, process, tasting notes, prices and stock, the product description, and bag images.

How to identify it

User agent: CafuneBot/1.0 (+https://cafuneapp.com/bot)
robots.txt token: CafuneBot

How it behaves

• Obeys robots.txt (RFC 9309), including Crawl-delay.
• One request at a time, at least 10 seconds between requests to a site, roughly weekly.
• Logged-off public pages only. No CAPTCHA solving, proxies, or block evasion.
• Backs off on 429 and 5xx responses.
• If a site's robots.txt returns a server error, it does not crawl that site.

Opting out

Add this to your robots.txt:

User-agent: CafuneBot
Disallow: /

Takedown and questions

Email legal@cafuneapp.com. On request we stop crawling your site and remove the pages, images, and descriptions we stored from it.

Credits

Crawling is done with Crawl4AI. This product includes software developed by UncleCode (https://x.com/unclecode) as part of the Crawl4AI project (https://github.com/unclecode/crawl4ai).