Scribe

What is the Scribe user agent?

Scribe is the name of a simple web crawler that indexes useful and high-quality articles and documentation from around the web.

How often does Scribe crawl my pages?

Scribe is in an alpha stage and will not crawl your pages often. It will never request more than one page every 10 seconds from your website.

What if I face any issues with Scribe?

I’m very sorry that the robot caused you discomfort. Please email scribe@brytecode.be, and we’ll figure things out.

What if I don’t want Scribe to crawl my website?

Scribe respects the Robots TXT specification, and you can block the robot by adding the following to your robots.txt file at the root of your web domain.

User-agent: Scribe
Disallow: /

You can block the robot from accessing specific portions of your website. Check the Robots TXT website for more information.