Generate an llms.txt for any website

llms.txt tells language models what a site contains and where to look, the way robots.txt does for search crawlers. Enter a domain and we will write one, then keep it current.

How it works

  1. 1

    Enter a website

    We check it resolves and responds before doing any work.

  2. 2

    We build the file

    Pages are discovered, ranked, summarised, and written as a spec-conformant llms.txt.

  3. 3

    It stays current

    We re-check every three minutes and rewrite the file only when the site actually changes.