Skip to content
Skip to main content

Crawler and AI usage policy

How search crawlers, AI assistants, and automated clients may use MacWall content, which machine-readable endpoints to prefer, and how to request access.

This page is the human-readable companion to /robots.txt and /ai.txt. It states plainly what automated clients may do with this site, and points at the endpoints designed for them so nobody needs to scrape rendered HTML.

Text content: allowed, with attribution

Documentation, guides, blog posts, and product and pricing information may be crawled, indexed, quoted, summarised, and used to answer questions, including inside AI search and assistant products. We only ask that you attribute the source and link back to the page you used.

Wallpaper media: not licensed for reuse

The wallpaper video and image files are a different matter. They are licensed for use inside the MacWall app. They may not be redistributed, re-hosted, bundled into datasets, or used as training data for generative image or video models. The details are in Terms; if you need something beyond that, ask.

Use these endpoints instead of scraping

  • /llms.txt: curated Markdown index of every content page, in the llms.txt format.
  • /llms-full.txt: the full text of the site in one document.
  • Append `.md` to any content URL, for example `/docs/install-macwall.md`. Same content, clean Markdown with YAML frontmatter, no layout markup.
  • /rss.xml, /atom.xml, /feed.json: blog syndication in RSS 2.0, Atom 1.0, and JSON Feed 1.1, with full article content.
  • /sitemap.xml: every indexable URL with last-modified dates.
  • /.well-known/api-catalog: RFC 9727 catalog of the public API.
  • /openapi.json: OpenAPI 3.1 description of those endpoints.
  • /api/wallpapers: the catalog as JSON, documented in the public API docs.

What is off limits

  • `/admin/`, `/api/admin/`, and `/auth/` are administrative surfaces, disallowed in robots.txt.
  • `/activate`, `/thank-you`, and `/open` are post-purchase and deep-link bridges that contain no useful content and may carry personal parameters.
  • Bulk downloading of wallpaper video files, or hotlinking them into another site or app.
  • Ignoring cache headers, or requesting at a rate that degrades the service for other clients.

Rate limits and identification

  • Send a descriptive `User-Agent` that includes a contact URL or email so we can reach you before taking action.
  • Honour `Cache-Control`. Most of this site is CDN-cached and changes infrequently.
  • Keep concurrency modest. A couple of requests per second is plenty for a site this size.
  • Prefer conditional requests and the feeds over re-crawling everything.

Corrections and access requests

If an AI product is describing MacWall incorrectly, or you need access beyond what is described here, email support@macwall.app. We would rather give you a clean feed than have you scrape around a rate limit.

Frequently asked questions

Can I train a language model on MacWall's written content?
Text content may be used for indexing, retrieval, and answering questions with attribution. For model training, email us first so we know who is using it and can keep you on current content.
Can I use MacWall's wallpaper videos in a dataset?
No. Wallpaper media is licensed for use inside the MacWall app only and is not available for redistribution or generative model training.
Is there an API key or rate limit I should know about?
The public endpoints are unauthenticated and have no published quota. Cache responses, keep request rates reasonable, and identify yourself in the User-Agent.
Why do the .md pages return noindex?
They exist for agents and pipelines, not for search results. The HTML page is the canonical version, so the Markdown twin is marked noindex to avoid competing with it.