Outway documentation
Crawl and map
Explore websites with bounded crawls and discover URLs with Outway map runs.
Crawl a website
Submit POST /v1/projects/{projectId}/crawls:
{"url":"https://example.com","maxDepth":1,"maxPages":20}
Set explicit bounds. Crawls support a maximum depth of three and a maximum of 500 pages. The requested page budget counts toward allowance; it is not simply the number of successful captures.
Render JavaScript pages
Add "render": {"waitAfterLoadMs": 1000} to a crawl request to use the native browser. Rendered crawls allow up to 50 pages and depth three. Each page saves rendered HTML, a screenshot, Markdown and collection metadata.
Scope and robots checks apply before every browser request, including redirects and assets. Out-of-scope assets can make a page incomplete; inspect partial results before using them in an audit. Existing per-page time, request and byte limits still apply. Map runs do not support rendering.
Inspect progress
Use the returned run identifier with GET /v1/crawls/{crawlId}. Inspect individual page outcomes and captures. A partial run can contain useful evidence and failed pages; do not describe it as complete coverage.
Discover URLs
Map runs discover site URLs through POST /v1/projects/{projectId}/maps. Inspect the required request fields in the API reference and read the result with GET /v1/maps/{mapId}. URL discovery does not guarantee a full capture of every URL.
Plan collection
Start with a small bounded run, inspect coverage and errors, then increase the budget deliberately. Neither crawl nor map guarantees access to authenticated, blocked or undiscoverable pages.