Skip to main content
FI INDEX   /   Curated corners of the internet   THE WEB IS STILL OPEN
Further IndexLOOK FURTHER.
← BACK TO THE DIRECTORY
SOURCE PROFILE /

Common Crawl

Explore an open repository of web crawl data, including raw pages, metadata, and text extracts collected since 2008.

SOURCE

commoncrawl.org

LAST CHECKED

2026-11-06

ACCESS

Free access to the copy hosted on Amazon; cloud computing you run yourself may carry its own costs.

VISIT ORIGINAL WEBSITE ↗

Best used for

Researching the web at scale: analyzing page text, links, and metadata, or looking up which pages were captured with the URL index.

What to know before using it

This is raw web content; the pages belong to their original publishers, and the project’s Terms of Use apply. The files are very large and suit people comfortable with cloud tools.

Our review covers the source as checked on the stated date. Availability, pricing, and third-party policies may change. Verify important claims with original evidence.