Editing Incubator escapee wiki:FAQ/Technical (section)

==Is it possible to download the contents of Wikipedia?==

:Yes, the complete text and editing history of all Wikipedia pages can be downloaded. See [[Wikipedia:Database download]].

:Note that downloading the database dumps is '''much''' preferred over trying to spider the entire site. Spidering the site will take you much longer, and puts a lot of load on the server (especially if you ignore our [http://www.wikipedia.org/robots.txt robots.txt] and spider over billions of combinations of diffs and whatnot). Heavy spidering can lead to your spider, or your IP, being barred with prejudice from access to the site. Legitimate spiders (for instance search engine indexers) are encouraged to wait about a minute between requests, follow the robots.txt, and if possible only work during less loaded hours (2:00–14:00 UTC is the lighter half of the day).

:The uploaded images and other media files are not currently bundled in an easily downloadable form; if you need one, please contact the developers on the [http://mail.wikipedia.org/mailman/listinfo/wikitech-l wikitech-l mailing list]. Please ''do not'' spider the whole site to get images.