Editing Web crawler (section)

====Path-ascending crawling====
Some crawlers intend to download/upload as many resources as possible from a particular web site. So ''path-ascending crawler'' was introduced that would ascend to every path in each URL that it intends to crawl.<ref>{{Cite journal | doi = 10.1002/asi.20078 | volume = 55 | issue = 14 | pages = 1228–1238 | last = Cothey | first = Viv | title = Web-crawling reliability | journal = Journal of the American Society for Information Science and Technology | year = 2004 | url = http://www.scit.wlv.ac.uk/~in7803/publications/cothey_2004.pdf | citeseerx = 10.1.1.117.185 }}</ref> For example, when given a seed URL of <nowiki>http://llama.org/hamster/monkey/page.html</nowiki>, it will attempt to crawl /hamster/monkey/, /hamster/, and /. Cothey found that a path-ascending crawler was very effective in finding isolated resources, or resources for which no inbound link would have been found in regular crawling.