ACHE by New York University provides web crawler capabilities capable of collecting web pages that satisfy domains or user-specified patterns. This service uses page classifiers to distinguish between relevant and irrelevant pages in a given domain. The REST API can be utilized to retrieve crawler metrics, and effectuate crawler actions.
The following is a list of sample source code snippets that matched your search term. Source code snippets are chunks of source code that were found out on the Web that you can cut and paste into your own source code. Whereas most of the sample source code we've curated for our directory is for consuming APIs, we occasionally find something interesting on the API provider side of things. If you know of some sample source code that would be of interest to the ProgrammableWeb community, we'd like to know about it. Be sure to check our guidelines for making contributions to ProgrammableWeb.