The Robot Text as a Service API enables developers to keeps track of rules for web crawlers or SEO tools on different websites. Developers can use GET or POST requests to check resources to see whether they can be crawled using a given user agent. The alpha release of this API is being offered for free.
The following is a list of how-to and tutorial content that matched your search term. ProgrammableWeb's how-to content comes from two sources; full-blown tutorials that we publish ourselves and other highly relevant tutorials that we find elsewhere on the Web. This list represents on combination of both tutorial types and if you go to ProgrammableWeb's API University, you'll not only be able to find more, they are organized based on your role (API providers or developers who consumes APIs). If you know of a tutorial that would be of interest to the ProgrammableWeb community, we'd like to know about it. Be sure to check our guidelines for making contributions to ProgrammableWeb.