Microsoft Adds Cognitive Services, Speech, and Image Search APIs to Bot Framework

Earlier this year, Microsoft introduced its Bot Framework which empowers developers to build bots for Microsoft platforms like Skype, Messenger, Telegram, etc. Since the launch, Microsoft has introduced a number of new features, most recently adding new APIs that allow developers to create intelligent bots. The APIs include the Computer Vision API, and two Bing APIs  (Bing Speech API and Bing Image Search API).

The Computer Vision API, part of Microsoft's Cognitive Services portfolio, will enable bots to describe an image by itself. For instance, a bot could create an image caption by using the Cognitive Services Computer Vision API to analyze an image and provide a detailed description. The API uses tags, descriptions, and domain-specific models to identify and label content.

Including the Bing Speech API within the Bot Framework will allow bots to consume audio input via an audio file and output text. Essentially, bots can become speech to text engines without human intervention. The reverse use is also available. Bots can convert text input into audio to "talk back" to users. The same speech services are utilized by Cortana, which can be demoed here.

The Bing Image Search API will allow bots to search the Web for certain images and enable suggestions for additional information (e.g. image metadata, similar images, etc.). For example, a bot could consume a product image as input and respond with a list of similar products. This use case scenario can be expanded to suggest internet merchants that carry the suggested products. Search results include thumbnails, full image URLs, publishing website information, image metadata, and more.

Microsoft describes the Bot Framework and the associated API integrations as "the next great conversational experiences." Microsoft encourages developers to share code samples and future ideas at the cognitive user voice forum. Head to the Microsoft Bot Framework site to learn more, and follow the Bot Framework blog for updates. 

Be sure to read the next Images article: Daily API RoundUp: DigitalGlobe GBDX, Aeries, YaCy, QBOS