Microsoft Cognitive Services Text Analytics API

Introduction

In my last blog post, I introduced Microsoft Cognitive Services and highlighted the key areas (Vision, Speech, Knowledge, Search, and Language) and looked at some of the features and benefits each area has on offer.

In this blog post, we take a closer look at the Language API and how it can help you take unstructured text and help you identify actionable and meaningful insights.We’ll cover the following topics:

• look at key features of the Text Analytics API
• explore how you can easily integrate the API with your existing applications
• run through some sample uses cases
• explore a sample application that shows how easy it is to implement

Key features

We touched on some of the key features of the Text Analytics API in the introductory blog post, however it’s worth having a recap. There are four main offerings in the API:

• Sentiment Analysis
• Key Phrase Extraction
• Language Detection
• Entity Identification (in Preview Mode)

Each of the features can add rich new functionality to your existing applications and save you from having to learn algorithms such as Bayesian Theorem. For example, the API can help you easily determine underlying sentiment in a stream of text or document.

As the Text Analytics API is built on Azure Machine Learning, you don’t even need to supply your own training data – its already been trained by Microsoft!

Sentiment Analysis

This returns a score between the range of 0 and 1. A score closer to 1 suggests a more positive sentiment whereas a score closer to 0 suggests the text contains negative language or phrases.

Key Phrase Extraction

Simply put, this API returns a list of strings that form the key points in a string.

Language Detection

With support of 120 languages, this API returns a confidence scoring between 1 and 0 to indicate how confident the API has been in identifying the language being used in a stream of text.

Entity Identification

Being able to identify entities such as places, people or organisations in streams of text can be valuable, you can use this API to do just that.

A few years ago, prior to the release of the Cognitive Services APIs, I had to apply sentiment analysis and name entity recognition to Twitter data. Being able to plug in the Text Analytics API would have saved me weeks, if not months of work!

Integration

To integrate this API, you need a Cognitive Services API account with the associated API ACCESS key and credentials. After you provision these, you then have two options:

• Consume the API directly in C# using the HTTP endpoints
• Use the Azure.CognitiveServices.Language SDK. This takes away some of the additional plumbing that you need to handle when calling the service and/or processing JSON the API will return.

Consuming the API using HTTP endpoints is done using standard POST and GET calls, naturally, you can use any language you’re comfortable with to construct these calls. You define the resource you want to use in your request (sentiment, keyPhrases, languages or entities), supply the endpoint API URL and go!

For example, here you can see the endpoints location in the US:

The service endpoints differ from region to region to make sure you pick one that’s most suitable for you.

Consuming the Text Analytics API using Postman

With Postman, you can create a request in just a few mouse clicks and see the API in action. In this example, you can see a request I’ve built that invokes the Text Analytics APIs sentiment endpoint:

Then, in the Body section of the request, we supply the text we want the Analytics API to classify, “Microsoft Build was great this year!”.

Finally, we send the request we’ve just built over to the Text Analytics API in Postman and get the following response:

In the JSON response you can see that the score of 0.89 was returned by the Text Analytics API after being processed. I think you’ll agree, that “Microsoft Build was great this year!” is a positive statement and the API has successfully identified this!

Consuming the Text Analytics API using C#

Postman is a quick way to test the API and get a feel for the endpoints, credentials and parameters you need to send. This is fine, but for the developers out there, here is some sample code that consumes the Text Analytics using C#.

For simplicity, I’ve created a Console Application and used the NuGet Package Microsoft.Azure.CognitiveServices.Language. This NuGet Package makes is simpler to invoke the Text Analytics API and encapsulates the calls you need to make in an easy to use client. (Note: this NuGet Package is versioned as Pre-release.)

In the following code sample, a few things happen:

1. We supply the credentials and add these to the request header.
2. We specify the Cognitive Services region that’s been configured.
3. An instance of the Text Analytics API client is instantiated.
4. The text we want the API to classify is supplied.
5. We loop through the results that are returned by the API and display these in the console.

In the following screenshot, you can see the output of the console application and we can validate that “document 0” (I had a great session at the gym this morning!) has been correctly identified as positive in sentiment as it has a score of 0.89, whereas “document 1” has been categorised as being negative in sentiment and has a score of 0.05.

In this simple example, we’ve seen how we can pass string to the Text Analytics API in C# with just a few lines of code and use it to determine the sentiment. Consuming the other endpoints is just as straightforward.

I wish I had this API a few years ago as I had to build a Bayesian Classifier, gather training data, pre-process it, and remove stop words before I could begin to identify the sentiment in Twitter data.

You get all of this out of the box with the Cognitive Services Text Analytics API which saves valuable time!

Final thoughts

Big Data, user and system generated datasets are growing exponentially and show no signs of slowing down. There is often too much data to process using conventional techniques, this, coupled with varying data structures can make it increasingly difficult to surface actionable insights for businesses.

Using the Text Analytics API, you can find these insights quickly by integrating it with your existing software products.

• Run a service desk? Use the API to identify key phrases in support tickets to help you reconfigure resourcing.
• Developing chatbots? Use the API to identify the mood throughout a conversation and help determine breakdowns in conversation.
• Processing social data? Use the API to surface key topics, phrases and mood. Use these insights to drive your marketing activities.
• Want a litmus test of your internal corporate email? Use the API to identify problematic communication or key topics being discussed.

These are just some ideas, I’m sure you’ll have your own!

Summary

In this blog post we’ve looked at the Microsoft Cognitive Services Text Analytics API in more detail.  We’ve seen how easy it is to integrate with your existing software projects and explored how it can help your product of business find meaningful insights in your data.

We’ve also looked a sample application and seen how easy it is to consume the API using C#.

Are you using the Text Analytics API in any of your projects?  What has your experience been?

The Grey Matter Managed Services team are Azure experts and can be contacted to discuss Azure options and costs: +44 (0)1364 654100.