How to use the Bing APIs to make your own business search better

Last year, Microsoft announced that organisations could customise Bing to work better for their business, showing internal information as well as results from the public web, with the option to customise results for specific searches where there was information the business wanted to highlight – like making sure everyone got the details of the latest version of a product instead of older lines.

Some of the most useful information in the customised Bing views turned out to come from Office 365, like being able to see how you were connected to someone through the business org chart, or finding relevant documents from internal file shares. Some of the 180 customers Microsoft worked with on the private preview of what was called “Bing for Business” wanted to be able to bring in more information from Office 365 like being able to search conversations inside Teams and Yammer. Seth Patton, general manager for Office 365, called Bing for Business the precursor for the unified Microsoft Search in Bing announced at the Ignite conference recently, which includes the same search tools for people, documents and internal tools but also has a much broader scope and more opportunities for developers.

“We started with the Office Graph, and then the Microsoft Graph and now bringing in the Bing AI technology as well as connectors to third party tools. That means over time, we really do unify all the entities and the signals so that we can use AI as a way to drive contextualised and personalised search both within the end points in Microsoft 365 – so Windows, Edge, Microsoft Search in Bing, as well as Office and SharePoint, having a single experience. We’re going to introduce a new unified search header in Word and PowerPoint and Excel. We’re going to deliver it in a consistent way but also in a way that’s personalised and in context for you.”

The first connectors are for software like Wrike that integrates with Office, but developers will be able to build connectors for internal line of business applications. Connectors are one part of personalising the experience not just to individual users but to the business, just as context is about what you’re doing but also where you work, Patton explained.

“It’s in the context of the app that you’re in; knowing that you’re in Word, we’re surfacing Word files or document elements before other files or other sites and people in your search results. But the other piece of it is being able to bring not just what you have traditionally in search – files and sites – but also bringing in people and also line of business information, and that’s where the connectors come in. The Microsoft Graph is both the place where entities are, and we apply AI and surface those in context across those end points, but then we also have an API that developers can add information to and extend.”

One of the reasons that enterprise search is more complicated than standard web search is that you don’t have widespread patterns of behaviour showing what websites people visit to get information about specific topics for your internal documents. But Microsoft Search in Bing can use the information in your Office 365 tenant – the org chart in Azure Active Directory, the groups and teams people participate in, document contents and authors, email contents and the pattern of communications in email and the other Office tools – to learn what’s relevant. “We’re even starting to get smarter about acronyms; we know that GDP means one thing in one organisation and something different in another one.”

There are also new Cognitive Services for developers who want to use Bing search features inside their own applications, in particular in the Bing Visual Search API and the Bing Custom Search API.

Bing Visual Search extends Bing’s image interpretation capabilities to the API. Developers can build applications to provide visually similar shopping resources and similar images along with image annotations. This offering also provides capabilities such as OCR, barcode/QR code detection, and celebrity, landmark, animal, flowers, and other object detection.

Bing Custom Search v2 already covered searching for images on your own domain, complete with autocorrect; the v3 release can search for video results from the public web sources you’ve selected in your custom search instance (which can be specific pages, subpages, domains or subdomains). If you want to pin a specific site to the top of the search results for a particular query, perhaps to highlight internal or official resources, you can now do that for queries that start with or end with phrases and general terms rather than the default of only exact matches (although this doesn’t yet work for image or video searches). You can turn off the autosuggest option that suggests alternative search terms but if you only want to stop autogenerated suggestions for what terms to use for specific queries – if your users will be typing some very specific searches and the autogenerated suggestions aren’t going to be helpful – you can now block just those from being shown.

You can also customise the three colour themes (classic, dark and skyline blue) by setting the RGB HEX value for specific colours in the theme.

Bing Local Business Search API, available in public preview in the US and soon UK, is the latest addition to the Bing APIs portfolio. It helps find local businesses in the specified region of interest and provides access to information on millions of local businesses from various categories such as Food & Drink, Professionals & Services, Healthcare, and others. Business information contains full name, address, phone number, website, business category, and latitude/longitude. This API could be helpful to build various experiences such as validating local business information, contacting a local business, or providing navigation to the place of interest using Bing Maps.

Developers building conversational apps or online forums can now enrich conversations by letting users search for GIFs to express their feelings with, by calling the GIF search in the Bing Image Search API using C#, Java, Node.js, PHP, Python, or Ruby. Set the image type in the API call to imageType=AnimatedGif, or AnimatedGifHttps to restrict it to results available via https (iOS apps need to use HTTPS for connections to web services).

The advantage of Bing GIF search is that it removes a lot of the offensive, adult content that can otherwise show up which would be inappropriate, especially in business applications. Bing uses computer vision to look for content that’s unsuitable or offensive, and common queries are reviewed by human moderators to make sure the results are safe to use.

The same computer vision tools help to rank GIFs to put the most relevant but also most engaging animations at the top of the search results. Word vectors look for related concepts, OCR makes the text in GIFs searchable (even if the display of a word is animated so not all of the letters appear on every frame), sentiment analysis looks for a matching emotion (including emoji), facial analysis helps to detect exaggerated faces and even looking at the poses, actions and expressions in the image.

If you have questions on Office 365, Bing APIs, or Azure related projects, the Grey Matter Managed Services team can be contacted on +44 (0)1364 654100.