I'm working on a new garage project, or better said "cellar project", as that's the place my home office is actually located, which I want to share with you. I gave it the code name "EMMO".
It's all about making deep sense on Microsoft Teams conversations in different media formats by unlocking the potential of the Microsoft Cognitive Services pre-built AI models.
Enterprises are communicating in a wide range of internal and external digital channels. EMMO easily connects to them all and starts to listen to conversations for further analysis.
Conversations include text, spoken and video formats, so we are not limited to text and broke the language barrier.
Modern and cloud based real time data processing technologies are making sure your conversations are safely collected, stored and even encrypted while processing and data retention period. Our service is running fully on Microsoft Azure and is available in any Azure region worldwide.
Use case:
Let's have a look, what we want to implement and the use cases the solution is covering. Then we walk step by step trough the sample setup.
Users have conversations in Microsoft Teams channels in text, audio and video.
Messages are captured are centrally stored to make the data work later on.
Audio and video messages are transcribed to text and translated if the input language is not English.
Messages are analyzed with cognitive skills and key characteristics like language, keywords, sentiment, entities and more are extracted.
Data is enriched with more user and context data like gender, position, office, teams channel, timestamp and other relevant properties.
Enriched data is stored and made action ready
Data alerts notify the user about relevant data driven events and make data actionable.
Data is visualized in Power Bi dashboards and made available for Power Bi Auto QnA Experience allowing to talk to data in natural language.
A little preview of some of the fun we are making with the conversation data:
Technologies
So, what to we need to build a solution to make deeper sense of internal or external communications in more or less real time?
Azure Event Grid
Azure Table Storage
Azure Cognitive Services
Text Analytics
Sentiment Analysis
Speech to Text
Video Indexer
Microsoft Teams
Microsoft Flow
Microsoft Power BI
Microsoft Bot Framework
Table of contents
Implementation steps
Our solution consists of multiple worker Flows acting a micro services in the overall solution architecture.
Think of "Worker Flows" as small, dedicated Flows, executing specific process steps or activities. Kind of like a function in grown-up software development. As all state logic is managed by the Controller, the main idea of a Worker Flow is to encapsulate actions and groups of actions into small, stateless sub processes.
I use Flow in this sample, obviously this has some limitations. Depending on your scale needs you would make use of Graph Data Connect, Cosmos DB, Azure SQL DWH, Azure Data Factory and Azure Event grid for example.
The focus of the article is on Cognitive Services and specifically the Text Analysis API and Flow is a wonderful tool to showcase the overall concept and orchestrating the different Lego bricks.
Step 1: Create Azure Event Grid Domain and configure a new Topic
Step 2: Create the Flow to capture the Microsoft Teams conversations
We call this first micro service the "Conversation Listener Flow". What it basically does is listening to new messages in Teams and capture them as JSON message to send to an Azure Storage Queue for analysis. If you have Logic Apps available, this could favorably be implemented there, as it's a backed only process and doesn't have any user interaction.
Flow provides us with the trigger "New Message in Teams Channel". This is a great and cost effective option to capture conversations, as this connector is included in the standard connectors with an Office 365 E3 plan and doesn't cost extra. Scale is limited, but additional Flow runs are much cheaper than P1 for each user.
This Flow has a very clear and easy to read and understand structure and some basic error handling implemented.
Connectors:
Microsoft Teams, Azure Queue Storage
Trigger: New Message in Microsoft Teams Channel
Payload: JSON Message containing conversation data
Please find a sample message JSON here, so you can just copy it right away and adopt it to your needs.