Microsoft • 2018 • User Experience Design
Design of an AI-based acronym search for enterprises
An employee of a medium to large-scale enterprise spends a significant portion of their time looking for documents, asking for files from a colleague, looking for subject matter experts, information on company portals, etc.
Microsoft Search enables employees to search their work-related information like documents, meetings, people, locations, intranet and cloud portals, meanings and acronyms, etc. through any search surfaces in the Office suite.
A new employee or an employee onboarding onto a different team/project needs to get up to speed with the project and vocabulary of the team. It’s not uncommon to find an employee spending a significant amount of time looking for documents, people, or the meaning of internal technical terms.
The Microsoft search system consists of 3 key stakeholders
- Company admin
Can add new acronyms and moderate existing results. - Employee
Employees of the enterprise search and view acronyms. They can also give feedback on search results and suggest additional acronyms. - Development team
Microsoft is the service provider that builds and moderates the AI results based on inputs from other stakeholders.
Designing search answers powered by AI
Design principles for search answers
- Limit distractions
- Be credible and trustworthy
Designing for different confidence levels
Unlike traditional software, systems with AI/ML components are probabilistic in nature and tend to change over time. Their design needs to account for this probabilistic and changing nature.
A confidence level is a statistical measurement that indicates how confident the AI system is about a particular outcome. Design of the enterprise acronym answer adapted to the confidence of the AI system.
Answers that were added by admins (human counterparts) were given the highest confidence followed by the AI-mined answers. Results with very low confidence were not shown at all.
For the purpose of understanding, I’ve categorized the AI confidence levels into 4 types:
- Very high confidence
- High confidence
- Low confidence
- Very low confidence
Very high confidence answer
High confidence answer
Low confidence answer
Very low confidence answer
Feedback mechanisms
User feedback is the communication channel between your users, your product, and your team. Leveraging feedback is a powerful and scalable way to improve your technology, provide personalized content, and enhance the user experience.
The AI system is guided by a reward function that helps the model determine whether the results are correct or incorrect (or a percentage value of how correct it was). The feedback mechanism was designed keeping in mind the goal of improving the AI model.
Privacy and trust
The enterprise search answers appear alongside web search results. This makes it imperative to build trust with users being shown sometimes what could be sensitive data. For the system, trust and privacy were established in two ways:
- Reiterating that their data is secure
- Building explainability into the design
Reiterating that their data is secure
Telling users that their search results are private.
Building explainability into the design
Ensuring that users are aware of the source of a mined acronym to help them make a conscious choice. Acronyms that were published by the company admin were given prominence and relevant attribution.
Other
Adding missing acronyms
If an acronym is not automatically extracted, an administrator can add new acronyms from the company’s O365 admin portal.
Onboarding new users
The team made a conscious choice of only introducing the feature to new users when they typed a related query. For e.g. Searching for a word definition might be the right place to introduce this feature.
Due to the confidential nature of unreleased projects and products, much has been left out.