Use Azure OpenAI And ChatGPT with Azure Cognitive Search
Create FAQ Chat App
Deploying a FAQ chatbot is a great way solve common issues and problems encountered by employees. Chime V5 provides a full FAQ system, with a chat service and connections to all Azure AI services. Deploy a Q&A chatbot, from the Microsoft Teams Store, to solve employee issues and immediately improve your existing service desk.
AI can create text, image, audio content, and produce enterprise FAQ chat bots and Q&A chat applications. Developers all across the world are incorporating AI into their applications for the benefit of their users. New AI models and SDKs pop out of the woodwork, it can be challenging for the enterprise to figure out how to get started and use it all especially when their curated content is located within SharePoint, ITSM knowledge bases or internal content management systems.
Microsoft Azure OpenAI offers high-performance AI models at a production scale, boasting industry-leading uptime. Complementing this, Chime provides enterprises with a platform to expediently leverage their pre-existing curated content to train Azure OpenAI models. Chime's chatbot understands natural human language and responds with content tailored to the enterprise, ensuring responsible, specific, and privacy-conscious interactions. Importantly, enterprises maintain complete control over the content delivered to their audience, preserving their unique brand identity and messaging. Here is an example of how Chime uses FAQs as part of a university deployment to populate ChatGPT chat responses as well as display a portal UI.
Bring Your Own Data: Azure OpenAI and Cognitive Search
Step 1: Preparation
Before you embark on this journey, ensure you have the following components ready:
Azure Storage Account: This serves as the repository for your data.
Azure Cognitive Search Account: The foundation for creating an index and enabling efficient searches.
Azure OpenAI Service: A crucial tool to utilize ChatGPT for enhanced responses.
Step 2: Creating an Azure OpenAI Service
Create a FAQ ChatBot and chat system using FAQs and Azure OpenAI with a cognitive index
Use our CMS to create a high value Q&A Chatbot, using your FAQs, to help answer employee questions and solve issues
Initiate the creation of an Azure OpenAI service, opening the doors to a world of possibilities. You can use the same service for all tenants in various environments or set up separate services based on your unique requirements.
Step 3: Creating a Azure Storage Account
Azure AI Playground is an easy way to build a FAQ Chatbot
Take advantage of new AI services, use your FAQs as content, and even import SharePoint articles to create a FAQ chatbot
Set up an Azure Storage Account, providing a structured foundation for data storage. You can create multiple storage accounts to segment data for different Chime tenants, but each tenant should have two uniquely-named Blob Containers within the same storage account.
Step 4: Create a Storage Container
Create an Azure OpenAI storage container to maintain the index
Lots of changes with Azure OpenAI but the work provides massive benefits and immediate value
Within your storage account, create an empty Blob Container. This container is pivotal for the Azure Cognitive Search setup, and it's crucial to assign a unique name to it.
Step 5: Adding a Seed Content File
To kickstart the creation of essential resources, add a "seed" file to the storage container. This file should be in HTML format and contain minimal content. Without this seed file, the Cognitive Search datasources, skillsets, and indexers won't be created.
Step 6: Creating an Indexer for the Container
Our FAQ chat system uses Azure OpenAI index as a tool to help answer employee questions
Index your company FAQs and KB to quickly create a Q&A chat service, in Microsoft Teams, to help answer common employee questions
Set up an indexer for the storage container. This indexer will periodically check for new documents in the container, ensuring that your knowledge base remains up to date.
Step 7: Creating a Cognitive Search Service
Powered By AI generally involves a cognitive index - as with most things, these are provided as tiers in Azure
We automatically deploy an Azure Cognitive Search index to create our chatbot service desk application -
Now, create a Cognitive Search service. You can choose the Basic tier to handle up to seven Chime tenants. This service will be used to build the knowledge base for ChatGPT.
Step 8: Setting Up the Cognitive Search Service
Add the power of AI - or just index your FAQs - to add chat to your service desk
The Azure Cognitive Index is new and very powerful - it starts to incorporate OpenAI across Azure
To configure the Cognitive Search service, you can utilize a user-friendly wizard available in the Azure OpenAI Chat Playground under the "Add your data (preview)" tab. Here, you'll need to connect your Azure Storage Account, Blob Container, and Cognitive Search service, streamlining the setup process.
OpenAI makes a high value addition to Azure
By combining a standard index, with a base LLM, FAQ chatbots are high value and easy to build using our product
Enable various search types, including keyword-based searching, semantic search, vector search, and combinations of these.
For a comprehensive knowledge base, it's recommended to enable all available options.
Review your selections and save the configuration. The Cognitive Search resources will be created.
Step 9: Verifying Setup
After completing the setup, you should observe two storage containers in your Azure Storage Account: one for raw documents and another for indexed chunks. Additionally, in the Cognitive Search resource, data sources, indexers, skillsets, and indexes should have been created, signifying that your knowledge base is now operational.
Step 10: Configuring Cognitive Search Settings in Chime V5
With the Cognitive Search setup complete, it's time to configure Chime settings to enable this transformative feature.
Configure Upload Storage Account:
Connect your chatbot to Azure OpenAI
We provide all of the wiring, automatic indexing, and connections to connect your chatbot with Azure OpenAI
Provide the Storage Container name (e.g., example-container) and the Storage Connection String, which can be found in the Azure Portal under the Access Keys tab. Test the connection to ensure its functionality.
Configure Cognitive Search Index Settings:
Connect your Q&A chatbot to Azure OpenAI
Once connected, our application provides all data synchronization, index, queries to the large language model (LLC) to create a great chatbot
Provide the Cognitive Search Endpoint (visible in the Azure Portal's overview blade for Cognitive Search Service), Cognitive Search Key (admin key is required for full access), Index Name (use the augmented index, e.g., example-index), and optionally, the indexer name. Test these settings to ensure a successful connection, and then save the configuration.
Step 11: Cognitive Search Status Page
Provide the secret key to connect your chatbot to the Azure Cognitive Service
It’s your data, your index, and the chatbot will use your FAQs and SharePoint content to provide answers to your employees using chat
From the Cognitive Search Setting Page, you can access the Cognitive Search Indexing Status Page, which shows an overview of your connection to the Storage Account and Search Index, including the number of uploaded and indexed documents. If configured, it displays the last time the Search Indexer ran.
Step 12: Marking Chime Documents as Indexable
Mark your FAQs and have them automatically indexed for chat
Your chat service will now use FAQs for Q&A and help answer employee problems - just update your FAQs and your chatbot will improve
Each content item has metadata attached to track if it should be uploaded. Update content items, and the new version will be automatically uploaded and indexed.
Step 13: Force Reindexing from the Cognitive Search Indexing Status Page
Index the latest knowledge base content in the Cognitive Search Indexing Status page
The indexer can be prompted to run within the service desk Admin UI Azure Cognitive Search settings
If you want to force the index to update, you can do it from the Cognitive Search Indexing Status page. Manually run the indexer to process new and modified documents in the datasource. Refresh to verify that the indexer has run and indexed the new document.
Step 14: Using Cognitive Search in the Chime Agent AI Chat Assist Feature
With Cognitive Search configured and FAQs indexed, you can enable Chime's Agent Assist feature to use indexed FAQ items in generating suggestions for agents.
Enable Cognitive Search in Agent AI Chat Assist Settings:
The first checkbox adds Cognitive Search to Chat GPT service calls.
The second checkbox ensures Chat GPT only uses information from indexed FAQ items. If no relevant information is found, it won't use the base model for responses.
Enable Cognitive Search in Agent AI Chat Assist Settings
Restrict whether your AI Chat Assist uses the base model for responses or only knowledge base content
Step 15: Cognitive Search Agent AI Assist in Action
With FAQs indexed, Chat GPT can provide responses that include information from the indexed documents, along with a link to the original FAQ for reference.
Step 16: Using Cognitive Search with the Chat GPT Pipeline Stage
Enable Cognitive Search for responses from the service desk chatbot
Responses can be restricted to only knowledge base content
You can also enable the Chat GPT pipeline chatbot to use indexed knowledge base items. With these settings enabled, the chatbot can respond based on the knowledge base, ensuring more accurate and relevant responses to guest queries.
Step 17: Customizing your ChatGPT & Cognitive Search Experience
Configure chatbot response settings for the Chat GPT model in the service desk chat flow
This is the area where length, number of responses generated, randomness, and creativity can be adjusted
Set custom parameters from a list of advanced parameters for the Chat GPT Model:
Choice Count: Specifies the number of alternate responses to return for a prompt.
Max Tokens: Defines the maximum number of tokens to generate in the response.
Temperature: Sets the sampling temperature, influencing the apparent creativity of generated completions. Higher values result in more randomness, while lower values lead to more focused and deterministic results.
Top P: Governs the randomness of the response. Lower values make the response more deterministic.
Frequency Penalty: Regulates repetition in the response. Positive values decrease the likelihood of tokens appearing as their frequency increases, reducing the model's tendency to repeat the same statements.
Presence Penalty: Manages repetition in the response. Positive values decrease the likelihood of tokens appearing when they already exist, thereby increasing the likelihood of the model outputting new topics.
Configure the Cognitive Search query FAQ search strictness and how referenced content displays in service desk chat
Enable or disable the MS Teams client pop-up task module window to display referenced FAQs
Set additional Cognitive Search settings to allow for a custom user experience when referencing your documentation.
Restrict Bot Responses to Knowledge Base: When enabled, the chatbot will exclusively suggest responses based on information retrieved from knowledge base resources such as FAQs. If no relevant resources are found, the user will be informed. When disabled, the chatbot can also respond using data from its base model if no relevant information is available in the knowledge base.
Strictness: This parameter determines how closely the chatbot matches searched documents to the user's query. Increasing the value raises the threshold for relevance, filtering out less-relevant documents for responses. Setting this value too high may limit the model's ability to generate responses due to a scarcity of available documents.
Maximum Retrieved Documents: Defines the maximum number of top-scoring documents provided to the chatbot as background information to generate a response. Increasing this value may enhance the quality of responses, particularly if there are numerous short documents in the knowledge base.
Search Query Type: Users can choose between 'Simple' and 'Semantic' search query types.
FAQ Citation Link Options: Users can opt to display cited documents as footnotes, where URLs are included at the end of the message, inline within the response, or choose not to link the source documents at all.
Conclusion
Enriching your AI models with your data using Azure Cognitive Search is a powerful method to enhance the accuracy and relevance of responses. By following these comprehensive steps, you can set up the integration of Azure Cognitive Search with Azure OpenAI ChatGPT to create a more informed and capable AI assistant for your Chime V5 platform. This not only enriches the guest experience but also makes your agents more efficient in providing information and support.