Responsible AI FAQs for Factory Safety Agent in Copilot Studio (preview)

What is Factory Safety Agent in Copilot Studio?

Factory Safety Agent in Copilot Studio is designed to enhance workplace safety for factory personnel. This low-code, easily deployable copilot aims to showcase the potential of Copilot Studio, inspiring partners to create more templates. Seamlessly integrated within the Microsoft 365 ecosystem, these copilots can be utilized in everyday tools like Microsoft Teams to provide answers to Occupational Health and Safety guidelines. Additionally, the agent offers features to streamline safety inspections and personalize safety training for the workforce. 

The agent can't answer questions outside of the knowledge information provided during setup and sticks to providing answers to questions based on it. 

What can Factory Safety Agent in Copilot Studio do?

Factory Safety Agent in Copilot Studio is built on Azure OpenAI GPT-3.5. Trained on a vast number of text samples from the internet, GPT-3.5 generates new text that looks and sounds like text written by a human. System also uses AI search technologies to get right responses from the knowledge information provided. 

During Setup, the admin/publisher can provide different knowledge information like internal/public websites, documents related to health and safety guidelines in using a SharePoint folder or directly uploaded in the Copilot's knowledge Base. Factory employees can then ask questions and get answers. We also provide an image to text capability using Azure Cognitive Search useful for converting handwritten notes during use cases like safety inspections to be converted into digital format with lesser friction. To use this capability, the publisher needs to update the Image to Text topic with their Azure Cognitive Search deployment details and a link to SharePoint folder where the factory persona can attach an image of their handwritten notes.

What is the intended use for Factory Safety Agent in Copilot Studio? 

The intended uses of the Factory Safety Agent in Copilot Studio are:

• Answering Safety related Queries - Extracting and summarizing data from public website and documents uploaded. These knowledge sources are to be specified by user.

• Summarizing the chat and preparing an email draft for user.

• Streamlining safety inspections - Convert image to text and use it for different use cases like for safety inspections, incident reporting etc.

• Employee Trainings - Helps to create content for Training Quizzes based on attached documents

How was Factory Safety Agent in Copilot Studio evaluated? What metrics are used to measure performance? 

Microsoft has an evaluation stage as part of our design and development process. Microsoft uses a set of evaluation methods that include a collection of sample questions, answers, and relevant documents to test the quality of the AI answers vs the human answers baseline.

We evaluate the solution by following Microsoft quality guidelines. It's advisable for customers to conduct an evaluation check for each new implementation, as the product is currently in preview, and we warmly welcome feedback. It's important to note that the evaluation is conducted thus far for English.

As part of the development process the product is going through a Responsible AI assessment following Microsoft RAI guidelines.

Features in the product also go through multiple quality reviews such as privacy review, security review, and threat model reviews before launching to customers. 

What are the current limitations of Factory Safety Agent in Copilot Studio? How can users minimize the impact of these limitations when using the system? 

Factory Safety Agent in Copilot Studio is built on GPT-3.5 models in Azure OpenAI. Underlying technologies are trained on wide range of sources, and some answers might not seem perfect. It's important for customers to test it with internal users and provide feedback.

Factory Safety Agent in Copilot Studio undergoes two-step process of launch- Preview, and General Availability. Since Factory Safety Agent in Copilot Studio is in preview, we recommend users to use the solution in test environments only. 

Admins can minimize the impact by providing access to limited set of users. 

What operational factors and settings allow customers to use Factory Safety Agent in Copilot Studio effectively and responsibly?

There are no extra settings provided to end user, which can impact the system behavior. For content filtering, we're relying on Content filters (Preview) provided by Azure OpenAI studio. Refer to the details here, https://go.microsoft.com/fwlink/?linkid=2236519.