AI Email Classification Performance Report

The AI Email Classification Performance Report displays information relating to EnateAI Email Classification, specifically showing how well this AI is performing in assessing and accurately classifying incoming emails which launch Tickets. Users may use this information as input into deciding if any adjustments should be made to AI settings, e.g. confidence levels set for requesting manual input.

Setup required to use the AI Email Classification Performance Report

In order for the AI Email Classification report to show data, the corresponding ‘EnateAI Email Classification’ integration must be enabled in your system. This can be done in Enate's Marketplace in Builder. When this integration is running, incoming emails which result in a new ticket will generate the data needed for the report.

It is important to note that data displayed in this report relates only to:

• Incoming Emails that trigger Ticket creation AND

• which have been classified by EnateAI Email Classification.

Accessing the AI Email Classification Performance Report

Access to and the ability to edit the AI Email Classification Performance Report is controlled via User Roles, in the same way as all other reports within Enate. To edit the report users will need the 'Create Custom Reports' feature enabled.

Here are some of the type of information which can be found within the report

AI Classification Performance & Accuracy Report Structure

Tickets from inbound emails

This graph displays the number of Tickets created by incoming emails.

Percentage of emails assessed by AI

This graph displays the percentage of incoming emails that created Tickets and were also classified by EnateAI Email Classification.

AI predictions accepted

This graph displays the number and percentage of AI classifications that were accepted by users.

AI predictions overridden

This graph displays the number and percentage of AI classifications where the Ticket Category selected was ultimately changed by an Agent.

Accuracy by AI confidence level

This graph displays the accuracy percentage levels across various level of AI confidence level. For example: “Ticket Categories selected with a confidence level of less than 50% were overridden 76.2% of the time. Ticket Categories selected with a confidence level of 80-90% and over were accurate 92.3% of the time”. This data is a good way to assess how accurate the AI’s assessment of its own findings are.

AI volume and accuracy trend

This graph displays the volume of Tickets categorized by EnateAI Email Classification over a given time period, and the Accuracy levels for them, allowing you to view how the AI’s accuracy is changing over time.

Top AI predicted L3 categories

This graph displays the most frequent Ticket categories predicated by EnateAI Email Classification, split by correct vs re-categorized outcomes.

Most overridden L3 categories

This graph displays the AI predicted Ticket categories that are most frequently corrected by users.

What if EnateAI Email Classification has not been enabled?

If you try to access the AI Classification Performance Report but do not have EnateAI features enabled, you will not be able to view the report and will be met with a pop-up explaining you will need to enable EnateAI features to access it.

If you try to access the AI Classification Performance Report having enabled EnateAI features but not the EnateAI Email Classification integration, you will not be able to view the report and will be met with a pop-up explaining you will need to enable the integration.