Email Receipt & Deduplication
Details regarding email deduplication - common Email Server behaviour with how they can create duplicate emails, and subsequent behaviour in Enate
As part of day-to-day processing in Email systems and integration with other systems, it is possible for errors to result in duplicate emails. To combat this Enate provides built-in capabilities to deduplicate received emails. Let’s take a look at the details for this.
As a core principle, Enate will generate or update 1 Work Item per received Email after deduplication.
To guard against accidental creation of multiple work items caused by duplicate incoming emails, Enate checks for such duplicates and deduplicates them so that only one work item is created. This is how Enate is set up as standard, and is termed ‘Global Level deduplication’.
However in some business operations, it is possible that the same exact email will be explicitly forwarded into two different parts of the business, with the intention of creating two different work items in Enate. In order to allow this, Enate supports the following options:
- Global Level Deduplication (default) – where emails deduplication logic is run globally on all emails, and any mails considered duplicate are always deduped.
- Per-Connector Level Deduplication – where emails are deduped if received into the same individual Email Connector, but are not deduped if the two mails are sent into two separate Email Connectors in Enate.
In Per-Connector mode, Enate will only search for existing Emails that have been received to the same Email Connector as configured in Enate.
In a scenario where a sender includes multiple addresses in the recipient list (To and CC) which are picked up by Enate, the system would normally receive 2 (or more) copies of the email, and the Email routing outside of Enate would determine whether it is caught as a duplicate or not. However, the duplication taking place in the Email Server infrastructure cannot be guaranteed.
Regardless of Global vs per-Connector deduplication, 2 (or more) copies of the email must be generated by the Email Server infrastructure otherwise the core principle stated above results in just a single Work Item.
Whether multiple emails are created or not in this scenario is outside the control of Enate and should be investigated with your Email Server provider or Email administration team. In our experience factors which may affect this include (but are not limited to):
- If the email is sent by internal or external senders
- If the recipients are real mailboxes or aliases
- If transport rules are used (Exchange/Office 365)
- If 3rd party antispam/antivirus products are involved in email delivery.
When Enate receives and processes multiple copies of the same Email, but with different headers, then multiple Work Items will be created. However each Email is necessarily processed and is completely unaware of the others.
- If the Emails are received into the same mailbox, then the same Enate routing will be applied resulting in all the created Work Items being in the same context (and Ticket Category for Tickets).
- If the Emails are received into different mailboxes, then each will follow the configured routing within that mailbox - these may obviously be different, resulting in the created Work Items potentially being in different contexts (and Ticket Categories for Tickets).
This limitation is caused by Email standards outside of Enate’s control, as explained in the Example below.
When an email is received because it has been BCC’d to the email address of the mailbox, or one of its aliases, configured in Enate, the system may not be able to route the email. This is because the BCC (Blind Carbon Copy) details are intentionally hidden from all recipients as part of standard Email behaviour. The result of this is that Enate receives an Email that does not identify itself as being destined for any configured Email address.
In this scenario an automated rejection Email will be sent informing the sender that they emailed an unconfigured address.
If the email is sent by a colleague (an internal email) then it is likely, depending on Email Infrastructure, that Joe would receive 1 email, Tom would receive 1 email and Kevin would receive 1 email.
If the email is sent by an external contact (an external email) then it is likely, depending on Email Infrastructure, that Joe would receive 2 emails, Tom would receive 1 email and Kevin would receive 1 email. Joe would see 2 identical emails in his Inbox with no discernible differences; only when viewing the Email headers would differences be identified and even then it is unlikely either email could be attributed to [email protected] vs [email protected].
In both examples Kevin would receive the email but wouldn’t really know why – he would not be able to identify himself as a recipient in the To or CC fields and the BCC field is not shown.
The impact of these 2 examples if received by Enate is that for the first example (sent from internal):
In the second example (sent from external):
- 2 emails would be received in the mailbox [email protected] which would follow the configured routing rules resulting in 2 Work Item in the same Context/Ticket Category. This is because Enate looks for a Routing Rule by evaluating the combined To/CC list reading from left-to-right with To taking precedence over CC. There is nothing in the two emails to identify each one as being intended for a different address; in that regard they are identical.