Anomaly Detection

This page describes the Anomaly Detection Alert.

Anomaly Detection is one of the possible Data Analysis techniques that Crystal can apply to Topics to monitor changes in time, and for which you can receive a specific Alert notification.

This is how it works.

What is Anomaly Detection

Pills of Data Analysis

Anomaly Detection is a Data Analysis technique used to identify patterns or occurrences that deviate significantly from the usual or expected behavior of the given dataset.

Some types of anomalies can be errors, outliers, or unusual patterns.

You can use this type of Alert when you don’t have a precise data change in mind but are interested in keeping an eye on a specific Topic, in general.

After setting this type of Alert, Crystal’s expertise will detect the type of Anomaly out of several possible options.

Set an Anomaly Detection

There are two alternative methods to set an Anomaly Detection:

• Asking in a Conversation

• Setting it from a Topic Card

Remember

As of now, the maximum amount of Anomaly Detection Alerts that you can set is 10.

Ask in Conversation

You can set an Anomaly Detection by directly asking Crystal in a conversation, either via text or voice.

Simply ask:

1. "Notify me / advise me / let me know etc" + "about"

2. Specify the Topic of interest

3. Specify Filters, if any

Please Note

• Alerts are filter-sensitive, meaning that if the monitored Topic has Dynamic Filters applied, it will be added to the Alert (however, this is not true for Time Ranges!)

• You do not need to ask about the Topic of interest on the Advisor for this request to work, as Crystal will understand the scope of the request based on your question alone.

Examples

• "Notify me about sales by week over time"

• "Advise me about sales by week over time for Italy"

Ask for an Anomaly Detection

Remember

This capability is only available for Configured Topics, whereas Auto-generated ones don't support it.

Set from Topic Card

You can also set an Anomaly Detection from the Topic Card, like this:

1. Ask for the Topic you want to monitor.

2. Once the requested Topic returns on the Advisor, navigate through the Topic Card and click on the "Alert" icon on the bottom.

3. Choose the correct Alert Type - in this case, "Anomaly detection" - and click on it to select it.

4. Click on “Select” to confirm and start the Anomaly Detection on the chosen Topic.

Set Anomaly Detection from Topic Card

Remember

This capability is available for both Configured Topics and Auto-generated ones.

Check for the Activation

In both cases, regardless of the activation method you choose, Crystal will give you a feedback on the result of your request.

Feedback on Request

These are the possible scenarios:

• Alert set correctly

• Alert already active

• Alert limit reached

• Technical errors (not enough data, not applicable, no network, etc...)

• Disambiguation: choose the correct Topic to continue (if asked in a Conversation)

To make sure that you set the Alert correctly, you can also check for it in the Alerts Section of your Account: you should find it among other active Alerts.

Check for Alert Activation

Once the Anomaly Detection is active, you simply need to wait until Crystal detects an anomaly.

Receive an Anomaly Detection Notification

When Crystal detects an Anomaly, you'll receive a notification in the Notification Center.

Please Note

Sometimes you won't receive any Anomaly Detection notification: don't worry, it just means that your data did not undergo relevant changes (which can be good news!)

This is what happens in this case:

1. When the notification arrives, open the Notification Center and read it: this notification signals a data variation (of any sort).

Detected Anomaly

1. To delve deeper, simply tap on the notification to be redirected to the Advisor.

2. You will now see the Topic monitored with an intro message that accurately describes which kind of anomaly has been detected. Read the text and explore the analysis!

Anomaly Description

From exploring your analysis, you can then make better decisions by quickly adapting to the turns of events!

How Does It Work?

This is how Crystal detects anomalies.

Once an Anomaly Detection Alert is set, Crystal immediately captures a first Data Snapshot, i.e. a picture of the status of the dataset underlying the Topic.

This first Data Snapshot represents the starting point from which Crystal will check all historical changes within all the Entities in the Topic.

To check historical changes, Crystal applies a strategy that modules the frequency of the checks based on frequency of the changes observed.

For example, if frequent data changes are observed, Crystal will continue to check at a certain frequency, whereas if the data don't change much, the frequency will decrease.

Rules

• Anomaly Detection is available for all existing Topic Objectives.

• For each Topic Objective, Crystal is capable of detecting a wide range of possible anomalies.

• If Crystal detects more than one anomaly in the data, you will only receive one Alert notification, related to the anomaly with the highest variation in the data.

• For Lists and Rankings Objectives, Alerts will continue to search for anomalies across the entire dataset, even if there are more than 1000 rows.

• If an Anomaly Detection is requested on a Topic with one or more Dynamic Filters, Crystal will only consider the filtered version of the Topic over time and not its original, unfiltered form.

• If an Anomaly Detection is requested on a Topic with Time Filters, Crystal will consider all historical Topic in its original, unfiltered form.

Limitations

• There's a limit of 10 Anomaly Detection Alerts running simultaneously.

• You can request Anomaly Detections on one Topic at a time only.

• Be careful when modifying or deleting a Topic with an active Anomaly Detection, as the following exceptional scenarios may happen:

  • If the Topic gets modified and re-published, this may either generate an Anomaly, generate an error, or not have an impact at all, depending on the specific modifications applied

  • If the Topic gets deleted, the Anomaly Detection may generate an error, without sending any notification.

Anomaly Catalog

Below is a complete list of the Anomalies that Crystal can detect.

Please Note

This list includes the general list of Anomalies along with their description.

However, in practice, the types of Anomalies for each Topic strictly depends on the Topic Objective.

N.	Anomaly	Description
1	New Entries on Main Group By	On a specific date, there was a change in a list, with the number of unique values in a key category increasing by a certain amount.
2	New Drops on Main Group By	On a specific date, there was a change in a list, with the number of unique values in a key category decreasing by a certain amount.
3	Main Group By Updates	On a specific date, a list experienced a shift when a certain number of unique values in a key category was added, while a different number was removed.
4	Mean Percentage Variation	On a specific date, a key metric experienced an unprecedented percentage change, marking a notable shift since the start of monitoring.
5	New High or Low in the Mean	On a specific date, a key metric reached a milestone value, a level never observed since the start of monitoring.
6	Top 3 Updates	On a specific date, there were notable shifts in a ranking based on a key metric, which highlighted a particular entity's significant upward movement and the decline of another previously higher-ranked entity.
7	Top 3 Positive Updates	On a specific date, there were notable shifts in a ranking based on a key metric, which highlighted the movement of specific individuals on the list.
8	Top 3 Overtime Updates	On a specific date, there were notable shifts in a ranking based on a key metric, highlighting a particular entity's significant upward movement and the decline of another previously higher-ranked entity.
9	Proportion Change	On a specific date, a key metric for a designated category experienced a notable change, emphasizing both absolute and proportional variations.
10	Clustered Proportion Change	On a specific date, a metric for a designated category experienced a significant shift, capturing both absolute and relative changes.
11	Ranking Change	On a specific date, one metric surpassed another in ranking, emphasizing the shift in values and positions of both metrics.
12	Breakdown Ranking Change	On a specific date, within a particular category, one metric that was previously lower in value surpassed another, marking a notable reversal in positions.
13	Correlation Change	On a specific date, there was a notable change in the correlation between two topic metrics.
14	Interesting Event	On a specific date, a metric displayed a value outside the anticipated range, exceeding the expected upper limit.

Keep analyzing your data!

---