Anomaly detection analyses the traffic to your domain looking for sessions (tokens) representing anomalous behavior. This can help to identify anomalous behavior stemming from:
- Bots
- Human users attempting to game the queue or your website
- Sessions engaging in collaborative behavior, such as token sharing between users
- Legitimate software services getting caught up in queues
Our anomaly detection algorithms use statistical analysis to analyze the traffic in real time, using a variety of metrics including:
- Variety of URLs requested, and tendency towards URLs of interest to bots
- Velocity of requests
- IP addresses, language and agent strings associated with tokens
- Counts of tokens originating from associated IPs
- Session age
- Geolocation
- Agent string anomalies
- IP reputation, and flags from similar IPs from other sites using CrowdHandler
These factors are used to build up a picture of statistical norms for your website and waiting rooms. Anomolous sessions are then flagged. You can investigate individual sessions to take decisions in individual cases, or set a threshold to ban IPs exhibiting anomalous behavior.
Getting Started
Anomaly detection is available on the Professional and Enterprise plans. To find anomalous sessions log into the Admin panel and click on the Sessions tab.
The sessions screen will show you all users on your website or in a waiting room queue, in queue order. The Risk score (from 0 to 100) indicates how anomalous particular sessions are.
You can use the minimum score filter to zone in on the most anomalous scores. (For example, set it to 50, and hit refresh, to find all sessions scoring above 50).
Or, you can sort your sessions using the anomaly column, so that the most anomalous sessions are at the top of the screen.
Investigating Sessions
Other than the anomaly risk score, you will see some other information about the session that can help you indicate how risky this session is. That includes details about the agent screen, and information about the IP, including geo-location, ISP, and whether the IP is originating in a data-center, or using a network such as TOR, or is listed on any abuse databases.
Click on the magnifier icon, to look at the session in more detail. This will show you the history of the session; polls from waiting rooms, progression through the queue, clicks on URLs, changes in IP and agent information. You will also see a breakdown of the risk factors contributing to the risk score.
Taking Action
You should be in a good place to take a decision about whether this is a legitimate session or a dubious one. Based on your view, there are a number of instant actions you can take from this screen:
Delete the token
If you have an active queue, deleting the token will result in this user being put to the back of the queue. This will happen whether they have already made it through the queue and onto your site, or if they are in the queue but progressing toward the front. The user will not be notified with a reason, but will find themselves at the back of the queue. If you have a substantial queue, and you want to speed up transit for your legitimate users, this is the best action to take.
Block the IP
Blocking the IP will mean that any session originating from this IP will be blocked, including existing live ones. Users will receive a waiting room page with a blocked message. This is the best action to take with regard to dubious behavior when you do not have an active queue. For example, scraper and spinner bots. Blocking an individual IP may not be effective where the token is originating from multiple IPs, although automated IP blocking can help in this scenario.
Ignore the IP
We're detecting Anomalous behavior. As well as bad actors, this may flag legitimate services that are making requests to your website. For example: Uptime monitoring services, payment gateways confirming transactions. In those cases you may wish to set the IP to be ignored or bypassed. As well as preventing these services from being caught up in queues, this will stop tokens from those IPs being flagged by anomaly detection in the future.
Automated Protection
Anomalous sessions and bot traffic make up a high proportion of traffic to typical websites. So keeping on top of individual sessions is likely to become tiresome quickly. So we recommend using the Firewall controls to automatically block sessions with an anomaly score above 95% and monitor regularly to see if you can take that threshold down.
If you choose to not block at all based on Anomaly thresholds, for fear of false positives, then that intelligence will still be used to inform IP block recommendations, and to help maintain CrowdHandler's Global block rules (which you can opt into.)