CAPTCHA is an acronym for “Completely Automated Public Turing test to tell Computers and Humans Apart.” Websites and applications owners use different types of CAPTCHA tests — based on text, picture and sound — to decide whether a user is legitimate or not, and act accordingly. The CAPTCHA is often used as a stand-alone bot mitigation solution that is added on specific endpoints, such as a login or checkout page. It is sometimes also leveraged as part of a feedback loop to complement a bot mitigation solution and served when a user exhibits suspicious or unexpected behavior.
The CAPTCHA concept has existed for many years, but it is continually changing and evolving — much like the bots it is designed to protect against. Up until recently, the CAPTCHA test was enough to determine users' legitimacy. A successful solve means the user is human, and a failed solve indicates a bot or automated tool. Unfortunately, it’s not that simple anymore. Attackers are finding new ways to bypass these tests, sometimes even better than real humans.
In this blog, I’ll follow the “evolution” of CAPTCHA-solving attacks and offer some useful tips on how HUMAN created a strong yet user-friendly CAPTCHA solution.
A brief history of CAPTCHA-solvers
In the past, most bots weren’t able to run JavaScript code and operated via API calls only. Therefore, loading a CAPTCHA from client-side JavaScript was impossible for these “primitive bots.” Another obstacle for bots was predicting the task’s solution, such as changing the test or identifying vague images. According to technical author Phil Haack in 2006, “Most comment bots currently don’t have the ability to evaluate JavaScript and thus will not be able to submit the form correctly.”
That was over 15 years ago, and things have changed since then. In the last few years, the “CAPTCHA Solving” attack vector has accelerated. There are new solving services appearing every day, and hackers can leverage these services in their attacks. Today’s cyber criminals don't necessarily need to know how to bypass CAPTCHAs themselves; instead, they can use a specific solver that they can add to their own attack. Over time, attackers have found numerous ways to overcome both JavaScript and task-prediction obstacles.
How CAPTCHA-solvers operate
There is an ongoing arms race between attackers using botnets and automated tools and the security defense side: we improve our defenses, the attackers improve their offenses, and so on. One of the most important changes we’ve seen in recent years is the ability to create bots that imitate human behavior and bypass CAPTCHA tests.
As CAPTCHA-solvers become more sophisticated, human-like and able to overcome previous obstacles, the CAPTCHA challenges must evolve as well. Researchers in a Cornell University study were able to get to a 99.8% success rate in solving reCAPTCHA using machine learning methods such as Deep Convolutional Neural Networks. A 2011 study found various types of CAPTCHA tasks that are no longer effective and easy to bypass.
One of the ways attackers were able to overcome the JavaScript loading obstacle is using Headless Browsers. Headless Browsers are web browsers without a graphical user interface, which provide automated control of a web page in an environment similar to popular web browsers like Chrome, Firefox or Safari. However, they are executed via a command-line interface or using network communication. Using Headless Browsers, attackers are able bypass different types of CAPTCHA tests. For example, there are documented ways to solve various CAPTCHAs using "Selenium" Headless Browser and "Headless Chrome".
Another method attackers use to automatically solve one or more CAPTCHAs is Payload Manipulation. In this method, attackers perform one or more “legitimate” solves as humans and record the payload sent after this solve. They analyze the required keys and values, and learn the patterns for dynamic payload parts. Then they assemble the important parts of the payload, add functions that will automatically apply these parts and create a false payload that resembles a legitimate one. This way, attackers can send many requests using the recorded payload without running JavaScript. This solution is easier and more cost-effective, as no browser involvement is needed. 
In addition, there are dedicated services that provide real-time human labor to bypass CAPTCHA challenges. The attacker sends a URL or screenshot of the CAPTCHA page to the solving service for a real human to solve. Then, the service sends the attacker the required code for an OCR or auditory CAPTCHA or X,Y coordinates of where to click on a visual CAPTCHA.
This solution can also work for CAPTCHAs that rely on a “humanity” rating. The attacker sends the CAPTCHA URL, a real human with valid rating passes the test and the human receives a valid token that they pass back to the attacker.
Any attack that involves CAPTCHA-solving is more complex than an average attack. As a result, attacks of this type are operated by knowledgeable attackers who create scripts that identify changes on a web page and react accordingly without being caught. This requires either in-depth technical capabilities or fees to pay someone with said expertise.
The escalation of a CAPTCHA attack
A single malicious CAPTCHA solve can lead to a significant amount of breached traffic. This is because CAPTCHAs were originally designed with the idea that a single solve indicated one legitimate user. By most CAPTCHA services’ logic, once a user proved their “innocence” and exonerated themselves, the web/app operator should let them pass smoothly and give a “grace period” for a certain amount of time. This means that after users solve a CAPTCHA test, they get a valid token from the website or app for a certain amount of time — usually around 5-30 minutes. This allows the user to browse the website or app and utilize its functions without interruption.
For attackers, solving a CAPTCHA and getting a valid token is the key that opens the door to a range of attacks: account takeover (ATO), buying limited edition items or scraping websites for data. Hackers often choose to start their session with a CAPTCHA and use it as an attack vector — a new way to obtain a valid cookie. Instead of waiting for the CAPTCHA to arrive and risking being marked as problematic in advance, attackers start their session with a solve, get a valid token and perform their attack for however long the grace period allows. Then, they move on to solve another CAPTCHA and use its grace period for another attack without interruption. This method is especially popular among sneaker bot users.
For example, HUMAN tracked traffic to one vendor on Black Friday 2021 and saw a single captcha solve that led to thousands of attempted malicious requests. The “user” solved a single CAPTCHA and then tried to use the “allowlist” grace period token to perform thousands of requests using that token, without success. After less than 15 minutes, the token expired and the user was gone. If we weren’t limiting “allowlist” tokens by number instead of by time, this could result in a breach.
An attacker solved one CAPTCHA on 06:08:00 and then tried to use the grace period to perform over 20K requests within less than 15 minutes
An attacker solved one CAPTCHA on 20:53:00 and then tried to use the grace period to perform over 40K requests within less than 15 minutes
In light of these evolving attack methodologies, the approach to implementing CAPTCHA needs to change. Instead of relying on the solve to assure this is a real human, we need to ask: why was this user suspected as malicious in the first place? And granting users a grace period framed in time can be dangerous, even if it’s only a few minutes.
Preserving user experience and conversion rate
When fighting with CAPTCHA solvers and designing a strong CAPTCHA test, keep in mind that there will be real users affected by it. In a Stanford University study, researchers show that CAPTCHA tests can be very difficult for humans, and that demographic factors have some influence on the difficulty of a CAPTCHA to a user.
Moreover, even when the user is able to solve a CAPTCHA successfully, it has been proven that only presenting CAPTCHAs can hurt conversion rate up to 30%. A 2009 study found that when CAPTCHA was turned off, the site's conversion rate increased by 3.2%.
CAPTCHAs always have “bad timing,” kicking the user out of their flow on the website, driving cart abandonment and reducing conversions. That is why it’s important to keep it user-friendly and short.
One way to make CAPTCHA tests harder for attackers without hurting the human’s user experience is to use invisible honeypots. There are many ways to add such “traps'' within the CAPTCHA page.
For example, developers can add embedded fields that a real user won’t fill or interact with because they are invisible due to CSS or JavaScript. Bots, on the other hand, will fill in a field that valid users can’t see, alerting us to their activity. If the honeypot field is filled in, we can confidently reject the form as spam. Note that such honeypots might affect visually impaired users, so it’s important to add some kind of warning.
Stop CAPTCHA-solvers
Most CAPTCHA-solvers we handle at HUMAN appear on high-profile customer accounts, where the profit from an attack will be more worthwhile. Based on the knowledge we have acquired handling such attacks, we assembled 14 tips to create a strong CAPTCHA, while preserving user experience.
- Randomize both client-side and back-end code
- Make the verification tasks random and hard to predict
- Use dynamic payloads
- Measure as much information as possible on the solve to characterize different solves
- Leverage AI and machine learning
- Create several difficulty levels and serve CAPTCHAs based on specific user data
- Check how and why the user got to the CAPTCHA page
- Consider reducing grace period on specific cases
- Enforce some rate-limiting detections over exonerated users to make sure they don’t abuse the ”Allowlist” token
- Ensure CAPTCHA tasks are clear and well explained
- Hide invisible honeypots that affect only automated tools and not real users
- Make sure the time it takes to complete the task is short and bearable
- Don’t use tasks that rely on knowledge that rely on language, age, education level, etc.
- Avoid presenting tasks that are exhausting and repetitive
It is important to remember that solving a CAPTCHA is not the goal, but a way to reach the goal. CAPTCHAs are just one small piece of the larger web attack lifecycle. Although users are generally hit with CAPTCHAs at login and/or checkout, automated threats exist everywhere throughout their digital journey. Enabling strong user verification without adding friction is critical to protect your revenue and reputation.
