US eDirect Deploys Bot Fighting Technology to Protect Access to California Campsites

Aug 25, 2021 | Blog, News

Executive Summary

Due to the strong demand for premium California State Parks campsites, a resale market had emerged to “assist” campers with reservations. Unauthorized resale and availability alerting services such as First Choice Reservations leveraged ReserveCalifornia.com’s public user interface to harvest availability and to automate bookings when the clock struck 8 AM daily.

This wreaked havoc for the regular user as they could not navigate fast enough to beat the bots used by these unauthorized companies.

The democratic ideal of a park open to everyone was thwarted by the reseller’s efforts to make a buck from users who were willing and able to pay more.

California State Parks campground leaders turned to US eDirect to combat the dreaded bots and diminish their ability to snap up the highly coveted campsites. US eDirect deployed a CAPTCHA system that’s able to discern between bots and real human users, prioritizing the actual users and improving their ability to reserve their campsite of choice. 

While demand for the best sites remains incredibly high, and visitors should plan to book many months in advance to reserve their desired site, the collaboration between US eDirect and the California State Parks system prevented the secondary market from continuing. 

This effort proved so success that the California governor issued a formal document saying the issue had been resolved technologically and didn’t require government intervention.

 

Project Description

California State Parks has battled campground reservation resellers and their bots for years. These bots gobbled up all of the high demand campsites when they became available and sold the reservations at a premium.

These unauthorized practices made it very difficult for campers to get a site of their choosing and stampeded on the fairness of “first come, first serve” within the campground reservation system. Starting in November 2018, we found a winning formula using multiple technologies to combat the resellers that ultimately led to the biggest reseller closing its virtual doors for good.

How were unauthorized companies doing this?

While we don’t know the exact tools that First Choice Reservations and others used, we can speculate that they used a “headless” browser such as Headless Chrome.

Headless simply means that the visual user interface isn’t used but the rendered response is returned as text and data and then made available for additional programmatic processing. Unauthorized resellers had walked their way through Reserve California as humans and gathered what actions were necessary to find availability and then to book.

They could have played back these steps using Headless Chrome with different credentials, campsites, and credit card numbers in a semi or fully automated fashion.

Interested in Learning More?

Executive Summary

Due to the strong demand for premium California State Parks campsites, a resale market had emerged to “assist” campers with reservations. Unauthorized resale and availability alerting services such as First Choice Reservations leveraged ReserveCalifornia.com’s public user interface to harvest availability and to automate bookings when the clock struck 8 AM daily. This wreaked havoc for the regular user as they could not navigate fast enough to beat the bots used by these unauthorized companies. The democratic ideal of a park open to everyone was thwarted by the reseller’s efforts to make a buck from users who were willing and able to pay more.

How were unauthorized companies doing this?

While we don’t know the exact tools that First Choice Reservations and others used, we can speculate that they used a “headless” browser such as Headless Chrome. Headless simply means that the visual user interface isn’t used but the rendered response is returned as text and data and then made available for additional programmatic processing. Unauthorized resellers had walked their way through Reserve California as humans and gathered what actions were necessary to find availability and then to book. They could have played back these steps using Headless Chrome with different credentials, campsites, and credit card numbers in a semi or fully automated fashion.

Why was it so difficult to stop them?

Bots looked no different than legitimate users when using Reserve California. There was not enough network activity to flag them as an aggressive bot whenever they “surgically” interacted with the website. Web application firewalls (WAF) are phenomenal blockers and tacklers of malicious activity but when it comes to bots that are performing human recorded legitimate activities, they can’t tell the difference.

Stopping the bots

The best method found by California State Parks to stop bots was through the use of CAPTCHA. CAPTCHA stands for “completely automated public Turing test to tell computers and humans apart”. It was built from the ground up to thwart bots. Google’s reCAPTCHA has become the industry standard and is what California State Parks selected as the foundation of the efforts to mitigate undesirable bot activity. The WAF residing in front of Reserve California also played a significant role in knocking down noisy bots through the use of IP reputation and behavioral filters.

Results: before and after

From January 1st, 2018 through January 17th, 2019, almost 600 suspicious bot reservations occurred. This is very small compared to the total number of reservations during this time; however, for high demand beach front camping during peak periods of the year, 600 was 600 too many. We estimated that during this time period, over $23,000 was taken from California campers by unauthorized resellers. On January 18th, the completion date for this project, everything changed. Since that date, there have been 0 suspicious reservations and fairness has returned to the bookings of highly sought-after campsites.

Project Details

Purpose of the project

This project of mitigating bots had multiple goals from the onset: to make camping fair, to provide value to campers and to engage with them directly instead of through unauthorized resellers, and lastly to mitigate any system overhead caused by runaway bots. California State Parks reservations should be available to everyone, not just to unauthorized reselling bot herders. By mitigating bot activity, all campers have equal access to book a cottage at Crystal Cove, a campsite on the beach in Carlsbad, or any of the other 15,000+ campsites within the 300 park system.

The second purpose of this project was to eliminate the unauthorized “middleman” and directly engage with the campers. Unauthorized resellers have taken tens of thousands of dollars from campers while providing no tangible value to the camping experience. When campers leveraged an unauthorized reseller, they may not have known what they were getting, and they may have divulged sensitive information such as credit card information and passwords. Reserve California is the one-stop-shop for activities and camping within the state park system. It provides real time information about campground availability and amenities, allows campers to book and modify reservations, and offers a secure experience.

The last major purpose of mitigating bots was to reduce or eliminate the unnecessary overhead on the system caused by bot activity. Generally speaking, bots create a lot of unwanted traffic as the result of scraping information. While Reserve California met its SLA target points, eliminating bot activity could only increase the system’s overall performance and provide a better experience for California State Parks’ campers.

Details of the implementation

Overview

For some time, Reserve California had been impacted by bots that hold and reserve campsites. These bots, sophisticated scripts that login to the reservation system and navigate through the checkout process, became very problematic at popular parks during peak reservation periods. To mitigate the impact of bots, California State Parks deployed Google’s reCAPTCHA service to key areas of the public reservation system: new customer, login, reserve unit, checkout, and contact us. reCAPTCHA worked in concert with the already deployed web application firewall (WAF) to thwart nefarious actors and to ensure appropriate campsite availability.

What is Google reCAPTCHA?

reCAPTCHA is a free service from Google that protects websites from spam and abuse. Using Google’s proprietary risk analysis engine, reCAPTCHA mitigates bots from engaging in unwanted activities. While Google uses complex techniques behind the scenes to determine the likelihood of a visitor being a human, the process to implement reCAPTCHA is straight forward. Google offers this service for free as it assists them in improving their own products such as Google Maps.

How Does Google reCAPTCHA work?

When a visitor creates an account, logs in, goes to checkout, or submits the contact form, the visitor is tested to ensure that they are human. California State Parks chose to implement reCAPTCHA version 2’s “I’m not a robot” checkbox. When a visitor clicks on the checkbox, reCAPTCHA determines the likelihood of the visitor being a human. If the human confidence is below Google’s threshold, the visitor is presented with a visual challenge. For visitors using screen readers, there is an audio challenge that can be taken. The reCAPTCHA response is then validated server side prior to any action occurring.

Web application firewall

Web application firewalls (WAF) differ from other firewall solutions in that they primarily focus on layer 7 HTTP traffic. They have the ability to detect and block malicious traffic by inspecting HTTP request data such as URL parameters, headers, and the request body by using deep packet inspection. California State Parks leveraged the existing WAF to identify and block suspicious IP addresses, user agents, and malicious requests to mitigate bots at the wholesale level. The WAF did its duty by stopping much of the noise and chatter before ever reaching the computing resources that serve Reserve California. The blocking and tackling approach provided by the WAF proved invaluable during the implementation phase of reCAPTCHA.

Why is this project innovative?

The bot mitigation project is innovative because it’s definitely a first in the state and local campground / public land reservation space to effectively halt bot activity. Being first doesn’t necessarily equate to innovation without results (see chart attachment). Secondly, our one-two approach to combat the problem using both a WAF and reCAPTCHA technologies together was innovative. Without the WAF, we would not have had the necessary visibility or control and without reCAPTCHA certain bots would have continued to slip through as they simply appeared as humans to the WAF and other controls.

Project Accomplishments

  • Campers have an equal opportunity to reserve high demand campgrounds
      • Booking a state park campsite should be fair and equitable for everyone. Above all else, this was our #1 goal of the project and are proud to say we achieved it. The following is an example of the praise that came from our customers:
        • “Just wanted say a huge THANK YOU for putting a stop to First Choice Reservations with the new restrictions put in. I was never able to get a reservation at the more popular campgrounds such as San Elijo, now I can enjoy them!!”
    • Resale market has evaporated for California campsites
      • The barrier to enter into the California campsite reservation market skyrocketed once the bot mitigation efforts went into production. Currently, there are no known methods to defeat Google’s reCAPTCHA using just computing resources alone. Fly-by-night services exist that will farm out reCAPTCHA challenges to humans but at a significant cost and failure rate.
  • Directly engaged with customers during the reservation process and providing value
    • California State Parks wants to be engaged and a part of the camper’s experience every step of the way. By having some of the most enthusiastic campers come back to Reserve California, California State Parks is providing value by eliminating the unauthorized middleman. In addition, campers get to enjoy the other amenities that comes with being a legitimate user within Reserve Calif

 

Resources

Like what you see? Learn more below.

Call of the Wild

Call of the Wild

Call of the Wild Sleeping under the stars. Breathing in the forest air. Unplugging and decompressing. These are just some of the benefits of spending time in the great outdoors, which is proving...

We use cookies to understand how you use our site and to improve your experience. By continuing to use our site, you accept our use of cookies, revised Privacy Policy and Terms of Use. more information

We use cookies to understand how you use our site and to improve your experience. By continuing to use our site, you accept our use of cookies, revised Privacy Policy and Terms of Use.

x