What to Test in the New Age of Black Friday

In 2020, there were dire predictions of the “end of Black Friday.” With the day itself losing significance over the years with e-commerce and earlier sales windows, experts feared that COVID-19 would put the final nail in Black Friday’s coffin. 

Boy, were they wrong.

Online spending on Black Friday 2020 increased roughly 22%, with consumers spending $9 billion the day after Thanksgiving. 

According to Adobe Analytics, consumers spent $6.3 million per minute online during Black Friday. Spending on smartphones increased 25.3% from 2019 to reach $3.6 billion, representing 40% of total e-commerce spending.

In short, Black Friday season is bigger than ever, making it especially critical for companies to prepare for the surge in online traffic and purchases well in advance. 

This creates tremendous amounts of pressure for DevOps teams. Application downtime on Black Friday can cost companies thousands of dollars in lost revenue, a brand reputation hit, and the low morale that comes with being one of “those” companies that fumbled Black Friday. 

That’s why at Perfecto, we’re here to help.

If you are starting your Black Friday preparations (and if you haven’t, you should be), make sure to keep a few important guidelines in mind. Following these guidelines, as well as taking advantage of our comprehensive Black Friday testing checklist, can make the difference between your Black Friday being a major success or a complete disaster.

Testing Checklist for a Better Black Friday

Prepare your app with this 37-point omni-channel checklist for the biggest day of the year.  

See the Checklist

Determine Your Test Coverage Strategy

The first step in getting ready for Black Friday is to test on the devices and browsers that are most relevant to your audience. There are many devices and browsers to test out there, so it’s important to be strategic with your test coverage to ensure that your prospective customers are getting the best user experience.

Perfecto's Test Coverage Guide is a great resource that can help you choose which devices, browsers, and operating systems are the best for executing tests for your team. This guide also takes into consideration upcoming mobile & web release dates, your comfort with risk, and where your users are located.

It is also important to understand the digital trends that are making an impact on today’s testing landscape, such as virtual devices for development and early-stage testing and the new Android 12 and iOS 15 releases. 

This combination of factors will help you determine the best strategy and get you set up for Black Friday success.

Run End-to-End Automated Tests for Core Processes

Another important factor for successful Black Friday testing is to run automated end to end tests for your core user interactions. These tests need to start well in advance of Black Friday, as many retailers roll out offers weeks before Thanksgiving to handle the surge in online traffic. 

Teams need to determine which aspects of their applications have the highest value and pose the highest risk if they go down. Every aspect of key business transactions must be tested – from login to inputting credit card details. 

These tests should be part of stable test automation suites that run continuously to ensure that they are running properly. These test automation efforts also need to include proper feedback and consistent results to minimize MTTR (mean time to resolution).

Core processes to test on Black Friday include, but are not limited to:

• Add to cart
• Log in or guest account creation
• Address and contact information input
• Payment processing
• Email confirmation
• Tracking emails
• Delivery confirmation

Another important consideration to keep in mind is that each vertical and market segment brings its own unique features and functionalities. In the retail industry, there are unique flows and user journeys that must be well covered and tested, such as:

• Making a purchase through the Apple/Google wallet
• Using the camera AR to virtualize an item 
• Buying items in specific locations
• Mobile geo location-specific offerings and alerts

Performance Testing

Performance testing is another important area to cover as part of your Black Friday checklist. Despite running offers in advance, there is typically higher traffic on Black Friday. With many consumers still minimizing in-person shopping experiences, it is critical to test your application’s speed, stability, and scalability while handling an increased workload.

There are various checks you will need to do as part of your performance testing, and they will differ for web and mobile applications. 

For web performance testing:

• Page load time across browsers.
• Varying screen sizes.
• Resolution across different browser OEMs.
• Refresh rates.

For mobile performance testing:

• Varying network conditions, such as poor 4G or 3G networks.
• Apps running in the background.
• Sensor latency, such as location and camera. 

Test Discounts & Promo Codes

Another major focus of your Black Friday testing should include your site’s discounts and promo codes. The drive to get the best discounts is at the heart of Black Friday, so giving your users faulty discount codes on one of the most important eCommerce days of the year would be nothing short of catastrophic.

In 2020, Black Friday also made a notable shift by embracing referral opportunities. Instead of homing in on Black Friday’s competitive nature, brands like PrettyLittleThing, Ted Baker, and Kurt Geiger have used referrals to both acquire new customers and incentivize existing ones to return. 

This shift in focus on referral programs has made Black Friday a more lucrative approach to customer retention instead of a one-and-done strategy that happens each year. Therefore, testers should also focus on testing referral opportunities in tandem with other, one-time-only discounts and promo codes. 

Account for Complex User Conditions With Real User Simulation Testing

It is also important for your mobile app testing to reflect the same environmental profiles as your users.

With real user simulation testing, teams can test for all conditions users will experience and eliminate the cascade of extra work that accompanies escaped defects.

This makes your testing smarter and more targeted while preventing escaped defects that add extra time and resources to resolve. 

Important user conditions to account for include:

• Biometric scanning
• Varying quality of mobile networks
• Airplane mode
• Camera
• Audio injection
• Call and text interruptions
• Device vitals
• GPS tracking
• Conflicting apps
• Packet loss
• Accessibility constraints

Regression Testing 

To make sure these additional Black Friday testing efforts don’t go to waste, it is also critical to keep up with regression testing. Running regression tests will ensure that any new Black Friday updates, such as banners, discounts, popups, or promo codes don’t cause bugs in other parts of the application and upset your shoppers.

While regression tests can be done manually, regression testing in Agile is best done with test automation and continuous testing tools. Continuous regression testing reveals defects early in the process and helps teams focus on innovation while releasing new features or updates for Black Friday. 

Bottom Line

Black Friday may have shifted since the start of the COVID-19 pandemic, but it is still a critical time of year for eCommerce companies everywhere. With the right testing well in advance, your organization will be poised to make Black Friday a raging success.

Get a jump start on your Black Friday testing with Perfecto. Our robust testing platform offers everything you will need to get started, from thousands of real devices in Perfecto’s Smart Lab to integrations with all the major solutions making up your DevOps toolchain.

See how Perfecto can get your application ready for Black Friday. Sign up for a free trial, or get instant access to the Perfecto platform with flexible monthly or annual subscription plans starting at just $83.

See plans & Pricing

14 Days of free testing