A/B Testing : Keys To Flawless Team Problem Solving

By Diamond Lambert

Imagine this: your development, creative, and product management teams spend countless hours to create what they deem, the perfect product. But there’s one group who wasn’t included in the last nod of approval: the customer. A/B testing will solve that minor problem.

For starters, going to market without assessing the customer’s perceived value would be quite frankly,  foolish. We all aim to keep products moving in a prosperous direction and ensure that your team remains on-task while sifting for answers. Consider these organizational essentials while working within your own team.

A/B Testing Tip #1: Establishing Process

Before delegating tasks and splurging precious marketing dollars, consider the need to craft the perfect process. Each team has its own unique ratio of resources and available time before final deliverables are due. Within our own team we try to incorporate the following mix:

  • Write a clear, concise hypothesis that needs can be justified (or disproved) via A/B testing.
  • Prepare a walk-through that offers test points and best practices for engaging with a test group.
  • Consider time needed to set up a test group. Have backup participants at the ready for any last-minute cancellations.
  • Incorporate times of rest. These are non-planned moments for open conversation among the test groups and even within your company’s marketing team. I have found that taking a break from the pomp-and-circumstance leads to frank conversations. Frank conversations, in turn, lead to new ideas for your team to back pocket for later.
  • Plan for sifting through data. Data that springs from one-off occurrences could be far-flung from data that are key to improving UI design or a development task. In short, some feedback could just be complaints due to general unfamiliarity or personal reasons. When dealing with people, anything can happen!

A/B Testing Tip #2: Watch The Time

Next is everyone’s least favorite: the concept of time. This bit of advice is key for the product managers out there. Knowing how to balance the expectations of executives with the demands of key stakeholders is tough stuff.

Additionally, when dealing with some of my most intense projects, I always scheduled padding time before delegating tasks. I explained to team members several dates:

  • Projected Start Date – Give team members anything from a week to a few days of prep time. Each member deserves an opportunity to get organized and wrap up straggler tasks already on their plate(s).
  • Weekly Check-In – Check-ins should occur no more than a week apart. This timing is ideal for reviewing if someone has an upcoming vacation, family on-goings, or conflicting projects.
  • Execution of A/B Test.
  • Assessment of A/B Test.
  • Pre-Showcase (1.5-2 weeks out) – Key stakeholders have everything to lose when it comes to a resource-intensive project. Large scale stakeholders could be Directors, VIPs or District Managers. Small scale stakeholders include business owners, angel investors, or department managers. Look at this meeting as a PR stunt. This is your shot at getting the executives amped up for your final presentation. Stakeholders will become your mouthpiece sharing the highs and lows. They’re also a great temperature gauge for how well your presentation will deliver to the key decision makers.
  • Final Delivery – You’re ready to present the best of the best. In a perfect world, the final presentation is flawless.

A/B Testing Tip #3: Drawing Conclusions

Let’s be honest: conclusions are never truly conclusive in the world of product development. Market segments change their interests, and pesky competitors are always influencing new changes.

Remember how I mentioned incorporating moments of frank, open discussion? Ideas generated from those moments of the A/B test process become handy next-step projects to keep your team’s product moving forward. The best part of the A/B testing process is taking a peek into the minds of consumers. Designers are sensitive to feedback. Try to reposition each suggestion as a new challenge.

Inviting your team to conduct a good ol’ SWOT analysis (strengths, weaknesses, opportunities, and threats) can help with preparing for your next go-round of testing. The only way to grow is to experiment, test limits, and never be afraid to outsmart the customer.

The great David Ogilvy himself said, “Never stop testing, and your advertising will never stop improving.”