end-to-end A/B testing course:
A/B Test like a Pro!

You can become an A/B testing expert, too. Usually, it takes years. But this A/B testing course will give you a huge head start. You’ll learn the science, the statistics, the business logic, the research framework, the typical mistakes… everything you need to run successful experiments.

More info...

90% of online businesses are doing A/B testing wrong!

Here's a typical A/B testing story.

Opening consultation for a data science project. The client proudly mentions:

"We run A/B tests all the time! And we get pretty nice results… Last time we just hit a +60% conversion increase on our main page."

Then we sit down to review those test results. And in 9 out of 10 cases it turns out that the client made a huge mistake somewhere in the process… And the promising conversion boost was only the result of improperly implemented or wrongly evaluated tests. The 60% increases they saw are, in reality, hovering around 0%. (Or worst-case, they turned out to be 20% decreases in conversion.)

Today, everyone can run an A/B test. There are easy to use point-and-click tools. In just an hour you can create and start your first experiment... Sure. But here's the thing: very few online experts can run A/B tests properly.

And as I have written: 90% of online businesses are doing A/B testing wrong! Extremely dangerous… (for them, anyway.)

It's not their fault though. It's hard to get great at A/B testing. It can take years to learn everything:

  • the basic concepts
  • the science (aka the statistics)
  • the business thinking (when and why to run A/B tests at all)
  • the best practices and
  • the typical A/B testing pitfalls (there are many!)

I created this A/B testing course so you can skip the trial and error phase and get into the race with a head start. You don't have to go through all the typical mistakes to become an A/B testing expert. You can learn from others' successes and failures!

In this course, I'll provide you everything.

Not just everything you need to know about A/B testing... But frameworks, checklists and step-by-step guides that I use myself in my conversion optimization projects.

So when you run your experiments in the future, they will be professionally executed, scientifically correct -- and most importantly, you will deliver real, long-term and outstanding results!

You'll A/B test like a pro!

Data Science + A/B testing

For running proper A/B tests, you don't have to be a data scientist. But you have to understand the scientific background of experiments.

Don't worry, it’s not as complicated as you’d think!

You have to master 3 main domains:

  • 1
    The A/B testing business logic and the strategy. How can you make a real impact with A/B testing? How can you deliver real value (and real profit) for your online business?
  • 2
    The statistical and mathematical background of A/B testing. How can you avoid the typical statistical errors in your experiments (e.g. stopping and evaluating them too early)?
  • 3
    The A/B testing research framework. Instead of testing random gut-feeling ideas (like many do), how can you prioritize your experiments and consistently keep the focus on the most promising, most impactful next steps?

Once you understand these, all you have to do is to learn a few more best practices and watch out for the typical mistakes... And you'll start to see more and more great results in your experiments!

A/B test like a data scientist!

Tomi Mester - author of data36.com

Hi, I'm Tomi Mester, a data professional since 2012. I've seen, managed and run hundreds of A/B tests. Besides, I was lucky enough to get to the industry's most important conferences and learn from the best data analysts.

In the last few years, I’ve collected the best methods (and improved them further in practice); I've seen all the typical mistakes; and I've tested all the best practices on real life projects.

I have a smooth, simple and efficient A/B testing framework -- and a knowledge base that contains everything you need to know. Not to mention the cheat sheets and checklist I use in my own projects.
And you can get access to all these!
I'm interested!

A/B testing course... How does it work?

The A/B Test like a Pro! course is an online course. When you enroll, you'll get instant access to the course curriculum, where you will find:

  • 38 videos. (~3 hours in total) These form the backbone of the course. In these 5-10 minute long videos, I'll guide you through everything you need to know about A/B testing, step by step. (See the exact curriculum below.)
  • Step-by-step guides and checklists (in printable format). When you run your first experiments, you might need some extra support to execute everything properly. So I'll also give you a few "cheat sheets" that you can print and keep next to you for the first few months. Following them, you won't overlook anything - guaranteed.
  • Extras in the Vault. I have a Vault section in the course curriculum, too. I’ll put more and more extra learning materials there. In the first iteration of the course, this means further readings (my favorite articles from other professionals), case studies and more. (If you enroll now, you'll get access to all future updates, too!)
  • Action modules. And at the end of most modules, you'll also find lectures called: ACTION. Each of these action lectures contains a task that will help you to put the things you've learned into practice. (After all, the ultimate goal of the course is to be able to run successful A/B tests in real projects, right?)
ab testing course content

This is a compact yet comprehensive course, by design. In other words, there is everything in it that you'll need and nothing that you won't.

I intend it to be the only A/B testing course you need to take. You take this course, you listen to it, you internalize it -- and then you'll simply know everything that you have to know to run experiments properly and with great results.
I'm interested!

Curriculum

Here's the exact curriculum of the course:

MODULE #1 - INTRODUCTION:

  • What is A/B testing exactly?
  • Why do you need it?
  • The correlation vs. causation issue
  • What can you A/B test?
  • EXTRA: Hand-picked A/B Testing case studies

MODULE #2 - IMPORTANT INITIAL BUSINESS QUESTIONS:

  • Mindset question: CRO (Conversion Rate Optimization) vs. Research?
  • When do you need to run an A/B test?
  • When don’t you need it?
  • When is it a really bad idea?
  • EXTRA: the "can I run this A/B test" checklist

MODULE #3 - STATISTICAL BACKGROUND:

  • The science behind A/B testing.
  • Typical mathematical mistakes.
  • Interpreting and understanding statistical significance and its true importance.
  • The right level of statistical significance (and how to calculate it)
  • EXTRA: online calculators (statistical significance and sample size)

MODULE #4 - EXECUTION:

  • STEP 1: How to prepare an A/B test properly. Initial research and prioritization. (It's at least ~80% of the job!) Success metric(s). Setting a hypothesis.
  • STEP 2: How to implement a test. Website A/B testing tool demo. About the "one-thing-at-a-time" myth and other typical mistakes.
  • STEP 3: How to run an A/B test. How long should it run? How big of a sample do you need? (+typical mistakes)
  • STEP 4: How to evaluate your results. How not to lie to yourself with statistics. What should you do when your new variation is worse than the original?
  • EXTRA: A/B Testing Hypothesis Form
  • EXTRA: The "can I start the A/B test?" checklist

MODULE #5 - A FEW SMALLER ADDITIONS:

  • Other A/B testing pitfalls.
  • The limitations of A/B testing
  • The art and science of A/B testing
  • EXTRA: Further reads and videos by other A/B testing experts

MODULE #6 - SUMMARY:

  • The right mindset to be successful with A/B testing.
  • Frequently asked questions and answers
  • Summary
I'm interested!
ab testing course - statistical significance

You will get evidence-based answers to the most common questions

In the last few years, I’ve gotten a lot of questions on A/B testing. I'll answer all of them in the course. A few (but not all) of these:

  • "How long should I run a test?"
  • "How many visitors (how big of a sample size) do I need to make sure my results are real?"
  • "What's the appropriate level of statistical significance? (What is statistical significance exactly?)"
  • "What is it worth it to test? The copy? Colors? Buttons? Images? A whole landing page?"
  • "Which of my ideas should I test first? How do I prioritize?"
  • "What should I do if a test doesn't deliver the expected results?"
  • "What numbers should I analyze for evaluating the test results? Revenue or conversion rate? The sales of product A or product B? Micro-conversions or macro-conversions? What are the key metrics to look at?"
  • "Should I focus exclusively on A/B testing or can I run A/B/C/D/... tests, too?"
  • "Should I run multivariate tests?"
  • "Should I segment my users by device type (e.g. mobile vs. desktop)?"
  • "What is/are the best A/B testing tool(s)?"
  • "What should I do if my company doesn't have the resources to run experiments continuously? (E.g. if I'm the only one in the organization who runs tests?)"
  • "What alternative research methods exist if my traffic is too low for A/B testing?"
  • And more...
ab testing experiment lesson

I've previously tested and fine-tuned the course materials in live workshops, too. So I know most of the questions that might be in your head and problems you'll face when starting out. I'll address them all in this A/B testing course!

This course is based on real life knowledge (and examples of real business cases)

In the last few years, over 1000+ students have enrolled in my data science courses. And I have two golden rules: (1) keep it simple and (2) keep it practical!

So I always put a huge emphasis on explaining everything (even the most complex concepts) in a way that enables everyone to understand it, internalize it and then apply it in real life.

In this A/B testing course, you will see examples from real-life business use cases. Most of the base concepts will be introduced through my (or through my data scientist friends' and colleagues') projects.

ab testing class
I'm interested!

Who is this course for?

I guarantee this: This is not another marketing BS course.

Here, we will focus on the practical parts of running experiments - and more importantly how you will be able to use them in real projects.

I recommend this course for people who don't expect A/B testing to be a fancy, one-size-fits-all marketing tool for overnight conversion boost. (Because that'd be a lie anyway.)
It's for online professionals who want to understand the science and the strategy behind A/B testing. Who see that this is a research method. And who are ready to improve their results methodically for real, long-term success.

Do you want to hear a proper "target audience," too? Fine. Here it is.

The A/B Test like a Pro! course is for:

  • data analysts and data scientists who plan to work on similar projects in the near future
  • digital marketing professionals and agencies (SEM, SEO, PPC, Social, content marketing, etc.)
  • UX researchers and designers
  • online business professionals (for those who implement A/B tests as much as for those who lead CRO/Analytics/Web teams)
  • anyone who's interested - and wants to invest in furthering her career in data science or data-driven marketing
I'm interested!

FAQ: "How deep do we go into A/B testing? Is this interesting for experienced online professionals, too?"

I got this question above from a course participant (before he registered). It might be a relevant question for many, so I’ve just copy-pasted the relevant part of the discussion here:

hi Tomi, quick question: How deep do we go into the topic? Is this interesting for experienced online professionals, too?

hey! Good question -- just in general: we start from scratch, and by the end of the course, we get far enough that everything regarding A/B testing will be completely clear. So I'd say that if someone has been running experiments for years, with great results, it's unnecessary for her to enroll. But for someone who just wants to get started with this field, or has been doing it for, say, half a year - but isn’t sure if she's doing it right, it's 100% that she will profit a lot from the course.

Two more frequently asked questions

Does this course require coding skills? What coding languages will we use?

As of the current version of the course we don't use any coding languages in it. So you don't have to know any Python, SQL, R or bash.

We focus on understanding the key-concepts: the statistics, the business logic and the processes behind A/B testing. And for that we use the right point-and-click tools.

In this 3 hours of video (+PDFs +extras), do you cover all the details of A/B testing?

Yes, I think, everything important is covered in the course. Well, I wanted it to be compact and to-the-point... I personally don't like unnecessary-long learning materials when I learn, so I kept this in mind when I created this course.


Anyway, I keep the right to be wrong about this -- and it's possible that I missed something that should be in the course.


But that's why I added an important extra to the course: a 1-hour 1-on-1 consultation - online - for each participant of this class. There, we can discuss every question that you have after finishing the course. (I've also done this before and I've built in the typical questions/answers already into the current course curriculum... Regardless, I think the 1-on-1 call is a good opportunity to go deeper into the "nitty details" if needed -- also to discuss/brainstorm your specific project, etc...)

The A/B Test like a Pro! course will take you here...

  • You will completely understand the theoretical and the practical fundamentals of A/B testing. You'll know when you need it and when you don't.
  • You'll be able to build, run and evaluate experiments professionally.
  • You'll know what you should watch out for -- and you won't ever make any of those common mistakes that many online professionals fall for.
  • You'll understand how experiments are able to support your business and how they can bring you higher conversion rates and more revenue.
  • You won't get confused by questions like "how long should the test run" or "how many visitors do we need." You'll understand the concept (and the importance) of statistical significance.
  • And of course, you'll know why it’s so crucial that A/B testing is not a stand-alone method -- but part of a comprehensive research framework.
I'm interested!

Presenter

Tomi Mester is a practicing data analyst and researcher with 7+ years of experience.

He has worked for Prezi, iZettle and several smaller companies as an analyst/consultant.

He’s the author of the Data36 blog where he writes posts and tutorials on a weekly basis about data science, AB-testing, online research and data coding.

He's an O'Reilly author and presenter at TEDxYouth, Barcelona E-commerce Summit and Stockholm Analytics Day. More info about Tomi: check out the intro video. >>

Registration Details

Price:

$197 (plus your country’s VAT if you live in the EU)

You'll get immediate life-time access for the course and for all future updates.

Registration

This course became the part of the Junior Data Scientist's First Month bundle. You can get access to it, if you enroll in the Junior Data Scientist's First Month course, here:

More info about the JDS course...

Clicking this button will take you to the check-out page where you can pay safely using your credit card or your Paypal account! (If you are registering from the EU as an individual - in accordance with EU law - you have to pay the applicable VAT of your country, too.)

gurantee ab experiment course

Guarantee

I would never want you to be unhappy! If you are unsatisfied with the course, contact me (tomi@data36.com) in the first 30 days and I will give you a full refund.

Frequently Asked Questions

Is it a problem if I don't have too much initial knowledge on the topic?

No. We will start from scratch. It's useful, though, if you are familiar with the basics of online marketing.


What if I am unhappy with the course?

I would never want you to be unhappy! If you are unsatisfied with your purchase, contact me in the first 30 days and I will give you a full refund.


When does the course start and finish?

This A/B testing course is a completely self-paced online course - you decide when you start and when you finish. Also, you'll have lifetime access to all materials -- and you will get all future updates, too.

How much time does the A/B testing course take?

Watching the videos and going through the learning materials will take around 3 hours. If you go deeper into the extra materials and if you follow all recommended action steps, it might take ~8-10 hours in total.


How long do I have access to the course?

How does lifetime access sound? After enrolling, you have unlimited access to this course for as long as you like - across any and all devices you own.


Will I get a receipt/invoice?

Yes. Individuals get a receipt via email. Companies get an invoice. EU-based companies get VAT-invoices. If you need something even more special, just reach out to me (tomi@data36.com) and we will solve your administrative issues!

I'm interested!

Privacy Policy

Terms of Use

Data36.com