Photo by Pietro Jeng from Pexels

Statistical Power Calculation of an A/B test result in Excel

A friend of mine has recently asked if I can help him with creating an Excel file to calculate the power of A/B test results. I found one calculator online here which I think is pretty sleek but the only problem is, it is online. So if you want to do your own calculation using your excel file, you will need to know the maths behind it.

Rather than getting into the details of what the statistical power is, in simple terms, the statistical power determines the possibility of Type II error (or false negative) within the result that you are seeing from the test.

I have created this Excel file which follows the same calculation that has been used by the AB Test Guide (https://abtestguide.com/calc/) to calculate the statistical power of any A/B test.

Instruction to use this excel file to calculate statistical power is pretty self explanatory:

  • Select whether the test is 1-Tail or 2-Tail (in most cases, if you are running A/B tests using major tools that are in the market – it should be 2-Tail)
  • Select the confidence level (90% / 95% / 99%)
  • Add the visitor numbers for your control and variation
  • Add the conversion numbers (NOT Conversion Rate) for your control and variation

That’s it! It will show the power of the test – in other words – the possibility of Type II errors (the lack of it).

Download Statistical Power Calculation of AB Test Results- EchoLogyx

Let's have a chat

If you want to find out a bit more, just get in touch. We love a chinwag, and we'd love to help you out.

Contact us

Don’t just take our word for it…

Our customers are our biggest advocates. That’s why we let them do the talking.

  • Working with EchoLogyx has simply changed the development lifecycle of our website. Their experienced A/B testing developers and QA engineers have significantly increased our efficiency

    Farhan Hussain

    eCommerce Director, eSpares
    4 3 2 1 0
  • I’d 100% recommend EchoLogyx – they have really sort of raised the bar of my expectation, of how you should work with developers and also their efficiency of completing the project

    Lucy Collins

    UX Consultant, Web Usability
    4 3 2 1 0
  • They come to me saying, this can be done better. This proactiveness is what I most like about the EchoLogyx team. They have got a strong QA team as well so that nothing goes out live without going through a thorough QA

    Nabil AlJannati

    Brand Manager, Ninety Percent
    4 3 2 1 0
  • Our experience of Echologyx is resoundingly positive. They’re very thorough and detailed. We’ve got a great working relationship with our optimisation managers.

    James Amborse

    Head of Operations, RedEye Optimisation
    4 3 2 1 0