Power

In statistics the concept of power means:

• Say there is an effect of a certain size,

  • for example the true population coefficient ρ = 0.21

• ... and I run an experiment with n subjects lots of times ....

• ... on what proportion of those experiments would I detect a significant effect?

In other words, power concerns the probability of detecting an effect in my sample given that there is a real effect in the population

So power is the converse of significance - as we just saw:

• Significance is the probability of detecting an effect of a certain size in my sample
  • eg a correlation of r=0.21 in a sample of size n = 50
...Given the true population correlation coefficient is ρ = 0


• Power is the probability of detecting a significant effect in my sample
  • eg p<0.05 in a sample of size n = 50
...Given the some value of the population correlation coefficient, e.g. ρ = 0.21

Example:

Say there is a true population coefficient ρ = 0.21.

If I draw samples of size n=50, how many of them will have a significant r value?

• r will be significant if it is greater than the critical value we worked out, r_crit=0.24

Let's try it!

Generate 1000 samples from a population with ρ = 0.21, work out their correlation coefficients, and plot a histogram.

• You can do this using sections 2 and 3 of the provided script
• What proportion or samples have a correlation coefficient of 0.24 or greater?

Try it again with a sample size n=10 or n=100 - to do this you would need to recalculate r_crit as this depnds on n.

• What sample size do you think would I need to detect my effect with a power of 80%


• You may be able to work this out my making a loop that does the above power calculation for a range of values of n but this would be a lot of work so skip unless you have lots of time

How many subjects

Power calculations are useful for working out how many subjects I would need to test to give a certain probability of detecting an effect (if it was there).

How many subjects would I need to test, so that if I repeated the experiment lots of times, 80% of those experiments would have a significant result at p<0.05?

The sample size can be calculated using an equation

But for the purpose of this tutorial, let me tell you that:

• For a power of 80%, and p<0.05, you need a sample size of n = 176

• For a power of 95%, and p<0.05, you need a sample size of n = 260

I can also tell you that for a sample size of 50, you need a sample correlation of at least r=0.24 for the correlation to be significantly greater than zero at p<0.05.

• Try running section 2 again with the sample sizes given for 80% and 95% power.

• Do you see a significant effect (r>=0.24) in the expected number of cases?

The snag

You may have noticed that to do the power calculations above, we needed to know in advance the population correlation coefficient ρ.

Which is tricky, because that would normally be the one thing we don't know - that's rather the point of doing the experiment

This may be one reason why power calculations are not commonly used in many branches of science!

►►►