When testing one page (A) against another (B), enter how many actions vs non-actions
you have
for each page.
E.g., You have two sales pages that you are testing against each other. Both
pages have received
100 visitors. Page A has produced 1 sale, and page B has produced 3 sales. Can
you say with
certainty that B is better than A?
Of the 100 visitors to page A, 99 did not take your desired action, so fill
in 99 in Non-actions of A.
1 did take the desired actions, so fill in 1 in Actions
Of the 100 visitors to page B, 97 did not take the desired action, so fill
in 97 in Non-actions for B.
3 did take the desired actions, so fill in 3 in Actions.
Now, if the p-value is less than 0.05, page B performs significantly better
than page A. In this example,
the value is 0.3, which is NOT less than 0.05, so we cannot say that B is better
than A.
However, this doesn't have to mean it is not better -- it just means
we cannot, at this stage, prove
that it is better. So you might want to wait until you have more data.