Sensitivity Analysis - Ritvvij Parrikh Sensitivity Analysis | Ritvvij Parrikh Humane ClubMade with Humane Club

Sensitivity Analysis

Also called Power Analysis, this technique helps you determine how many observations are needed for the results of A/B Testing or Multi-armed Bandit to be considered reliable.

To use the method, you need to provide two input parameters:

  • The size of the change you expect to measure, for e.g., 5% increase in CTR
  • The numerical confidence you want in your results, for e.g., 95%

You’ll typically need fairly large samples to get trustworthy results.


Links to this Evergreen Note

None yet