Made with Humane Club
Many aspects of your business will change with the seasons, and the better you can understand those changes the better you will be able to predict the future.
For example, Revenue from advertisements is seasonal.
CopiedIn the real world, attributing an outcome to a specific action is very hard.
CopiedThis training helps journalists, product managers, and business people build intuition for data so that we can collaborate and co-build more meaningfully with our technically savvy colleagues on data journalism, business analysis, and artificial intelligence projects.
Also called Power Analysis, this technique helps you determine how many observations are needed for the results of A/B Testing or Multi-armed Bandit to be considered reliable.
To use the method, you need to provide two input parameters:
- The size of the change you expect to measure, for e.g., 5% increase in CTR
- The numerical confidence you want in your results, for e.g., 95%
You’ll typically need fairly large samples to get trustworthy results.
CopiedUnlike A/B Testing, multi-arm helps you test multiple options on one aspect of your product or service at the same time. Additionally, it also takes the action of scaling the best performing option by itself.
How it works: Let’s explore one of the methods: Epsilon-Greedy.
- On 20% of your customers, it explores or tests out multiple options to see which one performs best. On 80% of your customers, it exploits or adopts at scale the options that has the best performance.
- This mode kicks in once you’ve at least one decent option. Prior to it, 100% of the customers are on explore mode.
The details: You’ve to choose interchangeable options. For example, you can use it for posts headlines or for cover image in a post. However, you cannot use it for things like different pricing tiers.
CopiedA/B Testing lets you test two different options on one aspect of your product or service at the same time in production to determine which one is performing better.
How it works: It uses segmentation to divider the selected user base into two buckets. The Experimental Group sees the change, while the the Control Group doesn’t. The outcome of the A/B test is determined by measuring the difference in performance of both the groups.
On the downside: A/B tests can test only two options. Hence, if you’ve multiple options to choose from, then you can end up wasting a lot of time determining the best performing option. That’s where multi-armed bandit testing comes in.
CopiedClustering helps find hidden patterns in your data by grouping similar things for you, such as customer personas. There are two types of clustering techniques:
- Partitioned: Each and every item is clustered in one, and only one, cluster. One such method is K-means.
- Hierarchical: Creates the clusters by continually grouping or breaking items.
K-means clustering
How it works: How this algorithm operates is in its name. K is an input to the algorithm and refers to the number of buckets or clusters that will be created. Each item is assigned to a cluster based on the closest mean.
- Choosing the initial values is one of the toughest problems. Often, the algorithm might choose centroids as the initial k points.
- Then it clusters every point based on the closest mean.
- Then it evaluates if the initial values were the best central values.
- If not, then it reassigns the initial k points and repeats the process.
The details: Given that the algorithm is guessing the initial values, you can get different results for the same value of k.
Hierarchical clustering
There are two types of hierarchical clustering: agglomerative and divisive.
- Agglomerative clustering is bottom up, i.e., each item is its own cluster and then they are joined together to create larger clusters.
- Divisive clustering is top down, i.e., all items start in a single cluster and then are broken apart to create smaller clusters.
The details: Hierarchical clustering always produces the same results every time because the distance between items doesn’t change.
CopiedA persona is a manually labeled fictional character or a real user that can represent a segment of users that share attributes or traits that allow for clear bucketing, for example, age, gender, location, etc. Since this is a manual exercise it is recommended to build only a handful of personas.
Why it matters: Personas act as stand-ins for large segments of your customer base. Your team uses personas to see decisions (features, communication, marketing) from the eyes of the customers.
CopiedSimilar to segmentation, Cohorts also divides customers into different groups based on shared characteristics. However, unlike segmentation, cohorts are built from dynamic attributes (number of purchases, total amount from purchases, etc.) and represent groups of individuals who share those characteristics for a certain period of time.
Example: One of the most common methods is to cohort users by the date or month they started using your product for the first time.
- Such cohorts help because your product or service keeps changing over time. Hence, a new user today will have a vastly different product compared to a user who started six months ago.
- The expectations and clickstream patterns of a year old user will be vastly different.
- Each cohort will have its own churn patterns.
CopiedSegmentation is dividing customers into different groups based on shared characteristics — typically current state of attributes.
Why it matters: Any successful large business has many customers, each with different needs and interests. Never analyze them in aggregate.
How to create segments?
You can create segments by a variety of factors:
- Who they are: These traits capture some aspect of why customers behave the way they do and can help predict future behavior.
- Segment by location or demographics or interests.
- What they do: These traits are a result of customer behavior, and are likely an artifact of either how your product operates or how customers perceive your company.
- How often: Google Analytics naively segments users into New Users and Returning Users. Beyond that, segment users by activity into DAU, WAU, and MAU.
- Where: Segment users by the Channels they came from.
- How well: A better segment is by Customer Lifecycle. To enable this transition, actively use onboarding. Such segmentation can help with pricing tiers..
- Why: Cohort users by why they are using your product. For example, Google Maps could bucket users by whether they are daily commuters, tourists in a new city, taxi driver or delivery agent, etc.
- Happiness: You can segment by Net Promoter Score and see which segment is using which features.
Related
If you bucket users based on dynamic attributes then you are building cohorts. If you create a fictional user to drive empathy, you are creating a persona. Finally, if you automate the process, then you are clustering.
CopiedFaceting: Quickly comprehend what’s inside your data
Build a culture of asking questions and proposing proposed explanation based on the limited information available before delving into complex problems.
Here’s how you write: “If we change… (action), then we will see… (result), because… (clarify rationale).
The details: Hypotheses are generated by human intuition based on the collective intelligence and experience of stakeholders and their understanding of the business and their environment. This is often generated by asking questions:
- What are you trying to do?
- What is happening that is not supposed to happen?
- What could drive it?
- Why is it happening?
- What is happening?
- Why is it happening?
- What are your hunches?
- What opportunity are you exploring?
- What problem are you looking to solve?
- Are their gaps in your knowledge that you are trying to fill-in?
CopiedData audit and cleaning