Model performance metrics
Sometimes the model can overfit the training data and hence underperforms with real data. To reduce complexity, we use regularization.
Derive new features
Predict numeric values
Linear Regression builds relationship between inputs (features) and outputs (targets). It is a simple yet effective model to start with to get a benchmark.
Polynomial Regression: Linear Regression can also be used to model a non-linear relationship. To achieve this, we transform an input feature by a non-linear transformation function and create a new feature that can be used as input in the model.
Chinese proverb: One chopstick is fragile, but a group of chopsticks is robust.
A big challenge with ML models is overfitting to training data. And hence, we can use ensemble models — average multiple models into a meta model, which in turn reduces variance.
Identify unusual cases
Drive Top of the Funnel from various Channels.
Best practice: Use UTM parameters appropriately so you can attribute traffic to each channel.
When retention fails, you churn, i.e., lose customers.
Why it matters: In any business you want to make sure that you are adding customers faster than you are losing them and at the right Customer Acquisition Cost.
Types: E-commerce businesses will measure transaction churn. In contrast, Subscription businesses will measure two forms of churn:
Challenges: Churn is a delayed backward looking and you cannot get a live number.
Examples. Everyone has faced challenges with churn:
Any new, and better way of doing things is technology.
Why it matters: Technology allows you to do more with less.
How: Technology has never been an automatic feature of history. We have to build it.
Cost Per Acquisition
Once someone is on your Internet Products, always push them to consume one more piece of content. Pages Per Session captures how interested are audiences in exploring your website beyond the landing page.
You can do this by capturing the audience’s intent of coming to your platform and serving them basis that intent using:
To understand their intent you can rely on the piece of content they are consuming right now or the organic keywords they were searching for prior to landing on your platform.
Cost Per Mille: Businesses book a fixed number of impressions within a date range. Once the order is accepted, publishers must honor the impressions while maintaining reach and view-ability% within acceptable limits.
Net Promoter Score (NPS) is a simple metric to measure customer satisfaction that users can fill up quickly on a regular basis and it doesn’t require any analysis to digest. However, NPS is backward looking. For forward looking, ask Qualitative Feedback.
How to ask.
How to analyze
Benchmarks. Look for blogs talking about NPS to get an idea about benchmarks.
Advertisements sold via an intermediary ad networks like Google Ads, Taboola, etc.
Examples of arbitrate opportunities:
Sessions Per User communicates how frequently users return to your product. At a macro level, this idea is also communicated via DAU/MAU. To improve it, use following levers:
Once they land on your platform, use Audience Engagement and Onboarding to get them to:
Eventually, with Identified Audience, run event-triggered and personalized communication using Newsletters and Emails.
The time you spend courting the media so you can get Referral Traffic.
A customer lifecycle is a form of segmentation or funnels that maps customers based on how proficient they are at finding and using your product and features.
Why it matters:
What caused the conversion? Establishing exact causality is very hard. For example, unknown to us, a conversion event might have occurred after the lead read one of our Candid Communication blog posts, seen a Digital Advertisements, received our Newsletters as a forward, or saw a post in their Social media feeds.
So, who gets the credit? There are few models that can help estimate attribution: