Ritvvij P.

Google Sheets stores the formatting applied on the spreadsheet file separately from the core information — the CSV. Similarly, the advent of Generative AI will force information products (Search Products, Social Media, etc.) to separate the core information value and the form, format, style, and tone it is presented in. Without this, we are in for a flood of similar content that is written just slightly different.

Historically, modern CMSes like Washington Post’s Arc CMS went headless and started storing the story and its presentation (HTML tags) separately. We’ll now have to extend this further.

You can escape being commoditized if you’ve competitive differentiation.

Why it matters: Differentiation allows you capture value for yourself without worrying about others copying you.

• You could create a lot of value without becoming very valuable yourself.
• Creating value isn’t enough — you also need to capture some of the value you create. In his book Zero to One, Peter Thiel talks about how the US airline companies serve millions of passengers and create $160 billion of value each year. But they capture very less value themselves.
• Beware: Rivalries cause us to overemphasize old opportunities and slavishly copy what has worked in the past.

Types: Defensible differentiations are also called moats. There are different types of moats.

Complexity is the amount of things you’ve to keep in mind before/when doing something. There are two types of complexity: accidental complexity and essential complexity. It is the job of the product organization to drop complexity.

Why it matters: As a system grows, it becomes complex. You should care about it because Complexity slows down execution. Hence, ruthlessly identify and shed as much accidental complexity as possible.

How to handle it

• Always do pre-mortem to breakdown information you’ve into: Known Knowns (Facts), Known Unknowns (Uncertainties or Questions), Unknown Knowns (Blindspot or Intuition), Unknown Unknowns.
• Be on the lookout for Wicked Loops

Users are able to use products for free because they give their time and attention which is then monetized through advertisements.

The customer’s perspective

Companies invest in advertising for:

• Brand Development: Brands advertise to maximum folks should know about your product (Owned Identity) to drop their Customer Acquisition Cost. All print and TV advertisements were for brand development. In such campaigns, the focus is on CPM while maintaining a basic CTR.
• Performance marketing: In this brands get Paid or Acquired Traffic for a specific business objective, for example, app installs, lead generation, or conversions. Agnostic to the method you use, what matters is the outcome that you promised and hence such advertisements are paid in the form of CPC, CPL, CPA, and CPT.

Companies will pay a premium for:

• Reach
• Performance
• If you can prove that the audience that comes to your platform is their target audience. Alternatively, if you can use artificial intelligence to target advertisements.
• If you can prove that your platform is a premium space.

Limitations of the ad-funded model

Indirect programmatic advertisement is bad for business and users. Yet, most companies adopt it because it is a relatively easy plug and play revenue channel.

Bad for business

• Bottom feeding. Most companies have not been able to monetize advertisements as efficiently as Google, Facebook, Amazon, and Apple. Hence, even when large organizations do direct sales of advertisements, they are fighting for the peanuts left behind.
• Downturns can hit hard. Companies spend on advertisement from their marketing budget which tends to be a variable expense. Hence, during a downturn or recession, ad-funded businesses are hit hard as companies stop spending this variable expense.
• Lack of consistency. Revenue from advertisements is seasonal.
• Converts your business into a commodity.

Bad for users

• Ideally, you must under-optimize. Adopting ads as a revenue model is all about managing tradeoffs: How to serve the needs of advertisers while retaining user delight.
• Eventually most will compromises on the product. Ad-funded economy has fused with acquisition through SEO. Hence, organizations tend to prioritize quantity over quality, i.e., create many low-quality articles to game the algorithm. Additionally, each story is valued by the page views it gets. Given that, a well-researched article and a funny cat video are measured by the same metric. Finally, the UX is extremely cluttered. It is this bad UX that has led to the creation of ad blockers.

Historical context

What’s next

The pivot to privacy brings in new opportunities!

• The Internet is sunsetting the third-party cookie. Various governments are enforcing stricter privacy rules. Finally, platforms like Apple are by default disabling tracking.
• Companies that have implemented their own DMP that tracks First Party Data will be able to target advertisements better. For example, Vox’s Chorus.

Not all page views are made the same.

Fill Rate = Ad Impressions / Ad Requests

Render Rate = Ad Impressions / Ad Code Served

Viewability Rate = Viewable Impressions / Ad Impressions

Ad Density = Ad Impressions for a given Ad-unit / Pageviews

Ad Coverage = Total Ad Impressions of All Ad Units for a given page / Pageviews

How does that make you feel?

• But what about tomorrow? What about the improvements you are considering making that none of your customers have seen yet? No segmentation can help you evaluate these options since you have no data.
• The only way to understand how the future will affect your customers’ happiness is to speak with them. That might not sound very data driven, but you need to treat customer discussions like data collection exercises! If you don’t, you may unwittingly introduce bias into the feedback you collect which will drive you to false conclusions.
• When getting ready to talk to customers about new features and updates, make sure to always do the following:
  • Randomly Select Customers. Don’t just talk to the customers you already know well, or the ones that are the easiest to reach. Always randomize the customers you collect feedback from to ensure they are a good sample. Remember that not all customers will have time to talk to you so choose a large enough group that even though not all will participate you’ll still have enough interviews.
  • Always ask the same Questions. It can be tempting to simply sit down with customers to have a discussion about prototypes and designs. However, such unstructured discussions give you uneven feedback and make it difficult to avoid bias. Be sure to have the same set of questions for all customers.
  • Focus on Feedback, not Imagination. Customers often have a hard time imagining ideal solutions to their problems, but they are great at providing feedback about potential solutions you put in front of them. Instead of asking them how they would like you to solve something, show them some options and allow them to choose.

Those of you familiar with survey design will find these familiar! That’s because the same rules of good survey design apply to customer interviews as well. There is a great detailed breakdown of survey design available from the Pew Research Center.

Even after doing these things it might seem difficult to turn qualitative feedback into data driven decisions, so tomorrow we’ll cover more sophisticated models of processing feedback (known as Customer Satisfaction models). They will help you turn your customer interviews into a highly data driven exercise!

Questionnaire design – Pew Research Center Methods | Pew Research Center (archive.org)

Experiment-driven development entails running tiny tests to evaluate options/ideas that you’ve in an iterative and structured fashion to incrementally achieve results.

Why it matters

Strategically

• Always scale after achieving fitment. Optimizing too early stunts iterative discovery.
• Incrementally discover and improve towards a larger goal.

Operationally

• Speeds up discovery. Short iterative experiments will pop-up the unknowns. Additionally, the process forces you to stop and reflect.
• Encourages risk-taking: An experiment might fail to achieve what we set out to do, but it succeeded in teaching us something new about audience. Additionally, each experiment is tiny enough that it does not result in outright failure of the product.
• Helps convince. Gives you ammunition to convince others with evidence.
• Helps seek permission. It comes from the vantage point that you don’t know and hence you will experiment. This helps bypass egos and strongly held positions. Additionally, each step is tiny so its easy to convince stakeholders to try it out.
• Promotes customer-centricity: Each experiment’s output is based on behavioral analytics.

How

• Build a culture of asking questions and proposing explanations as hypothesis based on the limited information available.

• Estimate data for each of the hypothesis using the RICE model.
  • Reach: How many users will use it?
  • Impact: What outcomes (business metrics) will it grow and by how much?
  • Confidence: Given your capability and constraints, how difficult would it be to test this
  • Effort: What would it take to implement this
• Prioritize which hypothesis to focus on
• Design your tests
  • Define context: What do you know? What do you have access to? What do you think is going on? What do you want to find out?
  • Summarize prior research: How have you experimented in this area before? What did you learn from your results and analysis? What further research do you need to do before you start the experiment?
  • Obstacles and risks: What are the risks in the following areas: Business, Usability, Technical? How will this affect? How will you mitigate these results?
  • Success: What is the primary success metric? What is the secondary metrics that this experiment will impact? What is the benchmark today? How much do you expect it to change?
  • Who and how many: Who will you test this on? Use Sensitivity Analysis to identify how large should be the sample size.
  • When: When will you conduct this test? How long will it last?
• Conduct the tests
  • Do anyone need it? Qualitative Research, Interviews, Paper Sketch.
  • They need it. Would anyone use it? Build an unscalable version, for example, prototype in Figma.
  • Does a significantly large audience need it? Digital advertisement campaign with a landing page
  • How do we build the product properly? A/B Testing, Multi-armed Bandit

Events such as breaking news, summits, natural disasters, and notable deaths cause an immediate surge in interest, which then quickly declines.

• Many online editorial teams end up producing numerous short shelf-life stories because they tend to stay on top of current events by watching Live TV, observing wires copies, and monitoring social media.

Below is Google Trend on the keyword Bipin Rawat.

Topics, like Covid-19, Fitness, etc. maintain consistent levels of traffic over extended periods because of enduring relevance.

For every query, Google Trends returns data that is normalized in relation to a peak (100). Here’s the response when you search for the topic Yogi Adityanath.

Lets run a similar query for General Bipin Rawat

This makes it impossible to compare these two terms. Hence, we introduce a neutral term – say “Google News”. And now we get these results.

https://trends.google.com/trends/explore?date=2021-12-01%202021-12-31&geo=IN&q=google%20news,%2Fm%2F0h4cxd

How to normalize? We need to make the traffic data from Google Trends comparable between Google News, Yogi Adityanath, and General Bipin Rawat. 

Response from the query for General Bipin Rawat on December 8, 2021.

• Google News: 1
• General Bipin Rawat: 100

https://trends.google.com/trends/explore?date=2021-12-01%202021-12-31&geo=IN&q=google%20news,%2Fm%2F09k76ff

Response from the query for Yogi Adityanath on December 8, 2021.

• Google News: 100
• Yogi Adityanath: 28

Normalized result using min-max scalar.

• Google News: 1
• General Bipin Rawat: 100
• Yogi Adityanath: 0.28

The business case for launching Apps

• Apps are great for Digital Advertisements based media products because even if your app user base is a fraction of your browser user base, the app user base will give you drastically higher Sessions Per User. This is generally because it is easier to build Habit-forming on apps due to Push Notifications.
• Additionally, the eCPM rates on the app is much higher than browser eCPM.
• App users tend to convert better to buy Subscription.

Do competitor research with AppAnnie

Paid or Acquired Traffic: As of 2020 in India, it typically costs Rs. 50 to acquire mobile app downloads. For these users, you’ve to be confident that the product is strong enough to grow Sessions Per User so they stick around and give you enough revenue in the long run. Another way to track this is to look out for Uninstall to Install ratio.

It seems most media businesses, at least in India, have not built Profitable mobile apps.

Habit-forming products have a clear trigger-action-reward.

• In editorial products, push notifications, newsletters, and podcasts tend to be habit forming.

Why it matters:

• Habit-forming products increase sessions per user or DAU/MAU, i.e., they get users back to your platform regularly.
• This in turn drops churn.
• Consistency leads to trust.

Examples

How Supreme cultivated a purchase habit

You’ll not find Supreme’s goods on Amazon or in another store.

Trigger: Like clockwork, every Thursday at 11:00 am during the spring and fall season, Supreme drops a limited collection of merchandise on its website. This fixed time and location reinforce the habit. Customers know when to expect new merchandise.

Action: Hence, they keep coming back to the Supreme website every week. It also creates excitement, rumors, and conversations within the community.

Reward: Find out what’s new.

The Times of India’s Start Your Day With brought the newspaper’s funny pages online

Print newspapers have always had the back-of-the-page infotainment section page. This page typically includes comic strips, brainteasers, crosswords, horoscope, events listing. Traditionally, it has been one of the pages which has had high engagement and time-spent with newspaper readers.

Trigger: Before cell phones went mainstream, it was common to see people interacting with this page during their time off. While Page One was the morning trigger, the funny pages with cartoons, games and comic strips was the reason you picked up the paper later in the day or carried it around with you. It was also one of the early use-cases for young readers — many began their newspaper reading with the comic pages.

Action: Read or play for entertainment.

Reward: On reading this page, the reader got clear well-established rewards:

• The editorial cartoon informed and entertained
• The crossword and brain teasers gave a sense of challenge and achievement
• The listings and horoscope had straightforward utility

What do journalistic investigation, dating, sales, and hiring have in common? Each of them are funnels towards insight, partner, customer, and employee respectively. This same concept can be expanded to value generating outcomes like Conversion, Retention, Ad Click, etc.

A funnel is a series of events or actions that customers need to take leading up to a value generating outcome. You start broadly (Top of the Funnel) and then increasingly narrow down into what you want.

Why it matters: A funnel models reality and exposes steps in the funnel that need to be optimized.

How do you know it is good?

• Funnel conversion rate benchmarks vary based on geographies, product categories, and industries. Always be on the lookout for openly available benchmarks like here and here.
• For ease of analysis, use Segmentation to simplify multi-path funnels into single-path lines.

Optimizing funnels:

Actively use A/B Testing or Multi-armed Bandit to optimize your funnels:

• Increase Top of the Funnel but that costs money
• Change the user experience to drop drop-offs
• Use Propensity Modeling to provide Discounts and Promotions, etc. at the right spots
• Drop steps in the funnel much like Amazon’s one-click purchase

Build and maintain an Audience Funnel

Onboarding helps orchestrate evolving users from one segment to another.

Why it matters: Like all people, customers change over time and go through a Customer Lifecycle. The more a customer learns about your product or service the better they will make use of it and the more advanced their usage will become. Onboarding helps them evolve.

• In Subscription products, it helps users get the most out of their purchase. This in turn, increases retention rates and LTV. The converse is also true. People will churn if you’ve not established a relationship with them. Onboarding can help you activate sleepers — subscribers who pay but don’t use the product.
• For users who have recently installed your Mobile Apps, onboarding can help improve D1 retention.
• Additionally, as users perform actions, you’ll end up collecting First Party Data which in turn ends up cold-starting Personalization.
• Fitness products track and reward usage streaks to increase DAU/MAU and sessions per user.

How

Setup: On sign up or purchase, thank them immediately and offer help to engage further with your product.

• Sometimes this can be a ‘Complete your setup’ wizard with few mandatory steps. Other items, it is a sticky widget that stays until the user completes those steps.
• Typically, you’ll get them to engage with all your Channels: Websites, install Mobile Apps, sign up to Newsletters, subscribe to Push Notifications and even validate email or open a Push Notifications.
• You can also get them sample your features and content across the Information Architecture.
• Celebrate if the user completes all of these steps.

Auto-responder campaign: You also want to run a Newsletters campaign that introduces the user to your mission, introduces your staff, serve them your most popular content, and ask them questions about their motives and interests. This could be as simple as asking them about their favorite local restaurant. The goal is to kickstart a two-way communication.

On-platform nudge users to towards specific actions. For example, ask them to leave their first comment, poll their opinion about topics, after a story ask them what else they they would like us to cover or share tips on what they could have done better. Some platforms like Stack Overflow offer points to encourage usage.

Risk: Beyond this point, you cannot keep bombard or overload the user with everything in one go. Instead, your must progressively narrow down to specific personalized actions, features and topics.

• +1: Encourage your users to share the product with someone else in their Owned Network.
• Invite them to an exclusive event or for a Ask Me Anything with your staff
• Build Transformative work products for them so they can do deeper into certain topics
• Ask them to follow staff on Social
• Constantly communicate to them about the new product features that you are launching for them

How long: You could start off with a fixed period onboarding but ideally you should go on until user graduates to desired engagement level on the Customer Lifecycle.

Running push notifications

• Etiquette
  • Have a per user day cap, i.e., have a upper limit to the number of push notifications that can be sent to a user per day.
• Measuring performance
  • Setup a funnel to track: Push notifications sent, Delivered, Opened, and Clicked.
  • Be on the look out for audiences unsubscribing to push notifications or uninstalling your app after receiving push notifications.

What to optimize

• Consistency
• Optimize CTR
• Explainability in stories
• Reading Windows

These are audiences whose identity you know. It could be as simple as an email address or phone number. To acquire them, get them to login to your website, download mobile app, or subscribe to your newsletters. For such audiences, you can use personalization to drive up pages per session through smart homepage, newsletters, push notifications, and onboarding.

Why it matters: You should collect their data in a customer data portal and use it improve your engagement, drive subscriptions, relationship with audience, etc.