[London School of Economics JournalismAI] How to think about AI for news? - Ritvvij Parrikh [London School of Economics JournalismAI] How to think about AI for news? | Ritvvij Parrikh Humane ClubMade in Humane Club

[London School of Economics JournalismAI] How to think about AI for news?

26 JUL 2023 Wednesday 12:30 Asia/Kolkata
Duration :
3 hour(s)
Journalism Ai Training
Block Pattern: Regular

This session was originally written for a three-hour training that I conducted as part of JournalismAI Academy APAC cohort 2023 on implementing AI for news.


As leaders, it is our job to shape the product so every investment produces an ROI.


Grounding

There are no easy answers because on one hand you have to play the AI game and on another hand you have to balance out reality. News is a low ARPU commoditized business.

History of bad decisions

We are left behind with lesser and lesser variables in our control to do something about our own business. Hence, as an industry, sometimes we choose to spend hard-earned cash on things that don’t improve competitive differentiation. Below are some common decisions:

Supply-side efficient innovation: In absence of a clear strategy to grow revenue (demand-side innovation), we choose to invest on copy-cat products that do the same thing but a little twist. Hence, most cash rich news companies have built their own custom-built CMS, grown it, realized it isn’t adding much and finally abandoned it. This problem is compounded by the fact that news companies struggle to build technology.

Novelty: To demonstrate that the iPhone could be further slimmed, Steve Jobs famously threw an iPhone prototype in water highlight air gaps. If such qualitative feedback adds to your differentiation then it is useful. However, sometimes we invest hard-earned dollars building products to accommodate such niche feedback from our leadership even though our audiences might not really care about it. For example: UI instead of UX and your method of journalism.

Surfing BigTech Waves

Rule-based complexity: Finally, because we are a low margin business, often business and product folks over-complicate the product by adding many custom rules to extract an additional 0.5% revenue. This makes the organization and product so complicated that no one wants to take the risk of changing it after that.

Obsession with new: My hunch is that at some level the nature of our industry — covering what’s new — influences what excites us. Hence, the news ecosystem has a graveyard of shallow POCs that do not meaningfully move the needle on anything constructive.

Competing against BigTech

A media CXO on OpenAI using their archive to train GPT.

They came, they scraped, they went.

Cash-flow versus Balance-sheet businesses: Most BigTech are balance-sheet businesses. They raise a tonne of capital from VCs to invest today and earn revenue in the future. In contrast, news tends to be a cashflow business, where we need to balance our P&L every year and we spend from what we typically earn.

Beyond the obvious power and leverage disparity, the reality is that big tech companies have immense cash reserves and technical skills that have managed to build and deploy AI at scale. We don’t.


The Generative AI Wave

Let’s evaluate the Generative AI wave through the above lens

Obsession with new

  • Most major newsrooms have jumped into retooling their CMS with OpenAI.

Supply-side innovation

  • Do we’ve the technical manpower or skill to build our own LLMs? No.

Will Using LLMs Help Me Save Editorial Costs?

Surfing BigTech’s waves

  • By building on OpenAI, will a newsroom be able to build competitive differentiation? No. Because everyone else has access to it.
  • In fact, LLM is the latest technology that is forcing newsrooms to move more inland into cost-heavy operations like fact-finding and writing the first draft. Many of the You’ll spend days investigating and in seconds it will get scrapped out.

Novelty

  • Will we’ve newsrooms and editorial leaders obsessing over the nuances of prompt-writing so we can get it just right? We already do. Will it lead to competitive differentiation? No!

Economic viability

  • Can we afford to build our own LLMs? It would take upwards of a few million USD in GPU server costs to train a LLM.
  • Can a newsroom afford to build demand-side, i.e., audience-facing LLM products? No. Because the cost of serving a cached page from CDN is 1/1000th the cost of a meaningful Q&A with OpenAI APIs. The unit economics doesn’t add up.

The Subscriptions Wave

Here’s what we can learn from the subscriptions wave:

  • The underlying conditions — the edging out, loss of control, etc. — that mandate the need for reader revenue have been around since the 2010s. Yet, it is only in the last couple of years that everyone has jumped onto the subscriptions bandwagon!
  • Most of us jumped in without thinking extremely deeply about it and many of the limitations of the subscriptions revenue model are becoming imminent now.

How to bet on a Wave

There are two variables to consider:

  • How large is your investment? The larger the cash spent, the larger the risk of downside.
  • How large is your bet size? The larger the bet size, the larger the upside potential from the bet.

The most successful options have been at the edges (extremes).

  • Big bet. Small investment. Typically, this involves small newsrooms that rely entirely on ready made platforms like Substack for their technology costs.
  • Big bet. Big investment. These are the New York Times and Financial Times of the world who bet big on reader revenue and also cannibalized their existing revenue lines to achieve it.

Most options in the middle don’t tend to yield much:

  • Small bet. Small investment. Typically, this option is used by newsrooms for improving revenue diversity. For example, The Guardian has thrown in a donations nudge everywhere but doesn’t rely on it for cashflow.

Preparing for the AI Wave

I’d recommend picking one of two extreme paths — Compute-heavy or compute-light — while acknowledging that adopting AI is fraught with challenges.

I’m choosing to call it compute-heavy and compute-light instead of a big and small newsroom ‘coz LLMs can enable a small newsroom to have big reach, much like, how WhatsApp had a 35-person staff but was serving 465 million MAU.

Below are some suggestions for each group:

Compute-light

Before betting big on AI, you should challenge your existing conventional cost structures by dropping functions and outsourcing non-core functions.

Drop functions: Reduce complexity of your systems and processes or up skill editorial staff such that the core value creators — editors, reporters, journalists — can directly provide value to audiences. This can help shave off 40-70% costs.

Outsource all non-core functions: Yes, you’ll be under-optimizing and will have to live within limitations but that’s fine because it will free up the newsroom’s cognitive bandwidth to fully adopt AI. Below are some best practices:

  • Get vendors and suppliers who can take ownership of entire chunks of the various functions required to run a news company. For example, a vendor for your entire WordPress site.
  • Go for vendors that charge a fixed flat-fee so that you know your costs as early as possible.
  • Ensure you retain the flexibility to exit the vendor with your audience, identity, commerce, and content.
  • Avoid spending a fortune on complicated integration. For example, a mid-sized newsroom in India spent a fortune to deeply integrate their website with a subscriptions vendor. Now they want to exist but have spent most their cash available to dedicate to this specific problem.

Compute-heavy

Hire and build highly skilled team (top-heavy) that can design and scale AI at scale at low cost on in-house servers. Task them to build models that require extremely low training computation and provide inference extremely fast.

Humans should vacate most of the tasks that AI automates freeing up bandwidth for cost cutting or going up the value chain.


Strategy for the AI Wave

Let’s again revisit the realities of our business:

Next, let’s review what what mainstream artificial intelligence already solves for! Once you see this, you’ll realize a commonality between what we’ve been edged out of and what mainstream AI already solves for, a.k.a., amplification.

Given this, here’s my suggestion:

The most obvious point is that…

Both compute light and compute heavy should invest in reporting and news gathering.

Beyond that their strategies will vary:

Compute-light

Most compute light organizations should avoid building anything custom, move to WordPress and then integrate with tools that integrate with WordPress. Alternatively, you can piggy off one platform like SubStack — however, that’s risky! Most of these tools and platforms will give you clear tools eventually to automate or augment the amplification factors.

If you must invest your hard-earned cash in AI then I would highly recommend that you only invest in building competitive differentiation on the demand-side. Your audiences should recognize that differentiation and be willing to pay additional cash for it. For example:

  • Owned Identity: May be if you’ve a unique visual drawing style, like Sketchplanations has, then train an AI to deepen this unique style.
  • Owned Network: Nintee is a new media-enabled personal coaching that already uses AI to deliver content and audience engagement to audiences on Discord.
  • Owned Media: You can use AI to publish professional work products like The Information does. Alternatively, you can use LLMs to publish summaries and catch click-baits as Artifact News does. Finally, if you’ve paywalled content that has not been scrapped, then like Skift you can launch bots that audiences can chat with.

Compute-heavy

Compute heavy newsrooms should invest in optimizing both driving and amplification factors

For example:

  • A clear driving or differentiation factor that mainstream newsrooms can work on is time to market, i.e., how fast do they break news. For this, most functions post reporting and news gathering can be near automated using generative ai.
  • On the amplification side, they can go multi-format and provide the same news in 20 different formats to micro-optimize gains from algorithmic distribution for each format.

Jobs for the AI Wave

To answer this, I would like to pick up three slides from Ezra Eeman’s presentation at WAN IFRA where he used a clean framework for categorizing how AI can help newsrooms.

He splits all AI investments into three buckets: Automate, Augment, and Transform. We went through each of these tasks and each of the 9 newsrooms tagged whether they’d like to use AI for each of these tasks in their newsroom.

Automate with AI

These are those tasks that can potentially be fully automated leaving staff to pick up more sophisticated tasks. These tend to be repetitive tasks that if automated can boost productivity.

Automate news with AI
TaskWant to useDon’t want to useDin’t tag
Editorial Analytics88.89%11.11%11.11%
Automated tagging88.89%0.00%11.11%
Surface relevant stories88.89%0.00%11.11%
Transcription66.67%11.11%22.22%
Image cropping and editing66.67%11.11%22.22%
Predictive planning55.56%11.11%33.33%
Text to speech55.56%22.22%22.22%
Data structuring44.44%11.11%44.44%
Comment moderation33.33%22.22%44.44%
Automated stories33.33%44.44%22.22%
Print automation11.11%22.22%66.67%

Augment with AI

These are tasks where a human augmented with AI can perform much better than a human or AI alone. This typically involves combining the pattern recognition of AI in tasks that require deep judgment.

Augment news with AI
TaskWant to useDon’t want to useDin’t tag
Headline prediction88.89%0.00%11.11%
Personalization77.78%11.11%11.11%
Content ideation66.67%0.00%33.33%
Detect trends66.67%11.11%22.22%
Promotion monitoring66.67%11.11%22.22%
Bias Detection44.44%33.33%22.22%
Archive optimization44.44%11.11%44.44%
Content mix33.33%22.22%44.44%
Content variation33.33%22.22%44.44%
Intelligent paywall and propensity models22.22%66.67%11.11%

Transform with AI

Finally, there are tasks that can completely be rethought or reinvented for the coming years using AI.

Transform news with AI
TaskWant to useDon’t want to useDin’t tag
Contextual personalization of formats77.78%22.22%0.00%
Content performance55.56%0.00%44.44%
AI crowdsourcing33.33%33.33%33.33%
Generated interfaces33.33%22.22%44.44%
New bots33.33%22.22%44.44%
Conversational archives33.33%22.22%44.44%
Synthetic avatars11.11%44.44%44.44%

Other

Below are some other ideas that the cohort came up with:

  • Predict the best times to publish
  • Archive all PDFs published by the government
  • Scrape, clean, warehouse data published by the government
  • Recommend experts to interview for a story
  • Personalized greetings
  • Automate explanatory news
  • Automate story writing from datasets
  • Auto hyperlink content from archive in new stories and maintain it
  • Auto check the company’s style guide