The hidden data preparation tax

Today, investors are looking for proprietary AI: data and algorithms that can't be easily replicated within existing Foundational GenAI models. While vendor and solution providers' estimations on data preparation have changed from 80% to 40% over the last year, the most optimistic ones are still above 25% of the whole budget needed to be spent on data preparation only, without taking. As Oxford and Cambridge researchers show, synthetic data (generated by AI) is not a solution, "training on generated data makes models forget". Gartner predicts that a lack of data will kill up to 60% AI projects and 88% of POCs within the next year.

What can you do about it

While there is a range of technical solutions, working on partnerships to get the proper data for your training, suggesting pilots in exchange for data usage, using of synthetic data sets for demos and, trial sandboxed may save as well.

• Involve customers: offer free pilots to enterprise clients in exchange for data rights excluding sensitive information, in line with GDPR and UK GDPR
• Engage academia: universities are offering free data sets for students, and may be involved in data annotation. University of Toronto provides already partners with giants like Microsoft 365, Leganto, and Hypothesis
• Share API revenue: leverage assets of the data-rich companies and split model revenues

Operational costs, which grew too fast

“Early stage is focusing on survival, cost management is not a priority; however, for AI, you need to rethink unit economics and how you price your customers to ensure you will get to the series A,” shares a friend of mine who builds AI agents.

AI startups' failure rate is higher than the industry average, with only 15.4% successfully raising Series A. Infrastructure cost, compliance, and data are among the main reasons. Research shows that average computing/hosting costs at venture-backed AIs can grow at a 300% annual rate, which is 6x more than non-AI SaaS escalations. Even though AI startups get $2.6 million of early-stage funding instead of the $2 million industry average, and grew revenue 2x faster than SaaS peers, that gap is still there. I know founders who cut all sales and marketing activities, streaming all funding into development, ending up with a solution looking for a market situation. With AI you need to mind UK electricity costs, which are 2x bigger for powering data centres than USA for instance.

What can you do about it

Finance modelling is hard, tracking the features development and estimation cost is even harder, though we can try to:

• Design EDGE cases and load for both inputs and outputs to assess potential infrastructure cost gaps
• Track infrastructure cost in a spreadsheet, as well as other spending
• Split costs between features based on feature infrastructure usage
• Rank features to understand which ones eat budget and which ones are potential value drivers for customers. You may be surprised at how rarely this overlap occurs

The trial cost no one expected

Imagine working on automated compliance. Heavy standards like HIPAA and GDPR add 10-20% to the project cost alone. You may pay up to $60 for a 5000-word report the customer generates to try your product. Dealing with enterprise leads, up to 1 million messages per month will cost you $1250$ in Chat GPT-4 and up to $2000 with self-hosted LLM. Every move you make and every trial breath you take will cost you high computing costs.

What can you do about it

AI startups also need to design value-driven, cost-effective trials.

• Craft demo cases accurately (avoid spelling mistakes, use real-world examples) for reuse and marketing purposes, include ready-to-use prompts which already keep the context
• Record live demos for those who are still struggling to acknowledge the AI-driven concept
• Limit output power: deliver the first chapter of the report, 10 emails, 7 social media posts, etc. Aim for less, show better quality
• Design trials with UX/UI: buttons, popups, synthetic test data and trial scenarios to push customers to show off the least infrastructure-heavy features

Token Economy behind AI spending

Tokens are slightly different from your Tesco clubcard. There is no straightforward conversion between generated customer outputs and price you will pay LLM provider for them. Token prices vary between $0.10$ and $80, with a huge difference in inputs (what customers provide) and outputs (what your solution generates). A 30-word (≈ 40 tokens) patient query in your healthcare app generates a 500-word (≈ 667 tokens) response with a total price for 10000 transactions between $2.7 and $541.6. German, French and Spanish will cost 2.3x, and Hindi x6.4 more.

What you can do about it

Do a reality check when calculating EDGE cases of customer inputs and outputs. Statistically, it is not what is going to happen, though AI is moving rapidly from innovation to commodity, and scale may happen faster than you've expected. What do you expect to show the most value to the customer? What part is more infrastructure, token-consuming? Do you spend on what drives your customer adoption, or on what your investor or CTO wants to implement?

The bottom line

There is a change in classic tech and business founder dynamics. We need a shared understanding of the solution backbone and infrastructure to prevent the business from failing before it launches. Ask yourself what how much of your budget goes to actual innovation versus keeping the lights on?

For more startup news, check out the other articles on the website, and subscribe to the magazine for free. Listen to The Cereal Entrepreneur podcast for more interviews with entrepreneurs and big-hitters in the startup ecosystem.