Summary

70 million Americans show symptoms of insomnia

Evolve is a YC-backed company that's been building products for about nine years. It started with Podcast App, one of the most-downloaded podcast apps on iOS with more than 15 million downloads, and they're continuing to innovate today.

Podcast App usage spiked late at night because listeners were using it to fall asleep. That led Evolve to investigate sleep care, where roughly 70 million Americans show symptoms of insomnia but access to CBT-I, the clinically recommended first-line treatment, is scarce. Evolve's clinical partner, Ashley Mason, a CBT-I researcher and therapist at UCSF, has an 18-month waitlist and can offer only small-group or costly individual care. Evolve saw an opportunity to deliver CBT-I through voice AI at greater scale and lower cost.

Rest: a sleep expert that never sleeps

Rest is Evolve's AI sleep coach that runs people through a CBT-I program. The program, which was built with sleep experts, typically runs over about eight weeks.

Tracing a voice-and-text agent end to end

All of Rest's AI usage is traced in Langfuse. The voice experience runs on Vapi as the orchestration layer, with ElevenLabs for voice synthesis, Deepgram for transcription, and OpenAI as the underlying LLM. Vapi has a native Langfuse integration, so voice conversations are traced automatically at the click of a button. The text-chat experience is built and traced directly on OpenAI LLM APIs in Evolve's own backend. The text experience, and also non-conversational LLM work such as background extractors that process conversations after the fact, are traced through Langfuse as well.

Error analysis as a product strategy tool

For both Ignacio and Martin, tracing their agents in Langfuse is the foundation for their most important eval practice: error analysis. It's how they find failure modes and act on them, either by implementing and testing a fix directly or by setting up an evaluator to monitor for it going forward.

Martin was clear that looking into traces himself functions as a product strategy exercise. Reviewing real conversations in annotation queues, based on the traces in Langfuse, surfaces which product gaps are quick fixes and which are "a whole big component we need to build... a multi-week, maybe even a multi-month exercise." Staying close to those real conversations is one of the most important inputs into where the team invests next.

On top of manual review, Evolve runs LLM-as-a-judge evaluators in Langfuse to identify memory and forgetting issues.

Deep dive: fixing the AI's memory

The biggest failure mode they discovered through error analysis involved memory issues.

Users were telling Rest's coach things, and the AI would respond in the moment as if it understood and would carry that forward, "sure, I'll take that into account," but then fail to retrieve it in a later conversation. Users noticed. And said so directly. And often.

That kind of failure hits trust hard, even when the underlying product is otherwise working. Investigating it, the team realized memory has to be tackled from several angles and turned it into a project: a genuinely multi-month engineering effort, only now wrapping up. To work towards the right goal they needed to find the right examples from production data. To continuously broaden their understanding of how these failures happen, the team set up an LLM-as-a-judge in Langfuse to catch the moments where users are frustrated specifically because of memory issues. That gives them high-signal input to better understand how to fix it at scale. The efforts paid off: The team was able to get memory issues down from 10% to only 5% of cases over the first iterations already.

Prompt management at scale

Beyond tracing, Evolve manages more than 100 prompts in Langfuse Prompt Management. These aren't static prompts. They are composed on the fly, and Langfuse makes that composition easy to pull off: a foundational prompt sets up the overall conversation, and on top of that sits an agenda of two to four items, each its own prompt that can embed further prompts, including locale-specific variants (English and Spanish, adjusted for users in Argentina, Colombia, and Spain).

The main win, per Ignacio, is that this gives non-engineers, people writing and refining prompts who don't have (or want) deploy access, a safe way to edit and ship prompt changes without touching code or GitHub. Staging and production labels let a prompt be tested before it's promoted live.

Why Langfuse

Ignacio's team found Langfuse through Evolve's YC network, and it stuck. As Ignacio put it, "The centralization of tracing, reliably managed. And the prompt repository was a lucky punch, we jumped to Langfuse for tracing, and eventually we discovered the prompt repository as well, and it turned out to be a game changer for us."

Ignacio credits responsive support and steady, visible product improvement. Martin, running on Langfuse Cloud, is just as confident handling sleep-related conversations there: "we also know that you have very high standards." Self-hosting isn't a strict compliance requirement for Evolve, but he'd still like it as a future option, for more direct control over personal data.

What's next

Deeper eval coverage. The product has grown substantially in complexity over the past year, and Martin wants confidence that core conversation flows are properly regression-tested before shipping changes. For this the team plans to start building out more datasets in Langfuse.

More proactive issue detection, including using the Langfuse API, MCP server, and CLI inside Evolve's own internal tools and admin dashboard, to pull trace data into context faster and catch problems earlier.

Possibly bringing the agent in-house. Ignacio said Evolve is considering building a custom agent harness for more control over performance and latency of voice interactions. Observability in Langfuse stays on regardless - a key advantage of Langfuse's vendor-agnostic tracing.