“Carly, I did it. I finally tried ChatGPT!”

My mom called me on a Tuesday afternoon, proud of herself. She’s in her 70s, living minimally off Social Security and a small pot of retirement savings that she has watched like a hawk for years. She still walks into her bank branch, waits for her favorite teller, and manually deposits checks while catching up on everyone’s business. She is not, by anyone’s definition, an early adopter.

But she’d recently gotten a letter about changes to her Medicare costs. Instead of calling me in a panic, she opened ChatGPT and asked what the letter meant for her monthly budget.

I was thrilled! I was also a little nervous.

Here’s the thing about my mom: for years, she ran her retirement scenarios on a notepad. Not Excel. A notepad. Oftentimes at 3 a.m. Her financial anxiety is grounded in lived experience. She’s experienced real financial instability both as a child and a mom raising three kids, working minimum-wage jobs into her 60s, and navigating the soaring rents of the Bay Area rental market on a fixed income. And here she was, using a general-purpose LLM to help her understand one of the most consequential letters she’d received all year.

Carly Perera and her mother circa 1985.

Medicare costs aren’t simple. They’re income-dependent and interact with Social Security in ways that can catch people off guard. For my mom, interpreting changes to her monthly budget depends on understanding her full financial picture, including her income sources, benefit mix, housing costs, and overall savings. 

This time, the answer she got back from AI was solid, accurate, and helpful. But what about the next time she has a high-stakes question that keeps her awake at 3 a.m. and the model gets it wrong? 

Financial Health Network research tells us that 69% of Americans are not Financially Healthy. They’re carrying debt, navigating benefit cliffs, and making financial decisions with no margin for error. And they are increasingly turning to AI for financial guidance, whether it’s through an LLM or, very soon, their bank’s app. 

I’ve spent my career working at the intersection of financial health and innovation, and I know the people building AI tools right now genuinely care about their customers. Product teams are generally taught to design for the majority  and deprioritize everyone else. The problem is, when it comes to financial health, people like my mom are the majority—they just keep getting treated like edge cases. 

Many AI models are built on datasets that overrepresent financially stable, higher-income populations. This means the models learn patterns and behaviors that may not reflect the realities of Financially Vulnerable people living paycheck to paycheck. In this reality, even technically accurate financial guidance can miss something critical about a person’s financial life. 

If AI is going to power the way millions of people receive financial guidance, we need better ways to design and evaluate these systems.

Builders need representative data, real-world context, and better ways to identify where financial guidance can fall short. And that’s what we’re building. 

For 20 years, the Financial Health Network has been sitting inside the lives of people like my mom. We know the financial challenges that keep people up at night, what questions they’re too embarrassed to ask anyone else, and where the conversation loops don’t close after a family member says, “Sorry, mom, I have to go to bed now.” 

Most importantly, we know how a well-intentioned tool that gives technically accurate advice can land like a grenade in a household budget with almost no margin for error.

We’ve built in this space before. Through our Financial Solutions Lab, we partnered with over 80 fintechs and nonprofit organizations at a time when almost nobody was asking early-stage builders to think about financial health. The companies we worked with helped customers build $3 billion in savings, settle $200 million in debt, and avoid $420 million in fees. That experience taught us what can happen when builders are equipped with financial health insights and infrastructure from the very start.

AI is accelerating faster than fintech ever did. We don’t have decades—or even years—to play catch-up. Financial health must be the new normal for AI.

We know what it takes to bring a financial health lens into a builder’s room and we’re doing it again, for AI. We’re building the Financial Health Network AI Toolkit: a practical kit that gives builders what they need to design, train, and evaluate AI financial tools for this population.