AI Outcomes Reporting
Developing an AI-powered reporting solution delivering continuous value to customers to reduce churn.
TL;DR
The Opportunity
How might Resilia, a tech-for-good capacity-building platform, consistently demonstrate the impact of nonprofit engagement on the platform in furthering a grantmaker's mission, to reach their revenue goals of $10 million in Annual Recurring Revenue (ARR) through renewals and expansions?
The Outcome
An AI-powered solution focuses on boosting engagement among nonprofit users to meet capacity-building goals while automating outcome reporting to grantmakers to drive expansions and renewals, and to external donors to increase revenue through Resilia’s donations platform.
The Problem
Resilia is a venture-backed, "tech for good" startup developing SaaS solutions for nonprofit organizations to become high-performing and enable funder enterprises (corporations, cities, and private foundations) to measure and scale the impact of their grant deployments.
Reporting to grantmakers was focused on NPO user engagement within the Resilia ecosystem, misaligning with customer expectations on outcomes. Stories were collected anecdotally by the customer success team, they did not reflect the outcomes of the larger cohort’s engagement within the platform.
The business impact
$3M+ in churned accounts due to customer dissatisfaction
High manual overhead of reporting taking time away from relationship building with customer
Disengagement of grantmaker customers in-between monthly touchpoint, reducing the likelihood of renewals
My Impact
As the Director of Design at Resilia
People
Coached IC Design Lead to develop experimentation process for accelerated releases, and built cross-functional partnerships with key stakeholders in CS to achieve shared revenue and engagement goals.
Process
Hands-on in launching experiments to mitigate risks in Large Language Model (LLM) reporting while enhancing nonprofit engagement.
Designed POC to align executive stakeholders with strategy.
In partnership with CS Operations and the VP of Product, designed a scalable service model to enable automated reporting to grant makers,
Practice
Maintained a high standard of quality to increase user satisfaction by 40% on this high-value workflow to drive engagement.
Developed a repeatable framework for experimentation across the R&D function.
The Approach
Learning Fast to Mitigate Risks
The primary risk in our initiative was the misalignment between our assumed intended outcomes and the actual goals of nonprofit organizations. This could lead to low engagement from nonprofits and insufficient data for our Large Language Model to generate compelling outcome stories. Partnering with the Design Lead, we developed a strategy to simultaneously collect data on the needs of our nonprofit users, and train our LLM to produce summary reporting. Our approach leveraged tools like Appcuses and Amplitude to launch and measure experiments in less than a sprint, without engineering overhead.
Strategy to Collect Insights and Train LLM
Early Engagement Wins
In partnership with the PPM and Customer Success teams, we launched an updated Nonprofit Needs Assessment within the app, enabling both of our teams to set benchmarks for what a NPO user’s greatest needs areas, drive activation and retention through targeted campaigns, and iterate quickly to close gaps in offerings to prevent dropoff. The new needs assessment automated the goal and task creation process within the Objectives tool of the platform, driving a >60% increase of usage, leading to higher nonprofit engagement.
AI-Powered Reporting
We utilized Appcues to deploy a brief survey triggered after users completed platform activities, asking about intended outcomes toward their goals. The responses, combined with task-related data, contributed to training our LLM, establishing the foundation for future automated reporting to grantmakers and nonprofit users, that shows the link between platform outputs and mission-related outcomes.
Nonprofit Impact Story
Scalable, Automated Reporting for Grantmakers
The implementation of LLM reporting empowered the Resilia Platform to aggregate outputs and outcomes from a grant funder's portfolio of over 30 sponsored organizations, offering real-time insights and benchmarking for assessing overall impact. In partnership with CS operations and the VP of Product, we formulated a long-term service blueprint to ensure ongoing engagement with nonprofit and grantmaker users. Our solution streamlined the report generation process, minimizing the workload for the customer success team, and enabling them to establish deeper connections with grantmaker customers. On-demand reporting introduced a novel grantmaker value proposition, contributing to our ambitious goal of achieving $10 million in Annual Recurring Revenue (ARR) through renewals and expansions.
The Outcome
Early 60% increase in engagement
The launch of a needs assessment with auto-generated goals resulted in increased nonprofit engagement within the objectives feature and successfully facilitated the training of LLM for reporting purposes.
Competitive Advantage
AI solutions differentiate Resilia within the Tech for Good and nonprofit sectors, where adoption is still limited.
Blueprint for Repeated Success
The project's rapid experimentation process sets a precedent for future R&D team experiments, reducing overhead to de-risk solutions and providing a valuable learning experience for the broader design function in incorporating AI into design solutions.