Helping the Helpers: Evaluating a GenAI-powered assistive chatbot for caseworkers
This report evaluates the effectiveness and implementation of a generative AI-powered assistive chatbot designed to help caseworkers navigate complex public benefit programs like Medicaid and SNAP.
The evaluation utilizes a randomized controlled trial and a 14-week real-world pilot in Los Angeles to demonstrate that the chatbot improves caseworker accuracy by an average of 40%, with the most significant gains occurring in the most difficult client scenarios.
While the tool reduced learning costs by providing plain-language summaries of dense policy manuals, the study noted that successful adoption requires ongoing organizational support, as usage tended to decline over time without sustained engagement strategies. Additionally, the findings highlight an “accessibility gap,” showing that while the chatbot is more readable than original source materials, it still operates at a 10th- to 12th-grade reading level, suggesting a need for further simplification to support a broader range of staff and clients.
Share this Resource:
