Project Snapshot: California Employment Development Department, User Personas for Government Technology Procurement

Problem Statement

The California Employment Development Department (CA EDD) has over 10,000 employees across 8 branches. The agency administers multiple benefits programs including Unemployment Insurance (UI), Disability Insurance (DI), and Paid Family Leave (PFL). During the pandemic, CA EDD saw an increased volume of UI claims which exceeded their processing capacity leading to backlogs and delays for claimants. CA EDD also faced challenges appropriately detecting fraudulent claims, without blocking eligible claimants.

Coming out of the pandemic, CA EDD is pursuing a multi-year modernization initiative, EDDNext. The initiative aims to improve customer and employee experiences across EDD’s programs. As part of this initiative, EDD sought to find user-centric strategies to procure new technology and systems for an updated digital identity verification system.

Project Description

As part of the overall EDDNext initiative, CA EDD hoped to update the identity proofing process for claimants, minimizing friction and bias as much as possible to improve and streamline customer experience across the various programs that the agency administers. When procuring new identity proofing technology, the department wanted to find ways to center customer experience in the process of selecting an identity proofing product. CA EDD ultimately chose to deploy user personas in their evaluations of vendor products.

To learn more about the user persona approach, the Digital Benefits Network (DBN) team interviewed a project manager, IT manager, section manager, software architect, and a supervisor for application development at CA EDD involved in the project.

As learned from an interview with the IT Manager, CA EDD utilized California’s Software Licensing Program (SLP), a form of Leveraged Procurement Agreement (LPA), to conduct market research and explore whether they could procure technology for a new digital identity verification system from a vendor with a pre-existing state contract. Established by the Department of General Services, the SLP enables agencies to negotiate extensive software discounts with authorized vendors through SLP contracts. Part of the motivation for using the SLP was the flexibility that utilizing existing contracts provided to assess vendor technologies. This allowed CA EDD to leverage user personas when evaluating vendor offerings, a strategy they had used on previous projects.

CA EDD generated user personas for external users (e.g., claimants) and internal users (e.g., call center representatives, auditors, security managers) interacting with identity proofing technology at CA EDD. Sample personas included:  – An individual filing for disability insurance who is not “tech-savvy” and has not previously undergone online identity proofing – An individual filing for unemployment insurance who has identity documents with different spellings of their last name – An EDD call center responding to an Unemployment Insurance claimant who is unable to submit requested documentation

These personas were then used to assess whether vendor technologies could address the identity proofing needs of CA EDD users. User personas were developed with input from stakeholders across the agency, rather than just one program, and through consultation with front-line staff from the IT, public affairs, and fraud investigations branches.

The vendor selection process for EDDNext took place over multiple stages. 14 vendors participated in an initial Vendor Day in March 2023. Of that group, 11 vendors were registered with the state licensing system and were selected to respond to CA EDD’s challenge questions. CA EDD then selected 4 vendors to submit technical documentation and participate in a demo day to demonstrate how their products could address the needs of various user personas, considering the level of friction and if the user has to interact with the identity proofing product after entering their information.

The CA EDD team developed and scored vendors using a scoring matrix that was developed with input from different branches across the agency. Evaluation criteria to score vendors included determining the organizational impact of each user persona (how often each scenario occurs and the impact of having additional friction for the users) and cost considerations. Through the vendor selection process, CA EDD ultimately selected Socure to develop and implement the digital identity verification system.

Project Outcomes and Impact

Staff at CA EDD shared that this approach to identifying a new identity proofing product was significantly faster than a traditional request for proposal (RFP) process. One IT manager at EDD we spoke to highlighted that the user persona market research/procurement took 4 months, compared to the 9 months they would expect in a more traditional RFP process. The CA EDD team felt the flexibility of the approach allowed for quicker decision-making and execution.

The CA EDD team shared that they did face some challenges with this approach. A supervisor for application development at CA EDD, involved in developing the user personas, noted that project team members had to change organizational mindsets to obtain buy-in for a novel procurement approach. For privacy reasons, EDD was also not able to provide vendors with historical claimant or internal user data so that EDD could actually test how different vendor tools performed on that data. In order to understand how different products could meet user persona needs, EDD staff members had to evaluate vendor explanations of how their tools would handle each user persona, and what kinds of information or data their tools pulled on. EDD technical experts also needed to evaluate each product’s technical specifications. In some cases, it was also difficult to encourage vendors to respond directly to the user persona prompts, instead of providing a more traditional sales pitch.

From our interviews with CA EDD staff, we learned that the implementation of Socure’s identity proofing product is expected to go live in 2024. In preparation for the launch, an active data study with Socure is underway to assess risk and calibrate the system. Collaborative design meetings are concurrently taking place with stakeholders to ensure a cohesive and efficient user experience.

Replicable Takeaways

While user personas should be modified to fit a given agency’s population and service context, CA EDD’s use of user personas offers a concrete example for representing the needs of internal and external users in a procurement or market research project. The scoring matrix also offers a model for having multiple business units within agencies evaluate vendor responses. Other agencies interested in incorporating user personas into their procurement processes could also expand on the approach CA EDD took by adding more direct engagement with end-users (e.g., claimants or beneficiaries) into the process of developing user personas. If historical claims data can be sufficiently de-identified or otherwise meet privacy needs, other agencies might consider providing vendors with historical data to test performance of different products. CA EDD’s use of user personas in this project provides a useful example that can be adapted and expanded upon by other agencies.

While this research publication focused on California EDD’s procurement strategies, the DBN has been conducting research on digital identity in public benefits since late 2022. If you are interested in reading more about our research, we suggest checking out our publication Promising Practices for Digital Identity in Public Benefits.