Policy2Code Demo Day Recap
A recap of the twelve teams who presented during the Policy2Code Prototyping Challenge at BenCon 2024.
The Policy2Code Prototyping Challenge—organized by the Digital Benefits Network at the Beeck Center for Social Impact and Innovation and the Massive Data Institute, both based at Georgetown University—culminated in the Policy2Code Demo Day on September 17, 2024 during BenCon 2024. Twelve teams presented their experiments, prototypes, and other developments to an audience of technologists, data scientists, civic tech specialists, public benefits advocates, non-profit organizations, private industry consultants, researchers, students, and city government officials for feedback, awareness, and evaluation.
Key takeaways from the demos include:
- Currently available web-based chatbots, such as OpenAI’s ChatGPT and Google’s Gemini, have high error rates and are likely giving inaccurate information to people seeking benefits today.
- It is possible to improve results by integrating specific policy documents into the pipeline (as Retrieval Augmented Generation (RAGs) or through fine-tuning).
- Results can be further improved by giving a “template” of what to expect.
- Approaches that incorporate human-in-the-loop evaluation and verification will be more successful in the immediate future.
- We have a lot to learn and improve before we can fully automate policy to code.
Share this Resource:
