PROJECT BRIEF

BACKGROUND

The City of San Jose, CA is a pioneering municipality in acquiring and implementing artificial intelligence (AI) systems and services for the benefits of  employees and residents.  As the founding partner of the GovAI Coalition, the City leads the Coalition in proposing adaptable, standardized systems for ensuring safety by thoroughly  assessing and mitigating AI risks.  San José wants to create a systematic, standardized AI risk assessment process that is easily adaptable for all GovAI coalition partners within their respective procurement processes.

AIMS

This project proposed an AI system risk assessment framework, in a systematic and adaptable manner, for various use cases during the software lifecycle such as: procurement, annual software audits, and software department transfers. Preliminary research posited a standardized method to evaluate risk of AI systems with concrete indicators to be used by the City of San José, CA’s Information Technology department and GovAI Coalition members. The AI Risk Assessment Framework (RAF) is vendor-agnostic and can be used within the procurement process and post-procurement. 

METHODOLOGY

The project team reviewed the City of San Jose’s current documentation on AI risk review and also evaluated federal policy on AI risk management and existing public frameworks. Case study analysis on existing state policy and legislation highlighted discrepancies between existing documents on how risk levels and acceptable risks for AI systems are defined in current documents (e.g. personally identifiable information, opt-out policies, predicted harm) and other AI risk management frameworks. 

DELIVERABLES

Project deliverables included (1) AI risk assessment framework to standardize the risk review process, (2) a tiered AI risk assessment matrix, and (3) an AI risk training presentation and summary for information technology employee education.

IMPACT AND FUTURE WORK

The AI Risk Assessment Framework for San José, CA and the GovAI Coalition, represents a significant step to ensuring that the acquisition, deployment, and use of AI systems within City operations are conducted in a manner that is ethical, transparent, and aligned with the community’s values. This document implements a standardized benchmark for AI risk assessment, and the proposed policy changes highlight the City's Commitment to leading by example in the ethical use of AI technologies.