Pragmatic AI Advice from UC San Diego Health’s AI Expert
SNI’s Dr. Ashrith Amarnath, UC San Diego Health’s AI expert Dr. Karandeep Singh, and SNI’s Data Analyst Arlene Marmolejo at the AI Forum during the CAPH/SNI Annual Conference in Napa.
By Dr. Ashrith Amarnath, Chief Health Officer, California Health Care Safety Net Institute
For public health care systems in the early phases of their AI journey, Dr. Karandeep Singh*, MMSc, Chief Health AI Officer at UC San Diego Health, shared pragmatic advice at the recent CAPH/SNI Annual Conference. Speaking to fellow AI officers and CIOs, CMIOs, and CMOs, Dr. Singh discussed how to establish the foundations for AI initiatives and evaluate AI models for equity, which is summarized below.
*Dr. Singh is also the Joan and Irwin Jacobs Chancellor’s Endowed Chair in Digital Health Innovation and Associate Professor of Medicine in Biomedical Informatics at UC San Diego School of Medicine, as well as the leader of AI initiatives at the Joan and Irwin Jacobs Center for Health Innovation.
Four essential steps to building an AI foundation
“You don’t have to do this on your own. Partner with those who have expertise.”
– Dr. Karandeep Singh, Chief Health AI Officer, UC San Diego Health
1. Establish governance
Health systems first need governance structures to evaluate AI models and establish AI-specific policies and processes, including how to routinely monitor models for effectiveness. Singh recommended assembling a group or committee to govern AI that is willing to be held accountable for decisions made around these models.
Committee members do not need to be AI experts themselves but can partner with specialists instead. For example, external partners can help the AI governing committee read between the lines when assessing vendor offerings.
“The tools are secondary, and the problems we solve are primary.”
– Dr. Singh
2. Determine AI strategy
The cornerstone of an AI strategy is identifying and prioritizing solvable problems, rather than looking at an increasingly flooded marketplace of AI tools and unsystematically adopting what’s available. Once a health system has determined its most pressing challenges to address with AI, Singh suggested starting out by reviewing and cataloging the AI tools that are already available, such as those within the electronic health record (EHR).
“If I’ve learned one thing in this space, it’s that if a frontline clinician gives you qualitative insight – when they say something isn’t working – they’re usually right.”
– Dr. Singh
3. Prioritize clinician input, and seek their feedback
Singh recommended focusing on use cases of AI that interest clinicians or help meet an operational need. There also must be a system for consistently gathering their input.
“Look at whether the AI model will waste people’s time or create extra busy work.”
– Dr. Singh
4. Consider the amount of work that AI creates
When evaluating AI models, assess the work it creates and ask:
Is the extra work worth it?
Is there a way to decrease this new workload?
Singh said that health systems need to avoid adopting a “work creation tool” and instead ensure AI models are effective. If they do generate additional work, that tradeoff needs to be recognized and deemed worthy in service of upcoming goals.
The importance of gauging the effectiveness of AI tools cannot be overstated. Given the high cost of some of the tools, taking vendors’ effectiveness claims at face value can be a costly mistake.
“What are the potential harms of using the AI model?”
– Dr. Singh
How to evaluate AI models for equity
For public health systems, ensuring AI systems are free from bias and are equitable is paramount.
At UC San Diego Health, Singh said they have a questionnaire for those proposing AI models where they are initially asked, “Is your model equitable?” However, since respondents often struggled with this broad question, the AI committee at UC San Diego Health replaced it with more specific questions to better address bias and equity:
What are the potential harms of using the AI model?
Consider the harms that could occur if the model is inaccurate and state them.
Consider the harms if the model is accurate and state them.
Consider harms to potential groups, keeping in mind anti-discrimination laws, and state them.
What is your mitigation plan for these harms?
These targeted questions can help health systems more effectively evaluate an AI model’s impact on equity.
For more information, read Dr. Singh’s presentation here.
QIP PY8 Reporting Manual User Agreement AMA PQA CMS v20241211
End User Agreement
Please review this agreement and click “Accept” at the end of this agreement if you agree to the terms of this agreement. If you do not agree, please click “Cancel”; in this case, you will not be able to access the Reporting Manual.
LICENSE FOR USE OF CURRENT PROCEDURAL TERMINOLOGY, FOURTH EDITION (“CPT®”)
End User Point and Click Agreement: CPT codes, descriptions and other data only are copyright 2022 American Medical Association (AMA). All Rights Reserved. CPT is a trademark of the AMA.
You, your employees and agents are authorized to use CPT only as contained in the QIP PY8 Reporting Manual solely for your own personal use and only for participating in state governmental healthcare related programs supported by and/or administered by California Health Care Safety Net Institute (SNI). You acknowledge that the “AMA” holds all copyright, trademark and other rights in CPT.
Any use not authorized herein is prohibited, including by way of illustration and not by way of limitation, making copies of CPT for resale and/or license, transferring copies of CPT to any party not bound by this agreement, creating any modified or derivative work of CPT, or making any commercial use of CPT. License to use CPT for any use not authorized herein must be obtained through the AMA, CPT Intellectual Property Services, AMA Plaza, 330 Wabash Ave, Suite 39300, Chicago, IL 60611-5885. Applications are available at the AMA Web site (http://www.ama-assn.org/go/cpt.).
U.S. GOVERNMENT RIGHTS
This product includes CPT which is commercial technical data, which was developed exclusively at private expense by the AMA, 330 Wabash Avenue, Chicago, IL 60611. The AMA does not agree to license CPT to the Federal Government based on the license in FAR 52.227-14 (Data Rights - General) and DFARS 252.227-7015 (Technical Data - Commercial Items) or any other license provision. The AMA reserves all rights to approve any license with any Federal agency.
DISCLAIMER OF WARRANTIES AND LIABILITIES
CPT is provided “as is” without warranty of any kind, either expressed or implied, including but not limited to the implied warranties of merchantability and fitness for a particular purpose. Fee schedules, relative value units, conversion factors and/or related components are not assigned by the AMA, are not part of CPT, and the AMA is not recommending their use. The AMA does not directly or indirectly practice medicine or dispense medical services. The responsibility for the content of this product is with SNI, and no endorsement by the AMA is intended or implied. The AMA disclaims responsibility for any consequences or liability attributable to or related to any use, non-use, or interpretation of information contained or not contained in this product.
This agreement will terminate upon notice if you violate its terms. The AMA is a third party beneficiary to this agreement.
PHARMACY QUALITY ALLIANCE COPYRIGHT STATEMENT
By agreeing to this policy, user accepts the following limitations of the QIP’s use of the Pharmacy Quality Alliance (PQA) Opioid measures and code sets: To accurately calculate the measure rates, the PQA value sets are required. User acknowledges that the PQA measures and value sets will only be used for the sole purpose of evaluating and improving opioid use for populations served by Public Health Care Systems (PHS) and District and Municipal Hospitals (DMPH) and will not be used for other purposes. Except for the purpose indicated above, the PQA measures and value sets will not be used for any other commercial product, service or value-added benefit. User also acknowledges that the information and PQA value sets will not be forwarded or provided to anyone outside of SNI, the California Department of Health Care Services (DHCS), the PHS or the DMPH.
CMS MEDICAID ADULT AND CHILD CORE SET
License Agreements
Please review the licensing information below and indicate your agreement and acceptance.
LICENSE FOR USE OF PHYSICIANS' CURRENT PROCEDURAL TERMINOLOGY, FOURTH EDITION (CPT)
End User Point and Click Agreement:
CPT codes, descriptions and other data only are copyright 2021 American Medical Association (AMA). All rights reserved. CPT is a registered trademark of the AMA.
You, your employees, and agents are authorized to use CPT only as contained in the following authorized materials of Centers for Medicare and Medicaid Services (CMS) internally within your organization within the United States for the sole use by yourself, employees, and agents. Use is limited to use in Medicare, Medicaid or other programs administered by CMS. You agree to take all necessary steps to insure that your employees and agents abide by the terms of this agreement.
Any use not authorized herein is prohibited, including by way of illustration and not by way of limitation, making copies of CPT for resale and/or license, transferring copies of CPT to any party not bound by this agreement, creating any modified or derivative work of CPT, or making any commercial use of CPT. License to use CPT for any use not authorized herein must be obtained through the AMA, CPT Intellectual Property Services, AMA Plaza, 330 Wabash Ave., Suite 39300, Chicago, IL 60654. Applications are available at the AMA website, http://www.ama-assn.org/go/cpt.
Applicable FARS\DFARS Restrictions Apply to Government Use. This product includes CPT which is commercial technical data and/or computer data bases and/or commercial computer software and/or commercial computer software documentation, as applicable which were developed exclusively at private expense by the AMA, AMA Plaza, 330 Wabash Ave., Suite 39300, Chicago, IL, 60611- 5885. U.S. Government rights to use, modify, reproduce, release, perform, display, or disclose these technical data and/or computer data bases and/or computer software and/or computer software documentation are subject to the limited rights restrictions of DFARS 252.227-7015(b)(2) (November 1995) and/or subject to the restrictions of DFARS 227.7202-1(a) (June 1995) and DFARS 227.7202-3(a) (June 1995), as applicable for U.S. Defense Department procurements and the limited rights restrictions of FAR 52.227-14 (June 1987) and/or subject to the restricted rights provisions of FAR 52.227-14 (June 1987) and FAR 52.227-19 (June 1987), as applicable, and any applicable agency FAR Supplements, for non- Department of Defense Federal procurements.
AMA DISCLAIMER OF WARRANTIES AND LIABILITIES
CPT is provided "as is" without warranty of any kind, either expressed or implied, including but not limited to, the implied warranties of merchantability and fitness for a particular purpose. AMA warrants that due to the nature of CPT, it does not manipulate or process dates, therefore there is no Year 2000 issue with CPT. AMA disclaims responsibility for any errors in CPT that may arise as a result of CPT being used in conjunction with any software and/or hardware system that is not Year 2000 compliant. No fee schedules, basic unit, relative values or related listings are included in CPT. The AMA does not directly or indirectly practice medicine or dispense medical services. The responsibility for the content of this file/product is with CMS and no endorsement by the AMA is intended or implied. The AMA disclaims responsibility for any consequences or liability attributable to or related to any use, non-use, or interpretation of information contained or not contained in this file/product. This Agreement will terminate upon notice if you violate its terms. The AMA is a third party beneficiary to this Agreement.
CMS DISCLAIMER
The scope of this license is determined by the AMA, the copyright holder. Any questions pertaining to the license or use of the CPT should be addressed to the AMA. End Users do not act for or on behalf of the CMS. CMS DISCLAIMS RESPONSIBILITY FOR ANY LIABILITY ATTRIBUTABLE TO END USER USE OF THE CPT. CMS WILL NOT BE LIABLE FOR ANY CLAIMS ATTRIBUTABLE TO ANY ERRORS, OMISSIONS, OR OTHER INACCURACIES IN THE INFORMATION OR MATERIAL CONTAINED ON THIS PAGE. In no event shall CMS be liable for direct, indirect, special, incidental, or consequential damages arising out of the use of such information or material.
LICENSE FOR NATIONAL UNIFORM BILLING COMMITTEE (NUBC)
These materials contain the NUBC Official UB-04 Specifications Manual and data codes (UB-04 Data), which are copyrighted 2022 by the American Hospital Association (AHA), Chicago, IL. Reproduced with permission. No portion of the AHA copyrighted materials may be copied without express written consent of the AHA. AHA copyrighted materials including the UB-04 codes and descriptions may not be removed, copied, or utilized within any software, product, service, solution or derivative work without the written consent of AHA. If an entity wishes to utilize any AHA materials, please contact the AHA at 312-893-6816.
Making copies or utilizing the content of the UB-04 Manual, including codes and/or descriptions, for internal purposes, resale and/or to be used in any product or publication; creating any modified or derivative of the UB-04 Manual and/or codes and descriptions; and/or making any commercial use of the UB-04 Manual or any portion thereof, including the codes and/or descriptions is only authorized with an express license from the AHA.
The license granted herein is expressly conditioned upon your acceptance of all terms and conditions contained in the agreement. By clicking below on the button labeled “Accept” you hereby acknowledge that you have read, understood, and agreed to all terms and conditions set forth in this agreement.
If you do not agree with all terms and conditions set forth herein, click below on the button labeled “Do Not Accept” and exit from this computer screen.
If you are acting on behalf on an organization, you represent that you are authorized to act on behalf of such an organization and that your acceptance of the terms of this agreement creates a legally enforceable obligation of the organization. As used herein, “you” and “your” refer to you and any organization on behalf of which you are acting.
1. Subject to the terms and conditions contained in this agreement, you, your employees, and agents are authorized to use UB-04 Data only as contained in the following authorized materials and solely for internal use by yourself, employees and agents within your organization within the United States and its territories. Use of UB-04 Data is limited to use in programs administered by Centers for Medicare & Medicaid Services (CMS). You agree to take all necessary steps to ensure that your employees and agents abide by the terms of this agreement. You acknowledge that the AHA holds all copyright, trademark, and other rights in UB-04 Data. You shall not remove, alter, or obscure any AHA copyright notices or other proprietary rights notices included in the materials.
2. Any use not authorized herein is prohibited, including, by way of illustration and not by way of limitation, making copies of UB-04 Data for resale and/or license, transferring copies of UB-04 Data to any party not bound by this agreement, creating any modified or derivative work of UB-04 Data, or making any commercial use of UB-04 Data. License to use UB-04 Data for any use not authorized herein must be obtained through the AHA, 155 N. Wacker Drive, Suite 400, Chicago, IL, 60606. Applications are available at the NUBC website, http://www.nubc.org/.
3. The UB-04 Data included in this product is commercial technical data and/or computer databases and/or commercial computer software and/or commercial computer software documentation, as applicable, which was developed exclusively at private expense by the AHA, 155 N. Wacker Drive, Suite 400, Chicago, IL, 60606. U.S. Government rights to use, modify, reproduce, release, perform, display, or disclose these technical data and/or computer data bases and/or computer software and/or computer software documentation are subject to the limited rights restrictions of DFARS 252.227-7015(b)(2) (November 1995) and/or subject to the restrictions of DFARS 227.7202-1(a) (June 1995) and DFARS 227.7202-3(a) (June 1995), as applicable for U.S. Department of Defense procurements and the limited rights restrictions of FAR 52.227-14 (December 2007) and FAR 52.227-19 (December 2007), as applicable, and any applicable agency FAR Supplements, for non-Department of Defense Federal procurements.
AHA DISCLAIMER
The AHA hasn’t reviewed and isn’t responsible for the completeness or accuracy of any information contained in this material, nor was the AHA or any of its affiliates, involved in the preparation of this material, or the analysis of information provided in the material. The views and/or positions presented in the material do not necessary represent the views of the AHA. CMS and its products and services aren’t endorsed by the AHA or any of its affiliates.
AHA DISCLAIMER OF WARRANTIES AND LIABILITIES
UB-04 Data is provided "as is" without warranty of any kind, either expressed or implied, including but not limited to, the implied warranties of merchantability and fitness for a particular purpose. The sole responsibility for the software, including any UB-04 Data and other content contained therein, is with the Medicare/Medicaid Contractor or the CMS; and no endorsement by the AHA is intended or implied. The AHA expressly disclaims responsibility for any consequences or liability attributable to or related to any use, non-use, or interpretation of information contained or not contained in this file/product. This agreement will terminate upon notice to you if you violate the terms of this agreement. The AHA is a third-party beneficiary to this agreement.
CMS DISCLAIMER
The scope of this license is determined by the AHA, the copyright holder. Any questions pertaining to the license or use of the UB-04 Data should be addressed to the AHA. End users do not act for or on behalf of the CMS. CMS DISCLAIMS RESPONSIBILITY FOR ANY LIABILITY ATTRIBUTABLE TO END USE OF THE UB-04 DATA. CMS WILL NOT BE LIABLE FOR ANY CLAIMS ATTRIBUTABLE TO ANY ERRORS, OMMISSIONS OR OTHER INACCURACIES IN THE INFORMATION OR MATERIAL COVERED BY THIS LICENSE. In no event shall CMS be liable for direct, indirect, special, incidental, or consequential damages arising out of the use of such information or material.
Should the foregoing terms and conditions be acceptable to you, please indicate your agreement and acceptance by clicking below on the button labeled “I agree”.
