Using Computer Alert Systems in the Emergency Room to Screen for Child Abuse

This project has results

Abstracts

Public Abstract

Download this abstract: English (pdf) | Español (pdf) | Audio Recording (mp3)

What was the research about?

Young children who come to the emergency room, or ER, with bruises and other injuries may be victims of physical abuse. However, hospital staff don’t always order standard tests that help detect child abuse.

The research team did two studies looking at ways to improve identification and referral of children at risk for child abuse. The first study tested a new alert system at a children’s hospital. The team wanted to know if the system helped doctors

Identify children under age two who are at risk for physical abuse
Evaluate children for abuse
Check all children equally, without differences based on their race or income

In the second study, trained ER nurses filled out a short form for children under 13 years old. The research team wanted to know if using the form in ERs improved how often doctors reported possible abuse to Child Protective Services.

What were the results?

Study 1

Children at risk. The alert system effectively identified children under age two who were at risk for physical abuse.

Checking for abuse. ER doctors who saw the alerts checked for abuse about as often as doctors who didn’t see them. Whenever doctors ordered tests based on the alerts, they ordered all the recommended tests.

Checking all children equally. Doctors checked for abuse more often when children had public insurance compared with children who had private insurance. There was no difference in how often doctors checked for abuse based on children’s races.

Study 2

Checking for abuse. ER nurses completed the form for 68 percent of patients under 13 years old. ER doctors reported possible abuse to Child Protective Services more often when they had the results from the form than when they didn’t.

Checking all children equally. There was no difference in how often doctors reported abuse to Child Protective Services based on children’s race or their family’s income. In addition, there were no differences in how often doctors reported abuse between different hospitals.

Who was in the study?

Study 1

To make sure the alert system was effectively identifying children under age two who were at risk for physical abuse, the research team looked at 226 children who triggered the alert. Of these children, 68 percent were white, 61 percent were male, and the average age was nine months. The children got care at one children’s ER in Pennsylvania.

Next, the research team wanted to learn if doctors were checking for abuse, and checking all children equally. This part of the study included 306 children under age two who went to the same children’s ER in Pennsylvania and triggered the alert system. Of these, 66 percent were white, 56 percent were male, and the average age was nine months.

Study 2

The second study included health records for 17,163 children under age 13 who went to one of 13 ERs in a hospital system in Pennsylvania. Of these, 72 percent were white and 21 percent were African American. The average age was 6 years, and 46 percent were male.

What did the research team do?

For the first study, the research team created an alert in the hospital’s computer system to signal doctors to check for abuse. For example, if an infant had bruises, it would trigger the alert.

For the second study, the research team tested a five-question form at 13 ERs. The team asked nurses to fill out the form for all children under age 13. If the nurse answered yes to a question, the computer system would alert a doctor to the concern for abuse. The alert would suggest the doctor speak with a social worker and check for abuse.

What were the limits of the study?

The first study took place at a children’s hospital where doctors may be more familiar with signs of abuse and therefore more likely to look for it than at other hospitals. All the doctors at the hospital knew about the alert system. So, they may have been more likely to notice signs of abuse even when they didn’t see an alert.

For the second study, the research team didn’t have data on why ER nurses filled out the form for some patients and not others. For example, if a child had a history of ER visits, the nurse may have been more likely to fill out the form. This may have affected which children had a completed form in their record.

Future research could test these systems in different medical settings.

How can people use the results?

ER doctors and hospital managers can use findings from the first study to improve screening in the ER for physical abuse in children under age two.

Hospitals can use the findings from the second study to decide if they want to use the form to help identify possible abuse in children under 13 years old.

Professional Abstract

Objective

(1) To develop and validate an electronic health record (EHR)-based system for alerting pediatric emergency department (ED) clinicians of possible physical abuse and determine whether compliance with screening guidelines for suspected physical abuse in children less than two years old improves with use of the alert system compared with usual procedures; (2) To observe whether identification and reporting of child maltreatment in children under age 13 improves with use of a child abuse screen (CAS) at a network of 13 EDs

Study Design

Design Elements	Description
Design	Objective 1: randomized controlled trial Objective 2: prospective observational study
Population	Objective 1: 226 children under age 2 presenting with risk factors for physical abuse in a pediatric ED in the validation group and 306 children under age 2 in the randomized controlled trial Objective 2: 17,163 health records of children under age 13 presenting at 13 general EDs
Interventions/ Comparators	Objective 1: pop-up alert linked to order set in EHR system versus no pop-up alert in EHR system Objective 2: 5-item CAS versus no CAS
Outcomes	Objective 1: sensitivity, specificity, negative and positive predictive values for identifying physical abuse (primary); proportion of cases in which provider was fully or partially compliant with AAP guidelines for evaluating suspected physical abuse, proportion of cases in which provider was compliant by patient race and insurance status Objective 2: rate of reporting suspected child abuse to CPS, rates of reporting based on patient and/or hospital characteristics
Timeframe	Objective 1: immediate follow-up for primary outcomes Objective 2: immediate follow-up for primary outcomes

Clinicians, social workers, medical informatics specialists, parents, and patient advocates participated in all stages of this study.

Objective 1. Researchers developed and evaluated the Child Abuse Clinical Decision Support System (CA-CDSS), an EHR alert system that prompts pediatric ED providers to use an order set to evaluate for possible child maltreatment, including physical abuse, sexual abuse, and neglect, in children under age two who trigger the alert system. The team embedded CA-CDSS in the EHR system at the Children’s Hospital of Pittsburgh of the University of Pittsburgh Medical Center (UPMC) ED and trained providers to use it. Providers could also access the child abuse order sets directly, independent of the alert system.

Researchers measured the system’s accuracy for seven months, during which 226 children under age two triggered an alert. Of these children, 68% were white, 61% were male, and the average age was nine months.

Next, researchers conducted a randomized controlled trial to examine the system’s effect on provider adherence to the American Academy of Pediatrics (AAP) guidelines for evaluating suspected physical abuse. During the trial, if a patient triggered the CA-CDSS, the system randomized the EHR to show or not show the pop-up alert.

The trial included 306 children under age two who presented in the ED and triggered the CA-CDSS alert. Of these, 66% were white, 56% were male, and the average age was nine months.

Objective 2. Researchers compared physicians’ rates of reporting suspected child abuse to Child Protective Services (CPS) for children under 13 years old who did and did not have a completed child abuse screen (CAS), a five-item questionnaire to identify child abuse in their EHR.

Researchers embedded the CAS into the EHR system at 13 general EDs throughout the UPMC healthcare system. The study included 17,163 children who sought care at these EDs during a six-month period. Of these, 72% were white, 21% were African American, 46% were male, and the average age was six.

A nurse trained in completing the CAS examined children presenting in the EDs. A positive answer to any CAS item triggered an alert, which made the physician aware of the concern for abuse and suggested consulting a social worker and considering using the physical abuse order set.

Results

Objective 1

CA-CDSS accuracy. Sensitivity was 96.8%, specificity was 98.5%, and positive and negative predictive values were 26.5% and 99.9%, respectively, for identifying possible, probable, or definite physical abuse.

Trial. There was no significant difference in the proportion of providers who ordered physical abuse evaluations between the pop-up alert (85%) and control (77%) groups, even though providers in the control group were not prompted to use the physical abuse order set and had to search for it.

In every case in which a provider used the physical abuse order set, the provider was fully compliant with the AAP guidelines. The proportion of cases in which providers were only partially compliant decreased during the trial (p=0.04).

Providers were more likely to order evaluations when patients had public insurance than when patients had private insurance (p=0.02). There was no difference in provider compliance by patient race.

Objective 2. Providers completed the CAS with 68% of patients, and 1.9% were positive. The rate of abuse reported to CPS was significantly higher when providers completed the CAS compared with when they did not (p<0.0001). Younger children were more likely to be screened, but there were no race or income differences in the odds of being screened. There were also no significant differences in the odds of being screened by the hospital’s teaching status, size, or urban versus rural location.

Limitations

Objective 1. The trial took place at one pediatric ED. Providers, all of whom were aware of the study, may have been more attentive to signs of child abuse compared with providers in other settings. Their awareness may also have led to more providers using the order set overall. The study may have overestimated the sensitivity of the CA-CDSS because children with difficult-to-detect signs of abuse may not have been referred to the expert team that determined whether abuse was present.

Objective 2. Researchers were unable to assess whether variables such as reason for visit, severity of condition, past medical history, and history of repeated visits might have affected the likelihood of patients being screened. Therefore, there may have been differences between children who had a completed CAS compared with those who did not, which may have affected the likelihood of providers reporting abuse to CPS.

Conclusions and Relevance

Objective 1. The newly developed EHR-based alert system accurately identified children under age two in need of evaluation for physical abuse. The CA-CDSS pop-up alert did not significantly increase compliance with AAP guidelines compared with usual procedures, nor did it reduce disparities in evaluations based on type of insurance. However, providers seeing children in the control group, as well as in the pop-up alert group, accessed the order set for child physical abuse, suggesting that they found it useful.

Objective 2. Use of the CAS significantly improved rates of reporting suspected child physical abuse to CPS across multiple ED locations.

Future Research Needs

Future research could investigate the impact of these systems and conduct similar comparison studies in non-ED settings.

Journal Articles

Journal of the American Medical Informatics Association

Integration of physical abuse clinical decision support into the electronic health record at a Tertiary Care Children's Hospital

Child Abuse & Neglect

Implementation of routine electronic health record-based child abuse screening in General Emergency Departments

Journal of the American Medical Informatics Association

Development of an electronic medical record-based child physical abuse alert system

More on this Project

This project's final research report is expected to be available by April 2019.

Peer-Review Summary

Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.

The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments.

Peer review identified the following strengths and limitations in the report:

The reviewers found the report well written and the study timely.
The reviewers asked whether legislative changes about mandatory reporting of child abuse might have affected the outcomes of the study. The researchers explained that the legislative changes mostly expanded the list of people required to report a suspicion of child abuse. The researchers also explained that the legislative changes occurred several months before they collected most of the study data. The researchers said that these legislative changes probably did not affect data collection and outcomes.
The reviewers said that measures of the system’s accuracy for triggering referral of a child for suspected abuse were undoubtedly inaccurate because of selection bias in referring children for a definitive evaluation for abuse. The researchers acknowledged this limitation in the report but stated that they used accuracy measurement terms that were consistent with those in their published article about this research.
The reviewers asked whether the researchers could have used reports from child protective services (CPS) as a standard against which to measure false-positive and false-negative rates of the trigger system. The researchers contended that CPS criteria for child abuse cases vary by county and jurisdiction. The county in which the main study took place has one of the lowest rates of child abuse referrals in the country. Therefore, using CPS system criteria to make decisions about whether cases are child abuse would probably lead to an overestimate of false negatives in the study’s county. In another county with less-conservative CPS criteria but with the same triggers, using the criteria might lead to fewer false negatives.

Conflict of Interest Disclosures

View the COI disclosure form.

Related PCORI Dissemination and Implementation Project

Disseminating Child Abuse Clinical Decision Support to Improve Detection, Evaluation, and Reporting

Project Details

Principal Investigator

Rachel Berger, MD, MPH

Project Status

Completed; PCORI Public and Professional Abstracts Posted

Project Title

Using the Electronic Medical Record to Improve Outcomes and Decrease Disparities in Screening for Child Physical Abuse

Board Approval Date

May 2013

Project End Date

July 2018

Organization

University of Pittsburgh

Year Awarded

2013

State

Pennsylvania

Year Completed

2018

Project Type

Research Project

Health Conditions

Trauma/Injuries

Physical Abuse

Funding Announcement

Addressing Disparities

Project Budget

$1,119,045

Study Registration Information

HSRP20143284

Page Last Updated:

June 25, 2019