What was the research about?
Patients often take surveys about their health or quality of life. Results from these surveys can help doctors meet patients’ needs. Young children can’t fill out surveys by themselves. They may not be able to read or understand the questions. Most often, parents or hospital staff read the questions aloud, or parents answer the questions for their children. But this method may not give accurate results.
In this study, the research team tested three surveys for children ages 4 to 12 who are going to have or who recently had surgery. The first survey asks about general health. The second survey asks about feeling worried before surgery. The third survey asks about pain after surgery. A computer program reads the survey questions aloud. The surveys are animated and choices for the answers appear as cartoons.
The team wanted to learn if the surveys were
- Accurate, or correctly capturing how the children were feeling
- Reliable, or if children answered in a consistent way when asked similar questions
What were the results?
Overall, the three surveys were accurate and reliable. This result was true for children of different ages, genders, and races.
Who was in the study?
The study included 725 families with children getting care at Children’s Hospital of Orange County in California. All children were going to have surgery. Of the parents who took part in the study, 66 percent were Latino, 25 percent were white, and 9 percent were other races. The average parent age was 37, and 86 percent were women. Of the children, 71 percent were Latino, 21 percent were white, and 8 percent were other races. The average child age was 7, and 60 percent were boys.
What did the research team do?
The team asked children, paired with their parents, to complete the three surveys. Of the child–parent pairs, 420 pairs completed the surveys before surgery, and 242 pairs completed the surveys at two and seven days after surgery. Children answered animated questions by themselves. Their parents answered the same questions without the animations. Children and parents also completed other surveys on the same topic at the same time.
The team looked at survey responses of both children and parents to see if the responses were accurate and reliable.
Two groups of doctors, hospital administrators, and parents of children helped design the study and develop the survey.
What were the limits of the study?
The team designed the surveys for children ages 4 to 12. Younger children may not understand the survey questions. The study looked at recovery from surgery during a short time. The team did not test how well the surveys work if children take them at other times after surgery.
Future research could use other methods to test the surveys during a longer recovery time. Also, researchers could test if children younger than four years old are able to use animated surveys.
How can people use the results?
Doctors can use the animated computer surveys to understand their young patients’ health needs.
To assess the reliability and validity of the Child Health Rating Inventories (CHRIs) general health status measure and two newly developed preoperative anxiety and postoperative pain management measures among children ages 4–12 undergoing surgery
|Design||Survey development and testing|
|Data Sources and Data Sets||Data from 578 parents and 483 children ages 4–12 undergoing surgery. Survey data include child- and parent-reported CHRIs, FDI, PedsQL, STAI-CH, and PPPM scales, as well as parent reports of children’s days missed from school and healthcare utilization. Medical record data include type and length of surgery|
|Analytic Approach||Evaluation of reliability and validity|
|Reliability and validity of the child-reported CHRIs measures|
In a previous study, researchers developed the CHRIs measure of general health status for children ages 4–12, with age-appropriate vocabulary and response formats. The measure included animated cartoon survey response options and computer audio instructions. In this study, researchers developed two new CHRIs measures for children’s preoperative anxiety and postoperative pain management and assessed the reliability and validity of all three CHRIs measures.
Researchers recruited 725 families with children having surgery at Children’s Hospital of Orange County, California. Among parents, 66% were Hispanic, 25% were white and 9% were other races. The average age of parents was 37, and 86% were female. Among children, 71% were Hispanic, 21% were white, and 8% were other races. The average age of children was 7, and 60% were male.
Researchers enrolled eligible parent–child dyads to complete surveys preoperatively and at two and seven days postoperatively. Children completed the animated measures. Parents completed nonanimated CHRIs measures along with a variety of other measures. Of the dyads, 420 completed CHRIs preoperatively, and 242 completed CHRIs postoperatively.
Researchers assessed the internal consistency reliability of CHRIs measures using Cronbach’s alpha and the construct validity using exploratory factor analysis (EFA). To validate the child-reported CHRIs measures, the team examined their correlations with parent-reported CHRIs measures and legacy measures, including Pediatric Quality of Life (PedsQL), Functional Disability Index (FDI), State-Trait Anxiety Inventory for Children (STAI-CH), and Parents’ Postoperative Pain Measure (PPPM). The team also examined the correlations between child-reported CHRIs measures and parent-reported healthcare utilization and days missed from school.
Two advisory groups including clinicians, hospital administrators, and parents of children treated at the hospital made recommendations on the study design and provided feedback on the animated survey questions.
- CHRIs general health status. The child-reported composite of all subscales and six of the eight subscales showed good internal consistency reliabilities (α≥0.70) that were similar regardless of age, gender, race, or ethnicity of the child. EFA did not fully support the construct validity of the measure. Child-reported CHRIs scores moderately correlated with parent-reported CHRIs scores, child-reported PedsQL and FDI, parent-reported days missed from school, and parent-reported healthcare utilization.
- CHRIs preoperative anxiety and postoperative pain. The child reports of the two measures showed good internal consistency reliabilities (α≥0.70) that were similar regardless of age, gender, race, and ethnicity of the child. Both measures showed good construct validity based on EFA. Child reports of both measures were moderately to highly correlated with the corresponding parent reports. Child-reported preoperative anxiety correlated with STAI-CH, and child-reported postoperative pain management correlated with PPPM.
Researchers only examined recovery from some types of surgery and over a short time period. Therefore, they were unable to assess certain types of validity, such as discriminant validity and predictive validity. Designed for children ages 4–12, the surveys may not be understandable for children younger than 4.
Conclusions and Relevance
The animated CHRIs general health status measure and the new animated CHRIs preoperative anxiety and postoperative pain management measures can produce valid and reliable scores when used by young children. Clinicians and families can use information from CHRIs to help inform children’s care.
Future Research Needs
Future research could investigate whether children younger than age four provide accurate and reliable information using the animated approach.
Final Research Report
View this project's final research report.
More to Explore...
Related PCORI Dissemination and Implementation Project
Related Journal Citations
Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.
The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments.
Peer reviewers found the study to be statistically sound and to address an important issue. The reviewers did note the complexity of the language describing the study’s psychometric methods. The researchers said it would be hard to satisfy both experts in the field and general scientists while staying within the page limit, but they provided an expanded glossary.
Conflict of Interest Disclosures
Study Registration Information
- Has Results