Results Summary
What was the research about?
Attention deficit hyperactivity disorder, or ADHD, and autism spectrum disorder, or ASD, are health conditions that affect how people learn and relate to others. Students with either condition may have a hard time following classroom rules. These challenges may also make it harder for other children to learn.
In this study, the research team compared the support students with ADHD or ASD usually get with a new program called Unstuck and On Target, or UOT. In UOT, school staff actively teach students how to plan, set goals, and be flexible. The team looked at:
- Students’ classroom behaviors after being in the program
- Students’ problem-solving skills and parent reports about their behaviors after the program and one year later
What were the results?
After the program, classroom behavior improved both for students who were in UOT and those who received usual support. Across all students, UOT didn’t improve students’ behaviors more than usual support. For students with ADHD symptoms, both UOT and usual support improved students’ classroom behaviors. When only looking at students with ASD symptoms, UOT improved behaviors but usual support didn’t.
After the program and one year later, students’ problem-solving skills and parent-reported behaviors didn’t differ between UOT and usual support. Most of the program benefits remained.
Who was in the study?
The study included 148 students in grades 3–5 at 24 schools in Washington, DC, and northern Virginia. All schools served communities with limited resources. Of the students, 32 percent were Latino, 31 percent were white, 22 percent were African American, 11 percent were multiracial, and 3 percent were Asian. The average age was 10, and 82 percent were boys. In addition, 66 percent had ADHD symptoms, and 34 percent had ASD symptoms.
What did the research team do?
The research team assigned schools by chance to offer the UOT program or support as usual. Teachers chose students with ADHD or ASD symptoms to take part in the study. The team trained parents and school staff in both groups on rewarding flexible behaviors. For students in UOT, school staff led 21 group sessions over six months. Sessions taught students how to plan, set goals, and be flexible. For those getting usual support, school staff led 15 group and 5 in-class sessions. In these sessions, students practiced flexible behaviors but didn’t actively learn about them. In the second year, schools could choose to stay with their assigned program or switch programs.
The research team observed students before the program started and again after it ended. The team noted students’ behaviors, including
- Social skills
- Rule following
- Taking part in class
Parents took surveys and students did problem-solving tasks before the program started, after the program ended, and again one year later. The surveys and tasks helped the research team learn more about students’ behaviors.
Parents, school officials, teachers, and mental health providers gave input on the study and program materials. They also helped choose students to take part in the study.
What were the limits of the study?
During the second year of the study, four schools switched from their assigned group. Results may differ if schools hadn’t switched. This study took place in only one area of the country. Results may differ in other places.
Future studies could compare classroom behavior in other places.
How can people use the results?
Schools can use results when considering ways to support students with ADHD or ASD symptoms.
Professional Abstract
Objective
To compare the effectiveness of two school-based behavioral programs on improving executive functioning behaviors among students with autism spectrum disorder (ASD) or attention deficit hyperactivity disorder (ADHD) symptoms
Study Design
| Design Element | Description |
|---|---|
| Design | Randomized controlled trial |
| Population | 148 students with ASD or ADHD symptoms and executive functioning difficulties |
| Interventions/ Comparators |
|
| Outcomes |
Primary: classroom-observed executive functioning behaviors, including social reciprocity, rule following, appropriate transitioning, getting stuck, negativity or overload, and classroom participation Secondary: parent-reported executive functioning behaviors, behavioral and emotional problems, and flexibility; students’ problem solving; researcher-assessed executive functioning skills |
| Timeframe | Up to 10-month follow-up for primary outcome |
This cluster randomized controlled trial compared differences in executive functioning behaviors between a cognitive behavioral treatment program called Unstuck and On Target (UOT) and current standard of care through an adapted contingency management program called Parents and Teachers Supporting Students (PATSS). UOT included 21 weekly group sessions that taught students executive functioning skills, including flexibility, goal setting, and planning. PATSS included 15 weekly group and 5 in-class sessions that allowed students to practice executive functioning skills but did not actively teach these skills.
Researchers implemented the study throughout two consecutive school years. In the first year, researchers randomly assigned 24 schools serving students in low-income communities in Washington, DC, and northern Virginia to receive either UOT or PATSS. Teachers at each school identified students with ASD or ADHD symptoms and problems with executive functioning. School staff implemented their assigned programs over six months. In the second year, schools could choose whether to continue or change their assigned programs. Researchers trained parents and school staff in both groups on rewarding flexible behaviors.
The study included 148 students in grades 3–5 attending a participating school. Of these, 32% were Latino, 31% were non-Latino white, 22% were non-Latino black or African American, 11% were multiracial, and 3% were Asian. The average age was 10, and 82% were male. In addition, 66% had ADHD symptoms, and 34% had ASD symptoms.
To assess primary outcomes, researchers observed students in classroom settings for at least 15 minutes at baseline and postintervention. For secondary outcomes, parents completed surveys and students completed problem-solving activities at baseline, postintervention, and one year later.
A community advisory board of parents, school officials, teachers, and mental health and medical professionals provided input on the study design and measures and helped create program manuals and recruit participants.
Results
At postintervention, students in both programs improved. Improvement in classroom-observed executive functioning behaviors across all students did not differ between programs. Among students with ADHD symptoms, both programs improved classroom-observed executive functioning behaviors (p<0.05). Among students with ASD symptoms, the UOT program improved classroom-observed executive functioning behaviors (p<0.05) but the PATSS program did not.
At postintervention and one year later, the two programs did not differ in parent-reported executive functioning outcomes, students’ problem solving, or researcher-assessed executive functioning behaviors. Most of the treatment gains remained.
Limitations
During year two of the study, four schools switched programs, which limits the ability to draw conclusions about how the programs affected behavior. The study took place in two locations; results may differ in other geographic areas.
Conclusions and Relevance
Students in the UOT and PATSS programs did not differ in classroom-observed executive functioning behaviors. Results suggest that both school-based behavioral programs may benefit students with ASD and ADHD attending schools serving students from low-income communities.
Future Research Needs
Future research could compare these programs in other schools or in different geographic regions.
Final Research Report
View this project's final research report.
Implementation
Related PCORI Dissemination and Implementation Project
Journal Citations
Related Journal Citations
Peer-Review Summary
Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.
The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments.
Peer reviewers commented, and the researchers made changes or provided responses. The comments and responses included the following:
- Reviewers noted that the study was not entirely a randomized trial because a portion of the schools in the second cohort chose their intervention, yet researchers combined analyses for all cases. The researchers said that the fact that 4 out of 24 schools chose to switch from their randomly assigned intervention for the second year of the project was an unexpected modification of the trial design. But the researchers said, they found that whether schools stayed in the randomized intervention arm or switched did not affect outcomes, which they have now explained in the report.
- Reviewers asked for greater description of the participants, noting the small number of girls included in the autism spectrum disorder (ASD) and attention deficit hyperactivity disorder (ADHD) groups. Reviewers wondered whether this was a result of teacher expectations of student behavior, since teachers referred students to the study. Responding to the comment about teacher referral, the researchers said the study was community based and designed for implementation largely by school personnel, so it had to be compatible with school procedures. The researchers acknowledged that the gender distribution of participants may reflect some bias by school personnel but may also indicate that boys with ASD or ADHD and executive-functioning problems are more likely to have the target behavioral characteristics than are girls. The researchers said they collected extensive demographic data for the children but did not conduct a comprehensive diagnostic evaluation. Therefore, they could not assess the extent of comorbidity across ASD and ADHD or with other conditions.
- Reviewers asked for more information about school staff. The researchers said they only collected information about staff members’ professions, which was publicly available knowledge. They explained that their Institutional Review Board (IRB) did not allow them to gather other information, such as ethnicity, without treating staff as study subjects and requiring their signed consent. The researchers considered this too burdensome but noted their IRB did allow them to collect extensive feedback from staff on their opinions of the interventions.
- Reviewers commented that publications did not support the psychometric properties of the Executive Function Challenge Task (EFCT). The researchers responded that several projects have validated the EFCT and a paper is in preparation to report these results. The researchers added that they have found that performance on the EFCT predicts real-world outcomes, such as behavior in the classroom.
- Reviewers wondered how to evaluate the improvement in executive function in both study groups without a control group, indicating that such an improvement might then be because of chance or maturation. The researchers noted that previously they compared their Unstuck and on Target (UOT) intervention with a social skills training intervention. They found that only UOT delivered improvements in executive function, implying that improvements in executive function do not result only from developmental change. Also, their stakeholder partners strongly opposed having a non-intervention control group given their commitment to help underserved children.
Conflict of Interest Disclosures
Project Information
Key Dates
Study Registration Information
^Laura Gutermuth Anthony, PhD, was the original PI on this project.