Results Summary
What was the project about?
To diagnose rare genetic conditions, doctors look at patients’ genetic data and a phenotypic profile. A phenotypic profile is a record of all the physical traits of a condition. It uses a list of standard terms called Human Phenotype Ontology, or HPO. Doctors and clinic staff do a thorough exam with the patient to create the profile. The exam takes a long time and often more than one visit.
Patients may be able to create phenotypic profiles themselves using surveys. These surveys may take less time than clinic visits. But it is unclear whether patient surveys can provide enough details to correctly identify conditions.
In this project, the research team tested two surveys:
- Phenotypr. This survey asks patients to describe their symptoms and then matches the descriptions to plain language HPO or clinical HPO terms.
- GenomeConnect. This survey uses multiple choice questions to asks patients about their health and symptoms.
What did the research team do?
First the research team used a computer program to create phenotypic profiles from the surveys for all 7,344 known genetic conditions. For each condition, the program mimicked how 20 patients would respond to the surveys. The team checked these profiles against published profiles for each condition. Published profiles include a standard set of typical symptoms for each condition.
Then the research team tested the surveys with 282 patients. Patients had 1 of 257 different rare genetic conditions. The team assigned the patients by chance to take one or both surveys. The team also interviewed 17 patients who took both surveys to see which one they preferred.
Patients, doctors, and a genetic counselor helped design the study.
What were the results?
Both surveys were able to accurately identify rare genetic conditions. When comparing published profiles with survey profiles, use of the plain language HPO had more exact matches.
Patients preferred GenomeConnect over Phenotypr.
What were the limits of the project?
It was hard to compare how well the surveys worked because the number of patients with each condition was small.
Future research could test the surveys with a larger group of patients with rare genetic conditions.
How can people use the results?
Clinics can use these results when considering how to create phenotypic profiles to diagnose rare genetic conditions.
Professional Abstract
Background
Some genetic conditions are not only rare but hard to diagnose. Clinicians can use a phenotypic profile along with genetic information to diagnose these conditions. A phenotypic profile is a record of all the physical aspects of a condition using a standardized vocabulary called Human Phenotype Ontology (HPO). Self-phenotyping by patients may be more efficient and yield more complete information than phenotyping performed by multiple specialists. However, it is unclear whether current surveys for self-phenotyping accurately reflect a disease’s phenotype.
Objective
To validate two self-phenotyping surveys for generating phenotypic profiles for patients with rare genetic conditions
Study Design
Design Element | Description |
---|---|
Design | Simulation and validation study |
Data Sources and Data Sets |
7,344 HPO Monarch reference phenotypic profiles; survey data from 282 patients with rare genetic conditions; qualitative interviews with 17 patients who took both surveys |
Analytic Approach | Bayesian ontology query algorithm, sensitivity analysis using Fisher’s exact test, comparison of simulated profiles to actual patient profiles from surveys |
Outcomes |
Receiver operating characteristic curves for standard Monarch phenotypic profiles versus simulated Phenotypr and GenomeConnect profiles; similarity of simulated profiles to actual patient profiles |
Methods and Results
In the simulation study, researchers validated two web-based self-phenotyping surveys:
- Phenotypr. Researchers developed this survey, which asks patients to describe all their symptoms and then matches the descriptions to layperson HPO or standard HPO terms.
- GenomeConnect. Researchers adapted this preexisting survey for the web. It uses multiple choice questions to ask patients about their issues or symptoms for one body system at a time.
To validate the surveys, researchers compared the similarity of the phenotypic profiles generated by each survey to Monarch phenotypic profiles, a collection of published phenotypic profiles for each known genetic disease. First, for each survey, researchers simulated profiles of every known genetic condition in the Monarch collection based on how a patient with that condition might answer the survey’s questions. Researchers simulated 20 patient profiles for each of the 7,344 genetic conditions, resulting in 146,880 profiles per survey. Researchers analyzed the clinical utility of the GenomeConnect and Phenotypr simulated profiles. They used a Bayesian ontology query algorithm to generate receiver operating characteristic curves for published Monarch phenotypic profiles and simulated GenomeConnect and Phenotypr phenotypic profiles. The curves measured how well each survey differentiated between conditions. Analyses showed that the Monarch (area under the curve [AUC] = 0.985), Phenotypr (AUC = 0.957), and GenomeConnect (AUC = 0.913) phenotypic profiles all had good diagnostic power.
Then researchers randomly assigned 282 patients with rare genetic conditions to complete one or both surveys. Patients represented 257 different rare genetic conditions. Both the GenomeConnect and Phenotypr surveys were useful in collecting phenotype data directly from patients.
Compared with profiles from patients completing the Phenotypr survey, profiles from patients completing the GenomeConnect survey were more concordant with the simulated profiles; as such, GenomeConnect was more accurate. Among patients who completed both surveys, Phenotypr had a tighter distribution of similarity scores than GenomeConnect; as such, Phenotypr was more precise.
Researchers also interviewed 17 patients who took both surveys. They preferred GenomeConnect over Phenotypr.
Patients, clinicians, and a genetic counselor helped design the study and provided input throughout.
Limitations
The number of actual patients with any given genetic condition was small, which limited the ability to compare the two surveys’ performance for each condition.
Conclusions and Relevance
In this study, phenotypic profiles generated by patient surveys were effective at identifying rare genetic conditions.
Future Research Needs
Future research could test the surveys with a larger group of patients with rare genetic conditions.
Final Research Report
View this project's final research report.
Journal Citations
Related Journal Citations
Peer-Review Summary
Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.
The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments.
Peer reviewers commented and the researchers made changes or provided responses. Those comments and responses included the following:
- The reviewers noted that the study population had little diversity and asked the researchers to comment on ways that people from minority groups or of low socioeconomic status might participate in this research in the future. The researchers added the homogeneity of the study population to their limitations section and described ideas for addressing this issue in their section on future research, in the discussion.
- The reviewers liked the idea of producing a top 10 list of possible diagnoses and conditions based on patient-reported phenotyping but were disappointed to find that there could be hundreds of diseases ranked in the top 10 list. The researchers added information to the results section of the report to clarify this point. They also noted that the goal of the tool they created was to help clinicians diagnose patients using patient-reported data, not for patients to diagnose themselves. The researchers admitted that for some patients with more common disease characteristics, clinicians will still need to work through many possible diagnoses.