Results Summary

What was the research about?

Meta-analyses combine the results of many studies to find out how well a treatment or other healthcare intervention works. Most meta-analyses use public sources of data, such as published journal articles, as the main sources of information for study results. But journal articles are not the only sources of study results. Some results appear in other places, such as clinical study reports. Clinical study reports are documents that describe what researchers did and found in much more detail than journal articles. However, these reports may not be available to the public. As a result, meta-analyses may not include all available information about a treatment.

The research team wanted to learn whether adding or replacing public and nonpublic data sources changed the results of meta-analyses. To find out, the research team added and replaced data as they conducted two meta-analyses. The first looked at adult use of a nerve-pain medicine. The second meta-analysis looked at adult use of a medicine to treat bipolar depression.

What were the results?

The research team found that adding nonpublic data sources to public ones provided new details about study methods and results. Using all available data sources changed the size of the effect in the nerve-pain meta-analysis but doing so didn’t change the results for the bipolar-depression meta-analysis.

Nonpublic data sources, such as clinical study reports, often had information that medical journal articles lacked. Based on these results, the research team developed guidance for researchers who want to use different kinds of data sources in meta-analyses.

What did the research team do?

The research team asked patients and doctors to make a list of the health outcomes that matter to patients who take the medicines for nerve pain or bipolar depression. Then, the research team searched for studies about the two medicines. The research team looked at whether changing the sources of data on the outcomes that matter to patients changed the results of the meta-analyses.

The team looked at public and nonpublic data sources for the two meta-analyses. The research team found 80 sources for 21 studies about the nerve-pain medicine. The team found 51 sources for 7 studies about the bipolar-depression medicine.

What were the limits of the study?

The study looked at the effect of adding and replacing data sources in only two meta-analyses. Future research could look at whether adding and replacing data sources changes the results for meta-analyses focusing on other treatments or health problems. Also, future research could look at the effect of using nonpublic data sources on meta-analysis findings about treatment harms.

How can people use the results?

Researchers may want to use the results of this study when they look for data sources for meta-analyses.

Final Research Report

View this project's final research report.

Peer-Review Summary

Peer review of PCORI-funded research helps make sure the report presents complete, balanced, and useful information about the research. It also assesses how the project addressed PCORI’s Methodology Standards. During peer review, experts read a draft report of the research and provide comments about the report. These experts may include a scientist focused on the research topic, a specialist in research methods, a patient or caregiver, and a healthcare professional. These reviewers cannot have conflicts of interest with the study.

The peer reviewers point out where the draft report may need revision. For example, they may suggest ways to improve descriptions of the conduct of the study or to clarify the connection between results and conclusions. Sometimes, awardees revise their draft reports twice or more to address all of the reviewers’ comments. 

The awardee made the following revisions in response to peer review:

  • The awardee explained to reviewers that the project was a methods study and the clinical issues described in the methods section were not the focus and should not appear in the background section. The awardee reasoned that the background section should focus on the methodological gaps the study aimed to fill.
  •  The awardee shortened the description of the methods used to conduct meta-analyses and generally reduced the length of the methods section by replacing text with tables. The awardee increased the section’s readability by using shorter paragraphs.
  • In the discussion section, the awardee strengthened the argument for the study’s generalizability.
  • The awardee added a study limitation related to a lack of assessment of potential harms of the interventions. The awardee also noted the need for future research on the reporting of harms in different data sources.

Conflict of Interest Disclosures

Project Information

Kay Dickersin, PhD, MA
Johns Hopkins University
Integrating Multiple Data Sources for Meta-analysis to Improve Patient-Centered Outcomes Research

Key Dates

September 2013
October 2016

Study Registration Information


Has Results
Award Type
Health Conditions Health Conditions These are the broad terms we use to categorize our funded research studies; specific diseases or conditions are included within the appropriate larger category. Note: not all of our funded projects focus on a single disease or condition; some touch on multiple diseases or conditions, research methods, or broader health system interventions. Such projects won’t be listed by a primary disease/condition and so won’t appear if you use this filter tool to find them. View Glossary
Intervention Strategy Intervention Strategies PCORI funds comparative clinical effectiveness research (CER) studies that compare two or more options or approaches to health care, or that compare different ways of delivering or receiving care. View Glossary
State State The state where the project originates, or where the primary institution or organization is located. View Glossary
Last updated: March 4, 2022