• Download Summary

What was the project about?

Electronic health records, or EHRs, have information about a patient’s health such as test results, diagnoses, and treatments. EHRs also have clinical notes that doctors and patients can use to track goals and decisions.

Clinical notes may be useful for research or to help improve care. But it’s hard to get information from these notes across large groups of patients. The notes may use different ways to describe the same thing. For example, high blood pressure may be called hypertension. Also, the notes may use abbreviations or have spelling mistakes.

In this project, the research team designed and built a search engine to make EHR notes easier to search and use for patient care and research.

What did the research team do?

To develop a new method for searching clinical notes in EHRs, the research team used 66 million clinical notes from patient visits at Nationwide Children’s Hospital in Ohio from 2006 to 2016. Using the new method, the team built a search engine called QREK. QREK stands for Query Refinement by word Embedding and Knowledge base. QREK finds and pulls out EHR notes that are related to keywords entered into it. It can also suggest other relevant keywords and common alternatives.

The research team tested QREK in two ways. First, they asked three doctors to rate the relevance of terms suggested by QREK across 11 searches. Second, the team looked at how often QREK correctly suggested a synonym for a known medical term.

The research team tested the final version of QREK under nine different scenarios with people at Nationwide Children’s Hospital to get feedback about its usefulness. For example, some people used QREK to do research; others used QREK to help improve care.

Patients, hospital administrators, health insurers, health information technology specialists, researchers, and clinicians provided input during the study.

What were the results?

The research team found that about 72 percent of the terms suggested by QREK were relevant to the original search term. Also, of the first 60 terms suggested by QREK, 54 percent matched synonyms on a standard list of known medical terms.

In the nine scenarios tested, people reported that QREK improved their use of EHR notes.

What were the limits of the project?

Testing occurred at a children’s hospital using keywords for children’s care.

Future research could continue to refine QREK and test QREK with EHR notes from adult care settings.

How can people use the results?

Researchers and hospital staff can use QREK to search and use notes in EHRs. QREK is available free of charge.

Background

Researchers can use electronic health records (EHRs) as a source of data for comparative effectiveness research. Clinical notes in EHRs contain details about clinical decision making and patient perspectives. But these data often go unused due to a lack of robust search methods.

A text search engine can extract information from unstructured textual notes and make it usable for research. However, existing methods used by text search engines do not account for misspellings or the use of plain language for medical terms. Developing user-friendly, efficient, and precise search methods can help extract text from clinical notes for use in improving research and clinical care.

Objective

(1) To design and build a search engine to extract relevant clinical text from EHRs efficiently; (2) To assess the performance of the new search engine

Study Design

Methods

The research team first established a new methodological framework to efficiently search for relevant clinical text from EHRs using an original query term. To create the framework, the team used more than 66 million clinical notes documenting patient encounters at Nationwide Children’s Hospital in Ohio from 2006 to 2016. The framework included possible refinements for common queries and categorized relationships between original query terms and query refinements.

Next, the research team developed a web-based interactive search engine called Query Refinement by word Embedding and Knowledge base (QREK). Given a user’s input query, QREK generates a list of relevant keywords, including word variations such as formal or informal forms, synonyms, abbreviations, and misspellings, and other relevant words like related diagnoses, medications, and procedures.

The research team then assessed the performance of QREK in two ways. First, the team asked three hospital residents to conduct 11 predefined queries and assess the relevance of terms suggested by QREK. Second, the team assessed QREK’s ability to recall known synonyms from 6,682 terms in the Systematized Nomenclature of Medicine (SNOMED). The team calculated the percentage of SNOMED synonyms that QREK suggested among the first 60 search results.

The research team tested and refined the QREK user interface with six clinical residents. They then implemented the final version of QREK in nine use cases at Nationwide Children’s Hospital.

Patients, hospital administrators, health insurers, health information technology specialists, researchers, and clinicians provided input during the study.

Results

On average, among the 11 predefined queries, precision was 0.72 (i.e., 72% of QREK results were found to be relevant). Among the 6,682 synonym searches, recall was 0.54 (i.e., 54% of QREK’s first 60 search results matched a synonym in the SNOMED list).

In the use cases, clinicians, researchers, patients, and caregivers reported increased utility of EHRs with QREK.

Limitations

Testing occurred at a pediatric hospital using only pediatric keywords; performance with adult care keywords is unknown.

Conclusions and Relevance

QREK can help users extract textual clinical notes, with the goal of improving use of these notes in research and clinical care.

Future Research Needs

Future research could continue to refine QREK and test it in adult care settings.