Eligibility evidence often lives in the notes, not the codes

Where does the 80 percent figure come from?

Articles on this topic often open with the claim that 80 percent of healthcare data is unstructured. The number is older and vaguer than it looks. A widely cited origin is a 1998 Merrill Lynch report about data in organizations generally, which said "some estimates run as high as 80%" with no clear source.^⁠[1] A 2019 Healthcare Informatics Research editorial repeats the figure for medical data, citing a 2015 trade article rather than a measurement.^⁠[2]

For a site, the useful question is how many of a protocol's criteria a structured query can settle alone. That has been measured.

How many eligibility criteria need a clinical note?

Two studies counted criteria directly. In a 2014 Ohio State study of chronic lymphocytic leukemia (CLL) and prostate cancer trials, unstructured data was essential to resolving 59 percent of the CLL criteria and 77 percent of the prostate cancer criteria.^⁠[3] A 2024 preprint on the COPILOT-HF heart failure trial at Brigham and Women's Hospital reports that structured EHR data could determine 5 of 6 inclusion criteria but only 5 of 17 exclusion criteria, leaving the rest to chart review by study staff.^⁠[4]

The COPILOT-HF split follows how many protocols are written. Inclusion criteria often name a diagnosis and a lab value, which are coded. Exclusions ask about history, severity and prior treatment, which are usually written in prose.

Which criteria live in the note rather than the code?

Often the structured record holds a hint and the note holds the answer.

Some criteria, like the last one, cannot be settled from any record and belong on the pre-screening call.

What does a structured query miss?

A query built on codes, labs and medication lists fails in two directions. It flags patients whose notes rule them out, and it skips patients whose eligibility is written down but never coded.

• False positives. At Brigham and Women's Hospital, a rule-based EHR system flagged heart failure patients for guideline-directed therapy (a care program, not a trial). Of 5,460 flagged patients screened by hand, 1,754 (32.1 percent) were truly eligible. Over 38 percent of false positives came from misjudging symptomatic heart failure and medication history, and the 2024 paper calls for processing clinical notes.^⁠[5]
• Low precision. In a 2009 Columbia evaluation of 125 patients for the ACCORD trial, electronic screening excluded ineligible patients with 100 percent negative predictive accuracy, but only 13 percent of the patients it flagged were eligible.^⁠[6]
• Silent misses. A patient whose progression or prior therapy appears only in a note never matches a code-based filter, so nobody opens the chart, and the miss never reaches a screening log.

Can software reach the notes at all?

Increasingly, yes. USCDI version 1, the data set certified EHR APIs had to support, includes a Clinical Notes class with eight note types, from history and physical to pathology and imaging narratives.^⁠[7] Certified developers had until December 31, 2022 to roll out these FHIR APIs, and ONC reported in February 2023 that more than 95 percent met the deadline.^⁠[8]

A site's IT and privacy teams still decide which notes a vendor may read. The harder part is reading: tying each note to the right criterion and time window, and showing a coordinator the sentence behind each decision. The EHR recruitment guide covers interface and privacy options.

What does manual screening cost a site?

When a query cannot answer a criterion, someone reads the chart. A 2012 study that tracked screening effort at one academic cancer center found that the largest share of eligibility evaluations (35.8 percent) took 10 to 30 minutes, and more than 10 percent took 2 to 4 hours. Finding, screening and enrolling one patient took an average of 3.4 to 8.8 staff hours depending on study phase, and screening cost $129.15 to $336.48 per enrolled patient, largely nonreimbursed.^⁠[9]

What should a site do about it?

Treat the structured query as a first pass, not the screen, and plan for the notes on purpose.

• Sort each criterion into three groups: settled by coded data, by notes or reports, or only by talking to the patient.
• Build the candidate pool from the coded group, but never let a missing code exclude someone on a criterion that lives in notes.
• For each note-based criterion, name the documents that count as evidence, such as pathology reports for receptor status or imaging reports for progression.
• Track screen-failure reasons. If they cluster on criteria the notes could have answered, the pre-screen is under-reading the chart (more on screen failure).
• When evaluating software, ask to see the note sentence behind each criterion decision, not only a score.