TrialGPT is an algorithm developed by the National Institutes of Health (NIH) to make patient recruitment for clinical trials more efficient. Using modern AI technologies, TrialGPT analyzes patient records and compares relevant information such as diagnoses and demographics with the inclusion criteria of studies on ClinicalTrials.gov. The goal is to identify suitable studies faster and more accurately.
TrialGPT uses a four-step process to match patients with clinical trials:
1. Information retrieval: TrialGPT analyzes the patient record and identifies important medical and demographic data such as diagnoses, treatment history, age, gender, and ethnicity.
2. Retrieval of relevant studies: The algorithm searches the ClinicalTrials.gov database and filters out studies that are potentially relevant based on the extracted patient information. This step uses a combination of keyword generation and hybrid retrieval techniques to account for both lexical and semantic matches.
3. Suitability Assessment (TrialGPT-Matching): In this crucial step, TrialGPT assesses the patient's suitability for each of the selected studies by comparing the patient information with the specific inclusion criteria of the study. TrialGPT not only provides a binary classification (suitable or not suitable), but also generates detailed explanations as to why a patient is or is not suitable for a particular study. In addition, TrialGPT identifies the relevant text sections in the patient record that support the algorithm's decision.
4. Ranking (TrialGPT-Ranking): Based on the results of TrialGPT matching, the algorithm ranks trials according to the patient's probability of meeting the inclusion criteria. This step utilizes various aggregation methods, including linear aggregations (e.g., percentage of inclusion criteria met) and LLM-based aggregations, to produce meaningful scores.
TrialGPT's performance was evaluated in several studies with promising results:
· Matching accuracy: In a study with 183 “synthetic patients” and over 75,000 study annotations, TrialGPT achieved an accuracy of 87.3% in assigning patients to studies, which is close to the performance of experienced clinicians.
· Time savings: TrialGPT reduced screening time by an average of 42.6% compared to manual methods. A pilot study with human clinicians confirmed these results and showed that using TrialGPT reduced screening time by 40% without compromising accuracy.
· Criterion-level prediction accuracy: Evaluation by medical experts showed that TrialGPT can predict patient eligibility at the criterion level with 87.3% accuracy, which is comparable to the performance of human experts.
· Correlation of trial-level scores: The trial-level scores generated by TrialGPT correlate strongly with the manual assessment of patient suitability, indicating that the algorithm can effectively rank trials according to their relevance for a specific patient.
· Increased efficiency: TrialGPT can significantly speed up the time-consuming process of manually searching for suitable studies and relieve clinicians so that they can focus on other important tasks.
· Improved accuracy: TrialGPT's high accuracy in matching patients to studies minimizes the risk of patients being enrolled in unsuitable studies or missing suitable studies.
· Transparency and traceability: TrialGPT provides reasoned explanations for its decisions and highlights the relevant information in the patient record. This transparency promotes clinicians' trust in the system and supports informed decision-making.
· Potential to improve diversity in clinical trials: By identifying suitable patients more efficiently and accurately, TrialGPT could help increase diversity in clinical trials, leading to more representative study results.
The researchers plan to further evaluate the performance and fairness of TrialGPT in real-world clinical settings and optimize the model for use with electronic health records (EHRs). The incorporation of EHR data would significantly improve the scalability and practical utility of TrialGPT.
Furthermore, research into alternative open-source LLMs as a backbone for TrialGPT will increase the accessibility and sustainability of the system.
The continued development and refinement of TrialGPT promises to be transformative for patient recruitment for clinical trials, which could contribute to faster development of new therapies and improved healthcare for all.
Sources:
- https://www.ncbi.nlm.nih.gov/research/trialgpt/
- https://www.nature.com/articles/s41467-024-53081-z
- https://www.healthcareitnews.com/news/new-nih-tool-uses-genai-connect-volunteers-clinical-trials
- https://informaconnect.com/ai-tech-trialgpt-can-cut-screening-time-by-40-says-nih/
