BACKGROUND The rapid expansion of molecularly informed therapies in oncology, coupled with evolving regulatory FDA approvals, poses a challenge for oncologists seeking to integrate precision cancer medicine into patient care. Large Language Models (LLMs) have demonstrated potential for clinical applications, but their reliance on general knowledge limits their ability to provide up-to-date and niche treatment recommendations. To address this challenge, we developed a Retrieval-Augmented Generation (RAG)-LLM approach that integrates with a precision oncology knowledge resource, and evaluated whether this approach improved accuracy in biomarker-driven treatment recommendations relative to alternative frameworks.

METHODS We developed a RAG-LLM workflow that integrates Molecular Oncology Almanac (MOAlmanac) and evaluated this approach relative to alternative frameworks (i.e. LLM-only) in making biomarker-driven treatment recommendations using both unstructured and structured data. We evaluated LLM performance by calculating exact and partial match accuracies across 234 therapy-biomarker relationships. Finally, we assessed real-world applicability of the workflow by testing it on actual queries from practicing oncologists.

RESULTS While LLM-only achieved 54–69% accuracy in biomarker-driven treatment recommendations, RAG-LLM achieved 73–85% accuracy with an unstructured database and 91–99% accuracy with a structured database. In addition to accuracy, structured context augmentation substantially increased precision (49.0% to 79.9%) and F1-score (55.7% to 84.4%) compared to unstructured data augmentation. In queries recommended by practicing oncologists, RAG-LLM achieved 75–94% accuracy and high precision (91.67%).

CONCLUSIONS These findings demonstrate the effectiveness of the RAG-LLM framework in recommending FDA-approved precision oncology therapies based on individualized clinical information, and highlight the importance of integrating a well-curated, structured knowledge base in this process. While our RAG-LLM approach significantly improved accuracy compared to standard LLMs, further efforts will enhance the generation of reliable responses for ambiguous or unsupported clinical scenarios.

RG has equity in Google, Microsoft, Amazon, Apple, Moderna, Pfizer, and Vertex Pharmaceuticals; his spouse is employed by Carrum Health. ES receives research funding from Genentech/imCORE and Oncohost. CL receives research funding from Genentech/imCORE. EMVA holds consulting roles with Enara Bio, Manifold Bio, Monte Rosa, Novartis Institute for Biomedical Research, Serinus Bio, and TracerBio; he previously held consulting roles with Tango Therapeutics, Invitae, Syapse, Janssen, Genome Medical, Genomic Life, and Riva Therapeutics; he receives research support from Novartis, Bristol-Myers Squibb, Sanofi, and NextPoint; he has equity in Tango Therapeutics, Genome Medical, Genomic Life, Enara Bio, Manifold Bio, Microsoft, Monte Rosa, Riva Therapeutics, Serinus Bio, Syapse, and TracerDx; he received travel reimbursement from Roche and Genentech; he has filed institutional patents on chromatin mutations and immunotherapy response, and methods for clinical interpretation, and provides intermittent legal consulting on patents for Foaley & Hoag. Other authors have no relevant disclosures.

This study did not receive any funding.

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

Yes

I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.

Yes

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

Yes

I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.

Yes

Description This study evaluates a context-augmented large language model (LLM) workflow for recommending FDA-approved, biomarker-driven cancer therapies. By optimizing prompt design and integrating a structured precision oncology knowledge base, the approach demonstrates strong performance across both synthetic queries and real-world oncologist scenarios.

All data produced in the present study are available upon reasonable request to the authors.