Evaluating the performance of LYDIA: an AI-powered decision support system in detection of metastatic tumors in lymph nodes

Abstract

The integration of AI in histopathology represents a significant advancement in diagnosing metastatic cancers. LYDIA (LYmph noDe assIstAnt) is a commercially available AI-powered tool designed to annotate tumors in lymph node sections to aid histopathologists in diagnosing metastases. This study rigorously evaluates LYDIA’s standalone performance and the time and cost benefits on diagnostic workflow. In this study, LYDIA’s performance was rigorously evaluated on a blind image dataset comprising 366 whole slide images (WSIs) from metastatic breast, colon, lung, and skin cancers. Additionally, a clinical diagnostic workflow was simulated using an internal cohort of 105 WSIs, evaluated by four experienced histopathologists, to assess the impact of LYDIA when used as a decision support tool on diagnostic accuracy and case handling time. A Monte Carlo simulation was also conducted to estimate potential reductions in Immunohistochemistry (IHC) costs associated with AI assistance in sentinel lymph node evaluation. The analysis demonstrated that LYDIA achieved excellent diagnostic accuracy with high ROC-AUC scores (0.995 for breast cancer, 0.963 for colon cancer, 0.973 for lung cancer, and 0.983 for melanoma). In a clinical setting, AI-assisted diagnosis significantly reduced time-to-diagnosis across all metastasis sizes, with a maximum of 1.59-fold acceleration for micro-metastases (saving 26.5 seconds per WSI). LYDIA also enhanced pathologists’ diagnostic performance, increasing their sensitivity from 77.3% to 87.3%. The Monte Carlo simulation indicated an average saving of €7.89 per case, highlighting cost benefits from reduced IHC requests. In conclusion, the findings highlight LYDIA’s capabilities as a supportive tool that holds the potential to reduce diagnostic turnaround times and assist pathologists in identifying important regions within slides. This study illustrates the capability of AI-powered solutions like LYDIA to accelerate diagnosis and ultimately improve patient care. The observed time and cost benefits suggest that integrating AI into routine pathology practice can improve diagnostic quality, reduce costs and optimize resource utilization.

Competing Interest Statement

The authors have declared no competing interest.

Funding Statement

This study did not receive any funding

Author Declarations

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

Yes

The details of the IRB/oversight body that provided approval or exemption for the research described are given below:

Ethics committee of University Medical Center Utrecht gave ethical approval for this work. Anonymised images from anonymised cases where used from patients that have opted-in for medical research

I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.

Yes

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

Yes

I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.

Yes

Data Availability

The image dataset used to evaluate the performance of DeepPATH(TM)-LYDIA originates from anonymized patient cases, but it can be made available upon reasonable request for review purposes.

Comments (0)

No login
gif