Accuracy, readability, and understandability of large language models for prostate cancer information to the public

Bundorf MK, Wagner TH, Singer SJ, Baker LC. Who searches the internet for health information? Health Serv Res. 2006;41:819–36.

Article  PubMed  PubMed Central  Google Scholar 

Andreassen HK, Bujnowska-Fedak MM, Chronaki CE, Dumitru RC, Pudule I, Santana S, et al. European citizens’ use of E-health services: a study of seven countries. BMC Public Health. 2007;7:53.

Article  PubMed  PubMed Central  Google Scholar 

Cacciamani GE, Bassi S, Sebben M, Marcer A, Russo GI, Cocci A, et al. Consulting “Dr. Google” for prostate cancer treatment options: a contemporary worldwide trend analysis. Eur Urol Oncol. 2020;3:481–8.

Article  PubMed  Google Scholar 

Cacciamani GE, Dell’Oglio P, Cocci A, Russo GI, De Castro Abreu A, Gill IS, et al. Asking “Dr. Google” for a second opinion: the devil is in the details. Eur Urol Focus. 2021;7:479–81.

Cacciamani GE, Gill K, Gill IS. Web search queries and prostate cancer. Lancet Oncol. 2020;21:494–6.

Article  PubMed  Google Scholar 

Cacciamani GE, Stern MC, Medina LG, Gill K, Sotelo R, Gill IS. Cancer awareness crusades-pink ribbons and growing moustaches. Lancet Oncol. 2019;20:1491–2.

Article  PubMed  PubMed Central  Google Scholar 

OpenAI. Introducing ChatGPT 2022 [updated November 30, 2022. Available from: https://openai.com/blog/chatgpt.

Ruby D. 30+ Detailed ChatGPT Statistics - Users & Facts 2023 [July 7, 2023]. Available from: https://www.demandsage.com/chatgpt-statistics/#:~:text=ChatGPT%20currently%20has%201.16%20billion,February%202023%20to%20March%202023.

Davis R, Eppler M, Ayo-Ajibola O, Loh-Doyle JC, Nabhani J, Samplaski M. et al. Evaluating the effectiveness of artificial intelligence-powered large language models application in disseminating appropriate and readable health information in urology. J Urol. 2023;210:688–94. 101097ju0000000000003615.

Article  PubMed  Google Scholar 

Schardt D. ChatGPT is amazing. But beware its hallucinations! 2023 [updated March 20, 2023. Available from: https://www.cspinet.org/blog/chatgpt-amazing-beware-its-hallucinations.

Brashier NM, Schacter DL. Aging in an era of fake news. Curr Dir Psychol Sci. 2020;29:316–23.

Article  PubMed  PubMed Central  Google Scholar 

Rawla P. Epidemiology of prostate cancer. World J Oncol. 2019;10:63–89.

Article  CAS  PubMed  PubMed Central  Google Scholar 

Temsah O, Khan SA, Chaiah Y, Senjab A, Alhasan K, Jamal A, et al. Overview of early ChatGPT’s presence in medical literature: insights from a hybrid literature review by ChatGPT and human experts. Cureus. 2023;15:e37281.

PubMed  PubMed Central  Google Scholar 

Eppler MB, Ganjavi C, Knudsen JE, Davis RJ, Ayo-Ajibola O, Desai A, et al. Bridging the gap between urological research and patient understanding: the role of large language models in automated generation of layperson’s summaries. Urol Pr. 2023;10:436–43.

Google Scholar 

Cocci A, Pezzoli M, Lo Re M, Russo GI, Asmundo MG, Fode M, et al. Quality of information and appropriateness of ChatGPT outputs for urology patients. Prostate Cancer Prostatic Dis. 2024;27:103–8.

Eppler M, Ganjavi C, Ramacciotti LS, Piazza P, Rodler S, Checcucci E, et al. Awareness and use of ChatGPT and large language models: a prospective cross-sectional global survey in urology. Eur Urol. 2024;85:146–53.

Article  PubMed  Google Scholar 

Ganjavi C, Eppler MB, Ramacciotti LS, Cacciamani GE. Clinical patient summaries not fit for purpose: a study in urology. Eur Urol Focus. 2023;9:1068–71.

Shiely F, Daly A. Trial lay summaries were not fit for purpose. J Clin Epidemiol. 2023;156:105–12.

Lois A, Droullard D, Lee J, Flum D. Patient decision-making in symptomatic gallbladder disease. Surg Endosc. 2022;36:6271–7.

Article  PubMed  PubMed Central  Google Scholar 

Lloyd JC, Yen T, Pietrobon R, Wiener JS, Ross SS, Kokorowski PJ, et al. Estimating utility values for vesicoureteral reflux in the general public using an online tool. J Pediatr Urol. 2014;10:1026–31.

Article  PubMed  PubMed Central  Google Scholar 

Lee P, Bubeck S, Petro J. Benefits, limits, and risks of GPT-4 as an AI chatbot for medicine. N Engl J Med. 2023;388:1233–9.

Article  PubMed  Google Scholar 

Gabrielson AT, Odisho AY, Canes D. Harnessing generative AI to improve efficiency among urologists: welcome ChatGPT. Wolters Kluwer Philadelphia, PA. 2023. https://doi.org/10.1097/JU.0000000000003383.

Pan A, Musheyev D, Bockelman D, Loeb S, Kabarriti AE. Assessment of artificial intelligence chatbot responses to top searched queries about cancer. JAMA Oncol. 2023;9:1437–40.

Article  PubMed  Google Scholar 

Walters WH, Wilder EI. Fabrication and errors in the bibliographic citations generated by ChatGPT. Sci Rep. 2023;13:14045.

Article  CAS  PubMed  PubMed Central  Google Scholar 

Qu RW, Qureshi U, Petersen G, Lee SC. Diagnostic and management applications of ChatGPT in structured otolaryngology clinical scenarios. OTO Open. 2023;7:e67.

Article  PubMed  PubMed Central  Google Scholar 

Abi-Rafeh J, Xu HH, Kazan R, Tevlin R, Furnas H. Large language models and artificial intelligence: a primer for plastic surgeons on the demonstrated & potential applications, promises, and limitations of ChatGPT. Aesthetic Surg J. 2024;44:329–43.

Varas J, Coronel BV, VILLAGRáN I, Escalona G, Hernandez R, Schuit G, et al. Innovations in surgical training: exploring the role of artificial intelligence and large language models (LLM). Rev Col Bras Cir. 2023;50:e20233605.

PubMed  PubMed Central  Google Scholar 

Sallam M. ChatGPT utility in healthcare education, research, and practice: systematic review on the promising perspectives and valid concerns. Healthcare (Basel). 2023;11:887.

Article  PubMed  Google Scholar 

Musheyev D, Pan A, Loeb S, Kabarriti AE. How well do artificial intelligence chatbots respond to the top search queries about urological malignancies? Eur Urol. 2024;85:13–6.

Zhou Z, Wang X, Li X, Liao L. Is ChatGPT an evidence-based doctor? Eur Urol. 2023;84:355–6.

Article  PubMed  Google Scholar 

Talyshinskii A, Naik N, Hameed BZ, Zhanbyrbekuly U, Khairli G, Guliev B, et al. Expanding horizons and navigating challenges for enhanced clinical workflows: ChatGPT in urology. Front Surg. 2023;10:1257191.

Article  PubMed  PubMed Central  Google Scholar 

Nedbal C, Naik N, Castellani D, Gahuar V, Geraghty R, Somani BK. ChatGPT in urology practice: revolutionizing efficiency and patient care with generative artificial intelligence. Curr Opin Urol. 2024;34:98–104.

Caglar U, Yildiz O, Meric A, Ayranci A, Gelmis M, Sarilar O, et al. Evaluating the performance of ChatGPT in answering questions related to pediatric urology. J Pediatr Urol. 2024;20:26.e1–26.e5.

Shah YB, Ghosh A, Hochberg AR, Rapoport E, Lallas CD, Shah MS, et al. Comparison of ChatGPT and Traditional Patient Education Materials for Men’s Health. Urol Pract. 2024;11:87–94.

Article  PubMed  Google Scholar 

Musheyev D, Pan A, Loeb S, Kabarriti AE. How well do artificial intelligence chatbots respond to the top search queries about urological malignancies? Eur Urol. 2024;85:13–6.

Article  PubMed  Google Scholar 

Song H, Xia Y, Luo Z, Liu H, Song Y, Zeng X, et al. Evaluating the performance of different large language models on health consultation and patient education in urolithiasis. J Med Syst. 2023;47:125.

Article  PubMed  Google Scholar 

Thia I, Saluja M. ChatGPT: is this patient education tool for urological malignancies readable for the general population? Res Rep Urol. 2024;16:31–7.

PubMed  PubMed Central  Google Scholar 

Gabriel J, Shafik L, Alanbuki A, Larner T. The utility of the ChatGPT artificial intelligence tool for patient education and enquiry in robotic radical prostatectomy. Int Urol Nephrol. 2023;55:2717–32.

Huo B, Cacciamani GE, Collins GS, McKechnie T, Lee Y, Guyatt G. Reporting standards for the use of large language model-linked chatbots for health advice. Nat Med. 2023;29:2988. 1-

Article  CAS  PubMed  Google Scholar 

Good Lay Summary Practice 2021 [Available from: https://health.ec.europa.eu/system/files/2021-10/glsp_en_0.pdf.

Fox S Health Online 2013 Pew Research Center2013 [Available from: https://www.pewresearch.org/internet/2013/01/15/health-online-2013/.

Hopkins AM, Logan JM, Kichenadasse G, Sorich MJ. Artificial intelligence chatbots will revolutionize how cancer patients access information: ChatGPT represents a paradigm-shift. JNCI Cancer Spectr. 2023;7:pkad010.

Di H, Wen Y. Will generalist medical artificial intelligence be the future path for health-related natural language processing models? Prostate Cancer Prostatic Dis. 2023:1–2.

Usage Policies [Available from: https://openai.com/policies/usage-policies.

Singhal K, Tu T, Gottweis J, Sayres R, Wulczyn E, Hou L, et al. Towards expert-level medical question answering with large language models. arXiv preprint arXiv:230509617. 2023.

Cacciamani GE, Siemens DR, Gill I. Generative artificial intelligence in health care. J Urol. 2023;210:723–5.

Article  PubMed  Google Scholar 

Meskó B, Topol EJ. The imperative for regulatory oversight of large language models (or generative AI) in healthcare. npj Digit Med. 2023;6:120.

Article  PubMed  PubMed Central  Google Scholar 

Baydoun A, Jia AY, Zaorsky NG, Kashani R, Rao S, Shoag JE, et al. Artificial intelligence applications in prostate cancer. Prostate Cancer Prostatic Dis. 2024;27:37–45.

Morozov A, Taratkin M, Bazarkin A, Rivas JG, Puliatti S, Checcucci E, et al. A systematic review and meta-analysis of artificial intelligence diagnostic accuracy in prostate cancer histology identification and grading. Prostate Cancer Prostatic Dis. 2023;26:681–92.

Alarcón-Zendejas AP, Scavuzzo A, Jiménez-Ríos MA, Álvarez-Gómez RM, Montiel-Manríquez R, Castro-Hernández C, et al. The promising role of new molecular biomarkers in prostate cancer: from coding and non-coding genes to artificial intelligence approaches. Prostate Cancer Prostatic Dis. 2022;25:431–43.

Article  PubMed  PubMed Central  Google Scholar 

Checcucci E, Rosati S, De Cillis S, Vagni M, Giordano N, Piana A, et al. Artificial intelligence for target prostate biopsy outcomes prediction the potential application of fuzzy logic. Prostate Cancer Prostatic Dis. 2022;25:359–62.

Article  PubMed  Google Scholar 

Comments (0)

No login
gif