Unchecked sex bias in machine learning (ML) algorithms used in healthcare can exacerbate disparities in care and treatment. We aimed to assess the acknowledgment and mitigation of sex bias within studies using supervised ML for improving clinical outcomes in Rheumatoid Arthritis (RA). For this systematic review, we searched PUBMED and EMBASE for original, English language studies published between 2018 to November 2023. We scored papers on whether the authors reported, attempted to mitigate or successfully mitigated the following types of bias: training data bias, test data bias, input variable bias, output variable bias, analysis bias, and assessed the quality of ML research in all papers. This study is registered on PROSPERO with ID CRD42023431754. We identified 52 papers to include in our review. All but one had a female skew in their study participants, yet 42 papers did not acknowledge any potential sex bias. Three papers assessed bias in model performance by sex disaggregating their results. One paper acknowledged potential sex bias in input variables, and six papers in their output variables, predominantly disease activity scores. No paper attempted to mitigate for any type of sex bias. The findings demonstrate the requirement for increased promotion of inclusive and equitable ML practices in healthcare.

This systematic review has been conducted by employees of Haleon which produces and sells healthcare products, including some that may be mentioned in the research papers being reviewed. Whilst this anticipated conflict of interest is unlikely to appear often if at all, it is acknowledged, nevertheless. The authors declare no financial or other conflicts of interest that could influence the interpretation of the results or the conclusions drawn from this review.

Funding for research wholly derived from HALEON standing R&D. There were no additional sponsors or funding bodies.

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

Yes

The details of the IRB/oversight body that provided approval or exemption for the research described are given below:

N/A

I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.

Yes

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

Yes

I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.

Yes

All data produced in the present work are contained in the manuscript.