Abstract

Objective To validate a primary care electronic medical record (EMR) case definition for mood and anxiety disorders (including depression, anxiety, and bipolar disorder) and schizophrenia that can be used to estimate prevalence and co-occurrence.

Design Retrospective cross-sectional study.

Participants De-identified EMR data was used from 1574 primary care providers participating in the Canadian Primary Care Sentinel Surveillance Network (CPCSSN) from 1,692,987 patients who had 1 or more visits with a primary care provider. The reference set included 2488 patients, with 434 positive and 2054 negative for 1 or more mental health conditions of interest. A second reference set for schizophrenia represented 760 patients (30 positive and 730 negative).

Main outcome measures The agreement of 29 case definitions was assessed against a reference set by reporting sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy. Prevalence was estimated and co-occurrence was assessed in the CPCSSN dataset (N=1,692,987).

Results The strongest definition for mood disorders captured anxiety, depression, and bipolar disorder with a sensitivity of 80.7%, specificity of 88.7%, PPV of 59.9%, and NPV of 95.7%; and an estimated prevalence of 21.8% (95% CI 21.7 to 21.9). The inclusion of psychosis did not improve agreement (sensitivity 95.2%, specificity 80.7%, PPV 51.0%, NPV 98.8%), but schizophrenia alone had high agreement (sensitivity 93.3%, specificity 100%, PPV 100%, NPV 99.9%).

Conclusion High co-occurrence of anxiety, depression, and bipolar disorder was found. Algorithms validated to capture these conditions together produced stronger agreement compared with individual definitions. Schizophrenia was less likely to co-occur with other mental health conditions and produced higher agreement when validated separately. Application of validated algorithms to capture mental health conditions can inform disease surveillance and health system planning.

Mental health conditions—characterized by a disturbance in an individual’s cognition, emotional regulation, or behaviour—are increasing globally.1,2 Mental health impacts many aspects of an individual’s life, influencing health, social relationships, economic outcomes, and community involvement.1-4 Mood and anxiety disorders are the most common mental health conditions, encompassing a range of conditions such as generalized anxiety disorder and major depressive disorder.1-3 Previous studies have suggested 20% to 30% of people will experience a mental health condition in their lifetime.3-7

Early intervention in primary care has been shown to improve mental health outcomes.8 However, recognition of mental health conditions varies considerably in primary care settings. Globally, there are important gaps in treatment and access to care.1 The diagnosis and surveillance of mood and anxiety disorders is challenging because of overlapping symptoms, varying diagnostic accuracy, and lack of established diagnostic tests.2 High prevalence and co-occurrence, combined with delayed diagnosis and treatment increases the burden and impact of mental health conditions.1-3 Despite literature suggesting increases,8-18 the COVID-19 pandemic led to initial decreases in the occurrence of mental health visits and treatment.8,15-18 Delayed presentation in primary care can lead to greater severity of mental health disorders, as well as long-term impacts on population health and economic stability.18-20

Application of validated algorithms to accurately capture mental health conditions can inform research on prevalence and care trajectories. This study aimed to develop, validate, and apply electronic medical record (EMR)–based case definitions for mental health conditions, including mood and anxiety disorders (ie, depression, anxiety, bipolar disorder) and schizophrenia, to capture and characterize patients with mental health conditions in primary care settings. We explored the presentation of mental health conditions in primary care including the prevalence of and overlap between these conditions.

METHODSDesign

We conducted a retrospective cross-sectional study to develop, validate, and apply an EMR-based case definition for mental health diagnoses, including depression, anxiety, bipolar disorder, and schizophrenia, within a pan-Canadian representative patient population.21

Setting

This study used de-identified EMR data from 1574 primary care providers participating in the Canadian Primary Care Sentinel Surveillance Network (CPCSSN). Participating family physicians, nurse practitioners, and community pediatricians represent 1,692,987 Canadians located across 7 of the 10 Canadian provinces (British Columbia, Alberta, Manitoba, Ontario, Quebec, Nova Scotia, and Newfoundland and Labrador). Regional networks extract EMR data from 12 EMR products, which are merged into a pan-Canadian CPCSSN dataset. During this process, CPCSSN data are standardized to map prescribed medications to Anatomical Therapeutic Chemical (ATC) classification codes, and medical diagnoses to International Classification of Diseases (ICD), 9th Revision, Clinical Modification (ICD-9-CM) diagnostic codes. This study assessed billing, health condition (problem list), encounter diagnosis, prescribed medication, and patient and provider characteristics from CPCSSN.

Participants

We used EMR records for active patients, defined as those with 1 or more visits between January 1, 2019, and December 31, 2021. There were 689,301 active patients in CPCSSN with health records from the inception of the EMR to December 31, 2021.

A subset of active patients was randomly selected for chart review using the Structured Query Language (SQL) random function. This subset was reviewed by 2 medically trained individuals, with discrepancies reviewed by a third person (A.G.S.). In total, 2488 patients were included in the reference set. Reviewers assessed de-identified EMR records from the health condition, billing, and encounter tables set and documented the following: unique patient identification number; diagnostic text for depression, anxiety, bipolar disorder, and schizophrenia; confirmation of each diagnosis (eg, yes, no, unsure); and other relevant notes (eg, treatment, referral) to determine if the patient had 1 or more mental health conditions of interest. Using the created data extraction tables, which had high inter-rater agreement (98.8%), we flagged patients as positive or negative for each mental health condition. The reference set had 434 patients positive for 1 or more mental health conditions (ie, depression, n=249; anxiety, n=261; bipolar disorder, n=19; schizophrenia, n=6) and 2054 patients who were negative (ie, did not have any mental health indications including diagnoses, text, or medications) (Figure 1). In our reference set, 272 patients had 1 condition, 152 patients had 2 conditions, and 10 patients had 3 conditions of interest. Our reference set included EMR data from British Columbia (3.3%), Alberta (14.8%), Manitoba (11.2%), Ontario (44.6%), Quebec (11.2%), Nova Scotia (10.0%), and Newfoundland and Labrador (4.9%).

Those diagnosed with schizophrenia represented a small proportion of the reference set (n=6); therefore, a second medical record review was performed. To ensure we captured patients with schizophrenia, we used terms previously found by chart reviewers to identify records. These additional records underwent medical record review using the previously described data extraction process. Our final schizophrenia reference set included 760 patients (30 positive and 730 negative) (Figure 1).

Case definitions

Definitions derived from the literature search, medical terminology sources, and diagnostic coding structures were reviewed by physicians and researchers for face validity (Supplementary Table 1, available from CFPlus*).2,5,6,16,22-29 As mental health symptoms and diagnoses may overlap,22,23 we developed broad mood and anxiety definitions, as well as definitions for individual conditions, using ICD-9 codes from billing, encounter diagnosis, and health condition lists of the EMR as well as ATC codes representing prescribed medications. Initial assessment of 9 definitions led to additional definitions being developed and tested. In total we tested 29 definitions (mood and anxiety [11], depression and anxiety [3], depression and bipolar disorder [3], anxiety [3], depression [4], bipolar disorder [3], and schizophrenia [2]) (Table 1).28

Table 1.

Developed mental health disorder case definitions: HC represents the health condition or problem list from the EMR, billing represents the billing table in the EMR, and EDT represents the encounter diagnosis table in the EMR. Bold text indicates chosen definition.

Statistical analysis

We assessed agreement between each of the definitions and the reference sets using several metrics including sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and overall accuracy.

We assessed the prevalence and 95% confidence limits computed using an exact binomial test. Significance was assessed at P=.05. We characterized patients based on sex and age using descriptive statistics, including means and SDs, and frequencies and percentages, and assessed the co-occurrence of mental health conditions within the CPCSSN cohort. Patient age was calculated at the index date of December 31, 2021. We applied a previously validated case definition to capture posttraumatic stress disorder (PTSD).22,23 We assessed differences in patients with and without a mental health condition of interest using 2 and t tests. Approval for this study was obtained from the Health Research Ethics Board at the University of Manitoba in Winnipeg. Statistical analyses were conducted using SAS, version 9.4.

RESULTS

There were 1,692,987 patients in the CPCSSN repository, 54.0% of whom were female with a mean (SD) age of 43.5 (24.7) years. Our reference set had similar characteristics, with 54.2% female and a mean (SD) age of 42.6 (23.1). The mean (SD) annual visit rate for patients in our reference set was higher than the CPCSSN dataset (2.75 [3.0] vs 2.65 [3.0]; P=.004).

Mood and anxiety case definitions

We tested 11 definitions for mood and anxiety disorders (Table 2). Definitions 5, 6, 8, 9, and 11 demonstrated high agreement, with sensitivity and specificity greater than 80.0%. Definitions 1 and 3 had sensitivities and specificities of more than 80.0%; however, the PPVs were lower (53.3.% and 51.0%, respectively) due to high false-positive results. Similarly, the inclusion of medications (definition 5) produced high sensitivity (93.8%) but lower PPV (55.2%). The inclusion of adjustment reaction in definitions 6 and 8 did not improve capture and produced minor decreases in specificity (87.7% and 88.1%) when compared with definition 11 (88.7%) (Table 2).

Table 2.

Mood and anxiety case definition agreement: N=2488. Bold text indicates chosen definition.

Individual mental health condition case definitions

We assessed 3 definitions for depression and anxiety. Definition 13 provided balanced metrics, with a sensitivity of 79.5%, NPV of 95.5%, specificity of 88.9%, and PPV of 59.4%, indicating that the definition was able to identify patients with and without the condition. A similar algorithm produced high agreement for depression and bipolar disorder (definition 17: sensitivity 81.6%, NPV 97.8%, specificity 93.6%, PPV 60.0%) (Table 3).

Table 3.

Individual mental health condition case definition agreement: N=2488. Bold text indicates chosen definition.

Anxiety definitions (18 to 20) and bipolar disorder definitions (25 to 27) demonstrated poor agreement compared with broader mood and anxiety definitions (Table 3). Definition 19 (anxiety) and 26 (bipolar disorder) applied similar algorithmic approaches to previously mentioned definitions and had the following agreement: definition 19, sensitivity 53.6%, specificity 87.9%, PPV 34.2%, NPV 94.2%; definition 26, sensitivity 89.5%, specificity 98.3%, PPV 28.8%, NPV 99.9%). Among definitions capturing depression, definition 22 applied a similar approach and had moderate agreement (sensitivity 79.9%, specificity 94.2%, PPV 60.5%, and NPV 97.7%) (Table 3).

Mood and anxiety definition 3 incorporated diagnostic codes for schizophrenic disorder and had a sensitivity of 95.2%, specificity of 80.7%, PPV of 51.0%, and NPV of 98.8% (Table 2). When schizophrenic disorder was assessed independently (definition 29), it produced a sensitivity of 93.3% (77.9% to 99.2%), specificity of 100% (99.5% to 100%), PPV of 100% (87.7% to 100%), and NPV of 99.97% (99.0% to 99.9%) (Table 3). The inclusion of psychosis did not improve agreement (sensitivity 95.2%, specificity 80.7%, PPV 51.0%, NPV 98.8%).

Co-occurrence of mental health conditions

When applied to the CPCSSN population (N=1,692,987), mood and anxiety prevalence ranged from 13.6% to 37.8% (Table 4). Definition 11, which included depression, anxiety, and bipolar disorder, suggested a prevalence of 21.8%. We demonstrated prevalence estimates for anxiety (definition 19, 16.1%), depression (definition 22, 10.6%), bipolar disorder (definition 26, 2.2%), and schizophrenia (definition 29, 0.6%) in Table 4.

Table 4.

Exact binomial estimate: N=1,692,987. Bold text indicates chosen definition.

A higher proportion of those with mood and anxiety diagnoses were female (definition 11) (64.5% vs 35.5%, P<.0001). Female patients were more likely to be diagnosed with anxiety (65.2% vs 34.8%, P<.0001), depression (66.9% vs 33.1%, P<.0001), bipolar disorder (63.0% vs 37.0%, P<.0001), and PTSD22 (62.5% vs 37.5%, P<.0001); however, female patients were less likely to be diagnosed with schizophrenia (38.8% vs 61.2%, P<.0001). Figure 2 describes the overlap of mental health diagnoses. Anxiety had the largest estimated prevalence (16.1%), and less than half of anxiety diagnoses occurred alone (9.7%); 11.4% of patients with anxiety had another mental health diagnosis. Co-occurrence was largest with anxiety and depression; 5.0% of patients had both conditions. Similarly, more than half of those with bipolar disorder also had anxiety (0.6%), depression (0.3%), or anxiety and depression (0.6%). Among patients diagnosed with schizophrenia half did not have another mental health condition (Figure 2).

Figure 2.

Venn diagrams showing percentages of patients diagnosed with 1 or more mental health conditions

DISCUSSION

Using pan-Canadian primary care EMR data we developed definitions for depression, anxiety, bipolar disorder, and schizophrenia. We sought a definition with balanced metrics to accurately identify patients with and without a condition. Capturing depression, anxiety, and bipolar disorder independently with specific codes produced moderate agreement, represented in definitions 19 (anxiety), 22 (depression), and 26 (bipolar disorder). Agreement was higher when anxiety, depression, and bipolar disorder were captured together; this was related to high co-occurrence of diagnostic codes and an estimated prevalence of 21.8%. However, schizophrenic disorder had higher agreement when assessed individually, with an estimated prevalence of 0.6%.

The literature demonstrates high co-occurrence of mood and anxiety disorders, with anxiety having the highest prevalence.1,2 In our study, half of the patients with anxiety had another co-occurring mental health condition. Similarly, although depression definitions had moderate agreement, agreement was improved in combined definitions. Gleeson et al indicated that EMR diagnostic codes fail to capture a large proportion of patients with a mental health condition.5 Mental health diagnosis is challenging due to diagnostic overlap, diagnostic limitations, lack of established diagnostic tests, and variability and fluidity of symptoms.2,5 Anxiety and depression can be both symptoms of other conditions or conditions themselves. Furthermore, mental health–related stigma can impact those wanting to seek help and is a barrier to treatment access.30 Contrary to mood and anxiety disorders, most patients diagnosed with schizophrenia did not have another mental health diagnosis. Similar to other studies, we found that mood and anxiety disorders were more common among female patients whereas schizophrenia was more common among male patients.2

Prior to the pandemic, the Public Health Agency of Canada reported that approximately 9.4% to 10.5% of Canadians accessed health services for a mood and anxiety condition annually.2 Most mental health problems are managed entirely in primary care, comprising an estimated 40% of the primary care workload.8,31 Although many patients use medications to manage their mental health conditions,5 we did not find inclusion of medication codes improved capture. Medications may be used more broadly or have varying reliability in treatment. Mood and anxiety disorders frequently coexist with other chronic conditions and are situated in contextual environmental, social, and cultural factors that impact care-seeking behaviour, presentation, and health care interactions.1,2,15,18,31 Understanding prevalence and trends in primary care may prevent widening of health inequalities and ensure service provision can support growing primary care demands.2

Limitations

We relied on primary care provider documentation in EMRs, which could both overestimate or underestimate prevalence due to variation in provider coding, missing diagnoses, and incomplete documentation. Clinicians use EMR systems for clinical purposes and may not use specific ICD-9-CM codes. This study aimed to assess diagnostic codes; future research should assess the evolution in diagnostic accuracy that may present over the course of multiple visits. CPCSSN includes only primary care EMR data and does not represent specialist visits; future studies linking this dataset to representative cohorts of specialists could be helpful to improve the certainty and accuracy of our prevalence estimates.

Conclusion

Our study demonstrated a lifetime prevalence of 21.8% for mood and anxiety disorders. The co-occurrence and overlap between these diagnostic codes support broader mood and anxiety definitions. However, schizophrenia, characterized in a different patient population, should be captured independently. This work can support assessment of trends in prevalence and health service research to inform improvements in disease surveillance and support monitoring of trends and changes in care provision.

Acknowledgment

Funding for this study was provided by the Children’s Hospital Research Institute of Manitoba and the University of Manitoba Internal Grants Competition. Infrastructure and human resource funding for the Canadian Primary Care Sentinel Surveillance Network is provided by the Canadian Primary Care Research Network (the Strategy for Patient-Oriented Research initiative of the Canadian Institutes of Health Research) and the Public Health Agency of Canada. The research team is supported by in-kind funding and support provided by the Department of Family Medicine at the University of Manitoba in Winnipeg. We acknowledge Rita Costa and Emmanuel Adegbite for performing the chart validation reviews in this study.

Footnotes

↵* Supplementary Table 1 is available from https://www.cfp.ca. Go to the full text of the article online and click on the CFPlus tab.

Contributors

Leanne Kosowan, Dr Alexander G. Singer, Dr Elissa M. Abrams, and Dr Jennifer L.P. Protudjer conceptualized the study. Leanne Kosowan and Dr Singer supported data acquisition. Leanne Kosowan conducted the analysis and drafted the manuscript. All authors assisted with the interpretation of study results and manuscript revisions; and approved the final version of the manuscript for submission.

Competing interests

None declared

This article has been peer reviewed.

Cet article a fait l’objet d’une révision par des pairs.

Copyright © 2025 the College of Family Physicians of Canada