To identify our study cohort, we began with all VA clinics or sites (n = 1110) and excluded atypical such as community living centers or sites with fewer than 450 patients over the study period, similar to prior work.21 The final study cohort included 1050 VA sites (CRH-PC: 254; non-CRH-PC: 796). All patients assigned to a PACT team in each site were included; patient assignments were identified from the Reengineered Patient Care Management Module in the Corporate Data Warehouse (CDW).22

As adoption of CRH can impact both the number of care visits and patients seen at sites, we examined the quarterly number of visits and patients served per site for two types of care:

VA outpatient primary care—primary care visits (total across all modalities—phone, video, and in-person and by each modality) and primary care patients served.

VA emergency department (ED) visits and hospitalizations—visits to and patients seen in EDs and inpatient stays within the VA system.

We considered reporting change in visits per patient assigned to each site; however, many patients assigned to a PACT team are not expected to and do not use primary care in each quarter. We therefore reported results as the percentage change in volume of visits and patients served compared to baseline patterns of utilization, adjusting for site size. We also reported the number of visits per 1000 primary care patients served to indicate the extent of changes for patients who do utilize primary care in each quarter.

Note that if a hub provider had a telehealth visit for a patient assigned to a spoke site, the visit was attributed to the spoke site. Data on PC and ED encounters were obtained from Managerial Cost Accounting (MCA) OUT (Outpatient) National Data Extract (NDE),23 where we categorized primary and secondary clinic stop code pairs into categories and modalities of care. Hospitalizations were obtained from MCA TRT (Treating Specialty) NDE.23

We adjusted for differences in outcomes due to potentially differing site characteristics. We adjusted for site rurality (e.g., urban, rural, highly rural), site type (e.g., VHA Medical Center (VAMC), PC Community-Based Outpatient Clinic (PC-CBOC), multi-specialty CBOC (MS-CBOC), other), site size (number of patients with PACT team assignments by quarter), geographic region of each VA service network (East Coast, Southeast, Rocky Mountain/Gulf, Midwest, West Coast), and the quarterly average Elixhauser Comorbidity Score of patients assigned to each site. Importantly, we also included a binary indicator of whether a site ever adopted CRH-PC to adjust for any remaining unobservable differences across program and non-program sites that were time-invariant. In sensitivity analyses, we also adjusted for site-level summaries of other patient covariates such as mean age, as well as percents male, White, Black, Hispanic, and VA enrollment priorities 1 and 2.

In all models, we included the quarterly count of COVID-19 cases in each county24 and included quarter indicators to adjust for any shocks to health care systems or care use in each quarter, including lingering effects of the pandemic outside of the case count.

Covariate data were obtained from the CDW,22 VHA’s Geospatial Service Support Center,25 and New York Times’ COVID-19 county-year level data downloaded from GitHub.24

We first examined baseline site characteristics for CRH-PC and non-CRH-PC sites from FY2020Q1 (October-December 2019) before pandemic-related shutdown of in-person care began. We then examined unadjusted trends of CRH-PC and non-CRH-PC sites before and after the onset of COVID-19 to determine whether a difference-in-difference (DiD) framework was appropriate. Event studies improve on the traditional DiD estimator26,26,28 because they estimate differences between treatment and control group for each period prior to and after treatment (i.e., each quarter pre- and post-pandemic onset in our case).26, 28 This allows to visually and more transparently assess whether pre-pandemic model-adjusted differences between CRH-PC sites and non-CRH-PC sites were significant or trending upward or downward in a manner that could obscure or mask true differences in the post-pandemic onset period.26, 28, 29 An absence of pre-pandemic differences across CRH-PC and non-CRH-PC sites after covariate adjustment followed by abrupt differences post-pandemic signals attributability of findings to the pandemic.26, 28, 29 We also generated traditional DiD estimates to obtain the average effect of CRH-PC across all post-pandemic onset quarters (methods details in Appendix Section A).

As effects may vary across types of VA sites or site size, we conducted identical analyses, stratified by site type as site size is vastly different across site types and among these stratified analyses, further adjusted for site size. We also examined the number of visits per 1000 primary care patients served.

To strengthen attributability of findings to the CRH program, we conducted sensitivity analyses restricting the sample of CRH-PC sites to sites with pre-pandemic program implementation and to sites with program implementation during the majority of the post-pandemic onset period. We also examined reliance on CRH-PC services at CRH-PC sites (Appendix Section D).

All statistical analyses were conducted in Stata 17.0 (StataCorp, LLC).

This study followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines. It was funded by VA’s Office of Primary Care for quality improvement purposes and was therefore exempted from review by the Stanford institutional review board.