Introduction

Colorectal cancer (CRC) is increasingly prevalent in China, ranking as the second most common malignancy with significant morbidity and mortality.[1,2] In recent years, the advent of whole-slide images (WSIs) has revolutionized the field of pathology, enabling the comprehensive assessment of spatial relationships among various biological phenotypes within the tumor microenvironment.[3] The tumor-infiltrating lymphocytes (TILs) are vital components of the immune microenvironment that can predict prognosis and therapeutic effects in solid tumors.[4] In the context of CRC prognosis evaluation, specific markers such as CD3 and CD8 have gained prominence due to their exceptional staining quality and reliable antigenic stability.[5] These markers facilitate the precise quantification and comprehensive characterization of TIL populations within distinct regions of the tumor, namely the core of the tumor (CT) and the invasive margin (IM). The immunoscore, a composite measure derived from the quantification of CD3+ and CD8+ T cells densities within the CT and IM areas, has emerged as a pivotal tool for predicting the risk of CRC relapse.[6]

So far, there has been little discussion about single CD3 and CD8 markers comparison (at different locations), their association with CD3-CD8 (representing the combination of CD3+ and CD8+ T cells density within the CT and IM), and prognostic performance comparison. Previous studies have demonstrated that CD3+ T cells were correlated with prognostic value closely at both IM and CT, and it has been identified as an independent prognostic factor.[7,8] The findings provide evidence that the IM of CD3 could influence the prognosis of stage II CRC and CD3+ T cells density could predict the prognosis of stage III colon cancer (CC).[9–11] Besides, there were strong links between CD3+ and CD8+ T cells.[12,13] CD3+ lymphocytes represent whole T lymphocytes, including helper T lymphocytes, CD8+ T cell, CD4+ T cell subsets, memory subsets, and functional subsets. Meanwhile, CD8+ T cells play an essential role in anti-tumor immunity by expressing cytotoxic molecules to suppress tumor cells, representing strong prognostic ability.[14] Consequently, there is a strong possibility that either CD3+ T cells or CD8+ T cells alone could be as efficient as the combined assessment of CD3+ and CD8+ T cells. This approach would not only alleviate the workload of pathologists and reduce patients’ financial burden but also enhance efficiency by selecting a single representative variable.

With the advent of artificial intelligence (AI) technology, deep learning is increasingly applied to medical image analysis because it is quantitative, objective, and repeatable. WSIs technology further improves the resolution and clarity of slides and dramatically enhances work efficiency via high-throughput section scanning. Deep learning techniques are widely used to differentiate benign and malignant breast cancer through the training of histopathological WSIs.[15] Moreover, a deep learning-based classifier could predict common gene mutations like APC and KRAS in CRC histopathology images.[16] Therefore, through the utilization of AI technology, the spatial quantification of CD3+ and CD8+ T cells can be expedited and automated.

There are two primary aims of this study: (1) To quantify the spatial distribution of CD3+ and CD8+ T cells within tissue regions in WSIs by using AI technology. (2) To compare the prognostic performance of the individual and combined indicators of CD3+ and CD8+ T cells in two independent CRC cohorts.

Methods Patients

This study consecutively included pathologically confirmed stage I–III CRC patients who underwent surgical resection with curative intent and excluded patients who received preoperative therapy (radiotherapy, chemotherapy, or chemoradiotherapy). Patients from Guangdong Provincial People’s Hospital (GDPH, March 2009 to December 2014) were selected as a training cohort. Others were from the Sixth Affiliated Hospital of Sun Yat-sen University (SYSU6, May 2013 to October 2016) and were chosen as a validation cohort. The determination of sample sizes for both training and validation sets relied upon the availability of tissue samples. This retrospective study has been approved by the Research Ethics Committee of Guangdong Provincial People’s Hospital (No.KY-Z-2022-009-02) and the Research Ethics Committee of the Sixth Affiliated Hospital of Sun Yat-sen University (No.2019ZSLYEC-169). Clinicopathological data of these patients, including age, sex, tumor-node-metastasis (TNM) stage, tumor grade, tumor location, and carcinoembryonic antigen (CEA) were collected from the database of two hospitals. The start time of the study was when the patient was diagnosed with CRC. The endpoint of this study was overall survival (OS), defined as the time from the surgery date until either the date of the death (event) or the last follow-up (censored). The follow-up of patients is mainly conducted through the inquiries made via the hospital’s medical information system and telephone. Slides in the training and validation cohorts were prepared as part of the routine histopathological examination at their original laboratories, and all were made by staining a 3 μm formalin-fixed paraffin-embedded (FFPE) tissue block section with hematoxylin and eosin staining (H&E).

WSI acquisition

H&E-stained slides depicting the most invasive part of the primary tumor were chosen from the tissue block for analysis. The selection process was performed by experienced pathologists at each institute, who were blinded to patient clinical information and outcomes. To acquire immunohistochemical (IHC)-stained slides (CD3 and CD8), a series of steps, including dehydration, section cutting, staining, and sealing, were carried out. The detailed procedure for IHC staining of CD3 and CD8 can be found in the Supplementary Method 1, https://links.lww.com/CM9/B837.[17] The IHC-stained slides from both hospitals were digitalized to WSIs at 40× magnification using digital slide scanner (Aperio-GT450 or AT2, Leica Biosystems, Buffalo Grove, Illinois, USA).

Tissues segmentation via neural network

A convolutional neural network (CNN) was used to segment tissues in IHC-stained WSIs automatically. The entire workflow is represented in Figure 1. We used a CNN model (VGG-19) to perform the segmentation of CD3 and CD8 WSIs, which had been released with our previous work.[18,19] The tissue segmentation map can be illustrated briefly by the following steps: Firstly, the IHC-stained WSIs were tiled into overlapped patches. Then, the CNN model was used to classify patches, and the prediction probability of each image patch was arranged by patch X-Y position within WSI. Next, the tissue category with the highest prediction probability was chosen as the prediction label. Finally, according to the areas occupied by each type of tissue, we calculated their proportion (excluding background areas) and got nine tissue categories (tumor stroma, tumor epithelium, lymphocytes, mucus, normal mucosa, adipose, debris, smooth muscle, and background).

Figure 1:

The workflow consisted of four steps for the quantification of cells within both the CT and the IM. Step 1: A CNN model was selected from the classic VGG-19, and a 9-category classification layer substituted the last layer of the model. And each patient’s WSI of CD3 and CD8 was divided into nine tissue categories and ICH staining was used. Step 2: nine tissue types were merged into four types: tumor area (red color), normal area (green color), infiltrating front (blue color), and gland (yellow color). Step 3: CD3+ T cells and CD8+ T cells in the IM and tumor core were segmented. Step 4: the positive T cells were quantified by density. CNN: Convolutional neural network; CT: The core of the tumor; ICH: Immunohistochemical staining; IM: Invasive margin; WSI: Whole-slide image.

Automatic determination of invasive tumor margin

Deep learning algorithms were employed to differentiate between the tumor area and normal tissue, enabling the identification and localization of specific regions within the tissue samples. The normal tissue, glands, IM, and CT were accurately located using these algorithms. The tumor region was obtained by merging debris, stroma, and tumor epithelium areas in WSIs. Meanwhile, the normal part was acquired by combining adipose and muscle area. Regarding the IM area, red-green-blue (RGB) images were converted into grayscale images at first. Next, 2D Gaussian smoothing with a standard deviation of 10 was used to filter grayscale images to reduce image noise and detail. Following this, we got the images divided into the black background and white region of interest (ROI). After that, the area of each connected domain (white part) was calculated and arranged by pixel area size, and the most prominent area was kept as the ROI. The morphological closing operation, which means dilation and erosion, was used to fill up the small cracks in the image, while the overall position and shape remained the same. Eventually, the area where the tumor area overlapped with the normal area was identified as the invasive margin of tumors around 500 μm thickness. We successfully divided the slices into four tissue types: normal tissue area, tumor area, IM area, and gland area. For visualization, these areas were colored. The green color represents the normal tissue area, the red color represents the tumor area, the blue color represents the IM, and the yellow color represents the glands [Figure 1].

Calculating the TILs density in IM and CT

The CD3+ T cells and CD8+ T cells were segmented in both the CT and IM, with their densities calculated in these regions and regarded as candidate variables in this study using our previously developed cell segmentation software.[17] All density values underwent percentile transformation (range from 0 to 100%). The CD3CT represented CD3+ T cell density in the CT, while CD3IM represented that in the IM. CD8CT represented CD8+ T cell density in the CT, while CD8IM represented that in the IM. Moreover, CD3 represented the average of CD3CT and CD3IM (CD3CT + CD3IM), while CD8 represented the mean of CD8CT and CD8IM (CD8CT + CD8IM). Furthermore, the average of CD3CT and CD8CT denoted as CD3-CD8CT (CD3CT + CD8CT). Similarly, CD3-CD8IM (CD3IM + CD8IM) represented the positive T cell density for two markers in the IM. Meanwhile, the average density of the CD3+ T cells in both regions was also acquired and referred to as CD3 (CD3CT + CD3IM), with that of the CD8+ T cells referred to as CD8 (CD8CT + CD8IM). Lastly, the immunoscore-like CD3-CD8 value was calculated by taking the average of the four percentiles derived from the analysis ([CD3CT + CD8CT + CD3IM + CD8IM]/4). In this manner, nine distinct CD3 and CD8 related score were constructed based on the density of the variables, using a percentile-converted value.

Statistical analysis

All image processing was done in Matlab software (R2020a, MathWorks, Natick, MA, USA) environment. Data analysis was carried out using R software (www.r-project.org; Version 4.1.2). The packages of survival, surminer, receiver operating characteristic R language (ROCR), boot, Hmisc, rms, modern applied statistics with S (MASS), dplyr, pheatmap, risksetROC, and survivalROC were used. The distribution of clinicopathological characteristics between the two cohorts was tested using the Pearson chi-squared test. To analyze the survival curves for the three- or two-category variable score, the Kaplan–Meier method was employed. The thresholds were accomplished using the cut2 function from the Hmisc R package.

Univariate and multivariate analyses were conducted using the Cox proportional hazards model to calculate the hazard ratio (HR) for each variable score, along with other clinicopathological risk factors, including age, sex, TNM stage, location, CEA, and grade. The discriminatory performance of the variable models was evaluated using Harrell’s concordance index (C-index). Moreover, the discrimination ability of individual variable and models was assessed by calculating the integrated area under the ROC curve (iAUC). The iAUC of each variable and model was assessed using 1000 bootstrap resampling iterations.

The Pearson correlation coefficient was calculated to evaluate the association among the variables. Statistical significance was defined as a two-sided P <0.05.

Results Patients data

After data management and WSI’s quality control, a total of 492 patients were enrolled in this study, including 358 patients from Guangdong Provincial People’s Hospital and 134 patients from the Sixth Affiliated Hospital of Sun Yat-sen University [Supplementary Figure 1, https://links.lww.com/CM9/B837]. Statistically significant differences were observed between the two cohorts in terms of tumor stage, grade, and location. However, no significant differences were found between the two cohorts in relation to age, sex, and CEA. Table 1 summarizes the distribution of clinicopathological characteristics between the two cohorts.

Table 1 - Distributions of demoimagedata and clinicopathologic characteristics of CRC patients in the two cohorts. Characteristics Training cohort (N = 358) Validation cohort (N = 134) χ 2 P values Age (years) 3.388 0.066 ≤60 133 (37.2) 62 (46.3) >60 225 (62.8) 72 (53.7) Sex 0.002 0.967 Male 213 (59.5) 80 (59.7) Female 145 (40.5) 54 (40.3) TNM stage 12.063 0.002 I 45 (12.6) 34 (25.4) II 143 (39.9) 43 (32.1) III 170 (47.5) 57 (42.5) Location 44.068 <0.001 Right-sided 95 (26.5) 0 Left-sided 263 (73.5) 134 (100.0) CEA 0.897 0.639 Normal 213 (59.5) 86 (64.1) Abnormal 109 (30.4) 36 (26.9) NA 36 (10.1) 12 (9.0) Grade 13.403 0.001 Low 310 (86.6) 119 (75.3) High 38 (10.6) 4 (3.0) NA 10 (2.8) 11 (8.2)

Data are presented as n (%). CEA: Carcinoembryonic antigen; CRC: Colorectal cancer; NA: Not available; TNM: Tumor-node-metastasis.

Correlation analysis between CD3 and CD8-based variables

The correlation analysis results of nine variables of positive T cells densities in different regions are presented in Figure 2. It had been observed that CD3CT and CD3 were closely related (R = 0.87) [Figure 2F]. Similar trends were observed between CD3IM and CD3, CD8IM and CD8, CD8CT and CD8, separately (R = 0.92, R = 0.89, R = 0.92) [Figure 2E,G,H]. However, CD3IM and CD8IM, CD3CT and CD8CT, CD3 and CD8 were weakly correlated (R = 0.36, R = 0.43, R = 0.39) [Figure 2C,D,I]. The heatmap visually illustrates the distribution of the variables’ survival status. Each column represents one patient, and the density of the variable is higher as it approaches 1 [Supplementary Figure 2, https://links.lww.com/CM9/B837]. Meanwhile, the CD3CT and CD3-CD8 were highly expressed in group 1 (OS status: alive) and weakly expressed in group 0 (OS status: dead). And the survival status of CD3CT was more stable than that of the other eight variables.

Figure 2:

Positive T cells of CD3 and CD8 counting consistency and correlation analysis. (A) Correlation analysis between CD3CT and CD3IM (R = 0.60); (B) Correlation analysis between CD8CT and CD8IM (R = 0.64); (C) Correlation analysis between CD8IM and CD3IM (R = 0.36); (D) Correlation analysis between CD8CT and CD3CT (R = 0.43); (E) Correlation analysis between CD3 and CD3IM (R = 0.92); (F) Correlation analysis between CD3 and CD3CT (R = 0.87); (G) Correlation analysis between CD8 and CD8IM (R = 0.89); (H) Correlation analysis between CD8 and CD8CT (R = 0.92); (I) Correlation analysis between CD8 and CD3 (R = 0.39). CT: Core of the tumor; IM: Invasive margin.

Prognostic value of variables

The discrimination metrics of each variable for predicting OS are presented in Supplementary Table 1, https://links.lww.com/CM9/B837. CD3CT exhibited good performance in predicting OS (C-index: 0.65 in the training group, 0.69 in the validation group). Importantly, the C-index values of CD3CT were close to those of CD3-CD8 (C-index: 0.65 vs. 0.64) in the training group and (0.69 vs. 0.69) in the validation group. Additionally, the iAUC values of CD3CT and CD3-CD8 in the training group were 0.65 vs. 0.63, and the validation group was 0.68 vs. 0.69. Owing to the excellent performance of CD3CT, our subsequent research focuses on this specific variable.

Prognostic value of CD3CT cell density

In the three-category CD3CT analysis of the training cohort, a density ranging from 0 to 33.8% (121 cases) was categorized as low, a density between 33.8% and 67.0% (119 cases) was categorized as intermediate, and a density between 67.0% and 100.0% (118 cases) was categorized as high. These categories were determined using the thresholds of 636 cells/mm2 and 941 cells/mm2. As shown by the Kaplan–Meier curves of these three groups, there was a significant difference in OS among these groups in the training cohort (P <0.001) [Figure 3A].

Figure 3:

Kaplan–Meier curves analysis for three-category CD3CT score in (A) the training cohort, and (B) the validation cohort. CD3CT: CD3+ T cells density in the core of the tumor.

In the validation group, we used the same thresholds as those used in the training group to categorize patients into three groups. This resulted in 62 patients in the low-density group, 39 patients in the intermediate-density group, and 33 patients in the high-density group. Although the differences between the high-density and intermediate-density groups were not noticeable, the discrepancy between them and the low-density group was significant (P = 0.018) [Figure 3B]. Meanwhile, we plotted the survival curve of the CD3-CD8 using the same grouping method and observed similar trends between the training cohort and the validation cohort (training cohort: P <0.001; validation cohort: P = 0.180) [Supplementary Figure 3, https://links.lww.com/CM9/B837].

Moreover, the patients were categorized into two groups based on CD3CT density: the low-density group (0–33.8%) and the high-density group (33.8%–100%). The results indicated that patients in the high CD3CT density group exhibited a higher 5-year survival rate compared to patients in the low CD3CT density group (81.3% vs. 62.5%) within the training cohort. High CD3CT density was associated with significantly better outcomes compared to low CD3CT density in the training cohort (P <0.001). The results were validated in the validation cohort. In the validation cohort, high CD3CT density group patients had significantly higher 5-year survival rate compared to the low CD3CT density group (92.5% vs. 75.0%). Besides, high CD3CT density correlated with superior survival outcomes (P = 0.005) [Figure 4]. Likewise, patients displaying elevated CD3-CD8 density exhibited higher rates of survival (training cohort: P <0.001; validation cohort: P = 0.068) [Supplementary Figure 4, https://links.lww.com/CM9/B837].

Figure 4:

Kaplan–Meier curves analysis for two-category CD3CT score in (A) the training cohort, and (B) the validation cohort. CD3CT: CD3+ T cells density in the core of the tumor.

In addition to the density-based stratification, our survival study included the categorization of patients into distinct groups based on cancer type (colon or rectal). The observed trend in CRC patients within the training cohort held equivalent significance to that observed in rectal cancer (RC) patients within the validation cohort (training cohort: P <0.001; validation cohort: P <0.001) [Supplementary Figure 5, https://links.lww.com/CM9/B837]. Furthermore, when stratifying patients based on the location of their CRC (left-sided and right-sided), CD3CT remained significant in both left-sided and right-sided CRC cases, as depicted in Supplementary Figure 6, https://links.lww.com/CM9/B837. Overall, the CD3CT score met the proportional performance assumption. Further analysis indicated CD3CT could be equally effective than CD3-CD8 in predicting patient outcomes.

CD3CT density as an independent predictor

For multivariate analysis, univariate variables with P <0.05 were selected. HR with 95% CI was calculated by using the Cox model. Univariate and multivariate analyses were conducted to assess the association between clinicopathological factors including CD3CT and OS [Table 2]. In the training cohort, HR for high vs. low CD3CT was 0.22 (95% confidence interval [CI]: 0.12–0.38, P <0.001). Similarly, in the validation cohort, the HR for high vs. low CD3CT was 0.21 (95% CI: 0.05–0.92, P = 0.037). In multivariate analysis, CD3CT was emerged as an independent predictor of OS in both cohorts (training cohort: adjusted HR for high vs. low 0.35, 95% CI: 0.18–0.65, P <0.001; validation cohort: 0.21, 95% CI: 0.05–0.94, P = 0.041). The lack of statistical significance in TNM stage could potentially be attributed to the limited number of cases.

Table 2 - Univariate and multivariate analyses of CEA, Grade, CD3CT and other clinicopathological variables. Characteristics Univariate analysis Multivariate analysis Training cohort Validation cohort Training cohort Validation cohort HR (95% CI) P values HR (95% CI) P values aHR (95% CI) P values aHR (95% CI) P values Age (years) ≤60 Ref – Ref – Ref – Ref – >60 1.53 (1.01–2.32) 0.046 2.20 (0.90–5.35) 0.083 1.02 (1.00–1.04) 0.019 1.04 (1.00–1.08) 0.045 Sex Male Ref – Ref – – – – – Female 1.01 (0.69–1.49) 0.945 0.80 (0.34–1.88) 0.607 – – – – TNM stage I Ref – Ref – Ref – Ref – II 2.63 (0.93–7.46) 0.069 1.13 (0.32–4.02) 0.846 1.77 (0.61–5.10) 0.290 1.01 (0.28–3.67) 0.981 III 6.49 (2.37–17.8) <0.001 2.10(0.68–6.44) 0.195 3.21 (1.14–9.03) 0.027 2.35 (0.75–7.35) 0.144 Location Right-sided Ref – – – – – – – Left-sided 0.84 (0.56–1.22) 0.417 – – – – – – CEA* Normal Ref – Ref – Ref – Ref