Predicting the Ki-67 proliferation index in pulmonary adenocarcinoma patients presenting with subsolid nodules: construction of a nomogram based on CT images

Introduction

The introduction of low-dose computed tomography (CT) has facilitated the detection of pulmonary subsolid nodules (SSNs) in population (1,2). Many SSNs follow a general pattern of inert growth, with minimal changes that may be difficult for the naked eye to discern during annual follow-up (3). However, a recent study showed that the malignant rate of persistent pulmonary SSNs can be quite high, with 3–30% becoming malignant (4).

Uncontrolled cell proliferation is a hallmark of cancer, and the proliferation status is related to the tumor’s biological behavior, which affects the treatment and prognosis of patients (5,6). The nuclear-associated antigen Ki-67 is associated with cell proliferation and has been widely used as a prognostic indicator of disease-free survival in many malignant tumor types including glioma, breast cancer, and hepatocellular carcinoma (6-8). A recent study reported that 81.8% of oncologists indicated that the Ki-67 proliferation index (PI) is useful for informing patients about their prognosis, and more than 90% of oncologists indicated that they would use the Ki-67 PI to suggest a more guarded prognosis and/or initiate therapy after surgical resection (9). Indeed, predicting the Ki-67 PI may indirectly reflect the aggressiveness of tumors. Traditional immunohistochemical staining for Ki-67 is a widely accepted method for evaluating proliferative activity and prognosis in various tumor types. However, this technique requires postoperative specimens or preoperative fine-needle biopsies, which are both invasive and may increase the rate of pneumothorax by approximately 15% (10). Therefore, an alternative, noninvasive assessment would be highly valuable in daily clinical practice.

The morphology of a tumor may be related to its malignancy (11). However, the accurate determination of the malignancy of SSNs still remains a significant challenge (12). Radiomics is the process of extracting information from medical images with high-throughput methods and transforming them into mineable quantitative data, which provides more valuable information compared to traditional imaging analysis (13). Image-based biomarkers can noninvasively capture the radiographic phenotype and characterize the underlying histopathology of a tumor and are not subject to sampling bias. Several studies have shown the potential of radiomics in aiding cancer detection, diagnosis, tumor classification, and assessment of prognosis (8,14,15). In addition, the radiomics approach has been used for predicting the significance of the Ki-67 PI in lung cancer prognosis, disease-free survival, and lymph node metastasis (16-18). However, to the best of our knowledge, little work has been reported on using radiomic features for predicting the Ki-67 PI in lung cancer patients with SSNs (19).

Therefore, the objective of this study was to build and validate a nomogram for predicting the expression status of Ki-67 in lung adenocarcinoma (LUAD) patients with SNNs.

Methods

Patients

This study was conducted in accordance with the Declaration of Helsinki (as revised in 2013) and was approved by the institutional board of the First Affiliated Hospital of Zhejiang Chinese Medical University (no. 2019-KL-067-01), China. Individual patient consent was waived due to the retrospective nature of this study.

Between January 1, 2015, and December 31, 2018, 359 LUAD patients with confirmed histopathology results were retrospectively recruited from the First Affiliated Hospital of Zhejiang Chinese Medical University, China (institution 1) and the First Affiliated Hospital, Zhejiang University School of Medicine, China (institution 2). The diagnosis of LUAD satisfied the standards of the International Association for the Study of Lung Cancer, the American Thoracic Society, and the European Respiratory Society classifications (20). The inclusion criteria for this study were as follows: (I) a chest CT scan with thin-thickness performed within 1 month before surgery; (II) LUAD presenting with a SSN, including pure ground-glass nodules (pGGNs) and part solid ground-glass nodules (GGNs); (III) SSNs with a diameter ≤30 mm (window width 1,300 HU and window level –600 HU); and (IV) immunohistochemistry showing Ki-67-positive staining. The exclusion criteria were as follows: (I) patients who had any oncological treatment, biopsy, or noninvasive examination before surgery; (II) histopathological data of Ki-67 unavailable; (III) images that affected nodule sketching; or (IV) LUAD presenting as solid nodules. Finally, a total of 153 SSNs were enrolled (Figure 1). The 107 patients from institution 1 were randomly divided into a training cohort and an internal validation cohort in a 7:3 ratio. The 46 patients from institution 2 were used as the external validation cohort.

Figure 1 A flowchart showing the selection process of the study population.

Immunohistochemical analysis

All specimens were surgically resected tumors to ensure that the Ki-67 PI represented the whole tumor. The specimens were evaluated by experienced pathologists who were blinded to the CT information. Two to three sections of lung cancer with the hottest spots were selected, and at least 5 fields of view (400× magnification) were examined in each section. The Ki-67 PI refers to the percentage of malignant cells with positive nuclear Ki-67 immunostaining. In accordance with previous studies (21,22), the median of the 5th percentile was selected as the Ki-67 PI cutoff value. Samples were classified into 2 groups: the high Ki-67 PI group (Ki-67 ≥5%) and the low Ki-67 PI (Ki-67 <5%) group.

Image qualitative evaluation and segmentation

All patients underwent a conventional CT scan. Of the 107 enrolled patients from institution 1, 47 were scanned by a Somatom Sensation 64 machine (Siemens Healthineers, Erlangen, Germany), and 60 were scanned by an Aquilion ONE scanner (Toshiba Medical Systems, Tokyo, Japan). The CT scan parameters were as follows: tube voltage 100–120 kV, automatic tube current modulation, collimation 0.6 mm × 64 (Siemens) and 0.5 mm × 64 (Toshiba), matrix 512×512, and slice thickness 0.75 mm (Siemens) and 1.0 mm (Toshiba).

Of the 46 enrolled patients from institution 2, 15 were scanned by a Brilliance 64 scanner (Philips Medical Systems, Best, The Netherlands), 23 were scanned using a Lightspeed VCT, and 8 were scanned by an Optima CT 540 machine (General Electric Company, Boston, MA, USA). CT scan parameters were as follows: tube voltage 120 kV, automatic tube current modulation, collimation 0.625 mm, matrix 512×512, and slice thickness 1.0–1.25 mm.

The nodules were identified by 2 cardiothoracic radiologists with 5 and 10 years of experience. Any disagreements were resolved via consultation with a third cardiothoracic radiologist with nearly 30 years of experience. The region of interest (ROI) was manually segmented in a 3D space across all of the 2-dimensional thin-slice CT images in DICOM format using open-source software ITK-SNAP (version 3.6.0; www.itksnap.org). The radiologists were blinded to the pathological diagnosis but were provided with clinical information, such as age, and were given the option to vary the window and level setting within the software to efficiently annotate the nodule. Inter- and intraclass correlation coefficients (ICCs) were used to assess the agreement of feature extraction.

CT image feature extraction and radiomics signature establishment

The segmented ROI was imported into the AK software (Analysis Kit, GE Healthcare, Chicago, IL, USA). A total of 1,316 radiomic features, including first order, shape, gray level co-occurrence matrix (GLCM), gray level run-length matrix (GLRLM), and neighborhood gray tone difference matrix (NGTDM) parameters were calculated. All extracted radiomic features were normalized with z scores to remove the limitation of each unit. In addition, the outlier values were replaced by the median value of the column features.

The minimum redundancy maximum relevance (mRMR) and the least absolute shrinkage and selection operator (LASSO) were used to select the most useful predictive features in the training cohort and radiomics signature (rad-score) building. The mRMR was determined to eliminate the redundant and irrelevant features. Subsequently, LASSO (23) was conducted to select the optimized subset to construct the radiomics signature. The rad-score was generated by weighing the selected features according to their respective coefficients.

Building the nomogram and model validation

Univariable logistic regression analysis was performed to evaluate clinical variables. The significant clinical risk factors and the radiomics signature were then imported into a multivariable logistic regression analysis to develop the nomogram. The flowchart of the nomogram is shown in Figure 2. The nomo-score was generated by weighing the rad-score and the diameter {the formula is shown in the Appendix 1 [2]}. Subsequently, the internal validation cohort and the external validation cohort were used to test the nomogram model. A calibration curve was used to investigate the performance of the nomogram with a calibration plot using 1,000 bootstrap resamples and the Hosmer-Lemeshow test. The performance of different models was assessed using the receiver operator characteristic (ROC) curve, sensitivity, specificity, and accuracy.

Figure 2 A diagram showing the workflow of the development of the nomogram.

Decision curve analysis

To assess the clinical value of the nomogram, the radiomics signature, and the clinical model for predicting the expression of Ki-67, decision curve analysis (DCA) was performed. DCA is a method to evaluate clinical predictive models, diagnostic tests, and molecular markers (24,25). The probability threshold is a certain value at which the diagnostic patient would choose intervention. The net benefit represents the difference between the expected benefits and the expected damages.

Statistical analysis

The chi-squared test was applied for categorical variables, such as gender. The t test was used between groups of normally distributed variables, and the Mann-Whitney test was used for nonnormally distributed variables. The "RMS" package of the R software was used to build a nomogram. The LASSO logistic regression model was also used with penalty parameter tuning that was conducted by 10-fold cross-validation. The DeLong test was used to compare the difference in ROC curves between different models. Statistical analysis was performed with R software (version 3.6.1; https://www.r-project.org) and SPSS version 25 (IBM Corp., Armonk, NY, USA). A 2-tailed P value <0.05 was considered statistically significant.

Results

Clinical characteristics

A total of 153 consecutive patients with LUAD were enrolled in this retrospective study. There were 101 females and 52 males, with a mean age of 56.2±11.0 years (range, 26–79 years). Within the cohort, 87 patients had high Ki-67 PI and 66 patients showed low Ki-67 PI. There were 108 cases of pGGNs and 45 cases of part-solid GGNs (Table S1). The mean diameter of the nodules was 13.8±6.1 mm (range, 5–30 mm).

The detailed clinical characteristics of the training cohort and the validation cohort are shown in Table 1. A high Ki-67 PI was often observed in patients with part-solid GGNs and higher-grade clinical tumor node metastasis (cTNM) staging.

Inter- and intraobserver reproducibility of radiomic feature extraction

The interobserver ICC for reader A and reader B ranged from 0.776 to 0.871. The intraobserver ICC between reader A and reader B ranged from 0.790 to 0.927. The results indicated a favorable inter-and intraobserver reproducibility for feature extraction.

Radiomic signature development and validation

A total of 20 features were retained by using the mRMR. LASSO was then used to select the optimized features to construct the radiomics signature (rad-score). The detailed formula of the rad-score is shown in the Appendix 1 [1]. The value of the rad-score was higher in patients with a high Ki-67 PI compared to patients with a low Ki-67 PI. Compared with the clinical model, the radiomics signature showed a higher predictive ability for Ki-67 PI. The AUC of the training cohort, internal validation cohort, and external validation cohort for the radiomics signature was 0.86 (95% CI: 0.77 to 0.95), 0.81 (95% CI: 0.64 to 0.98), and 0.77 (95% CI: 0.62 to 0.91), respectively (Figure 3). The DeLong test between the radiomics signature and the clinical model showed statistical significance (P=0.02).

Figure 3 Receiver operating characteristic (ROC) curves of predictive Ki-67 proliferation index (PI) of the clinical model, radiomics signature, and nomogram between the training cohort, internal validation cohort, and the external validation cohort.

Building the nomogram and assessing its diagnostic value

Univariable logistic regression analysis in the training cohort revealed that the diameter of the nodule (OR 1.17; 95% CI: 1.06 to 1.31; P=0.003) and the clinical TNM staging (OR 3.77; 95% CI: 1.60 to 8.87; P=0.002) were independent predictors of high Ki-67 PI. Stepwise multivariable analysis was used to integrate the optimal predictor and develop the nomogram (Figure 4). Table 2 showed a superior predictive performance in Ki-67 expression compared to the clinical model in both the training cohort (AUC 0.86; 95% CI: 0.77 to 0.95), the internal validation cohort (AUC 0.80; 95% CI: 0.64 to 0.97), and the external validation cohort (AUC 0.79; 95% CI: 0.65 to 0.94). The Hosmer-Lemeshow test showed a good fit between the nomogram and the actual trend (Figure 5). The DeLong test showed that the AUC of the nomogram was significantly different to that of the clinical model (P<0.01), but there was no statistical difference in the AUC between the radiomics signature and the nomogram (P=0.77).

Figure 4 The nomogram incorporating the radiomics signature and the diameter to predict the risk of high Ki-67 PI expression. The value of the diameter and the rad-score can be converted into a risk score according to the “Points”. After adding up the individual risk scores of these predictors, the corresponding prediction probability of Ki-67 is shown as “Risk”. PI, proliferation index.

Figure 5 The goodness-of-fit curve of the nomogram in the training cohort, internal validation cohort, and external validation cohort. The 45° dotted line represents the ideal prediction, the dotted curve indicates the performance of the nomogram in Ki-67 PI prediction, and the solid curve indicates the bias-corrected nomogram. PI, proliferation index.

Assessment of clinical value using DCA

DCA was used to assess the clinical value of the 3 models (Figure 6). The DCA showed that for patients with threshold probabilities of 10% to 78%, the net benefit was maximized by the nomogram and radiomics signature approach.

Figure 6 Decision curves of the clinical value assessment of the 3 models in the training cohort, internal validation cohort, and external validation cohort. The X-axis represents the high-risk threshold and the Y-axis represents the net benefit. The black solid line represents high Ki-67 PI, and the yellow line represents low Ki-67 PI. The red, blue, and green lines represent the nomogram, radiomics signature, and clinical model, respectively. PI, proliferation index.

Discussion

The aim of this study was to develop a nomogram to predict the Ki-67 PI in pulmonary adenocarcinoma patients presenting with SSNs on a conventional CT scan. The investigation demonstrated that the predictive performance of radiomics could act as a noninvasive predictor of Ki-67 expression for early LUAD patients. The nomogram showed a superior performance in predicting the Ki-67 PI. This concurs with a previous study by Gu et al., which showed that the radiomic classifier could predict the Ki-67 PI in patients with non-small cell lung cancer (NSCLC) (19).

In the current study, there was no significant difference in gender, age, or nodule type between patients with high Ki-67 PI and patients with low Ki-67 PI. In contrast, previous studies have shown that a high Ki-67 PI was commonly observed in older male subjects (19). The differences in results may be attributed to the fact that the study population in the latter were early LUAD patients with SSN as an inert growth. However, in agreement with McWilliams et al. (26), our current study found that larger diameter nodules with a high cTNM stage were associated with elevated Ki-67 expression.

In accordance with a previous study (22), a cutoff value of 5% (the median of our data) was used for Ki-67 expression. Due to the heterogeneity of tumors, a tumor biopsy does not represent the whole tumor. Thus, to ensure that Ki-67 expression represented the entire SSN, our study only selected LUADs with SSNs that were completely surgically removed. Meanwhile, Warth et al. found that different types of lung cancer have different Ki-67 expression and different Ki-67 cutoff values (17). This may explain the lack of an agreed upon standard for the Ki-67 PI cutoff value.

Recently, texture analysis has become an emergent field in radiology. Radiomic features can provide a plethora of information, especially in oncology images (14,27,28). Radiomic features have been shown to reflect numerous clinically relevant characteristics of lung cancer (29) and have been shown to be associated with biomarkers which may act as potential clinical prognostic indicators (16).

The Ki-67 PI is clinically the most commonly used biomarker for proliferative activity in tumors. Several studies have shown that a high Ki-67 PI indicates tumor cell proliferation, metastasis, poor prognosis, and high grade in NSCLC patients, and all this information may be beneficial during the clinical decision-making process. Furthermore, Ishibashi et al. reported that NSCLC patients with a high Ki-67 PI were more sensitive to radiation (30).

This study hypothesized that the Ki-67 PI in tumors may be related to radiomics signatures and a predictive nomogram was constructed. By comparing patients with a high Ki-67 PI (≥5%) and patients with a low Ki-67 PI (<5%), the report demonstrated that both the radiomics signature and the nomogram showed a better AUC compared to the predicted values. The nomogram was able to provide a risk nomo-score for each patient. This method may represent a noninvasive strategy to dynamically monitor the biological behavior of SSNs and provide a guideline for individualized clinical treatment.

There were several limitations to this study. First, this retrospective study involving 2 centers had a relatively a small sample size. Future large-scale studies should be conducted with data from multiple centers. Second, the lesions in this study were from early-stage LUAD patients with SSNs, and future studies should also examine LUAD with solid nodules. In addition, the data gathered herein were obtained from different scanners with different slice thicknesses. However, the AUC analysis of both the training and validation cohorts indirectly indicated that the radiomics signature and the nomogram had better generalization ability than did the clinical model.

Conclusions

This report demonstrated that the radiomics signature extracted from SNNs of the lung may represent a potential method for predicting the Ki-67 PI. Furthermore, a nomogram was developed to predict the individual patient risk score and this may contribute to the personalized management of patients with LUAD.

Acknowledgments

We thank Mr. Michael C. Lin for language editing and Mr. Liang Li for technical support in the software code.

Funding: This work was supported by the Zhejiang Provincial Natural Science Foundation of China (No. LSY19H180003) and the Scientific Research Project of the Zhejiang Education Department (No. Y202045271).

Footnote

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://dx.doi.org/10.21037/qims-20-1385). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. This study was approved by the institutional ethics board of the First Affiliated Hospital of Zhejiang Chinese Medical University and the First Affiliated Hospital, Zhejiang University School of Medicine. Individual patient consent was waived due to the retrospective nature of this study.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Yankelevitz DF, Yip R, Smith JP, Liang M, Liu Y, Xu DM, Salvatore MM, Wolf AS, Flores RM, Henschke CIInternational Early Lung Cancer Action Program Investigators Group. CT Screening for Lung Cancer: Nonsolid Nodules in Baseline and Annual Repeat Rounds. Radiology 2015;277:555-64. [Crossref] [PubMed]
2. Chang B, Hwang JH, Choi YH, Chung MP, Kim H, Kwon OJ, Lee HY, Lee KS, Shim YM, Han J, Um SW. Natural history of pure ground-glass opacity lung nodules detected by low-dose CT scan. Chest 2013;143:172-8. [Crossref] [PubMed]
3. Hasegawa M, Sone S, Takashima S, Li F, Yang ZG, Maruyama Y, Watanabe T. Growth rate of small lung cancers detected on mass CT screening. Br J Radiol 2000;73:1252-9. [Crossref] [PubMed]
4. Hammer MM, Palazzo LL, Kong CY, Hunsaker AR. Cancer Risk in Subsolid Nodules in the National Lung Screening Trial. Radiology 2019;293:441-8. [Crossref] [PubMed]
5. Sherr CJ. Cancer cell cycles. Science 1996;274:1672-7. [Crossref] [PubMed]
6. Hu XX, Yang ZX, Liang HY, Ding Y, Grimm R, Fu CX, Liu H, Yan X, Ji Y, Zeng MS, Rao SX. Whole-tumor MRI histogram analyses of hepatocellular carcinoma: Correlations with Ki-67 labeling index. J Magn Reson Imaging 2017;46:383-92. [Crossref] [PubMed]
7. Xu M, Tang Q, Li M, Liu Y, Li F. An analysis of Ki-67 expression in stage 1 invasive ductal breast carcinoma using apparent diffusion coefficient histograms. Quant Imaging Med Surg 2021;11:1518-31. [Crossref] [PubMed]
8. Li Y, Qian Z, Xu K, Wang K, Fan X, Li S, Liu X, Wang Y, Jiang T. Radiomic features predict Ki-67 expression level and survival in lower grade gliomas. J Neurooncol 2017;135:317-24. [Crossref] [PubMed]
9. Marchevsky AM, Hendifar A, Walts AE. The use of Ki-67 labeling index to grade pulmonary well-differentiated neuroendocrine neoplasms: current best evidence. Mod Pathol 2018;31:1523-31. [Crossref] [PubMed]
10. Wiener RS, Schwartz LM, Woloshin S, Welch HG. Population-based risk for complications after transthoracic needle lung biopsy of a pulmonary nodule: an analysis of discharge records. Ann Intern Med 2011;155:137-44. [Crossref] [PubMed]
11. Liu F, Wang M, Li H. Role of perfusion parameters on DCE-MRI and ADC values on DWMRI for invasive ductal carcinoma at 3.0 Tesla. World J Surg Oncol 2018;16:239. [Crossref] [PubMed]
12. Zhao J, Zhang C, Li D, Niu J. Combining multi-scale feature fusion with multi-attribute grading, a CNN model for benign and malignant classification of pulmonary nodules. J Digit Imaging 2020;33:869-78. [Crossref] [PubMed]
13. Lambin P, Rios-Velazquez E, Leijenaar R, Carvalho S, van Stiphout RG, Granton P, Zegers CM, Gillies R, Boellard R, Dekker A, Aerts HJ. Radiomics: extracting more information from medical images using advanced feature analysis. Eur J Cancer 2012;48:441-6. [Crossref] [PubMed]
14. Bi WL, Hosny A, Schabath MB, Giger ML, Birkbak NJ, Mehrtash A, Allison T, Arnaout O, Abbosh C, Dunn IF, Mak RH, Tamimi RM, Tempany CM, Swanton C, Hoffmann U, Schwartz LH, Gillies RJ, Huang RY, Aerts HJWL. Artificial intelligence in cancer imaging: Clinical challenges and applications. CA Cancer J Clin 2019;69:127-57. [Crossref] [PubMed]
15. Gates EDH, Lin JS, Weinberg JS, Hamilton J, Prabhu SS, Hazle JD, Fuller GN, Baladandayuthapani V, Fuentes D, Schellingerhout D. Guiding the first biopsy in glioma patients using estimated Ki-67 maps derived from MRI: conventional versus advanced imaging. Neuro Oncol 2019;21:527-36. [Crossref] [PubMed]
16. Woo T, Okudela K, Yazawa T, Wada N, Ogawa N, Ishiwa N, Tajiri M, Rino Y, Kitamura H, Masuda M. Prognostic value of KRAS mutations and Ki-67 expression in stage I lung adenocarcinomas. Lung Cancer 2009;65:355-62. [Crossref] [PubMed]
17. Warth A, Cortis J, Soltermann A, Meister M, Budczies J, Stenzinger A, Goeppert B, Thomas M, Herth FJ, Schirmacher P, Schnabel PA, Hoffmann H, Dienemann H, Muley T, Weichert W. Tumour cell proliferation (Ki-67) in non-small cell lung cancer: a critical reappraisal of its prognostic role. Br J Cancer 2014;111:1222-9. [Crossref] [PubMed]
18. Chen C, Zhu WD, Zhang XH, Zhu YH, Huang JA. Value of Ki-67 and computed tomography in the assessment of peripheral lung adenocarcinoma. Br J Biomed Sci 2016;73:32-7. [Crossref] [PubMed]
19. Gu Q, Feng Z, Liang Q, Li M, Deng J, Ma M, Wang W, Liu J, Liu P, Rong P. Machine learning-based radiomics strategy for prediction of cell proliferation in non-small cell lung cancer. Eur J Radiol 2019;118:32-7. [Crossref] [PubMed]
20. Travis WD, Brambilla E, Noguchi M, Nicholson AG, Geisinger K, Yatabe Y, Powell CA, Beer D, Riely G, Garg K, Austin JH, Rusch VW, Hirsch FR, Jett J, Yang PC, Gould MAmerican Thoracic Society. International Association for the Study of Lung Cancer/American Thoracic Society/European Respiratory Society: international multidisciplinary classification of lung adenocarcinoma: executive summary. Proc Am Thorac Soc 2011;8:381-5. [Crossref] [PubMed]
21. Ahn HK, Jung M, Ha SY, Lee JI, Park I, Kim YS, Hong J, Sym SJ, Park J, Shin DB, Lee JH, Cho EK. Clinical significance of Ki-67 and p53 expression in curatively resected non-small cell lung cancer. Tumour Biol 2014;35:5735-40. [Crossref] [PubMed]
22. He LY, Zhang H, Wang ZK, Zhang HZ. Diagnostic and prognostic significance of E-cadherin and Ki-67 expression in non-small cell lung cancer patients. Eur Rev Med Pharmacol Sci 2016;20:3812-7. [PubMed]
23. Gui J, Li H. Penalized Cox regression analysis in the high-dimensional and low-sample size settings, with applications to microarray gene expression data. Bioinformatics 2005;21:3001-8. [Crossref] [PubMed]
24. Kattan M, Vickers AJ. Decision Curve Analysis. Encyclopedia of Medical Decision Making 2012;313:409-10.
25. Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making 2006;26:565-74. [Crossref] [PubMed]
26. McWilliams A, Tammemagi MC, Mayo JR, Roberts H, Liu G, Soghrati K, et al. Probability of cancer in pulmonary nodules detected on first screening CT. N Engl J Med 2013;369:910-9. [Crossref] [PubMed]
27. Liu H, Jiao Z, Han W, Jing B. Identifying the histologic subtypes of non-small cell lung cancer with computed tomography imaging: a comparative study of capsule net, convolutional neural network, and radiomics. Quant Imaging Med Surg 2021;11:2756-65. [Crossref] [PubMed]
28. Gillies RJ, Kinahan PE, Hricak H. Radiomics: Images Are More than Pictures, They Are Data. Radiology 2016;278:563-77. [Crossref] [PubMed]
29. Huang Y, Liu Z, He L, Chen X, Pan D, Ma Z, Liang C, Tian J, Liang C. Radiomics Signature: A Potential Biomarker for the Prediction of Disease-Free Survival in Early-Stage (I or II) Non-Small Cell Lung Cancer. Radiology 2016;281:947-57. [Crossref] [PubMed]
30. Ishibashi N, Maebayashi T, Aizawa T, Sakaguchi M, Nishimaki H, Masuda S. Correlation between the Ki-67 proliferation index and response to radiation therapy in small cell lung cancer. Radiat Oncol 2017;12:16. [Crossref] [PubMed]