Artificial intelligence diagnosis of intrauterine adhesion by 3D ultrasound imaging: a prospective study
Original Article

Artificial intelligence diagnosis of intrauterine adhesion by 3D ultrasound imaging: a prospective study

Xingping Zhao1#, Minqiang Liu2#, Susu Wu1, Baiyun Zhang3, Arvind Burjoo4, Yimin Yang1*, Dabao Xu1*

1Department of Gynecology, Third Xiangya Hospital of Central South University, Changsha, China; 2Software Solution Unit, Hunan KMS Medical Technology, Changsha, China; 3Department of Ultrasound, Hunan Guangxiu Hospital, Changsha, China; 4Department of Obstetrics and Gynaecology, Bruno Cheong Hospital, Central Flacq, Mauritius

Contributions: (I) Conception and design: D Xu, Y Yang; (II) Administrative support: None; (III) Provision of study materials or patients: None; (IV) Collection and assembly of data: X Zhao, B Zhang; (V) Data analysis and interpretation: X Zhao; (VI) Manuscript writing: All authors; (VII) Final approval of manuscript: All authors.

#These authors contributed equally to this work and should be considered as co-first authors.

*These authors contributed equally to this work and should be considered as co-corresponding authors.

Correspondence to: Dabao Xu, MD; Yimin Yang, MD. Department of Gynecology, Third Xiangya Hospital of Central South University, 138 Tongzipo Rd., Changsha 410013, China. Email: 600757@csu.edu.cn; 492503562@qq.com.

Background: There were a very large number of intrauterine adhesion (IUA) patients. As improving the classification of three-dimensional transvaginal ultrasound (3D-TVUS) of IUA or non-IUA images remains a clinical challenge and is needed to avoid inappropriate surgery. Our study aimed to evaluate deep learning as a method to classify 3D-TVUS of IUA or non-IUA images taken with panoramic technology.

Methods: After meeting an inclusion/exclusion criteria, a total of 4,401 patients were selected for this study. This included 2,803 IUA patients and 1,598 non-IUA patients. IUA was confirmed by hysteroscopy, and each patient underwent one 3D-TVUS examination. Four well-known convolutional neural network (CNN) architectures were selected to classify the IUA images: Visual Geometry Group16 (VGG16), InceptionV3, ResNet50, and ResNet101. We used these pretrained CNNs on ImageNet by applying both TensorFlow and PyTorch. All 3D-TVUS images were normalized and mixed together. We split the data set into a training set, validation set, and test set. The performance of our classification model was evaluated according to sensitivity, precision, F1-score, and accuracy, which were determined by equations that used true-positive (TP), false-positive (FP), true-negative (TN), and false-negative (FN) numbers.

Results: The overall performances of VGG16, InceptionV3, ResNet50, and ResNet101 were better in PyTorch as opposed to TensorFlow. Through PyTorch, the best CNN model was InceptionV3 with its performance measured as 94.2% sensitivity, 99.4% precision, 96.8% F1-score, and 97.3% accuracy. The area under the curve (AUC) results of VGG16, InceptionV3, ResNet50, and ResNet101 were 0.959, 0.999, 0.997, and 0.999, respectively. PyTorch also successfully transferred information from the source to the target domain where we were able to use another center’s data as an external test data set. No overfitting that could have adversely affected the classification accuracy occurred. Finally, we successfully established a webpage to diagnose IUA based on the 3D-TVUS images.

Conclusions: Deep learning can assist in the binary classification of 3D-TVUS images to diagnose IUA. This study lays the foundation for future research into the integration of deep learning and blockchain technology.

Keywords: Intrauterine adhesion; three-dimensional transvaginal ultrasound; deep learning; convolutional neural networks; PyTorch


Submitted Sep 14, 2022. Accepted for publication Jan 20, 2023. Published online Mar 22, 2023.

doi: 10.21037/qims-22-965


Introduction

Intrauterine adhesion (IUA) is a condition where damage to the endometrium (uterus lining) causes mutual adhesion between the uterine walls (1). IUA can occur for many reasons including uterine curettage (due to induced or spontaneous abortion) (2). IUA can negatively impact the health and quality of life of women of childbearing age. Indications for hysteroscopic surgery in patients with IUA include fertility issues, such as infertility and recurrent miscarriage, and periodic abdominal pain caused by menstrual outflow obstruction (3). Early diagnosis and treatment are vital in managing and improving the health of those with IUA (4). Many methods exist for diagnosing IUA, with the most common being hysteroscopy (5), ultrasound (6), and hysterosalpingography (HSG) (7). Although the diagnosis of IUA by hysteroscopy is the current gold standard, this is an invasive method and is associated with both a high cost and difficulty in initial diagnosis (8). HSG also has its limitations (9), as it may produce inconclusive or unsuccessful results when assessing the middle and upper uterine cavity in cases of moderate-to-severe lower segment adhesions or when the cervix is completely obliterated. The other drawbacks of HSG include radiation exposure and the use of iodine, to which some women may be allergic.

Ultrasound is one of the most frequently used imaging modalities and is recognized as a powerful and ubiquitous screening and diagnostic tool for physicians and radiologists (10). When compared with hysteroscopy and magnetic resonance imaging (MRI), a three-dimensional transvaginal ultrasound (3D-TVUS) has unique advantages when it comes to IUA diagnosis. This is because it is easy to operate, noninvasive, and relatively inexpensive. Furthermore, 3D-TVUS not only produces accurate images of the conditions within the uterine cavity (including the angle of the uterus and the opening of the fallopian tube) but also gives surgeons a 3D image of the entire uterine cavity. These results are theoretically beneficial for later diagnostic and operative hysteroscopy (9). Ultrasonic images have special disadvantages, such as low image quality and high dependence on the doctor's experience in diagnostic interpretation (11). Therefore, we need to develop automatic ultrasonic image analysis methods to ensure that ultrasonic diagnosis and evaluation are objective and accurate (12).

Deep learning is a branch of machine learning that can directly process and automatically learn intermediate and high-level abstract features obtained from raw data (13). Deep learning has been recently used with medical 3D-TVUS to analyze various facets of anatomy, including those of the breast, prostate, liver, heart, cardiovascular system, brain, carotid artery, thyroid, intravascular structure, fetus, lymph node, kidney, spine, bone, muscle, nerve structure, and tongue (14). Despite this, few studies have been published that apply deep learning to 3D-TVUS images.

Our study aimed to test the deep learning approach of using state-of-the-art convolutional neural network (CNN) architectures for the binary classification of 3D-TVUS images. We present the following article in accordance with the STROBE reporting checklist (available at https://qims.amegroups.com/article/view/10.21037/qims-22-965/rc).


Methods

Data set

General information

This was a prospective and nonrandomized controlled study which included a total of 4,401 patients [IUA group, n=2,803 (1,615 from the Third Xiangya Hospital and 1,188 from Hunan Guangxiu Hospital); non-IUA group, n=1,598 (1,231 from the Third Xiangya Hospital and 367 from Hunan Guangxiu Hospital)] . This included the records of 1,615 patients between 20 and 40 years of age who were newly diagnosed with IUA by 3D-TVUS examination and hysteroscopy at the Third Xiangya Hospital of Central South University between January 2018 and June 2019. At the same hospital throughout the same period, we recruited another 1,231 women aged between 20 and 40 who were found to be tubal infertility or myoma of uterus but not have any IUA after 3D-TVUS and hysteroscopy examination. For our external data set, we also collected the 3D-TVUS images of another 1,188 IUA patients and 367 non-IUA patients from Hunan Guangxiu Hospital. The images of this external group were taken between October 2019 and October 2020.

The inclusion criteria of IUA were as follows (15): (I) patients with IUA (confirmed by hysteroscopy); (II) patients who had undergone 3D-TVUS evaluation before hysteroscopic adhesiolysis (HA); (III) patients aged 20–40 years with a strong desire to conceive; and (IV) patients with normal hormone levels and ovulation. The inclusion criteria of no-IUA were as follows: (I) patients without IUA; (II) patients who had undergone 3D-TVUS with tubal infertility or myoma; (III) patients aged 20–40 years.

The exclusion criteria of IUA and no-IUA were as follows: (I) patients with cervical or endometrial malignant lesions; (II) patients with serious heart and liver conditions or renal insufficiency; (III) patients with serious nervous system diseases, who were unable to care for themselves in daily life or unable to undergo relevant treatment; (IV) patients with surgical intolerance or an inability to follow the doctor’s advice to review or follow-up; (V) those with congenital malformation of the uterus.

All 3D-TVUS images and hysteroscopy videos of participants were reconfirmed by 2 or more senior sonographers and doctors specializing in gynecology. Some of the sample images obtained are shown in Figure 1 and exemplify how 3D-TVUS images can exhibit the following morphological characteristics of IUA clearly: irregular margins, uneven thickness, a defective interrupted endometrial line, obliteration in the form of an undetectable endometrium suggesting extensive adhesion, fibrosis in the form of a hyperechoic lesion without posterior shadowing, and poor intimal activity.

Figure 1 Samples of the adhesion and nonadhesion ultrasound images used. Adhesion sample 1: the lower segment of the uterine cavity was in the shape of a barrel, there were lots of endometrial deletions in the middle and upper segment of the uterine cavity, and the endometrium of the bilateral wall and fundus wall were uneven. Adhesion sample 2: the lower segment of the uterine cavity was narrow, the endometrium of the bilateral wall and fundus wall were uneven, and the bilateral fallopian tube ostia were invisible. Adhesion sample 3: the internal opening of the cervix was in the form of a pinhole, the middle and lower segment of the uterine cavity was in the form of a narrow barrel, the middle segment was significantly narrowed, and the endometrium of the bilateral wall had disappeared. Samples 4, 5 and 6: all these three images show a normal form of non-adhesive uterine cavity.

3D-TVUS examination

The 3D ultrasound examination was conducted using the GE Voluson E8 ultrasound system (GE Healthcare, Chicago, IL, USA) with a two-dimensional (2D) volume probe and a real-time 3D volume probe. Using a 7.5 MHz IC5-9D vaginal probe, the 4,401 female participants conducted the 3D-TVUS examination during the secretory phase of the menstrual cycle. Participants were asked to empty their bladders before to the examination, and they were then positioned in the lithotomy posture. A standard 2D ultrasonographic examination was completed first. In order to find disruptions of the endometrial-myometrial junction, the integrity of the endometrial layer was evaluated during the 2D ultrasound. After the real-time 3D volume probe was rotated, the target area was chosen and the overall image information was obtained using panoramic technology. After the inspection, the data was saved on a portable hard drive for later analysis.

Image preprocessing

Since the original ultrasound images contained large amounts of imprecise and incomplete information, image preprocessing was essential to ensure data consistency and accuracy. We applied data processing and data augmentation to each image (e.g., through cropping, resizing, and normalization). Data augmentation is a common technique for improving image results and prevent overfitting. We used flip, grayscale, saturate, and rotate data augmentation techniques on our data sets. The original image showed a large echo region. In order to eliminate the interference of irrelevant regions and appropriately reduce the calculation amount of the model, we adopted manual clipping of the diseased region as the region of interest (ROI) and set the size to a uniform 192×192 pixels in the JPG format according to the input size of the selected models.

We classified the 3D-TVUS images using state-of-the-art convolutional neural network (CNN) models with transfer learning and fine-tuning. Each image was processed with high-error retention [a high-pass filtering algorithm in Open Source Computer Vision Library (OpenCV)] and sharpened. The original image details retained after sharpening were adjusted using the threshold value of the high-pass filtering algorithm. The exposure and color of the picture were adjusted, and a clear image of the uterine cavity was obtained. The key features of the IUA are shown in Figure 2. To identify IUA, many 3D-TVUS images of the uterine cavity contour were first extracted and sorted. The information was then integrated by experts who repeatedly performed physical area identification, relationship extraction, event extraction, and other operations on the adhesions in the images. Finally, the key features of IUA were obtained by extracting a feature library through self-learning techniques.

Figure 2 Extraction of key features of the IUA. To identify IUA, numerous 3D-TVUS images of the uterine cavity contour were first extracted and sorted in the following fashion: (A) acquisition of the 3D-TVS image: coronal view of the uterine cavity, (B) identification of the outline of uterine cavity, (C) image of the endometrium, (D) identification of the adhesion contour, and (E) annotation of the original 3D-TVS image. Finally, the key features of IUA were obtained by extracting the feature library with self-learning techniques. IUA, intrauterine adhesion; 3D-TVS, three-dimensional transvaginal ultrasound.

Transfer learning strategies

The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). Approval was given to the study by the Institutional Review Board of the Third Xiangya Hospital and Xiangya Hospital, Central South University (No. 2019-S455). Informed consent was received after the procedure was fully explained to all participants and their legal guardians.

Transfer learning strategies have been widely used to exploit information learned from multiple domains. This cross-domain approach to learning can be contrasted to the approach of training a model from scratch with randomly initialized weights. Ultimately, transfer learning is the technique by which knowledge gained from an already trained model is used to learn about another data set. Figure 3 shows the transfer learning framework.

Figure 3 Transfer learning framework.

The batch size was set to 16, the learning rate was set to 0.0001, and all models were trained for 50 epochs. For fine-tuning, the network weights were initialized from weights that were trained on ImageNet. The training and testing processes of the proposed architectures were implemented in Python using the TensorFlow and PyTorch packages which were run on a Nvidia GeForce GTX 1660 Ti GPU graphics card.

Evaluation criteria

A computer-aided design and drafting (CADD) system was used to indicate how close the system came to correctly diagnosing a patient with IUA. In this study, the patients with IUA were considered the true class, while the patients without IUA were the false class. In this way, the easiest way to summarize the CADD system performance was with evaluation metrics. Using true-positive (TP), false-positive (FP), true-negative (TN), and false-negative (FN) numbers,the performance of the proposed classification models was evaluated based on sensitivity = TP/(TP + FN), precision = TP/(TP + FP), F1-score = 2× (precision × sensitivity)/(precision + sensitivity), and accuracy (16). TP indicates the number of correctly labeled positive cases, TN the number of correctly labeled negative cases, FP the number of falsely labeled positive cases, and FN the number of falsely labeled negative cases.


Results

The demographic and clinical characteristics of the study groups were showed in Table 1. In this study, our task was to classify IUA into 2 types. We chose 4 well-known CNN architectures, Visual Geometry Group16 (VGG16), InceptionV3, ResNet50, and ResNet101, to classify the IUA images obtained, and we used these pretrained CNNs on ImageNet by applying TensorFlow (17) and PyTorch (18). This process first involved collecting and processing the ultrasound images. We then modified the last layers of the CNN network to adapt it to our task domain. We fine-tuned of the modified CNN network by training it with our data sets. Figure 4 shows the overall framework of the methods used for this study, while Figure 5 shows each model’s architecture. There were 3 types of training used in our study. The patients of type 1 and type 2 were from the Third Xiangya Hospital. The differences between type 1 and type 2 were the date set of train, validation and test, the date sets were randomly selected. Type 3 was the external data set, the patients are from another hospital (Hunan Guangxiu Hospital).

Table 1

Demographic and clinical characteristics of the study groups

Basic clinical characteristics IUA group (n=367) Non-IUA group P value
Age (years)
   Mean ± SD 32.42±4.77 32.59±4.96 0. 516
   Median [min, max] 32 [23, 43] 32 [25, 46]
Gravidity (times)
   Mean ± SD 3.40±1.79 3.41±2.12 0.617
   Median [min, max] 3 [0, 10] 3 [0, 12]
Parity (times)
   Mean ± SD 0.67±1.06 0.66±0.80 0.742
   Median [min, max] 1 [0, 9] 1 [0, 5]
Abortion (times)
   Mean ± SD 2.53±1.62 2.51±1.92 0.453
   Median [min, max] 2 [0, 9] 2 [0, 11]
IUD, n (%) 0 0 NE
Other gynecological disease, n (%) 0 0 NE
Endocrine disorders, n (%) 0 0 NE

IUA, intrauterine adhesion; SD, standard deviation; NE, not estimable (due to nullity of category in both groups); IUD, intrauterine device.

Figure 4 Overall framework of the methods used for this study.
Figure 5 Architectures of the CNN models: (A) architectures of VGG16; (B) architectures of InceptionV3; (C) architectures of ResNet50; and (D) architectures of ResNet101. VGG16, Visual Geometry Group16; CNN, convolutional neural network.

The first type involved 2,422 3D-TVUS images being randomly selected to build prediction models using TensorFlow. We split this data set using 1,695 (IUA group, n=856; non-IUA group, n=839) images for the training set, 363 (IUA group, n=185; non-IUA group, n=178) for the validation set, and 364 (IUA group, n=170; non-IUA group, n=194) for the test set. The area under the curve (AUC) results of VGG16, InceptionV3, ResNet50, and ResNet101 were 0.950, 0.950, 0.875, and 0.906, respectively (Figure 6). The accuracy, F1-score, precision, sensitivity, specificity, kappa, Pearson’s correlation, and P value of these 4 models are shown in Table 2.

Figure 6 The AUC of VGG16, InceptionV3, ResNet50, and ResNet101 determined by using TensorFlow. VGG16, Visual Geometry Group16; AUC, area under curve.

Table 2

The accuracy, F1-score, precision, sensitivity, specificity, kappa, Pearson’s correlation, and P value of VGG16, InceptionV3, ResNet50, and ResNet101 by using TensorFlow of the first involved 2,422 3D-TVUS images

Models (tensorflow) Accuracy F1 score Precision Sensitivity Specificity Kappa Pearson’s correlation P value
VGG16 0.898 0.910 0.900 0.920 0.875 0.793 0.793 <0.001
InceptionV3 0.890 0.897 0.953 0.847 0.946 0.781 0.787425985 <0.0018
ResNet50 0.859 0.873 0.886 0.861 0.857 0.715 0.715772945 <0.001
ResNet101 0.860 0.878 0.855 0.903 0.804 0.712 0.713462 <0.001

VGG16, Visual Geometry Group16; 3D-TVUS, three-dimensional transvaginal ultrasound.

The second type of training involved 2,846 3D-TVUS images being randomly selected to build prediction models using PyTorch. We split this data set using 1,997 (IUA group, n=1,130; non-IUA group, n=847) images for the training set and 849 (IUA group, n=485; non-IUA group, n=364) for the validation set. The AUC results of VGG16, InceptionV3, ResNet50, and ResNet101 were 0.959, 0.999, 0.997, and 0.999, respectively (Figure 7), which were better than those obtained by TensorFlow. Training and validation accuracy of VGG16, InceptionV3, ResNet50, and ResNet101 is shown in Figure 8. The best CNN model was InceptionV3 with 97.3% accuracy, 96.8% F1-score, 99.4% precision, and 94.2% sensitivity (details shown in Table 3).

Figure 7 The AUC of VGG16, InceptionV3, ResNet50, and ResNet101 with PyTorch. VGG16, Visual Geometry Group16; AUC, area under curve.
Figure 8 Training and validation accuracy of the CNN models with PyTorch. VGG16, Visual Geometry Group16; CNN, convolutional neural network.

Table 3

The accuracy, F1-score, precision, sensitivity, specificity, kappa, Pearson’s correlation, and P value of VGG16, InceptionV3, ResNet50, and ResNet101 with PyTorch of the second type of training involved 2,846 3D-TVUS images

Models (tensorflow) Accuracy F1 score Precision Sensitivity Specificity Kappa Pearson’s correlation P value
VGG16 0.879 0.848 0.914 0.791 0.944 0.748 0.754 <0.001
InceptionV3 0.973 0.968 0.994 0.942 0.996 0.944 0.945 <0.001
ResNet50 0.945 0.931 0.994 0.876 0.996 0.885 0.890 <0.001
ResNet101 0.963 0.956 1.000 0.915 1.000 0.925 0.927 <0.001

VGG16, Visual Geometry Group16; 3D-TVUS, three-dimensional transvaginal ultrasound.

Our third and final training type used 1,188 IUA and 367 non-IUA 3D-TVUS images collected from Hunan Guangxiu Hospital as an external test data set. With PyTorch, the 4 models still had good performances, with the AUC results of VGG16, InceptionV3, ResNet50, and ResNet101 being 0.937, 0.982, 0.966, and 0.973, respectively (Figure 9). Each model’s accuracy, F1-score, precision, sensitivity, specificity, kappa, Pearson’s correlation, and P value can be seen in Table 4. We also invited different levels of sonographers to discriminate mixed three-dimensional ultrasound images, and the accuracy of different levels of sonographers to judge uterine adhesions is shown in Table 5.

Figure 9 The AUC of the external test data sets of VGG16, InceptionV3, ResNet50, and ResNet101 with PyTorch. VGG16, Visual Geometry Group16; AUC, area under curve.

Table 4

The accuracy, F1-score, precision, sensitivity, specificity, kappa, Pearson’s correlation and P value of VGG16, InceptionV3, ResNet50, and ResNet101 with PyTorch of third and final training type used 1,188 IUA and 367 non-IUA 3D-TVUS images

Models (tensorflow) Accuracy F1 score Precision Sensitivity Specificity Kappa Pearson’s correlation P value
VGG16 0.882 0.759 0.735 0.785 0.912 0.681 0.682 <0.001
InceptionV3 0.947 0.894 0.850 0.943 0.949 0.859 0.861 <0.001
ResNet50 0.898 0.811 0.719 0.929 0.888 0.742 0.754 <0.001
ResNet101 0.911 0.832 0.747 0.940 0.902 0.772 0.782 <0.001

VGG16, Visual Geometry Group16; IUA, intrauterine adhesion; 3D-TVUS, three-dimensional transvaginal ultrasound.

Table 5

Summary of performance metrics for sonographers specializing in gynecology

The doctor title Accuracy F1 score Precision Sensitivity
Junior sonographers 0.807 0.805 0.810 0.801
Intermediate sonographers 0.850 0.852 0.843 0.861
Senior sonographers 0.917 0.916 0.930 0.902

Discussion

3D-TVUS imaging has many advantages: it is convenient, noninvasive, radiation-free, cheap, and has the option of multisectional views (19). At present, concomitant intraoperative transabdominal ultrasound (TAS) monitoring during HA is widely used in IUA treatment (9). However, some studies have shown that this is not the optimal period to evaluate the condition of the endometrium and uterus. The advantages that come with preoperative 3D-TVUS can help surgeons assess and interpret ultrasonographic data and apply this knowledge intraoperatively (20). 3D-TVUS provides more detailed and complete information on the uterine cavity than does TAS. Other research has also demonstrated that 3D-TVUS can delineate the anatomical relationship of the whole endometrium and myometrium (the endometrial-myometrial junction) and can detect the coronal section, which cannot be explored by 2D ultrasound (21). Furthermore, as 3D-TVUS incorporates various angles, it can clarify the extent, degree, and localization of lesions, along with the condition of the endometrium. Thus, adhesion identification by this method effectively reduces the frequency of misdiagnosed IUA cases (22). Additionally, there is no time limit on the length of operation for preoperative 3D-TVUS. By virtue of its proximity to the pelvic organs, the high-frequency transvaginal probe can also improve image resolution, thus providing 3D-TVUS with more informative and comprehensive imaging (23).

The best time to perform a 3D-TVUS is during the secretory phase of the menstrual cycle when the endometrium and the muscular layer strongly contrast, which enables a better display of the uterine cavity morphology. Some studies suggest (24) that a 3D-TVUS cross-sectional view showing endometrial thickness asymmetry or a mass-like echo in the cavity can indicate the possibility of IUA (19,25). Still, preoperative 3D-TVUS has its limitations, including a dependence on the experience of the operator or diagnostician and the high interobserver and intraobserver variability arising from different institutional practices and ultrasound systems (26).

Current artificial intelligence (AI)-powered models used with ultrasound diagnosis technology are only able to exploit 2D cross-sectional images. The information contained in these 2D cross-sections is limited and does not capture the full images of the lesions. However, AI models trained with 3D ultrasound data, ultrasound cine clips that capture lesions from multiple views, or spatiotemporal data could potentially improve the accuracy of diagnosing IUA and be considered completed models (27).

CNNs approach can improve diagnostic accuracy, reduces computational time, and significantly assists pathologists in diagnosing (28). As the provided data sets of 3D-TVUS of IUA are not sufficiently large for training a CNN model, transfer learning method for CNN model was be used, which can accelerate the convergence of the network, reduce computational power, and optimize network performance (29).

Our experiment used 4 well-known CNN architectures, ResNet50, ResNet101, InceptionV3, and VGG16, to develop an advanced automatic 3D-TVUS imaging analysis method which was objective, accurate, and rational. We also found that better prediction models were built with PyTorch than were built with TensorFlow. The best CNN model was InceptionV3 with 97.3% accuracy, 96.8% F1-score, 99.4% precision, and 94.2% sensitivity. The AUC results of VGG16, InceptionV3, ResNet50, and ResNet101 were 0.959, 0.999, 0.997, and 0.999, respectively. PyTorch also successfully transferred information from the source to the target domain where we were able to use another center’s data set as an external test data set. Moreover, as this method obviated the need for handcrafted features, it required minimal preprocessing and can therefore be widely applied to an array of detection tasks.

We built on past research using our unique data set of specific abnormal cases so that algorithms were able to discriminate between IUA and non-IUA cases. This is an improvement upon those systems that simply use a normal vs. abnormal image classification method. According to the excellent performance of the CNN models with PyTorch, we were able to successfully establish a webpage to diagnose IUA based on 3D-TVUS images. We believe that the wider use of AI-powered ultrasound will assist in future large-scale prevalence studies, which in turn will increase the medical knowledge concerning risk factors related to the causes and development of IUA. Improved early detection and effective management of IUA will significantly improve treatment efficacy and pregnancy prognosis for these patients. The use of deep learning with 3D-TVUS to provide an objective, accurate, and rational method of diagnosing IUA lays the groundwork for more in-depth studies of the surgical treatment and medical management of this condition.

Finally, it is essential to note that whether AI-assisted or not, ultrasounds will occasionally produce an FP result. Therefore, this method cannot be used independently of experts in the field, and training is still required to interpret these images. Indeed, the methods that employ deep learning are intended to assist clinicians rather than replace them.


Conclusions

Deep learning approach can assist in IUA diagnosis via the binary classification of 3D-TVUS images. It can achieve similar diagnostic accuracy as that of expert clinicians. This research provides a platform for future studies which seek to further integrate deep learning and blockchain technology.


Acknowledgments

Funding: This study was supported by the Hunan Provincial Clinical Medical Technology Innovation Guiding Project (Nos. 2021SK53704 and 2020SK53606), the Natural Science Foundation of Hunan Province (No. 2021JJ40956) and National Key Research and Development Program of China (No. 2018YFC1004800).


Footnote

Reporting Checklist: The authors have completed the STARD reporting checklist. Available at https://qims.amegroups.com/article/view/10.21037/qims-22-965/rc

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://qims.amegroups.com/article/view/10.21037/qims-22-965/coif). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). Approval was given to the study by the Institutional Review Board of the Third Xiangya Hospital and Xiangya Hospital, Central South University (IRB No. 2019-S455). Informed consent was received after the procedure was fully explained to all participants and their legal guardians.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.


References

  1. Shi X, Saravelos SH, Zhou Q, Huang X, Xia E, Li TC. Prevention of postoperative adhesion reformation by intermittent intrauterine balloon therapy: a randomised controlled trial. BJOG 2019;126:1259-66. [Crossref] [PubMed]
  2. Zhao X, Liu Y, Zhang A, Gao B, Feng Q, Huang H, Zhu X, Sun X, Xu D. Logistic regression analyses of factors affecting fertility of intrauterine adhesions patients. Ann Transl Med 2020;8:49. [Crossref] [PubMed]
  3. Kou L, Jiang X, Xiao S, Zhao YZ, Yao Q, Chen R. Therapeutic options and drug delivery strategies for the prevention of intrauterine adhesions. J Control Release 2020;318:25-37. [Crossref] [PubMed]
  4. Deans R, Vancaillie T, Ledger W, Liu J, Abbott JA. Live birth rate and obstetric complications following the hysteroscopic management of intrauterine adhesions including Asherman syndrome. Hum Reprod 2018;33:1847-53. [Crossref] [PubMed]
  5. Shen M, Duan H, Chang Y, Lin Q, Chen C, Wang S. Impact of concomitant intrauterine adhesions on pregnancy outcomes and obstetric complications in women with a septate uterus. Int J Gynaecol Obstet 2022;159:875-81. [Crossref] [PubMed]
  6. Devine K, Dolitsky S, Ludwin I, Ludwin A. Modern assessment of the uterine cavity and fallopian tubes in the era of high-efficacy assisted reproductive technology. Fertil Steril 2022;118:19-28. [Crossref] [PubMed]
  7. Dong R, Ma S, Zhao X, Wang B, Roy M, Yao L, Xia T, Liu Y. Recent progress of Bioinspired Hydrogel-based delivery system for endometrial repair. Front Bioeng Biotechnol 2022;10:1013217. [Crossref] [PubMed]
  8. Nieuwenhuis LL, Hermans FJ, Bij de Vaate AJM, Leeflang MM, Brölmann HA, Hehenkamp WJ, Mol BWJ, Clark TJ, Huirne JA. Three-dimensional saline infusion sonography compared to two-dimensional saline infusion sonography for the diagnosis of focal intracavitary lesions. Cochrane Database Syst Rev 2017;5:CD011126. [Crossref] [PubMed]
  9. Burjoo A, Zhao X, Zou L, Liu X, Lei L, Zhang B, Xu D. The role of preoperative 3D-ultrasound in intraoperative judgement for hysteroscopic adhesiolysis. Ann Transl Med 2020;8:55. [Crossref] [PubMed]
  10. Mu W, He M, Lei T, Zhang L, Du L, Xie H. Measurement of the Cobb angle by 3D ultrasound: a valuable additional method for the prenatal evaluation of congenital scoliosis. Quant Imaging Med Surg 2022;12:2805-12. [Crossref] [PubMed]
  11. Fuchs F, Burlat J, Grosjean F, Rayssiguier R, Captier G, Faure JM, Dumont C. A score-based method for quality control of fetal hard palate assessment during routine second-trimester ultrasound examination. Acta Obstet Gynecol Scand 2018;97:1300-8. [Crossref] [PubMed]
  12. Yap MH, Pons G, Marti J, Ganau S, Sentis M, Zwiggelaar R, Davison AK, Marti R. Moi Hoon Yap, Pons G, Marti J, Ganau S, Sentis M, Zwiggelaar R, Davison AK, Marti R. Automated Breast Ultrasound Lesions Detection Using Convolutional Neural Networks. IEEE J Biomed Health Inform 2018;22:1218-26. [Crossref] [PubMed]
  13. Esteva A, Robicquet A, Ramsundar B, Kuleshov V, DePristo M, Chou K, Cui C, Corrado G, Thrun S, Dean J. A guide to deep learning in healthcare. Nat Med 2019;25:24-9. [Crossref] [PubMed]
  14. Zhu M, Pi Y, Jiang Z, Wu Y, Bu H, Bao J, Chen Y, Zhao L, Peng Y. Application of deep learning to identify ductal carcinoma in situ and microinvasion of the breast using ultrasound imaging. Quant Imaging Med Surg 2022;12:4633-46. [Crossref] [PubMed]
  15. Zhao X, Yang Y, Liao D, Traoré A, He S, Xu D. Correlative study of preoperative three-dimensional transvaginal ultrasound findings and ongoing pregnancy/live birth in patients with intrauterine adhesions following hysteroscopic adhesiolysis: a retrospective study. Quant Imaging Med Surg 2022;12:2441-53. [Crossref] [PubMed]
  16. Haenssle HA, Fink C, Schneiderbauer R, Toberer F, Buhl T, Blum A, et al. Man against machine: diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists. Ann Oncol 2018;29:1836-42. [Crossref] [PubMed]
  17. Bai W, Sinclair M, Tarroni G, Oktay O, Rajchl M, Vaillant G, et al. Automated cardiovascular magnetic resonance image analysis with fully convolutional networks. J Cardiovasc Magn Reson 2018;20:65. [Crossref] [PubMed]
  18. Zimmer L, Lindauer M, Hutter F. Auto-Pytorch: Multi-Fidelity MetaLearning for Efficient and Robust AutoDL. IEEE Trans Pattern Anal Mach Intell 2021;43:3079-90. [Crossref] [PubMed]
  19. Zhang T, He Y, Wang Y, Zhu Q, Yang J, Zhao X, Sun Y. The role of three-dimensional power Doppler ultrasound parameters measured on hCG day in the prediction of pregnancy during in vitro fertilization treatment. Eur J Obstet Gynecol Reprod Biol 2016;203:66-71. [Crossref] [PubMed]
  20. Amin T, Saridogan E, Dooley M, Jurkovic D. Morphological appearance of uterine cavity on ultrasound prior to development of intrauterine adhesions. Ultrasound Obstet Gynecol 2018;51:142-3. [Crossref] [PubMed]
  21. Barel O, Krakov A, Pansky M, Vaknin Z, Halperin R, Smorgick N. Intrauterine adhesions after hysteroscopic treatment for retained products of conception: what are the risk factors? Fertil Steril 2015;103:775-9. [Crossref] [PubMed]
  22. Mohamed Amer MI, Omar OH, El Sherbiny Hamed M, Dahroug EG. Subendometrial blood flow changes by 3-dimensional power Doppler ultrasound after hysteroscopic lysis of severe intrauterine adhesions: preliminary study. J Minim Invasive Gynecol 2015;22:495-500. [Crossref] [PubMed]
  23. Freytag D, Günther V, Maass N, Alkatout I. Uterine Fibroids and Infertility. Diagnostics (Basel) 2021;11:1455. [Crossref] [PubMed]
  24. Casadio P, Magnarelli G, La Rosa M, Alletto A, Arena A, Fontana E, Morra C, Talamo MR, Fabbri M, Giovannico K, Virgilio A, Raimondo D, Guasina F, Paradisi R, Seracchioli R. Uterine Fundus Remodeling after Hysteroscopic Metroplasty: A Prospective Pilot Study. J Clin Med 2021;10:260. [Crossref] [PubMed]
  25. Prost E, Reignier A, Leperlier F, Caillet P, Barrière P, Fréour T, Lefebvre T. Female obesity does not impact live birth rate after frozen-thawed blastocyst transfer. Hum Reprod 2020;35:859-65. [Crossref] [PubMed]
  26. Dreisler E, Kjer JJ. Asherman's syndrome: current perspectives on diagnosis and management. Int J Womens Health 2019;11:191-8. [Crossref] [PubMed]
  27. Akkus Z, Cai J, Boonrod A, Zeinoddini A, Weston AD, Philbrick KA, Erickson BJ. A Survey of Deep-Learning Applications in Ultrasound: Artificial Intelligence-Powered Ultrasound for Improving Clinical Workflow. J Am Coll Radiol 2019;16:1318-28. [Crossref] [PubMed]
  28. Sarıgül M, Ozyildirim BM, Avci M. Differential convolutional neural network. Neural Netw 2019;116:279-87. [Crossref] [PubMed]
  29. Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, van der Laak JAWM, van Ginneken B, Sánchez CI. A survey on deep learning in medical image analysis. Med Image Anal 2017;42:60-88. [Crossref] [PubMed]
Cite this article as: Zhao X, Liu M, Wu S, Zhang B, Burjoo A, Yang Y, Xu D. Artificial intelligence diagnosis of intrauterine adhesion by 3D ultrasound imaging: a prospective study. Quant Imaging Med Surg 2023;13(4):2314-2327. doi: 10.21037/qims-22-965

Download Citation