深圳大学学报理工版

肝包虫病是一种严重的地域性寄生虫病,其病灶分型主要依靠临床医生对超声图像的主观判断,疾病筛查十分耗时,且容易造成误判.提出一种基于超声图像的肝包虫病病灶智能分型方法,首先从肝脏包虫病超声图像中直接裁剪得到病灶区域图像,利用深度卷积神经网络(convolutional neural network, CNN)提取图像多尺度特征,然后结合视觉注意力模型,通过分类网络的主分支和辅助分支分别学习图像的整体和局部细节特征,最后使用度量学习来表征同类别之间样本的相似特征,实现对9种类型的包虫病病灶进行全自动分类.构建了一个18层CNN网络,通过7 000张图像完成训练,在2 000张图像上测试得到的平均准确率为82%,平均F1分数为82%.实验结果表明,该方法能有效用于肝包虫病超声图像分型.

Hepatic echinococcosis is a serious regional parasitic disease. The lesion classification mainly relies on the subjective judgment of clinicians on the ultrasonic images, which is very time-consuming and prone to miscalculation in disease screening. This study proposes an intelligent classification method for hepatic echinococcosis lesions based on the ultrasound images. Firstly, the lesion images are cropped from the hepatic echinococcosis ultrasound images directly. The deep convolutional neural network(CNN)extracts the multi-scale features. Then the multi-scale features are combined with the visual attention model to learn the global and local detail features of the image through the the main and auxiliary branches of the classification network, respectively. Finally, we use the metric learning to represent the similar features of the samples of the same category, so as to realize the automatic classification of nine types of echinococcosis lesions. An 18-layer CNN network is constructed, which is trained on 7 000 images. The average accuracy on 2 000 test images is 82% and the average F1 score is 82%. The results demonstrate that the proposed method can be effectively applied to the ultrasound image classification of hepatic echinococcosis.

引言
1 方法
2 实验和结果分析
3 讨论
4 结语

图1 不同类型的肝包虫病超声图像病灶示例<br/>Fig.1 Ultrasound images of different types of hepatic echinococcal lesions

图1 不同类型的肝包虫病超声图像病灶示例
Fig.1 Ultrasound images of different types of hepatic echinococcal lesions

图2 CE1、CL和CE3型肝包虫病病灶难样本示例<br/>Fig.2 Difficult samples of CE1, CL and CE3

图2 CE1、CL和CE3型肝包虫病病灶难样本示例
Fig.2 Difficult samples of CE1, CL and CE3

图3 双分支分类网络结构(主分支将特征扩展成一维向量,接2个全连接层,第1个全连接层通过中心损失函数约束特征,第2个全连接层通过交叉熵损失函数进行学习,辅助分支将对部分擦除的特征进行分类)<br/>Fig.3 (Color online)Dual-branch classification network structure (The main branch expands the feature into a one-dimensional vector, which is connected to two fully connected layers. The first fully connected layer constrains the feature through the central loss function, and the second fully connectedlayer learns through the cross entropy loss function, and the auxiliary branch classifies the partially erased features.)

图3 双分支分类网络结构(主分支将特征扩展成一维向量,接2个全连接层,第1个全连接层通过中心损失函数约束特征,第2个全连接层通过交叉熵损失函数进行学习,辅助分支将对部分擦除的特征进行分类)
Fig.3 (Color online)Dual-branch classification network structure (The main branch expands the feature into a one-dimensional vector, which is connected to two fully connected layers. The first fully connected layer constrains the feature through the central loss function, and the second fully connectedlayer learns through the cross entropy loss function, and the auxiliary branch classifies the partially erased features.)

表1 各类别数据量<br/>Table 1 Number of images in each category

表1 各类别数据量
Table 1 Number of images in each category

表2 不同方法对肝包虫病分型的实验结果对比<br/>Table 2 Experimental results on classification of hepatic echinococcosis by different methods

表2 不同方法对肝包虫病分型的实验结果对比
Table 2 Experimental results on classification of hepatic echinococcosis by different methods

表3 各类别分型结果<br/>Table 3 Classification results for each category

表3 各类别分型结果
Table 3 Classification results for each category

图4 按样本量归一化的混淆矩阵<br/>Fig.4 (Color online)Confusion matrix normalized by sample size

图4 按样本量归一化的混淆矩阵
Fig.4 (Color online)Confusion matrix normalized by sample size

图5 病灶与对应热度图<br/>Fig.5 (Color online)Various types of lesions and the corresponding heatmaps

图5 病灶与对应热度图
Fig.5 (Color online)Various types of lesions and the corresponding heatmaps

[1] MCMANUS D P, ZHANG W, LI J, et al. Echinococcosis[J]. The Lancet, 2003, 362(9392): 1295-1304.
[2] LI Tiaoying, CHEN Xingwang, ZHEN Ren, et al. Widespread co-endemicity of human cystic and alveolar echinococcosis on the eastern Tibetan Plateau, northwest Sichuan/southeast Qinghai, China[J]. Acta Tropica, 2010, 113(3): 248-256.
[3] ECKERT J, DEPLAZES P. Biological, epidemiological, and clinical aspects of echinococcosis, a zoonosis of increasing concern[J]. Clinical Microbiology Reviews, 2004, 17(1): 107-135.
[4] BRUNETTI E, KERN P, VUITTON D A. Expert consensus for the diagnosis and treatment of cystic and alveolar echinococcosis in humans[J]. Acta Tropica, 2010, 114(1): 1-16.
[5] WHO Informal Working Group. International classification of ultrasound images in cystic echinococcosis for application in clinical and field epidemiological settings[J]. Acta Tropica, 2003, 85(2): 253-261.
[6] DEPLAZES P, RINALDI L, ROJAS C A A, et al. Global distribution of alveolar and cystic echinococcosis[J]. Advances in Parasitology, 2017, 95: 315- 493.
[7] TORGERSON P R, KELLER K, MAGNOTTA M, et al. The global burden of alveolar echinococcosis[J]. PLOS Neglected Tropical Diseases, 2010, 4(6): e722.
[8] LECUN Y, BENGIO Y, HINTON G. Deep learning[J]. Nature, 2015, 521(7553): 436- 444.
[9] HE Kaiming, GKIOXARI G, DOLLÁR P, et al. Mask R-CNN[C]// Transactions on Pattern Analysis and Machine Intelligence.[S. l.]: IEEE, 2017: 2980-2988.
[10] REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection[J]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 779-788.
[11] KERMANY D S, GOLDBAUM M, CAI Wenjia, et al. Identifying medical diagnoses and treatable diseases by image-based deep learning[J]. Cell, 2018, 172(5): 1122-1131.
[12] ZHANG Qi, SONG Shuang, XIAO Yang, et al. Dual-mode artificially-intelligent diagnosis of breast tumours in shear-wave elastography and b-mode ultrasound using deep polynomial networks[J]. Medical Engineering and Physics, 2019, 64: 1- 6.
[13] ESTEVA A, KUPREL B, NOVOA R A, et al. Derma-tologist-level classification of skin cancer with deep neural networks[J]. Nature, 2017, 542(7639): 115-118.
[14] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[C]// Proceedings of the 25th International Conference on Neural Information Processing Systems. Lake Tahoe, USA: Curran Associates Inc., 2012, 1097-1105.
[15] LIU Shengfeng, WANG Yi, YANG Xin, et al. Deep learning in medical ultrasound analysis: a review[J]. Engineering, 2019, 5(2): 261-275.
[16] LIN T Y, ROYCHOWDHURY A, MAJI S. Bilinear CNN models for fine-grained visual recognition[C]// Proceedings of the IEEE International Conference on Computer Vision. Santiago, Chile: IEEE, 2015: 1449-1457.
[17] YANG Ze, LUO Tiange, WANG Dong, et al. Learning to navigate for fine-grained classification[EB/OL].(2018- 09- 02). https://arxiv.org/abs/1809.00287v1.
[18] LIU Weiyang, WEN Yandong, YU Zhiding, et al. Large-margin softmax loss for convolutional neural networks[C]// Proceedings of the 33rd International Conference on International Conference on Machine Learning. New York, USA: JMLR.org, 2016, 48: 507-516.
[19] WEN Yandong, ZHANG Kaipeng, LI Zhifeng, et al. A discriminative feature learning approach for deep face recognition[C]// Proceedings in the 14th European Conference on Computer Vision. Amsterdam: Springer, 2016: 499-515.
[20] YANG Liu. Distance metric learning: a comprehensive survey[D]. East Lansing, USA: Michigan State Universiy, 2006.
[21] SIMONYAN K, Zisserman A. Very deep convolutional networks for large-scale image recognition[EB/OL].(2014- 09- 04)[2015- 04-10]. https://arxiv.org/abs/1409.1556
[22] ITTI L, KOCH C, NIEBUR E. A model of saliency-based visual attention for rapid scene analysis[C]// IEEE Transactions on Pattern Analysis and Machine Intelligence. Washington D C: IEEE, 1998(11): 1254-1259.
[23] ZHANG Jianming, BARGAL S, LIN Zhe, et al. Top-down neural attention by excitation backprop[J]. International Journal of Computer Vision, 2018, 126(10): 1084-1102.
[24] LI Wanjun, YAO Yuan, NI Dong, et al. Placental maturity evaluation via feature fusion and discriminative learning[C]// IEEE 13th International Symposium on Biomedical Imaging. Prague, Czech Republic: IEEE, 2016: 783-786.

备注

引言

1 方法

2 实验和结果分析

3 讨论

4 结语

期刊信息

备注

引言

1 方 法

2 实验和结果分析

3 讨 论

4 结 语

期刊信息

1 方法

3 讨论

4 结语