基于深度学习的微出血自动检测及解剖尺度定位|文献速递-视觉大模型医疗图像应用

Title

题目

Toward automated detection of microbleeds with anatomical scale  localization using deep learning

基于深度学习的微出血自动检测及解剖尺度定位

01

文献速递介绍

基于深度学习的脑微出血（CMBs）检测与解剖定位

脑微出血（Cerebral Microbleeds, CMBs）是由血管壁损伤引起的小血液产物的慢性沉积，通常位于动脉和毛细血管附近（Roob et al., 1999; Tajudin et al., 2016）。CMBs常见于老年人及脑血管疾病患者（Werring et al., 2005），尤其是在阿尔茨海默病（AD）、痴呆、缺血性及出血性卒中患者中更为普遍。近年来，研究表明CMBs与认知衰退、脑出血、脑梗死以及短暂性脑缺血发作复发相关（Akoudad et al., 2016; Park et al., 2017）。此外，CMBs被认为是高血压或脑淀粉样血管病（CAA）引起的小血管病理损伤的有价值生物标志物（Gregoire et al., 2009; Vernooij et al., 2008）。

CMBs的病理学意义不仅取决于其存在，还与其解剖学定位（如脑叶区、深部区域、小脑幕下区域）密切相关。例如，脑叶区的CMBs与CAA相关，并与阿尔茨海默病相关（Martinez-Ramirez et al., 2015）；深部区域的CMBs与腔隙性脑梗、高血压和舒张压波动相关；小脑幕下区域的CMBs则与收缩压波动和无先兆偏头痛相关（Arkink et al., 2015; Gao et al., 2018; Liu et al., 2012; Renard, 2018）。在认知衰退方面，脑叶区CMBs影响一般认知功能、执行功能、记忆力和处理速度，而深部及小脑幕下CMBs则与心理运动速度和注意力相关（Akoudad et al., 2016）。磁共振成像（Magnetic Resonance Imaging, MRI）是检测CMBs最广泛使用的方式。通过从梯度回波（Gradient-Recalled Echo, GRE）MRI脉冲序列生成的磁敏感加权图像（Susceptibility-Weighted Images, SWI），可以检测直径小至200微米的微小血管出血（Tanaka et al., 1999）。然而，CMBs的检测面临一些挑战。这些圆形或椭圆形病灶尺寸在2至10毫米之间，与整个脑组织相比非常稀疏且微小（Greenberg et al., 2009; Wardlaw et al., 2013）。另一个挑战是SWI图像中存在众多与CMBs具有相似低信号特征的伪影（如钙化和软脑膜血管）。研究表明，钙化可以通过相位图像与CMBs区分开来，因为其在相位图像中的信号强度相反（Yamada et al., 1996）。然而，软脑膜血管在SWI和相位图像中表现为相同的信号强度，因此需要通过多方向（如冠状面和矢状面）的连续切片检查加以区分（Greenberg et al., 2009）。基于这些原因，CMBs的手动检测和解剖学定位过程费时费力且主观性强，而自动检测工具可以作为辅助手段，提升检测效率。已有文献中，深度学习卷积神经网络（CNN）用于CMBs检测主要分为单阶段检测器和双阶段检测器两类。单阶段检测器通过单一深度学习模型直接检测CMBs，其分类模型的性能优于检测模型。然而，由于分类模型只能对输入图像中心目标进行分类，需要多次推断并逐步移动裁剪区域中心，这导致计算成本高且执行时间长。双阶段检测器由两个顺序模型组成：检测模型和分类模型。第一阶段用于筛选潜在候选区域，而第二阶段负责区分真实CMBs与伪影（即减少假阳性）。与单阶段检测器相比，双阶段检测器计算成本较低，因为仅需对第一阶段检测出的候选区域进行分类。此外，在假阳性率方面，双阶段检测器优于单阶段检测器。然而，双阶段检测器无法以端到端方式训练，导致第二阶段的性能受限于第一阶段，尤其是在处理漏检病例时存在不足。本文提出了一种单阶段3D深度学习检测模型，用于自动检测CMBs。我们采用SWI和相位图像作为3D输入，以高效捕获三维信息。本文的主要贡献如下：通过结合3D U-Net和Faster R-CNN的区域提议网络（Region Proposal Network, RPN），构建了一个端到端的3D CMBs检测器（Çiçek et al., 2016; Ren et al., 2015）。在模型中引入了特征融合模块（Feature Fusion Module, FFM），以高效学习上下文信息（Cao et al., 2019, 2018）。

提出了一个无需分类模型的单阶段检测器，通过难样本原型学习（Hard Sample Prototype Learning, HSPL）和卷积原型学习（Convolutional Prototype Learning, CPL）添加了新的损失项“集中损失”，性能优于双阶段检测器（Yang et al., 2018）。此外，我们将最初在MICCAI2022中提出的单阶段检测器扩展为一个框架，不仅能够检测CMBs，还能通过分割脑结构实现解剖学定位（Kim et al., 2022）。本文的新贡献包括：首次开发了解剖学定位任务，可识别CMBs所在的解剖区域（如脑叶、深部和小脑幕下区域），并进一步减少检测中的假阳性。在我们的数据集上，将检测器性能与当前最先进方法进行了比较。本文其余内容结构如下：第二节提供详细的文献综述。第三节详细介绍了所提出的深度学习框架，包括单阶段检测器和解剖学定位工具。第四节展示并讨论了框架的实验结果。最后，第五节总结了本文的结论。

Aastract

摘要

Cerebral Microbleeds (CMBs) are chronic deposits of small blood products in the brain tissues, which have explicit relation to various cerebrovascular diseases depending on their anatomical location, including cognitive decline, intracerebral hemorrhage, and cerebral infarction. However, manual detection of CMBs is a time consuming and error-prone process because of their sparse and tiny structural properties. The detection of CMBs is commonly affected by the presence of many CMB mimics that cause a high false-positive rate (FPR), such as calcifications and pial vessels. This paper proposes a novel 3D deep learning framework that not only detects CMBs but also identifies their anatomical location in the brain (i.e., lobar, deep, and infratentorial regions). For the CMBs detection task, we propose a single end-to-end model by leveraging the 3D U-Net as a backbone with Region Proposal Network (RPN). To significantly reduce the false positives within the same single model, we develop a new scheme, containing Feature Fusion Module (FFM) that detects small candidates utilizing contextual information and Hard Sample Prototype Learning (HSPL) that mines CMB mimics and generates additional loss term called concentration loss using Convolutional Prototype Learning (CPL). For the anatomical localization task, we exploit the 3D U-Net segmentation network to segment anatomical structures of the brain. This task not only identifies to which region the CMBs belong but also eliminates some false positives from the detection task by leveraging anatomical information. We utilize Susceptibility-Weighted Imaging (SWI) and phase images as 3D input to efficiently capture 3D information. The results show that the proposed RPN that utilizes the FFM and HSPL outperforms the baseline RPN and achieves a sensitivity of 94.66 % vs. 93.33 % and an average number of false positives per subject (FPavg) of 0.86 vs. 14.73. Furthermore, the anatomical localization task enhances the detection performance by reducing the FPavg to 0.56 while maintaining the sensitivity of 94.66 %.

脑微出血（CMBs）的深度学习自动检测与解剖定位

脑微出血（Cerebral Microbleeds, CMBs）是脑组织中小型血液产物的慢性沉积，与多种脑血管疾病密切相关，其解剖学定位不同可引发认知衰退、脑出血和脑梗死等问题。然而，由于CMBs的稀疏性和微小结构特性，手动检测过程既耗时又容易出错。CMBs的检测通常受到许多伪影（如钙化和软脑膜血管）的影响，这些伪影会导致较高的假阳性率（FPR）。本文提出了一种新颖的3D深度学习框架，该框架不仅能够检测CMBs，还可以识别其在大脑中的解剖学位置（即大脑皮层、深部区域和小脑幕下区域）。在CMBs检测任务中，我们提出了基于3D U-Net的单一端到端模型，并结合区域提议网络（Region Proposal Network, RPN）作为主干网络。为了在单一模型中显著减少假阳性率，我们开发了一种新方法，包括利用上下文信息检测小型候选区域的特征融合模块（Feature Fusion Module, FFM）以及通过卷积原型学习（Convolutional Prototype Learning, CPL）挖掘CMB伪影并生成额外损失项“集中损失”（Concentration Loss）的难样本原型学习（Hard Sample Prototype Learning, HSPL）。在解剖定位任务中，我们使用3D U-Net分割网络对大脑的解剖结构进行分割。该任务不仅能够识别CMBs所属的脑区，还能利用解剖学信息消除部分检测任务中的假阳性。我们将磁敏感加权成像（Susceptibility-Weighted Imaging, SWI）和相位图像作为3D输入，以有效捕捉三维信息。

实验结果表明，利用FFM和HSPL的RPN优于基线RPN，灵敏度为94.66%，假阳性平均数量（FPavg）从14.73降至0.86。此外，解剖定位任务进一步提高了检测性能，将FPavg降至0.56，同时保持94.66%的灵敏度。

Method

方法

3.1. Datasets

We retrospectively collected brain MR images of patients with CMBs from Gachon University Gil Medical Center (GMC). A total of 114 subjects including 365 CMBs were acquired. The mean number of CMBs per subject was 3.2, with a standard deviation of 4.53. Twenty-three subjects, including 75 CMBs, were randomly selected for testing, while the remaining 91 subjects, including 290 CMBs, composed the training dataset. The size range of CMBs is predominantly under 5 mm, with a maximum size limit reaching 10 mm. The subjects comprised 59 patients with cognitively normal, seven patients with mild cognitive impairment, and 48 patients with dementia (e.g., Alzheimer’s dementia, frontotemporal dementia, and traumatic brain injury). All subjects were scanned using 3T Verio and Skyra Siemens MRI scanners with the following imaging parameters: echo time (TE): 20 ms; repetition time (TR): 27 ms; flip angle (FA): 15◦; bandwidth/pixel (BW/pixel): 120 Hz/ pixel; resolution: 0.50 × 0.50 × 2 mm3 ; and matrix size: 512 × 448 × 72.We collected additional data from a total of 94 subjects, including 311 CMBs. The mean number of CMBs per subject was 3.31, with a standard deviation of 2.3. This dataset was utilized for generalization assessment. This testing data were collected from Seoul National University Hospital (SNUH) and were scanned using a 3T Biograph mMR Siemens MRI scanner with the following imaging parameters: TE: 20 ms; TR: 28 ms; FA: 15◦; BW/pixel: 170 Hz/pixel; resolution: 0.5 × 0.5 × 3 mm3 ; and matrix size: 448 × 392 × 52.The data acquisition was conducted in accordance with relevant regulations and guidelines. The study received approval from the Institutional Review Board of both sites.

3.1 数据集

我们从韩国加藤大学吉尔医院（GMC）回顾性收集了包含脑微出血（CMBs）患者的脑部磁共振（MR）图像数据。总计采集了114名受试者的数据，其中包括365个CMBs。每位受试者的平均CMB数量为3.2个，标准差为4.53。随机选择了23名受试者（包含75个CMBs）作为测试集，其余91名受试者（包含290个CMBs）作为训练集。CMBs的尺寸范围主要小于5毫米，最大可达10毫米。受试者包括59名认知正常患者、7名轻度认知障碍患者，以及48名痴呆患者（如阿尔茨海默痴呆、额颞痴呆及外伤性脑损伤）。所有受试者均使用3T Verio和Skyra Siemens MRI扫描仪进行扫描，成像参数如下：回波时间（TE）：20毫秒；重复时间（TR）：27毫秒；翻转角（FA）：15°；每像素带宽（BW/pixel）：120 Hz/pixel；分辨率：0.50 × 0.50 × 2 mm³；矩阵大小：512 × 448 × 72。此外，我们从首尔国立大学医院（SNUH）额外收集了94名受试者的数据，包含311个CMBs。每位受试者的平均CMB数量为3.31个，标准差为2.3。此数据集用于模型的泛化评估。测试数据通过3T Biograph mMR Siemens MRI扫描仪采集，成像参数如下：TE：20毫秒；TR：28毫秒；FA：15°；BW/pixel：170 Hz/pixel；分辨率：0.5 × 0.5 × 3 mm³；矩阵大小：448 × 392 × 52。数据采集严格遵循相关法规和指南，并获得了两个研究机构伦理审查委员会的批准。

Conclusion

结论

In this paper, we present a framework that not only detects CMBs in the detection task but also identifies their anatomical location in the newly proposed anatomical localization task. In the case of the detection task, we proposed the Feature Fusion Module (FFM) that reduces false positives by incorporating contextual information into the final feature map, and the Hard Sample Prototype Learning (HSPL) that enables the model to concentrate on CMB mimics (i.e., hard samples). The proposed modules in our single-stage detector outperform the two-stage detectors without using any classification model. Further, the main purpose of the anatomical localization task is to identify the anatomical location of CMBs, contributing to elimination of false positives in regions where CMBs could not exist, decreasing FPavg.

在本文中，我们提出了一个框架，该框架不仅能够在检测任务中检测脑微出血（CMBs），还在新提出的解剖定位任务中识别其解剖学位置。在检测任务中，我们提出了特征融合模块（Feature Fusion Module, FFM），通过将上下文信息融合到最终特征图中以减少假阳性，同时引入了难样本原型学习（Hard Sample Prototype Learning, HSPL），使模型能够专注于CMB伪影（即难样本）。我们单阶段检测器中提出的模块无需使用任何分类模型，其性能优于双阶段检测器。此外，解剖定位任务的主要目的是识别CMBs的解剖学位置，通过排除CMBs无法存在的区域中的假阳性来降低平均假阳性数量（FPavg）。

Results

结果

We qualitatively validated the effect of FFM on CMBs detection in Fig. 6. To visually evaluate the feature maps at each level, the number of channels in the feature map at each level is set to 1. As the baseline RPN and the RPN with FFM were trained separately, a strict comparison of the feature maps between the two models is challenging. Nevertheless, it can be observed that the contrast difference in the final feature map affects the probability map in both models. In the baseline RPN, the feature map of the third level serves as the final feature map. As shown in the probability map of the baseline RPN, regions containing CMB mimic exhibit high probability scores. In the case of the RPN with FFM, the final feature map is generated by summing the feature maps from three distinct levels. As shown in Fig. 6(a) and (b) of the RPN with FFM, regions where the CMB mimics might exist were identified utilizing contextual information. Subsequently, these regions were summed with those incorrectly predicted as the CMB regions in Fig. 6(c) of the RPN with FFM, resulting in the corrected Fig. 6(d). Therefore, the probability scores of the RPN with FFM in the regions of CMB mimic are reducedcompared to the baseline RPN case. As shown in Fig. 7, we plotted dimension-reduced feature vectors of true positives, false positives, false negatives, and true negatives. We randomly extracted feature vectors of true negatives from regions not close to the regions of CMBs and false positives. When comparing baseline RPN and RPN with FFM, it can be observed that the RPN with FFM exhibits a lower feature vector density of false positives compared to the baseline RPN. Quantitatively, for the GMC dataset as shown in Table 1, incorporating FFM resulted in a 40.12 % reduction in FPavg while maintaining sensitivity. For the SNUH dataset, incorporating FFM led to a 0.32 % decrease in sensitivity, but achieved a 31.32 % reduction in FPavg.

我们在图 6 中定性地验证了 FFM 对 CMB 检测的影响。为了直观地评估每个层级的特征图，将每个层级特征图的通道数设置为 1。由于基线 RPN 和带有 FFM 的 RPN 是分别训练的，因此严格比较这两个模型的特征图具有挑战性。然而，可以观察到最终特征图中的对比度差异会影响两个模型的概率图。在基线 RPN 中，第三层级的特征图作为最终特征图。如基线 RPN 的概率图所示，包含 CMB 模拟区域的区域具有较高的概率分数。对于带有 FFM 的 RPN，最终特征图是通过将来自三个不同层级的特征图相加生成的。如图 6(a) 和 (b) 所示，在带有 FFM 的 RPN 中，利用上下文信息识别出可能存在 CMB 模拟的区域。随后，将这些区域与图 6(c) 中带有 FFM 的 RPN 错误预测为 CMB 区域的区域相加，从而得到修正后的图 6(d)。因此，与基准 RPN 情况相比，具有 FFM 的 RPN 在 CMB 模拟区域的概率得分降低。如图 7 所示，我们绘制了真阳性、假阳性、假阴性和真阴性的降维特征向量。我们从远离 CMB 区域的区域和假阳性区域随机抽取了真阴性的特征向量。通过比较基准 RPN 和具有 FFM 的 RPN 可以观察到，具有 FFM 的 RPN 的假阳性特征向量密度低于基准 RPN。定量而言，对于表 1 中的 GMC 数据集，引入 FFM 使 FPavg 减少了 40.12%，同时保持了灵敏度。对于 SNUH 数据集，引入 FFM 使灵敏度降低了 0.32%，但使 FPavg 减少了 31.32%。

Figure

图

Fig. 1. Comparison between CMB and pial vessel through consecutive slices in the axial, sagittal, and coronal planes. The scan resolution of these MR images is 0.5 ×0.5 × 2 mm3 .

图1. 通过轴状面、矢状面和冠状面的连续切片对比脑微出血（CMB）和软脑膜血管的区别。这些磁共振图像的扫描分辨率为 0.5 × 0.5 × 2 mm³。

Fig. 2. Overview diagram of the proposed deep learning framework.

图2. 提出的深度学习框架概览图。

Fig. 3. The architecture of the detection task. The input consists of SWI and phase images. conv(n): n × n × n convolutional layer, BN: batch normalization, transposed conv(n): n × n × n transposed convolutional layer, maxpool(n): n × n × n max pooling layer. The Lcls, Lreg, and Lcon are losses for classification, bounding box offset, and concentration learning, respectively.

图3. 检测任务的架构。输入包括磁敏感加权图像（SWI）和相位图像。conv(n)：n × n × n 卷积层，BN：批归一化（batch normalization），transposed conv(n)：n × n × n 反卷积层，maxpool(n)：n × n × n 最大池化层。Lcls、Lreg 和 Lcon 分别表示分类、边界框偏移和集中学习的损失。

Fig. 4. Overview diagram of the proposed Hard Sample Prototype Learning (HSPL)

图4. 提出的难样本原型学习（HSPL）概览图。

Fig. 5. The architecture of the anatomical localization task. The input consists of a cropped patch and three tensors which have absolute coordinate information for x, y, and z axes.

图5. 解剖定位任务的架构。输入包括裁剪的图像块以及包含x、y和z轴绝对坐标信息的三个张量。

Fig. 6. The feature maps and their generated probability map of the baseline RPN and RPN with FFM. The lesion in red circle is a CMB mimic. (a), (b), and (c) showthe feature maps from first to third levels. (d) shows the final feature map. The probability map is also shown on the far right. Note that RPN+FFM reduces thedetection probability for this CMB mimic.

图6. 基线RPN与结合特征融合模块（FFM）的RPN的特征图及其生成的概率图。红圈中的病灶为CMB伪影。(a)、(b) 和 (c) 分别显示了第一至第三层的特征图。(d) 显示了最终的特征图。最右侧为概率图。请注意，结合FFM的RPN显著降低了该CMB伪影的检测概率。

Fig. 7. Feature vectors of CMBs and non-CMBs for three models. The dimension of feature vectors is reduced to two dimensions using t-SNE. The blue, red, yellow, and green points indicate feature vectors of true positives, false positives, false negatives, and true negatives, respectively. The top row represents an instance from the GMC dataset, while the bottom row shows an example from the SNUH dataset.

图7. 三种模型中CMBs和非CMBs的特征向量分布。特征向量通过t-SNE降维至二维。蓝色、红色、黄色和绿色点分别表示真阳性、假阳性、假阴性和真阴性的特征向量。顶部展示的是GMC数据集的一个实例，而底部展示的是SNUH数据集的一个实例。

Fig. 8. Examples of the detected candidates using different methods: baseline RPN, RPN with FFM, and RPN with FFM and HSPL. The lesions in green boxes are CMBs and lesions in red boxes are CMB mimics. The values written over the probability maps indicate the probability scores, with thresholds set at 0.3, 0.5, and 0.4 for the baseline RPN, RPN with FFM, and RPN with FFM and HSPL, respectively

图8. 使用不同方法检测到的候选区域示例：基线RPN、结合FFM的RPN，以及结合FFM和HSPL的RPN。绿色框内的病灶为CMBs，红色框内的病灶为CMB伪影。概率图上的数值表示概率分数，基线RPN、结合FFM的RPN以及结合FFM和HSPL的RPN的阈值分别设为0.3、0.5和0.4。

Fig. 9. (a) shows the sensitivity vs. FPavg, while the (b) presents the PR curve for baseline RPN, RPN with FFM, and RPN with FFM and HSPL.

图9. (a) 显示了灵敏度与平均假阳性数（FPavg）的关系，(b) 展示了基线RPN、结合FFM的RPN，以及结合FFM和HSPL的RPN的精确率-召回率（PR）曲线。

Fig. 10. Distribution of false positives per subject for baseline RPN, RPN with FFM, and RPN with FFM and HSPL, presenting performance results on (a) the GMC dataset and (b) the SNUH dataset.

图10. 基线RPN、结合FFM的RPN，以及结合FFM和HSPL的RPN每位受试者的假阳性分布，展示了(a) GMC数据集和(b) SNUH数据集上的性能结果。

Fig. 11. The lesions in boxes are CMB candidates from detection task. The red, green, and blue regions indicate lobar regions, deep regions, and infratentorial regions, respectively. All these detected CMB candidates get eliminated after checking the segmentation results where the candidates exist out of the anatomical regions.

图11. 方框内的病灶为检测任务中的CMB候选区域。红色、绿色和蓝色区域分别表示脑叶区、深部区和小脑幕下区。在检查分割结果后，所有位于解剖区域外的CMB候选区域均被剔除。

Table

表

Table 1 Comparison between the proposed single-stage approach and the two existing works using GMC and SNUH datasets

表 1  使用 GMC 和 SNUH 数据集的所提出的单阶段方法与现有两种方法的比较

Table 2 Comparison between the proposed single-stage approach against the recent works in the literature on CMBs Detection.

表2 提出的方法与文献中最新研究的CMBs检测结果的比较。

Table 3 Performance on the anatomical localization task across three types of data

表3 解剖定位任务在三类数据上的性能表现。

Table 4 Performance of the overall framework.

表4 整体框架的性能表现。