Brain tumor is one of the most dangerous diseases. Automated brain tumor segmentation technology is particularly important in the diagnosis and treatment of brain tumors. Traditional brain tumor segmentation methods mostly rely on UNet or associate variants, and the segmentation performance is highly dependent on the feature extraction quality. Recently, diffusion probabilistic model (DPM) has received a lot of attention and achieved remarkable success in medical image segmentation. However, the existing DPM-based brain tumor segmentation method did not utilize the advantages of complementary information between multimodal MRI. Additionally, they all constrained the generation of DPM using the original images. In this work, we propose a DPM-based brain tumor segmentation method, which consists of DPM, uncertainty generation module and collaborative Module. The collaborative module takes the input MRI from multimodal information and dynamically provide conditional constraints for DPM. This allows DPM to obtain more detailed brain tumor features. Considering that Previous works mainly ignore the influence of DPM's uncertainty on the results, we proposed an uncertainty generation module. It calculates the uncertainty of each step of the DPM and assigns corresponding uncertainty weights. The results of each step are fused according to inferred uncertainty weights to get the final segmentations. The proposed method obtained 89.32% and 87.82% dice scores on the BraTS2020 and BraTS2021 datasets, respectively, which verified the effectiveness of the proposed method.
Affective image analysis aims to understand the sentiment of different images. The challenge is to develop a discriminative representation that bridges the affective gap between low-level features and high-level emotions. Most existing studies bridge the gap by designing deep models carefully to learn global representations in one shot directly or identify image emotion by extracting features at different levels in the model. They ignore that both local regions of an image and relationships between them impact emotional representation learning. This paper develops an affective image analysis method based on the aesthetic fusion hybrid attention network (AFHA). A modular hybrid attention block is designed to extract image emotion features and model long-range dependencies of images. By stacking hybrid attention blocks in ResNet-style, we obtain an affective representation backbone. Furthermore, considering that image emotion is inseparable from aesthetics, we employ a modified ResNet to extract image aesthetics. Finally, through a fusion strategy, the image's emotion is considered with the aesthetics conveyed. Experiments demonstrate the close relationship between emotion and aesthetics, and our plan has an excellent competitive effect compared with existing methods on the image sentiment analysis dataset.
Infrared and visible image fusion aims to fuse infrared targets and visible details in a composite scene. Recently, many fusion methods have been proposed, but most cannot balance characteristics of infrared and visible images well. We propose an infrared and visible image fusion method. First, to enhance the quality of the infrared and visible images to assist the subsequent prefusion and decomposition, a preprocessing method based on singular value decomposition combined with high dynamic range compression and guided filter (GF) is proposed. Then the preprocessing results are fused to produce the prefusion result by multiscale structural image decomposition-based fusion method. Next, the prefusion result is decomposed by the multilevel image decomposition method based on latent low-rank representation (MDLatLRR). Then the source images are decomposed by MDLatLRR. For the base layer, we combine the visual saliency map with GF to obtain the final fused base layer. For the detail layers, we propose a nonlinear function to highlight the infrared target and use weighted least square to optimize the details. We also adopt a structural similarity-based weight map via L2-norm to improve the fusion performance. The experimental results demonstrate that the proposed method outperforms some state-of-the-art methods both qualitatively and quantitatively.
3D Medical image segmentation is a basic and key task in computer-aided diagnosis, and glioma has a high degree of non-uniformity and irregular shape in multimodal MRI images. Therefore, accurate and reliable segmentation of brain tumors is still a challenging work in medical image analysis. U-Net has become the de facto standard in various medical image segmentation tasks and has achieved great success. However, due to the inherent locality of convolution operations, U-Net is generally limited in terms of explicitly modeling long-term dependencies. Although this problem is solved in Transformer, it has extreme complexity in terms of calculation and space when processing high-resolution 3D feature maps. In this paper, we propose the Trans-coder, which is embedded in the end of the U-Net encoder to improve the segmentation performance while reducing the amount of calculation. The Trans-coder takes the feature map from U-Net as the input sequence, and extracts the relative position information of the feature map, so as to get more detailed information of the image, and input it into the decoder to obtain good segmentation performance. At the same time, a variational autoencoder is used for regularization to prevent over-fitting problems. our method achieves superior performances to certain competing methods on the Multimodal Brain Tumor Segmentation Challenge (Brats) dataset.
Images can convey rich semantic information and arouse strong emotions in the viewer. With the growing trend of online images and videos to express opinions, evaluating emotions from visual content has attracted considerable attention. Image emotion recognition aims to classify the emotions conveyed by images automatically. The existing image sentiment classification studies using manual features or deep models mainly focus on low-level visual features or high-level semantic representation without considering all factors. In this paper, we adopt visualization to study the working principle of deep representation in emotion recognition. Research shows that the deep model mainly relies on deep semantic information while ignoring the features of shallow visual details, which are essential to evoke emotions. To form a more discriminative representation for emotion recognition, we propose a multi-level representation model with side branches that learns and integrates different depth representations of the backbone for sentiment analysis. Unlike the hierarchy CNN structure, our model provides a description from the deep semantic representation to shallow visual representation. Additionally, several feature fusion approaches are analyzed and discussed to optimize the deep model. Extensive experiments on several image emotion recognition datasets show that our model outperforms various existing methods.
To improve the results of image fusion, taking multi-focus color image as the research object, a self-adaptive pulse coupled neural network multi-focus color image fusion method is presented, which is based on the nonsubsampled contourlet transform (NSCT). And compute the hue component and saturation component of the fused image by the fused intensity component, and finish the whole fused process of the multi-focus color image. The results show that the proposed image fusion method has a better quality than the fused by the wavelet transform and the nonsubsampled contourlet transform, as the same time the fused results in HSI model is of better quality than that fused in RGB model.
A fusion algorithm based on target extraction for infrared image (IIR) and visible image fusion in the nonsubsampled contourlet transform (NSCT) domain is proposed. Commonly, the target information in IIR is important; in order to fully retain the target information in a final fused image, first, use maximum between-class variance method to segment IIR, such that the target regions with salient objects are extracted to produce the background and target images. Next, the visible and background images are decomposed to a series of low-pass and band-pass images by NSCT, respectively. Then, fuse the obtained images to produce the fused background image by different strategies in each band, in which Gaussian fuzzy logic is used to produce the low-pass coefficient; the spatial frequency of each band-pass image is used to determine the linking strength β value of pulse coupled neural network structure, and the result is used to fuse the band-pass images. Eventually, the fused image is produced combining the target image and the fused background image. The experiments show that this algorithm can retain more background details of the two images and highlight the target in the infrared image more effectively, as well as obviously improve the visual effect of the fusion image.
Medical image fusion plays an important role in biomedical research and clinical diagnosis. In this paper, an efficient medical image fusion approach is presented based on pulse coupled neural network (PCNN) combining multi-objective particle swarm optimization (MOPSO), which solves the problem of PCNN parameters setting. Selecting mutual information (MI) and image quality factor (QAB/F) as the fitness function of MOPSO, the parameters of PCNN are adaptively set by the popular MOPSO algorithm. Computed tomography (CT) and magnetic resonance imaging (MRI) are the source images as experimental images. Compared with other methods, the experimental results show the superior processing performances in both subjective and objective assessment criteria.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.