To solve the problem of low resolution of underwater polarization imaging images, a method to improve the resolution of active underwater polarization imaging images is proposed, which includes an optimal orthogonal polarization image group acquisition method, and an accurate estimation and optimization method of the polarization degree of backscattered light and target reflected light. An experiment is carried out to improve the resolution of underwater image restoration based on linear polarized light active illumination. The experimental results show that the image group with the lowest correlation degree is used as the orthogonal polarization image group among multiple groups of orthogonal polarization images, and the scattering depolarization coefficient is used to accurately estimate and optimize the polarization degree of backscattered light and target reflected light, and the final restored target image has higher resolution. Compared with the existing image restoration algorithms, it has obvious advantages and is suitable for water media with different material objects and different turbidity.

Most existing arbitrary shape text detection methods employ connected components and text center lines for grouping text instances, which assume that texts in adjacent positions belong to the same instance. However, many hard-to-group scene texts are too complex to be effectively processed in this way. To address this challenge, we propose a novel scene text-spotting method that utilizes feature-based clustering inspired by human cognitive principles of text perception. Our approach involves first utilizing a character spotter to obtain the location and the transcription information of the characters. Then, a lightweight recognition network extracts the visual features of the characters by their locations. These visual features are then grouped into instances through a K-means-fuzzy-net, which explicitly model visual feature similarity to effectively group the nested text, the large-margin text, the continuous text, and the one with overlapping characters. Finally, the recognition results of text instances are processed by a word correction module to improve the overall accuracy and reduce the vulnerability of individual character detection. Additionally, we have contributed a hard-to-group text dataset. Experiments demonstrate the state-of-the-art performance of our method in addressing scenarios. Hard-to-group text dataset is available at: https://github.com/baggio321/Hard-to-Group-Text-Dataset.

Transformer-based methods have achieved impressive performance in image super-resolution (SR). To reduce the computational cost and redundancy of global attention, most transformer-based methods adopt a localized attention mechanism, which diminishes the desirable characteristics of self-attention (SA), such as the effective modeling of long-range dependencies and the ability to capture a global receptive field. To alleviate this problem, we propose a dilated neighborhood attention transformer for image SR (DiNAT-SR) to improve SwinIR for image SR; in it, we replace SA with DiNA to capture more global data and allow the receptive field to grow exponentially. In addition, we also introduce a convolutional modulation block into the transformer to enhance the visual representation and facilitate smoother convergence during training. Our research has, for the first time, confirmed the feasibility of DiNA in the field of image SR. Experimental results have demonstrated the effectiveness of DiNAT-SR with better results compared with SwinIR on most benchmarks in terms of both quantitatively and visually. We also provide a comparison of light-weight image SR models, and our model performs better than SwinIR-light on all benchmarks, with similar total numbers of parameters and floating-point operations. The effectiveness of each introduced component is also validated by an ablation study.

Video-based point cloud compression projects dynamic point clouds to 2D videos called geometry videos and texture videos for compression and transmission. In the geometry video, the pixel values represent the positions of each point in the point cloud. Due to distortion of the geometry video, there are some holes and noise points in the reconstructed point cloud. The previous methods for enhancing 2D images or videos can be applied to improve the quality of geometry videos. However, there will still be limitations in the reconstructed 3D point cloud because these methods do not take into account the feature of the 3D point cloud, such as the shape changes and surface orientations. To overcome these limitations, we propose an enhancement network specifically designed for point cloud enhancement through restoring geometry videos, which can calibrate the shape variations and surface orientations of point clouds. Specifically, the gradient information of the geometry video reflects the changes in pixel values, which correspond to changes in shape of the point cloud. Additionally, the normal vectors of the points in the point cloud provide the point cloud surface orientation information. To attain similarity of the shape changes, we design a two-branch network structure and utilize an additional branch comprising of gradient information restoring module to restore the gradient information of the geometry video. To ensure the consistency of point normal vectors, surface fitting and estimation module is designed to calculate the normal vectors of the points in distorted point cloud and the original point cloud through the geometry video, and then the difference of these normal vectors can be used to guide the restoration of the geometry video. Our method has been proved to be satisfactory through the validation experiments on currently available point cloud sequences.

In the domain of image fusion, integrating infrared and visible images provides a more complete scene description by merging the unique strengths of each modality. Existing methods struggle with handling the differences between modalities, which is caused by the inherent entanglement of scene-common information and modality-specific information within each modality. In response, we propose MIRFuse, a model for infrared and visible image fusion based on disentanglement representation via mutual information regularization. The process of disentanglement, in which scene-common information and modality-specific information are separated, forms the basis for identifying both shared and exclusive features. First, mutual information maximization is used as consistency constraint, enabling scene-common encoders to effectively extract shared features. Second, the Hilbert–Schmidt independence criterion is employed as heterogeneity constraint, promoting modality-specific encoders to extract exclusive features. Finally, both shared and exclusive features are identified and combined using various fusion strategies to produce a fused image. The resulting fused image provides a comprehensive representation of the entire scene, allowing for more effective utilization of information from multiple modalities. Our experiments have validated the advanced nature and effectiveness of our method.

The technique of digital watermarking has emerged as a crucial method for safeguarding the intellectual property rights of image owners. Substantial advancements have been made in the field of embedding watermarks in digital images. However, meeting the requirements of high-capacity, imperceptibility, and robustness simultaneously poses a challenge due to the trade-off between these factors. We propose a model that establishes a mutual mapping between the cover image and watermarked image domains, which enables alignment and can accelerate the model convergence. Our model prioritizes the decoder structure by utilizing an encoder-like architecture to align the overall structure of the model. This design improves the decoder’s capacity to extract watermark information. In the process of image generation, we present an approach to enhance feature size through antispectral aliasing, which replaces the transposed convolution mechanism in the U-Net structure. This technique significantly reduces high-frequency artifacts, leading to an overall improvement in the quality of the watermarked image. To improve the ability of the model to deceive the discriminator, we introduce a discriminator that imposes constraints on multiple frequency bands, resulting in significantly improved imperceptibility. Our model demonstrates superior robustness compared to state-of-the-art methods while preserving a competitive level of image quality.

In recent years, generative adversarial networks have been widely used to generate realistic fake images, and the current research mainly focuses on deepfake detection in face images. However, we are now facing the emergence of fake satellite images, which could potentially mislead and threaten national security. To address this issue, we propose a hybrid network of convolutional neural network and Transformer for deepfake geographic image detection. Specifically, our method combines the local modeling capability of convolutional neural networks with the global modeling capability of transformers. This allows us to effectively extract both local features and global representations from the satellite images. Additionally, we enhance the detection performance of the model by introducing channel attention mechanisms. Experimental results demonstrate that our proposed method outperforms existing approaches. Furthermore, under common image processing scenarios, such as Joint Photographic Experts Group compression and Gaussian noise corruption, our method also performs better than other comparison methods.

In capturing images, cameras often encounter color-cast problems owing to variations in the illumination of different scenes. Existing algorithms attempt to estimate the illuminant information from the raw image data captured by a camera sensor to address color-casting problems. However, these algorithms tend to overlook the potential interference of scene brightness information present in the raw data. Therefore, we introduce a novel illuminant estimation architecture, Retinex color constancy generative adversarial network (ReCC-GAN). ReCC-GAN aims to mitigate the impact of scene brightness information on illuminant estimation tasks, enhance global illuminant estimation accuracy, and resolve color-cast problems. The ReCC-GAN architecture initially separates the reflectance component from the raw data using the Retinex module, reducing the interference caused by the scene brightness information. Subsequently, a specially designed GAN for illuminant estimation was employed to estimate the illuminant vector from this component more effectively. This approach improved illumination estimation accuracy. Experimental results on the Cube++ and NUS-8 dataset demonstrated that the proposed algorithm achieves competitive results compared with other solutions, with outstanding performance on the worst 25% metric. Additionally, tests on various datasets confirmed that the ReCC-GAN exhibited strong generalization performance, further validating its effectiveness and rationality in scene-wide illuminant estimation tasks.

Classification networks of degraded images need to deal with various strengths of degradation, referred to as degradation levels, in practical applications. However, there has been limited exploration of data augmentation techniques for degraded images with various degradation levels. We propose a data augmentation technique to apply distinct data augmentations to both clean and degraded image domains. Specifically, the proposed method uses random erasing and CutBlur data augmentations for a clean and degraded image, respectively. Experimental results show that the proposed method can effectively train a classification network of degraded images without losing the classification ability of clean images. Furthermore, the results also confirm the proposed method’s efficacy across various degradations, multiple network architectures, and several datasets.

To solve the problem of obtaining a higher accuracy at the expense of redundant models, we propose a network architecture. We utilize a lightweight network that retains the high-precision advantage of the transformer and effectively combines it with convolutional neural network. By greatly reducing the training parameters, this approach achieves high precision, making it well suited for deployment on edge devices. A detail highlight module (DHM) is added to effectively fuse information from multiple scales, making the depth of prediction more accurate and clearer. A dense geometric constraints module is introduced to recover accurate scale factors in autonomous driving without additional sensors. Experimental results demonstrate that our model improves the accuracy from 98.1% to 98.3% compared with Monodepth2, and the model parameters are reduced by about 80%.

Currently, most feature extraction methods lack generality and require specific feature extraction methods for images from different sensors, particularly in the detection of faults in power equipment. However, the challenge lies in how to restore realistic texture details while correcting the color distortion. To address this issue, we propose an innovative infrared-visible light image fusion technique that combines hierarchical attention modules and collaborative refinement modules to facilitate feature fusion by jointly preserving intricate details and correcting lighting conditions. The hierarchical attention module aims to provide two different attention weight maps, which help select the most salient information from the source images in the infrared and visible light domains. This ultimately produces intermediate fusion results with comprehensive and complementary data. The collaborative refinement module consists of an edge enhancement network and a lighting correction network, which works together to enhance edge details and correct color differences. Experimental results validate the effectiveness of this method in successfully fusing the two types of images. The results demonstrate that, compared with several mainstream fusion methods, this method exhibits significant advantages in publicly available datasets and scenarios, such as power equipment detection.

In practical scenarios, detecting and grasping objects accurately can be very challenging due to the uncertainty of their positions and orientations, as well as environmental interference. Especially when the target object is occluded by other objects, traditional machine vision methods have difficulty in accurately recognizing it. To address this problem, we propose the double-attention mechanism-based segmentation grasping detection network (DAM-SGNET). DAM-SGNET is a technique used for detecting and grasping objects accurately in cluttered environments. It utilizes a deep neural network that incorporates two attention mechanisms to predict the optimal grasping posture for RGB images at the pixel level without relying on depth images. The method begins by reannotating datasets, such as the Cornell dataset, cluttered scenes objects dataset, and VMRD dataset, with a new labeling method proposed by previous researchers. These datasets are then used to train an occlusion detection model. DAM-SGNET uses a residual network (SERESNET) with channel attention mechanisms to extract features from the images, and an adaptive decoder including a feature pyramid deformation network and an efficient channel attention module to enhance robustness in cluttered, unstructured open environments. DAM-SGNET ultimately achieves grasp detection accuracy of 99.43%, 99.24%, and 85.38% for the official Cornell grasp dataset, the cluttered scenes grasping dataset, and the VMRD grasping dataset, respectively. Real-world experiments demonstrate the efficacy of DAM-SGNET in self-built robotic arm platforms, achieving a single-target grasping success rate of 99.6%, and an average grasping success rate of 96.46% for cluttered stacked objects.

As a fundamental task in computer vision, object detection has long been a challenging visual task. However, current object detection models lack attention to salient features when fusing the lateral connections and top-down information flows in feature pyramid networks (FPNs). To address this, we propose a method for object detection based on an enhanced bi-directional attention feature pyramid network, which aims to enhance the feature representation capability of lateral connections and top-down links in FPN. This method adopts the triplet module to give attention to salient features in the original multi-scale information in spatial and channel dimensions, establishing an enhanced triplet attention. In addition, it introduces improved top and down attention to fuse contextual information using the correlation of features between adjacent scales. Furthermore, adaptively spatial feature fusion and self-attention are introduced to expand the receptive field and improve the detection performance of deep levels. Extensive experiments conducted on the PASCAL VOC, MS COCO, KITTI, and CrowdHuman datasets demonstrate that our method achieves performance gains of 1.8%, 0.8%, 0.5%, and 0.2%, respectively. These results indicate that our method has significant effects and is competitive compared with advanced detectors.

Facial micro-expressions are categorized into various types based on different criteria, and typically each major category is further divided into multiple subcategories of expressions. For traditional micro-expression recognition problems, multiple subcategories of the same emotions are indiscriminately learned and verified, leading to potential misclassification, especially with negative emotions. To address the issue of intra-class variation in micro-expressions, we propose a meta-clustering learning network for micro-expression recognition called MCNet. This approach integrates the ideas of meta-learning and clustering, hierarchically clustering subcategories within a micro-expression class to generate multiple class centers for metric-based classification. The proposed method diverges from the common strategy of metric-based meta-learning algorithms, which typically use the mean feature of all samples within the same class as the class center. Furthermore, we incorporate transfer learning into the meta-learning process to jointly alleviate overfitting caused by the scarcity of micro-expression data. We conduct extensive comparative experiments based on the leave-one-subject-out protocol on three widely used micro-expression datasets. The experimental results demonstrate the competitive performance and strong generalization ability of the proposed MCNet approach.

For the synthetic aperture radar (SAR) automatic target recognition problem, we propose a multiview method based on adaptive decision fusion. Considering the possible independence and correlation among multiview SAR images, a clustering algorithm is first performed to divide them into different view sets. In each view set, the included SAR images are assumed to be highly correlated, which can be effectively classified through joint sparse representation. For the output decisions from different view sets, they are assumed to be relatively independent and contribute disproportionally to the recognition result. Therefore, based on the decisions of individual view sets, adaptive weights are decided based on Shannon entropy theory, which are used to linearly fuse the multiple decisions to determine the target label. The MSTAR dataset is employed in experiments, in which both the standard operating condition and some representative extended operating conditions are set up. The performance of the proposed method is studied in comparison with representative state-of-the-art methods.

The two-branch real-time semantic segmentation network can quickly acquire low-level details and high-level semantics. However, the large contextual gap between them results in adverse impact on their fusion, and limits the further improvement of real-time segmentation accuracy. This paper proposes a tripartite real-time semantic segmentation network with scene commonality (TriSCNet) to address this problem. First, we add a parallel scene commonality branch based on the current two-branch architecture to learn intrinsic common features in similar street scene images, such as the spatial location distribution of various objects and the internal connections between them at the semantic level. Further, with the guidance of commonality, we propose an external branch attention module to enrich and enhance the feature information of traditional two branches. Finally, we utilize an alignment and selective fusion module to correct the misaligned context in the semantic branch and highlight the essential spatial information in the detailed branch. Our proposed TriSCNet achieves an excellent trade-off between accuracy and speed, yielding 77.9% mIOU at 67.2 FPS on Cityscapes test set and 75.8% mIOU at 127.4 FPS on CamVid test set, respectively.

Visual object tracking, one of the most fundamental and challenging tasks with several applications, has attracted a great deal of research interest and has been widely applied in unmanned aerial vehicle (UAV) missions. Despite the efficient correlation filter (CF)-based approaches that have shown prominent performance in visual object tracking for different UAV applications, the model drift, caused by undesired boundary effects, filter degradation, and severe appearance variations, remains a challenging problem. To solve the aforementioned issue, we propose saliency-enhanced background-aware CFs with dual temporal regularization for UAV tracking. In more detail, the dual temporal regularization term is introduced to reinforce inter-frame continuity using current and previous frames’ filter and sample information, respectively. As for the saliency enhancement term, it is integrated by multiplying the merged saliency map and the vectorized image patch element-wise to emphasize the noticeable object and suppress irrelevant background noise. Finally, a CF regression model is formulated and its closed-form solution is derived by utilizing the alternating direction method of multipliers (ADMM) algorithm. The saliency and temporal information in the tracking scene are fully considered to promote model discriminative ability and boost the overall tracking performance. Extensive experiments are conducted on three public UAV benchmark datasets UAV123@10 fps, DTB70, and UAVDT using 243 challenging sequences. The results demonstrate that the proposed method performs favorably against other state-of-the-art trackers while operating at a real-time speed on a single CPU, making it suitable for UAV tracking tasks.

Abstract paintings are produced by artists based on their concepts and employ color, texture, and other techniques. Determining the arrangement of abstract paintings is challenging given their ambiguous nature. Previous studies on image orientation recognition faced three major difficulties. First, they relied heavily on pre-existing convolution neural network models, such as VGG and AlexNet. Second, they focused largely on a single task—recognizing image orientation. Finally, ground truth data concerning visual perception regions of images were often obtained through manual annotation. To overcome these issues, we introduce OC–OD: a multitask approach fused across multiple feature layers for better performance. The orientation classification (OC) subtask is the primary task, whereas the visual perception region detection (OD) subtask is auxiliary. OC and OD utilize the same feature extraction layer; OD aims to enhance the efficiency of OC completion. At the same time, the ground truth data used in OD is obtained from gaze fixation density maps gathered by an eye tracker during the subject’s viewing of the image, rather than through manual annotation. Two datasets were chosen to compare the training impact of various model parameters and architecture. The experimental results were extensively compared, leading to the discovery that our proposed approach significantly enhances orientation recognition accuracy and outperforms other state-of-the-art methodologies.

Despite recent promising advances in thermal infrared image colorization for nighttime driving scenes, the content structure preservation and fidelity of colorized images remain ineffectively addressed. Therefore, we introduce a dynamic spatial structure guidance to automatically learn the task-oriented structure representation in the unpaired thermal infrared image colorization of nighttime driving scenes. First, patch-based spatial correlation maps are constructed for the nighttime thermal infrared image and the color image. The content structure preservation during thermal infrared image colorization is realized by reflecting the patch-based spatial structural correlations between the input and output images. To flexibly utilize the spatial correlations, structural representation learning was constructed to learn task-specific structural characterization. We innovatively treat the spatial structural similarity between the corresponding image patches as an interest metric to guide contrastive learning to optimize structural representation learning. Compared with state-of-the-art methods, our model achieved an advanced thermal infrared image colorization performance. Additionally, we performed semantic segmentation and object detection tasks on the colorization to further address the practicality. Extensive qualitative and quantitative experiments and evaluations demonstrate that our method achieves state-of-the-art results in the nighttime thermal infrared colorization.

Low-light images inevitably suffer from degradation problems during the enhancement process, such as loss of detail and local overexposure or underexposure. Many existing methods target only one of these issues, leading to suboptimal results. We propose a multistage Laplacian feature fusion network (MLFFNet) capable of simultaneously mitigating both degradation difficulties. MLFFNet employs a pyramid framework that incrementally learns the degradation functions across various frequency bands, leveraging Laplacian feature maps at each stage. A key innovation of our approach is the supervised refinement module, which refines features through a dual strategy: an attention mechanism that enriches the detail capture, including edges, textures, and colors, and a residual mechanism that adjusts the luminance for a balanced exposure. The resultant enhanced image benefits from channel-wise attention, ensuring superior enhancement. Finally, the enhanced image is acquired by several channel attention blocks. Extensive experiments on various datasets indicate that our proposed MLFFNet outperforms the state-of-the-art methods both qualitatively and quantitatively.

In few-shot object detection, the limited amount of labeled data fails to adequately represent all possible scenarios and objects. This limitation leads to the model’s inability to fully learn the features and attributes of novel classes. In some cases, novel classes may be confused with base classes due to feature similarity, resulting in inaccurate detection results. During the two-stage fine-tuning phase, when there is a significant difference between the novel class and the base class data, the candidate boxes generated by the backbone network trained on the base class may not be suitable for the novel class targets. Therefore, it is a forward-looking research problem to explore how to mine novel class knowledge during the feature extraction process to supplement the disadvantage of having limited feature samples and improve the sensitivity in recognizing novel classes. To enrich the feature representation of novel classes, we propose a self-supervised feature pyramid network. This approach explores novel class attributes in the lower-level network, thereby encouraging the feature extractor to generate candidate boxes that are consistent with the novel class targets. The goal is to enhance the sensitivity of the backbone network in recognizing novel classes. We validate the effectiveness of our proposed framework by comparing it with state-of-the-art methods on two popular datasets and achieve an improvement of up to +5.2% on the standard PASCAL VOC benchmark and a 1.4% boost on the challenging COCO benchmark.

Cross-modal retrieval aims to find alignment relationships between different modalities and then compute the semantic similarities used for ranking. Because of the data distribution difference and inherent heterogeneity gap between modalities, a classic solution is to learn common representations in the common space, which could preserve the discrimination among the samples from different categories and alleviate the cross-modal discrepancy. To achieve this, we propose a method, termed LDCA, to learn discriminative common alignments based on the modal representations. LDCA utilizes a modality invariance loss that pushes away the hardest negative sample to further reduce the cross-modal discrepancy at the feature level. In addition, LDCA seeks alignments in the label space to improve the intra-modal discrimination by an effective cross-modal label loss. Extensive experiments are conducted on five widely used cross-modal datasets to evaluate the proposed LDCA. The integral experimental results prove the method’s superiority, and the comprehensive analyses verify the effectiveness of the method.

We propose a dual-branch Siamese network for visual object tracking. Our network architecture comprises two distinct branches: a shallow network branch and a deep network branch. The shallow network branch focuses on precise object localization and improving resistance to interference from similar objects. Meanwhile, the deep network branch emphasizes capturing abstract semantic features of the object. To enhance localization accuracy, we integrate a multi-scale KFFM into the shallow network. In addition, we leverage the attention mechanism to further enhance the model’s robustness. Through extensive experiments on three publicly available datasets, we demonstrate that our method surpasses state-of-the-art tracking algorithms in terms of performance and accuracy. The source code of this work is available online at https://github.com/mbgzwn/SiamDUL.git.

Underwater images often suffer from color distortion, blurred details, and low contrast. Therefore, more researchers are exploring underwater image enhancement (UIE) methods. However, UIE models based on deep learning suffer from high computational complexity, thus limiting their integration into underwater devices. In this work, we propose a lightweight UIE network based on knowledge distillation (UKD-Net), which includes a teacher network (T-Net) and a student network (S-Net). T-Net uses our designed multi-scale fusion block and parallel attention block to achieve excellent performance. We utilize knowledge distillation technology to transfer the rich knowledge of the T-Net onto a deployable S-Net. Additionally, S-Net employs blueprint separable convolutions and multistage distillation block to reduce parameter count and computational complexity. Results demonstrate that our UKD-Net successfully achieves a lightweight model design while maintaining superior enhanced performance.

By simulating the human brain, artificial intelligence-generated image models have the ability to learn various features from text memory, such as color, shape, and corresponding semantics. However, the quality of the generated images can be inconsistent due to various influencing factors. Therefore, the objective of this study is to evaluate the color quality of generated images depicting natural objects, such as sky, grass, fallen leaves, and rocks, based on memory colors. The memory color of the object in the reference image, obtained through the Baidu crawler, serves as the benchmark for comparison with the generated image. Alternatively, the color quality assessment of the generated images was performed by constructing H-S polar scatterplots, analyzing lab color distribution, and calculating average color differences. The GauGAN and ERNIE-ViLG models demonstrate higher accuracy in reproducing the memory color of the four objects, with the exception of the “fallen leaves” model, which exhibits more significant color differences. Furthermore, a survey was conducted to assess the recognition of color reproduction effects. The results indicate that the color presentation of the objects “grass,” “fallen leaves,” and “rocks” aligns with the observed color differences. These findings provide valuable insights for optimizing the color-quality rendering of text-generated image models.

It is difficult to obtain real-time crop water shortage during spinach seedling production, so this study proposes a real-time spinach seedling water stress classification method based on hierarchical transformer encoder (HTE) MobileNet (HT-MobileNet). An efficient sandglass residual module is designed for the grading algorithm to significantly improve the model grading accuracy. Meanwhile, a lightweight attention mechanism is proposed. It can effectively suppress the influence of irrelevant features on the classification results. Furthermore, an HTE is introduced, classifying the degree of stress accurately under extremely complex spinach seedling conditions. By introducing grouped convolution, the number of model parameters are reduced greatly, which lays the foundation for practical application. The grading accuracy is 94.61%. The number of parameters is 1.78M and the model size is 4.57MB, which are 40.67% lower than the original network. Comparison experiments are conducted between HT-MobileNet and 10 mainstream convolutional neural networks, such as ConvNext. The experimental results demonstrate the outstanding advantages of this research in terms of accuracy and weight. In addition, an intelligent spinach seedling water stress grading system is designed and tested. This study can provide technical support for real-time monitoring of spinach seedling growth status.

In X-ray object detection, scale awareness and spatial awareness (SA) are two independent issues. Previous research has attempted to address these two issues separately using different methods but lacked a unified approach to solve both problems simultaneously. This paper presents a single-stage object detector, referred to as scale and SA-based prohibited item detection, which consists of the multi-level fusion module (MLF) and the SA module. MLF is designed to learn multi-level features to enhance scale awareness, and SA is designed to learn and focus on object spatial information to enhance SA. Compared with previous research, our method shows outstanding performance in detection accuracy. Moreover, our single-stage object detector outperforms the accuracy of two-stage object detectors by 1.5 AP.

In traditional stroke-based image-to-painting techniques, strokes are often repeated in already drawn areas, resulting in ineffective rendering, low utilization of strokes, and low image quality. This paper presents an approach for image-to-painting that dynamically assigns strokes based on the intrinsic characteristics of different regions in the target image. The proposed method uses an estimation technique to determine the amount of information in different image regions to generate a sequence of content richness. The number of strokes is then dynamically assigned based on the sequence of content richness, which results in a sequence of strokes. Afterward, the distribution of strokes on the canvas is flexibly adjusted based on their sequence, allowing for the accurate rendering of the target image. In addition, the original network model is optimized and improved to make the stroke rendering more accurate and thus obtain a final result that better fits the input image. The experimental results show that the proposed approach can scientifically allocate strokes based on the inherent differences in different image regions, which leads to effective rendering, improved utilization of strokes, and higher visual performance. Furthermore, many indicators, such as the peak signal-to-noise ratio, structural similarity, and Learned Perceptual Image Patch Similarity demonstrate higher image quality compared with those obtained by traditional technologies.

Weakly supervised semantic segmentation (WSSS) using only image-level labels is a challenging task. Most existing methods utilize class activation map (CAM) to generate pixel-level pseudo labels for supervised training. However, the gap between classification and segmentation hinders the network from obtaining more comprehensive semantic information and generating more accurate pseudo masks for segmentation. To address this issue, we propose TSD-CAM, a transformer-based self distillation (SD) method that utilizes CAM similarity. TSD-CAM uses the similarity between CAMs generated from different views as a distillation target, providing additional supervision for the network and narrowing the gap between classification and segmentation. SD supervision allows the network to acquire more semantic information and refine CAMs to generate higher precision pseudo-labels. In addition, we propose the adaptive pixel refinement module, which adaptively refines and adjusts images based on pixel variations, further improving the precision of pseudo labels. Our method is a fully end-to-end single-stage approach that achieves state-of-the-art 71.3% mIoU on PASCAL VOC 2012 and 42.9% mIoU on the MS COCO 2014 dataset, and the proposed TSD-CAM can significantly outperform other single-stage competitors and achieve comparable performance with state-of-the-art multi-stage methods. Meanwhile, the effectiveness of our method is demonstrated by a large number of ablation experiments, and we provide a new way of thinking to solve the problems of WSSS. Our code is available at: https://github.com/pipizhum/TSD-CAM.

This work provides a method for reducing dimensionality using a median-based discriminatory analysis, which considers the adverse impact on the class mean caused by outliers, referred to as an enhanced linear discriminant analysis (ELDA). ELDA is based on a point-to-median distance, used as a measure metric to describe the within-class and between-class median scatter. The standard LDA algorithm may not be effective when the class distribution is uneven, and the limited sample size for inequality datasets and outliers persists. To address all issues, the proposed ELDA method is applied for cashew leaf disease identification on the cashew crop disease database (CCDDB), and it is more accurate at identifying and classifying cashew leaf illnesses in different sets of training and testing samples on three datasets. Research exhibits greater accuracy on CCDDB with 97.7%, on image database of plant disease symptoms dataset with 99.8%, and on dataset for crop pest and disease detection with 81.7%.

The kernel graph cut method provides good performance in image segmentation. However, its efficiency strongly depends on the data term and regularization term in the objective function. The data term maps the standard deviation of the data of each region to the transformation space. The regularization term is responsible for smoothing the boundaries. The regularization term, based on a constant coefficient, is added to the data term. Allocating a fixed coefficient for all images leads to inappropriate image segmentation. In this regard, we deal with the automatic adjustment of the regularization term coefficient in the kernel graph cut method based on image energy. This approach is useful in image segmentation with thin structures. The laboratory results were taken on three datasets: fingerprint, coronary artery, and real. The efficiency evaluation of the algorithm, compared with other methods in the field of energy-based algorithms, shows a maximum score of 88.27 in Dice, 79.67 in Jaccard, 75.67 in accuracy, 61.06 in peak-signal-to noise ratio, and a minimum score of 0.06 in mean sum of squared distance on the mentioned datasets. The proposed method also takes advantage of appropriate computing load in the methods based on a graph cut.

Magnetic resonance imaging (MRI) is extensively utilized in psychiatric disorder diagnosis. However, the high-dimensional nature of the data poses challenges in network training. We propose a classification approach leveraging autoencoder and multilayer perceptron techniques. Initially, a convolutional autoencoder is designed to reduce the dimensionality and extract features from the input feature map using its encoder component. Subsequently, the extracted features are fed into the designed multilayer perceptron for training and testing, enabling joint classification based on dual features. Finally, we apply this method to classify attention deficit hyperactivity disorder (ADHD), and the experimental results demonstrate the effectiveness of our approach in significantly enhancing the accuracy of ADHD classification.

With the improvement in the quality of spoofed faces, there are significant challenges in terms of the security of face recognition systems. At present, methods based on multi-modality can further protect systems from spoofing. However, such approaches generally have a considerably large amount of parameters and calculations. To overcome such problems, a lightweight face anti-spoofing method based on cross-fused multi-modal features was proposed. Firstly, the lightweight network ShuffleNetV2 is improved, thereby improving the classification accuracy while reducing the number of model parameters. In addition, a cross-attention-based feature fusion module was designed, which enriches the feature representation by cross-computing with the depth modality to obtain the attention maps of RGB and infrared modalities. In the present study, the proposed method was tested in several datasets and had good accuracy and only 0.0892 M parameters, indicating that the method is completely suitable for deployment into resource-constrained mobile or embedded devices. The code is available at https://github.com/HeDan-11/LFAS-CFMMF.

With the ever-increasing use of the Internet, secure data transmission has become critical. The benefits that communication networks have provided are enormous but at the expense of data security. Privacy and security are becoming major concerns due to the rise in the use of public communication networks. Recently various encryption schemes have been proposed but most of them have high computational overhead and are also not resistant to recent attacks. For data transmission networks, a secure and lightweight encryption (LWE) scheme has been developed. Using pseudo-random number sequences (PRNS) and DNA arithmetic, the plain-text image (PI) is confused and diffused one at a time in the proposed scheme. A composite PRNG (CPRNG) uses different sets of keys to generate PRNSs. The proposed LWE uses the CPRNG, which is composed of the chaotic map and 256-bit linear feedback shift register. Both the components can run on low computational overhead. The large key space makes the scheme highly robust and scalable. This encryption scheme has been tested on a variety of gray-label images to demonstrate its robustness and broad applicability. The test results of the evaluation are extremely promising, indicating that the scheme can provide adequate security against attacks.

Information about vehicles plays an important role in intelligent transportation systems (ITSs). It can be applied in different areas such as vehicle monitoring, vehicle detection, auxiliary reconnaissance, etc. Among the different types of vehicle-related information, logo information plays an essential role in quickly identifying the vehicle and enabling relevant work to be carried out. However, existing logo detection methods face issues related to low training accuracy and difficulty in accurately locating the logo, which leads to inaccurate detection of vehicle logos. To address these challenges, we first propose a method for detecting vehicle logos, particularly at expressway exits. We created a dataset comprising seven categories of vehicles for this purpose. Our solution includes a lightweight model that bridges CNN and transformer and a creative method for locating the logo. Additionally, we also do data processing on the test images to make them robust to environmental changes. The network that we designed is simple yet effective, achieving improvements in both precision and speed. Furthermore, our vehicle logo localization algorithm can withstand environmental variations. Experimental results demonstrate that our algorithm achieves a 5% to 10% accuracy boost compared with other methods.

RGB-D salient object detection is a challenging task in computer vision, and deep architectures have been widely adopted in the previous studies. However, current convolutional neural network (CNN)-based models struggle with capturing global long-distance features efficiently, whereas transformer-based methods are computationally intensive. To address these limitations, we propose a nonconvolutional feature encoder. This encoder captures long-distance dependencies while reducing computation costs, making it a potential alternative to CNNs and transformers. Additionally, we introduce a spatial info enhancing mechanism to overcome weakened local information while capturing long-range dependencies. This mechanism balances local and global information at different expansion rates by exploring multiscale feature fusion in the feature maps. Furthermore, we introduce a spatial info sensing module to enhance the compatibility of multimodal features in long-range dependencies and extract informative cues from depth features. Through comprehensive experiments on four widely used datasets, we demonstrate that our proposed involution encoder significantly outperforms previous state-of-the-art RGB-D salient object detection methods based on CNNs in four key metrics. Compared to transformer-based methods, our approach balances speed and efficiency favorably.

The primary aim of human pose estimation involves accurately identifying key points on the human body, which is integral for various visual applications that require an in-depth understanding of human behavior. While high-resolution networks have excelled in this domain, their limitations, such as inadequate cross-dimensional information interaction and substantial computational costs, have prompted the need for more efficient solutions. To tackle these challenges, we introduce an adaptive cross-dimensional weighting high-resolution network (ACW-HRNet). This improved approach combines two key methods, cross-dimensional split convolution and adaptive context modeling (ACM). Cross-dimensional split convolution establishes effective cross-dimensional information exchange between spatial and channel, whereas ACM enhances the network’s ability to capture intricate spatial relationships through adaptive transformations and spatial weighting of input features. These make the network extract multi-scale context information and establish cross-dimensional dependencies, improving accuracy without introducing additional computational complexity. Our experiments on the COCO, MPII, and CrowdPose human pose estimation datasets illustrate its superior performance compared to mainstream lightweight networks.

Feature detection on multi-surface textureless metal parts is a common and crucial task in smart manufacturing. Traditional visual detection methods typically detect only one surface at a time, leading to inefficiency. In this work, we propose a framework for the rapid detection of multi-surface features on metal parts based on a non-parallel imaging method. The framework is built upon the principles of wireframe modeling and feature modeling, combining a lightweight neural network with a traditional template matching method. First, the line segment profile features of metal parts are extracted based on a lightweight neural network. The part is then represented by the correlation descriptors of the line segments. The initial bitmap of the part is obtained through template matching, followed by obtaining the accurate six-dimensional (6D) bitmap of the metal part through iterative matching of endpoints. Subsequently, utilizing the 6D pose information, the visible surface of the part undergoes transformation through orthographic projection, and the conformity of the transformed surface is assessed. Finally, experimental validation was conducted using a hydraulic valve as the object. The experimental results indicated that our method accurately estimates the bit-pose of textureless images. Moreover, it can concurrently perform visual detection tasks on multiple visible surfaces by employing a straightforward orthographic projection transformation.

The advancement of adversarial attack techniques, particularly against neural network architectures, is a crucial area of research in machine learning. Notably, the emergence of vision transformers (ViTs) as a dominant force in computer vision tasks has opened avenues for exploring their vulnerabilities. In this context, we introduce dual gradient optimization for adversarial transferability (DGO-AT), a comprehensive strategy designed to enhance the transferability of adversarial examples in ViTs. DGO-AT incorporates two innovative components: attention gradient smoothing (AGS) and multi-layer perceptron gradient random dropout (GRD-MLP). AGS targets the attention layers of ViTs to smooth gradients and reduce noise, focusing on global features for improved transferability. GRD-MLP, on the other hand, introduces stochasticity into MLP gradient updates, broadening the adversarial examples’ applicability. The synergy of these strategies in DGO-AT addresses the unique structural aspects of ViTs, leading to more effective and transferable adversarial attacks. Our comprehensive evaluations of a variety of ViT and CNN models, using the ImageNet dataset, demonstrate that DGO-AT significantly enhances the effectiveness and transferability of attacks, thereby contributing to the ongoing discourse on the adversarial robustness of advanced neural network models.

Digital images captured under insufficient lighting conditions may suffer from issues such as low contrast and poor visual quality. However, transformers treat images as one-dimensional sequential data, lacking the modeling of local visual structures, thus resulting in a shortage of feature extraction for degraded low-light images. In addition, transformer-based methods require longer training schedules to achieve better performance. We introduce a novel method, named local enhancement transformer (LET). By incorporating convolutions into transformer blocks, we improve our model’s capability to extract features from degraded low-light images. Furthermore, we propose a multi-level enhancement block to adaptively fuse features with learnable correlations among different levels. With the support of these two designs, LET can extract more useful features while requiring less training time. Experimental evaluations conducted on LOL and MIT-5K datasets prove that LET is superior to the state-of-the-art.

To realize the real-time detection of vehicle targets in an edge computing environment, we improved the YOLOv5Nano (YOLOv5n) model to develop a lightweight, high-precision, and real-time detection model called slimming, CBAM, distillation, YOLO (SCD-YOLO). By introducing the convolutional block attention mechanism, we aimed to increase the attention devoted to channel and spatial feature information, thereby improving feature extraction capabilities. The adoption of the slimming pruning algorithm further improved the weight and computational efficiency of the model. Finally, in the fine-tuning stage of the model, knowledge distillation technology was applied to use a model with a large number of parameters and high accuracy as a teacher model to guide the pruned model to compensate for a loss of accuracy. Experimental results demonstrate that compared with the original YOLOv5n model, on the University at Albany Detection and Tracking vehicle dataset, SCD-YOLO reduced the parameter count by 44.4% (approximately 4M parameters) and the calculation count by 40.4% while increasing processing speed by 14.7% with an accuracy loss of only 0.5%, which meets the requirements of real-time vehicle target detection in an edge computing environment.

Nighttime images suffer from exposure imbalance, leading to significant challenges in nighttime semantic segmentation. Given this, we propose a frequency-based dual attention network (FDANet) for nighttime semantic segmentation. To address the problem of targets hidden in the background caused by uneven exposure and low contrast that are difficult to distinguish, we propose a multi-spectral dual attention module to deeply explore hidden features from frequency information. In addition, to further utilize the relationship between frequency domain information and spatial domain features and guide the extraction of contextual features, spatial-frequency aggregation module is proposed to align and fuse features from different domains. Moreover, self-attention dilated convolution pyramid module is proposed to expand the effective receptive field and enhance the ability to identify multi-scale targets under uneven exposure. Extensive experiments on NightCity, ACDC, and BDD show that FDANet achieves an impressive balance between accuracy and speed. Specifically, FDANet achieves 53.76% MIoU, 49.47% MIoU, and 39.70% MIoU on NightCity, ACDC-Night, and BDD-Night, respectively. Notably, it attains these results with remarkable processing speeds of 111 FPS, 66 FPS, and 104 FPS for input resolutions of 512×1024, 1080×1920, and 720×1280, respectively.

In recent years, unsupervised person re-identification (re-ID) has attracted a lot of attention because it can save manual annotation costs and adapt to new scenes more conveniently in real-world applications. We focus on fully unsupervised learning-based re-ID, which aims to train a discriminative model based on unlabeled person images. In unsupervised learning, we need to generate pseudo labels by clustering convolutional features and then train convolutional neural network (CNN) models with these pseudo labels. The features used in the clustering process play an important role to ensure the quality of pseudo labels. Hence, we propose an enhancing feature extraction method to increase the reliability of generated pseudo labels and thus facilitate CNN model training. In order to enrich the obtained features, we carry out the feature extraction from both global and local aspects. The global features are extracted with ResNet50 backbone as many existing methods do. The local features are extracted by an additional part-based feature pyramid structure, in which the person image is divided into three parts and the features are extracted from each part with a pyramid structure. Then, we fuse the multi-layer pyramid features for each part as the local features. According to the joint global features and local features, the pseudo labels are predicted using clustering algorithms and further refined based on the similarity between global and local features. In addition, we design an inter-camera association learning component to effectively learn the ID discrimination ability across cameras. Extensive experiments on three large and representative person re-ID datasets demonstrate the effectiveness of the proposed approach.

Fourier single-pixel imaging (FSI) produces images through the acquisition of Fourier domain information using a single-pixel detector and a sequence of light modulation patterns. Conventional sampling approach in FSI tends to perform poorly when it comes to capturing the intricate details present in high-frequency components of the image. The variable density sampling method follows a predefined mechanism where the power of image information decreases when frequency increases. To enhance the sampling efficiency, we propose a self-adaptive sampling method to dynamically determine the order of the illumination pattern based on the scene’s spectrum distribution through the probability estimation of the low-frequency samples. The image is subsequently reconstructed through compressed sensing technique. Our results indicate improved image quality even at low sampling ratios. Unlike existing adaptive approaches, the proposed method does not require dataset training or redundant sampling. Instead, it exhibits remarkable versatility by adapting to various types of images.

Few-shot object detection aims to achieve object localization and recognition on novel classes with limited training instances. Due to the constraints of the two-stage fine-tuning mechanism, existing models lack the ability of knowledge reasoning. When transferring the base model to novel class detection, we add a region of interest feature transfer branch, which establishes a message transfer mechanism between complex instances, ensuring mutual attraction between instances of the same category while allowing for association across different categories. Specifically, a self-attention message transfer graph is constructed to facilitate the propagation of attribute information among target instances. Second, a box transfer loss function is proposed to combine the semantic relationships among instances to promote mutual exclusion among instances with significant category attribute bias, thereby constructing better category feature representations. Finally, we demonstrate the effectiveness of our proposed framework compared to other state-of-the-art methods on two popular datasets: PASCAL VOC and MS-COCO.

Data augmentation has been proven to be an effective regularization strategy that can reduce over-fitting risks in deep learning models. Erasure based data augmentation is one of the most advanced solutions; however, random region erasure inevitably leads to excessive loss of object information and the introduction of a large amount of negative noise. In this work, a data augmentation method based on self-transformation of encoded image subregions is proposed. This method first designs an encoding mechanism based on random strategy to determine the object sub-regions of the transformation. Then, self-morphing strategies such as random cropping, rotation, and channel transformation are applied to the sub-regions to maintain the original feature information conditions and increase the local variance characteristics of the samples. Finally, we perform erasure on the non-discriminative triangular region generated after rotation of the object region. The proposed method is easy to implement, and a large number of experiments were conducted on challenging datasets. The algorithm achieved significant improvements in classification performance. On the CIFAR-100 dataset, the top-1 error rate decreased by 2.83%. For object detection, the average detection accuracy on the PASCAL VOC dataset increased from 79.83% to 82.15%, with the highest average accuracy improvement of 5.32% for a single category.

Restoring images captured in adverse weather conditions can significantly enhance their visual quality, with widespread applications in fields such as autonomous driving. However, we observed a potential threat after image restoration: while the restored images are visually better, they may be adversarial for deep neural networks (DNNs). In light of this discovery, in this work, we explore a restorative attack approach, aiming to introduce adversarial perturbations in insensitive regions of the image while restoring images. In particular, we propose the deceptive deraining attack (DDA) method to generate more natural adversarial examples in the deraining scenario, denoted as DDA. In contrast to existing adversarial attack methods that degrade image quality, our DDA can efficiently introduce adversarial perturbations while enhancing the visual quality of rainy images. Specifically, we first restore the rainy image and derive the rain streak mask. Then the adversarial perturbations are added to the rain streak areas of the image according to the mask, and the perturbations are optimized by maximizing the classification loss to generate the adversarial image. Extensive experiments on synthetic rain datasets and real rain datasets indicate that the adversarial examples generated by our DAA are visually natural and maintain a high attack success rate on state-of-the-art DNNs.

Photo response non-uniformity (PRNU), a unique trait of image sensors, can efficiently trace the camera source of a digital video. In response to the poor identification effect of network compression subsampling videos, a multi-scale neighborhood value shrinkage filtering algorithm based on Stein’s unbiased risk estimation and an adaptive edge structure-preserving smoothing filtering algorithm are proposed, and a double-weighted PRNU extraction model is constructed. This model first performs a multi-scale transformation based on the dual-tree complex wavelet of the video frame that skips the loop filter, uses the multi-scale neighborhood value shrinkage filtering algorithm based on Stein’s unbiased risk estimation to estimate all high-frequency sub-bands, obtains the noise residual, and uses the adaptive edge structure-preserving smoothing filter to smooth the complex noise residual. The maximum likelihood estimation method of the double-weight coefficient matrix based on the noise variance and quantization parameters is used to aggregate the noise residuals to obtain the PRNU. Verification is carried out on the public Vision database. The proposed model outperforms the current PRNU extraction model in source identification of compressed videos, as evidenced by the receiver operating characteristic curve, area under the curve, and Kappa coefficient values. It exhibits an accuracy rate that surpasses the best competing algorithm by 17.4%.

You only look once (YOLO) target detection algorithms have garnered substantial attention in engineering inspection tasks due to their remarkable accuracy and swift detection speed. However, adapting the YOLOv7 algorithm to engineering construction presents challenges in balancing lightweight design and high precision. This work introduces a YOLOv7 lightweight detection algorithm, leveraging partial convolution to efficiently reduce network parameters without compromising detection speed. Integrating partial convolution into the network’s backbone is a highly effective parameter reduction strategy. Additionally, we propose the SE_Conv module following quantitative analyses through experiments, thereby significantly enhancing the algorithm’s detection accuracy. To address the impact of low-quality data on detection effectiveness in engineering tasks, the advanced bounding box loss function WIoU is employed. In a practical application scenario involving helmet detection, the algorithm achieves a notable 23.4% reduction in parameters and a substantial 28.2% decrease in operations through partial convolution. Subsequent experiments with the SE_Conv module yield a noteworthy 1.4% improvement in results. Significantly, in head detection, our algorithm surpasses YOLOv8 algorithm by 2.6% and single shot multibox detector algorithm by 9.4%. These findings underscore the efficacy of our proposed algorithm in engineering applications.

In recent years, convolutional neural networks have excelled in image Moiré pattern removal, yet their high memory consumption poses challenges for resource-constrained devices. To address this, we propose the lightweight multi-scale network (LMSNet). Designing lightweight multi-scale feature extraction blocks and efficient adaptive channel fusion modules, we extend the receptive field of feature extraction and introduce lightweight convolutional decomposition. LMSNet achieves a balance between parameter numbers and reconstruction performance. Extensive experiments demonstrate that our LMSNet, with 0.77 million parameters, achieves Moiré pattern removal performance comparable to full high definition demoiréing network (FHDe2Net) with 13.57 million parameters.

The application of machine vision in object identification and classification has significantly enhanced recognition efficiency. Nevertheless, for non-ferrous scrap metals with poor surface smoothness, the unevenness of reflected light results in the generation of dark regions in the images, obscuring a considerable amount of detailed information and reducing the recognition accuracy. Addressing these challenges, we propose a method for enhancing the details of dark regions based on the RGB-NIR image fusion theory, integrating detailed information from NIR images into RGB images. First, a robust deep residual denoising network is constructed to estimate and remove noise in images. Subsequently, to address the difficulty of extracting structural features in dark regions, a multi-scale spatial deep structure feature extraction module based on channel attention blocks is developed. This module effectively extracts the structural features of RGB and NIR image pairs, with the target image serving as the supervisory signal. Finally, guided by the theory of structural inconsistency, multi-scale feature maps are fused. The image fusion network adopts an encoder-decoder architecture embedded with residual channel attention blocks. The experimental results indicate that the approach proposed in this study demonstrates notable efficacy in image denoising and detail enhancement.

Quick and accurate detection of insulator defects from the complex aerial background (such as trees, hillsides, lakes, and buildings) is important work to ensure the safe operation of transmission lines. The existing detection methods have difficulty detecting the defect target due to the strong interference of complex backgrounds in aerial images. To solve this problem, we propose an insulator defect detection model based on a cascaded network. First, we introduce a hierarchical semantic segmentation network to separate the complex background from the target insulator, which is embedded into the main feature extraction branch to form a “segmentation-detection” cascade network to solve the interference problem of complex background when extracting target information; Second, aiming at the problem of direct fusion of conflicting information in different feature layers in the bi-directional path aggregation neck structure in the detection network, we propose an across-scale feature pyramid with feature refinement structure to enhance the information characteristics of insulator defect targets and improve the multi-scale expression ability of the network. Then, aiming at the problem of difficult samples and imbalance of positive and negative samples in the process of target detection, we propose a focal shape intersection over union loss (focal-SIOU-loss), which improves the precision and stability of the regression process by introducing the weight adjustment mechanism of focal loss and the structural similarity of SIOU Loss. Finally, the experimental results show that, compared with the standard detection models such as YOLOv5, YOLO7, and YOLOv8, the proposed detection model achieves a better performance in the precision, recall rate, and robustness in detecting insulator defects under complex backgrounds.

Motion information plays a vital role in video action recognition, serving as the fundamental building basis for the accurate interpretation of dynamic sequences. However, extracting accurate motion details remains a significant challenge for two-dimensional (2D) CNNs. To address this issue, we present an action recognition framework, named accurate motion understanding network (AMU-Net), designed to effectively perceive and aggregate valuable motion cues. Specifically, AMU-Net is a 2D CNN equipped with the proposed accurate motion perceptron (AMP) and action graph module (AGM). To capture finer local motion details, the AMP is introduced to handle motion noise in temporal differences. This module enables the extraction of critical local motion patterns from bidirectional temporal differences and enhances action-related features. Furthermore, to learn more precise global motion representations, the AGM is introduced to address the spatial sparsity of motion objects by detecting motion objects and selectively aggregating their features using a graph reasoning framework. Extensive experiments are conducted on three public benchmarks: ActivityNet-200, UCF-101, and Kinetics-400. Experimental results demonstrate that the proposed AMU-Net (based on ResNet-50) outperforms recent 2D CNN-based methods with a comparable computational overhead. In addition, the experimental results also show the effective transferability of the two modules to three popular lightweight convolutional architectures, emphasizing their versatility.

To guarantee the safety and efficiency of industrial production and prevent accidents or losses caused by personnel negligence or negligence, this work proposes a personnel on-duty status recognition method. The method combines a human pose estimation algorithm and a target detection algorithm, which can automatically discriminate six states of personnel on duty. First, the original image is processed using a high-resolution network (HRNet) to generate human pose keypoint maps. Then SE-VGG16 is constructed by combining the squeeze-excitation network and VGG16 for feature extraction of human pose keypoint maps. Finally, the design of the lightweight convolutional neural network for primary classification and you only look once version 5 is used for reclassification for behaviors with similar action features. The experimental results show that the method has an average recognition accuracy of 98.27% with good robustness and generalization ability for six kinds of personnel on-duty status in multiple environments.

Real-time semantic segmentation is critical in industries, such as autonomous driving and robotics, requiring both accuracy and speed. However, existing real-time segmentation algorithms often sacrifice low-level details to improve inference speed, leading to decreased segmentation accuracy. Therefore, we propose a new real-time semantic segmentation model dual interaction fusion network (DIFNet) to alleviate this problem. First, we propose a lightweight dual decoding fusion structure, which increases the focus on the low-level feature information and can extract richer edge details, while the structure reduces the computational overhead by decreasing the number of channels of the feature map during fusion. In addition, we construct a cross attention module to cross-weight fusion of high-level and low-level features through attention mechanism, which increases the interaction between features and effectively extracts features at different levels. Finally, we design a comprehensive perception module that introduces dilated convolution to expand the model’s receptive field, enabling it to better capture global features. Our network was validated on the Cityscapes and CamVid datasets. Specifically, on a single Nvidia GTX 2080 Ti, DIFNet achieves 77.6% mIoU at 83.9 frames per second (FPS) for 1536×768 inputs on Cityscapes test set and 77.0% mIoU at 135.8 FPS for 960×720 inputs on CamVid.

With the rapid development of object detection technology, underwater object detection has gradually become a hot topic. Due to the complex underwater environment, some object detection algorithms still encounter difficulties in detecting targets similar to the background. For the above problems, we propose a multi-scale feature-weighted dual-neck network (MFDNN) for underwater object detection. Our contribution is mainly divided into three parts. First, an enhanced feature extraction network, namely the dual-neck network, is designed to process and reuse the features extracted from the backbone network. Second, an attention mechanism is embedded in one of the neck networks to reweight features and pay more attention to important features. In addition, we introduce the adaptively spatial feature fusion mechanism to adaptively weight the features extracted at multiple scales. As demonstrated in comprehensive experiments, the mean average precision of our MFDNN can reach 87.79% and 86.51% on the underwater datasets URPC2019 and URPC2020, respectively.

Small-object detection presents a formidable challenge in object detection. While object detectors leveraging convolutional neural networks have shown remarkable advancements, the downsampling of images in current detectors results in the loss of spatial domain information. Addressing this issue, we propose SFDet, a small-object detection method that employs an attention mechanism shifting from the spatial to the frequency domain, specifically optimized for small-object detection in underwater images. Specifically, our approach incorporates a fusion mechanism that combines image enhancement networks for semantic enhancement and extracts a composite representation of spatial and frequency domain components to enhance small-object detection accuracy. We evaluate our proposed approach on four publicly available datasets, and the results demonstrate its superior performance compared with other methods. The code is available at: https://github.com/fadaishaitaiyang/SFDet.git

Precipitation forecasting has always been a challenging problem. Currently, meteorological radar echo data are used widely in the field of precipitation forecasting. From the radar echo images, the current precipitation situation can be obtained. Compared with the actual image, however, the radar echo extrapolation image has the disadvantages of echo loss and inaccurate echo tracks, which leads to low accuracy of the precipitation forecast. A more effective network based on the New-RainNet module for precipitation forecasting is proposed. The network is constructed by stacking New-RainNet modules that are established by adding a convolutional block attention module and a flexible switch from Adam to SGD optimization algorithm to the original convolutional long short-term memory network. The short-term and impending precipitation forecast in the next 0 to 60 min was successfully realized on the radar echo dataset of the Hong Kong Observatory and achieves satisfactory results. Experiments show that the network surpasses other methods in both network convergence and prediction accuracy. In addition, the radar echo image generated by the network prediction is also better than other methods in visual quality as it retains the image details and is closer to the real radar echo image than other methods.

Continuous sign language recognition (CSLR) is a challenging research task due to the lack of accurate annotation on the temporal sequence of sign language data. The recent popular usage is a hybrid model based on “CNN + RNN” for CSLR. However, when extracting temporal features in these works, most of the methods using a fixed temporal receptive field and cannot extract the temporal features well for each sign language word. In order to obtain more accurate temporal features, we propose a multiscale temporal network. The network mainly consists of three parts. The ResNet and two fully connected layers constitute the frame-wise feature extraction part. The time-wise feature extraction part performs temporal feature learning by first extracting temporal receptive field features of different scales using the proposed multiscale temporal block (MST-block) to improve the temporal modeling capability, and then further encoding the temporal features of different scales by the transformers module to obtain more accurate temporal features. Finally, the proposed multilevel connectionist temporal classification (CTC) loss part is used for training to obtain recognition results. The multilevel CTC loss enables better learning and updating of the shallow network parameters in CNN, which has no parameter increase and can be flexibly embedded in other models. Experimental results on two publicly available datasets demonstrate that the method can extract sign language features in an end-to-end manner without any prior knowledge effectively, improve the accuracy of CSLR, and achieve competitive results.

In military training, the shooting target reporting system faces several challenges, including difficulty grasping the target surface, low real-time judgment of bullet hole ring value, and susceptibility to external environmental factors. To address these issues, this work proposes an approach that combines deep learning target detection and semantic segmentation algorithms with the traditional YOLOv5 target detection model. The model is enhanced into a multi-task architecture, featuring a backbone network encoder and two decoders, to detect and segment the chest ring target area. To counteract the impact of external environmental interference, the work employs scale-invariant feature transform feature point matching and image correction to eliminate tilting and jittering. The TPH-YOLOv5 network framework is enhanced by integrating the Swin-transformer coding structure, enabling accurate detection of bullet holes and extraction of local and global features from the chest target image. In addition, a center distance judgment method is designed based on arc adjacency matrix ellipse detection to identify the boundary area of the tenth ring of the chest ring target. By measuring the distance from the center of the bullet hole to the center of the target, this work achieves precise judgment of the bullet hole score. The experimental results demonstrate the effectiveness of the proposed system. After 300 training iterations, the accuracy of target frame detection reaches 0.94, with a recall rate of 0.92. The precision and recall of bullet hole detection reach 0.98 and 0.97, respectively, with a frame rate of 45 fps. The shooting target reporting system developed in this study shows promising potential, as evidenced by testing and validation in a simulated real-world shooting environment.

Significant progress has been made in no-reference image quality assessment in terms of color harmony in recent years. However, most methods do not consider color contrast and lose some color information in the quantification process, so they perform poorly when evaluating the color harmony of color images. An image quality assessment model based on color descriptors is established under the inspiration of natural scene statistics. The model introduces local color contrast features of color fusion images and combines them with visual attention mechanism. The local color statistics coefficients of images in both RGB color space and CIELAB color space are calculated to reduce the loss of color information. Furthermore, a multi-scale content-aware network is used to extract multi-scale content features and establish the connection between color features and image content. Finally, the image quality score is obtained by the quality assessment network. The dataset validation and experimental results reveal that the proposed model, compared with the existing 15 methods, has good agreement with human visual perception and better performance.

Many deblurring and blur kernel estimation methods use a maximum a posteriori approach or deep learning-based classification techniques to sharpen an image and/or predict the blur kernel. We propose a regression approach using convolutional neural networks (CNNs) to predict parameters of linear motion blur kernels, the length and orientation of the blur. We analyze the relationship between length and angle of linear motion blur that can be represented as digital filter kernels. A large dataset of blurred images is generated using a suite of blur kernels and used to train a regression CNN for prediction of length and angle of the motion blur. The coefficients of determination for estimation of length and angle are found to be greater than or equal to 0.89, even under the presence of significant additive Gaussian noise, up to a variance of 10% (SNR of 10 dB). Using our estimated kernel in a nonblind image deblurring method, the sum of squared differences error ratio demonstrates higher cumulative histogram values than comparison methods, with most test images yielding an error ratio of less than or equal to 1.25.

Point cloud denoising presents several complex challenges. First, most existing denoising methods are capable of addressing either large-scale or small-scale noise, but not both effectively. Second, the parameters in various algorithms are often difficult to fine-tune. Third, certain noise points are so proximate to normal points that they are challenging to distinguish and remove. To tackle these issues, we introduce a point cloud denoising approach based on the analysis of a point’s neighborhood radius and gravitation. Through the analysis of a point’s neighborhood radius, this method can automatically discern the presence of large-scale noise within the point cloud. For the point cloud with large-scale and small-scale noise, this method can separate the large-scale noise by analyzing the distribution characteristics of the radius of a point’s neighborhood, leaving only small-scale noise to be addressed. For small-scale noise, the gravitation analysis method first analyzes the gravitation force on each doubtful point and then uses the force values to ascertain whether a point is indeed noise. Therefore, it can automatically identify and deal with point clouds with different noise scales. Experiments show this method can effectively remove multi-scale noise, and the average noise removal accuracy exceeds 98%. In addition, the absence of a need for manual parameter-setting enhances its practicality.

Erratum corrects errors in author’s name and Fig. 1.