Author name cluster

Zheng Zhu

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

19 papers

2 author rows

EAAI Journal 2026 Journal Article

Automatic estimation of lactate threshold heart rate and pace in real-world running based on transfer learning

Zheng Zhu
Wei Cui
Changda Lu
Yanfei Shen
Bingyu Pan

This study proposes a novel transfer learning-based approach for automatically estimating lactate threshold heart rate (LTHR) and pace (LTP) during real-world running. We first designed a graded exercise test (GXT) to collect physiological data. The model, constructed using a Recurrent Neural Network (RNN), employs hierarchical sampling during training to enhance accuracy. The model achieves mean absolute error (MAE) values of 4. 37 beats per minute (bpm) for LTHR and 0. 36 km per hour (km/h) for LTP. Subsequently, real-world running data undergo segmentation, filtering, and similarity-based selection to construct features, achieving the MAE of 9. 18 bpm and 1. 23 km/h for LTHR and LTP. Additionally, longitudinal tracking was conducted for several participants over 28 days, utilizing their daily running records to monitor the longitudinal changes in LTHR and LTP.

Details DOI

AIIM Journal 2026 Journal Article

Building trustworthy large language model-driven generative recommender system for healthcare decision support: A scoping review of corpus sources, customization techniques, and evaluation frameworks

Shuqi Yang
Mingrui Jing
Shuai Wang
Zongan Huang
Jiaqing Wang
Jiaxin Kou
Manfei Shi
Zhentao Xia

Introduction Large Language Model-Driven Generative Recommender Systems (LLM-GRSs) are playing a growing role in healthcare, particularly in clinical question-answering. This study reviews their corpus sources, customization techniques, and evaluation metrics. Methods We conducted a systematic search of PubMed, Embase, Scopus, and Web of Science for studies published between January 2021 and August 2025 that applied LLM-GRSs to deliver medical or healthcare information. Eligible studies included publications describing LLMs designed to emulate clinical decision-making by providing diagnostic or therapeutic recommendations through dialogue-based interfaces. Two reviewers independently screened studies and extracted data on corpus sources, model architectures, customization methods, and evaluation metrics. Results A total of 61 articles were included. Corpus sources were grouped into clinical data (n = 25), literature (n = 34), open datasets (n = 37), and web-crawled data (n = 15), with many using multiple types. Most studies (n = 43) combined multiple approaches. Customization techniques included prompt engineering, retrieval-augmented generation and model fine-tuning. Twenty-four studies used a single customization technique, while 37 studies combined these methods during model development. The evaluation metrics were classified into three main domains: process metrics, usability metrics, and outcome metrics. The outcome metrics included both model-based and manual-assessed evaluations. Conclusion LLM-GRSs hold considerable promise in healthcare; however, their safety and reliability hinge on the use of evidence-based training corpora, transparent system design, and standardized evaluation protocols within real-world clinical environments.

Details DOI

IROS Conference 2025 Conference Paper

Adjacent-view Transformers for Supervised Surround-view Depth Estimation

Xianda Guo
Wenjie Yuan
Yunpeng Zhang
Tian Yang
Chenming Zhang
Zheng Zhu
Qin Zou 0001
Long Chen 0005

Depth estimation has been widely studied and serves as the fundamental step of 3D perception for robotics and autonomous driving. Though significant progress has been made in monocular depth estimation in the past decades, these attempts are mainly conducted on the KITTI benchmark with only front-view cameras, which ignores the correlations across surround-view cameras. In this paper, we propose an Adjacent-View Transformer for Supervised Surround-view Depth estimation (AVT-SSDepth), to jointly predict the depth maps across multiple surrounding cameras. Specifically, we employ a global-to-local feature extraction module that combines CNN with transformer layers for enriched representations. Further, the adjacent-view attention mechanism is proposed to enable the intra-view and inter-view feature propagation. The former is achieved by the self-attention module within each view, while the latter is realized by the adjacent attention module, which computes the attention across multi-cameras to exchange the multi-scale representations across surround-view feature maps. In addition, AVT-SSDepth has strong cross-dataset generalization. Extensive experiments show that our method achieves superior performance over existing state-of-the-art methods on both DDAD and nuScenes datasets. Code is available at https://github.com/XiandaGuo/SSDepth.

Details

AAAI Conference 2025 Conference Paper

DetRF: Detachable Novel Views Synthesis of Dynamic Scenes Using Backdrop-Driven Neural Radiance Fields

Boyu Zhang
Zheng Zhu
Wenbo Xu

Representing and synthesizing novel views in real-world dynamic scenes from casual monocular videos is a long-standing problem. Existing solutions typically approach dynamic scenes by applying geometry techniques or utilizing temporal information between several adjacent frames without considering the underlying background distribution in the entire scene or the transmittance over the ray dimension, limiting their performance on static and occlusion areas. Our approach backdrop-driven neural radiance fields offers high-quality view synthesis and a 3D solution to detach the background from the entire dynamic scene, which is called DetRF. Specifically, it employs a neural representation to capture the scene distribution in the static background and a 6D-input NeRF to represent dynamic objects, respectively. Each ray sample is given an additional occlusion weight to indicate the transmittance lying in the static and dynamic components. We evaluate DetRF on public dynamic scenes and our urban driving scenes acquired from an autonomous-driving dataset. Extensive experiments demonstrate that our approach outperforms previous methods in rendering texture details and motion areas while also producing a clean static background. Code will be available soon.

PDF Details DOI

AAAI Conference 2025 Conference Paper

DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generation

Guosheng Zhao
Xiaofeng Wang
Zheng Zhu
Xinze Chen
Guan Huang
Xiaoyi Bao
Xingang Wang

World models have demonstrated superiority in autonomous driving, particularly in the generation of multi-view driving videos. However, significant challenges still exist in generating customized driving videos. In this paper, we propose DriveDreamer-2, which incorporates a Large Language Model (LLM) to facilitate the creation of user-defined driving videos. Specifically, a trajectory generation function library is developed to produce trajectories that conform to user descriptions. Subsequently, an HDMap generator is designed to learn the mapping from trajectories to road structures. Ultimately, we propose the Unified Multi-View Model (UniMVM) to enhance temporal and spatial coherence in the generated multi-view driving videos. To the best of our knowledge, DriveDreamer-2 is the first world model to generate customized driving videos, and it can generate uncommon driving videos (e.g., vehicles abruptly cut in) in a user-friendly manner. Besides, experimental results demonstrate that the generated videos enhance the training of driving perception methods (e.g., 3D detection and tracking). Furthermore, video generation quality of DriveDreamer-2 surpasses other state-of-the-art methods, showcasing FID and FVD scores of 11.2 and 55.7, representing relative improvements of ~30% and ~50%.

PDF Details DOI

NeurIPS Conference 2025 Conference Paper

EgoVid-5M: A Large-Scale Video-Action Dataset for Egocentric Videos Generation

Xiaofeng Wang
Kang Zhao
Feng Liu
Jiayu Wang
Guosheng Zhao
Xiaoyi Bao
Zheng Zhu
Yingya Zhang

Video generation has emerged as a promising tool for world simulation, leveraging visual data to replicate real-world environments. Within this context, egocentric video generation, which centers on the human perspective, holds significant potential for enhancing applications in virtual reality, augmented reality, and gaming. However, the generation of egocentric videos presents substantial challenges due to the dynamic nature of first-person viewpoints, the intricate diversity of actions, and the complex variety of scenes encountered. Existing datasets are inadequate for addressing these challenges effectively. To bridge this gap, we present EgoVid-5M, the first high-quality dataset specifically curated for egocentric video generation. EgoVid-5M encompasses over 5 million egocentric video clips and is enriched with detailed action annotations, including fine-grained kinematic control and high-level textual descriptions. To ensure the integrity and usability of the dataset, we implement a sophisticated data cleansing pipeline designed to maintain frame consistency, action coherence, and motion smoothness under egocentric conditions. Furthermore, we introduce EgoDreamer, which is capable of generating egocentric videos driven simultaneously by action descriptions and kinematic control signals. The EgoVid-5M dataset, associated action annotations, and all data cleansing metadata will be released for the advancement of research in egocentric video generation.

PDF Details

NeurIPS Conference 2025 Conference Paper

Pruning-Robust Mamba with Asymmetric Multi-Scale Scanning Paths

Jindi Lv
Yuhao Zhou
Mingjia Shi
Zhiyuan Liang
Panpan Zhang
Xiaojiang Peng
Wangbo Zhao
Zheng Zhu

Mamba has proven efficient for long-sequence modeling in vision tasks. However, when token reduction techniques are applied to improve efficiency, Mamba-based models exhibit drastic performance degradation compared to Vision Transformers (ViTs). This decline is potentially attributed to Mamba's chain-like scanning mechanism, which we hypothesize not only induces cascading losses in token connectivity but also limits the diversity of spatial receptive fields. In this paper, we propose Asymmetric Multi-scale Vision Mamba (AMVim), a novel architecture designed to enhance pruning robustness. AMVim employs a dual-path structure, integrating a window-aware scanning mechanism into one path while retaining sequential scanning in the other. This asymmetry design promotes token connection diversity and enables multi-scale information flow, reinforcing spatial awareness. Empirical results demonstrate that AMVim achieves state-of-the-art pruning robustness. During token reduction, AMVim-T achieves a substantial 34\% improvement in training-free accuracy with identical model sizes and FLOPs. Meanwhile, AMVim-S exhibits only a 1. 5\% accuracy drop, performing comparably to ViT. Notably, AMVim also delivers superior performance during pruning-free settings, further validating its architectural advantages.

PDF Details

AAAI Conference 2024 Conference Paper

DiffBEV: Conditional Diffusion Model for Bird’s Eye View Perception

Jiayu Zou
Kun Tian
Zheng Zhu
Yun Ye
Xingang Wang

BEV perception is of great importance in the field of autonomous driving, serving as the cornerstone of planning, controlling, and motion prediction. The quality of the BEV feature highly affects the performance of BEV perception. However, taking the noises in camera parameters and LiDAR scans into consideration, we usually obtain BEV representation with harmful noises. Diffusion models naturally have the ability to denoise noisy samples to the ideal data, which motivates us to utilize the diffusion model to get a better BEV representation. In this work, we propose an end-to-end framework, named DiffBEV, to exploit the potential of diffusion model to generate a more comprehensive BEV representation. To the best of our knowledge, we are the first to apply diffusion model to BEV perception. In practice, we design three types of conditions to guide the training of the diffusion model which denoises the coarse samples and refines the semantic feature in a progressive way. What's more, a cross-attention module is leveraged to fuse the context of BEV feature and the semantic content of conditional diffusion model. DiffBEV achieves a 25.9% mIoU on the nuScenes dataset, which is 6.2% higher than the best-performing existing approach. Quantitative and qualitative results on multiple benchmarks demonstrate the effectiveness of DiffBEV in BEV semantic segmentation and 3D object detection tasks.

PDF Details DOI

AAAI Conference 2024 Conference Paper

One at a Time: Progressive Multi-Step Volumetric Probability Learning for Reliable 3D Scene Perception

Bohan Li
Yasheng Sun
Jingxin Dong
Zheng Zhu
Jinming Liu
Xin Jin
Wenjun Zeng

Numerous studies have investigated the pivotal role of reliable 3D volume representation in scene perception tasks, such as multi-view stereo (MVS) and semantic scene completion (SSC). They typically construct 3D probability volumes directly with geometric correspondence, attempting to fully address the scene perception tasks in a single forward pass. However, such a single-step solution makes it hard to learn accurate and convincing volumetric probability, especially in challenging regions like unexpected occlusions and complicated light reflections. Therefore, this paper proposes to decompose the complicated 3D volume representation learning into a sequence of generative steps to facilitate fine and reliable scene perception. Considering the recent advances achieved by strong generative diffusion models, we introduce a multi-step learning framework, dubbed as VPD, dedicated to progressively refining the Volumetric Probability in a Diffusion process. Specifically, we first build a coarse probability volume from input images with the off-the-shelf scene perception baselines, which is then conditioned as the basic geometry prior before being fed into a 3D diffusion UNet, to progressively achieve accurate probability distribution modeling. To handle the corner cases in challenging areas, a Confidence-Aware Contextual Collaboration (CACC) module is developed to correct the uncertain regions for reliable volumetric learning based on multi-scale contextual contents. Moreover, an Online Filtering (OF) strategy is designed to maintain representation consistency for stable diffusion sampling. Extensive experiments are conducted on scene perception tasks including multi-view stereo (MVS) and semantic scene completion (SSC), to validate the efficacy of our method in learning reliable volumetric representations. Notably, for the SSC task, our work stands out as the first to surpass LiDAR-based methods on the SemanticKITTI dataset.

PDF Details DOI

IJCAI Conference 2024 Conference Paper

Unified Single-Stage Transformer Network for Efficient RGB-T Tracking

Jianqiang Xia
Dianxi Shi
Ke Song
Linna Song
Xiaolei Wang
Songchang Jin
Chenran Zhao
Yu Cheng

Most existing RGB-T tracking networks extract modality features in a separate manner, which lacks interaction and mutual guidance between modalities. This limits the network's ability to adapt to the diverse dual-modality appearances of targets and the dynamic relationships between the modalities. Additionally, the three-stage fusion tracking paradigm followed by these networks significantly restricts the tracking speed. To overcome these problems, we propose a unified single-stage Transformer RGB-T tracking network, namely USTrack, which unifies the above three stages into a single ViT (Vision Transformer) backbone through joint feature extraction, fusion and relation modeling. With this structure, the network can not only extract the fusion features of templates and search regions under the interaction of modalities, but also significantly improve tracking speed through the single-stage fusion tracking paradigm. Furthermore, we introduce a novel feature selection mechanism based on modality reliability to mitigate the influence of invalid modalities for final prediction. Extensive experiments on three mainstream RGB-T tracking benchmarks show that our method achieves the new state-of-the-art while achieving the fastest tracking speed of 84. 2FPS. Code is available at https: //github. com/xiajianqiang/USTrack.

PDF Details DOI

AAAI Conference 2023 Conference Paper

A Simple Baseline for Multi-Camera 3D Object Detection

Yunpeng Zhang
Wenzhao Zheng
Zheng Zhu
Guan Huang
Jiwen Lu
Jie Zhou

3D object detection with surrounding cameras has been a promising direction for autonomous driving. In this paper, we present SimMOD, a Simple baseline for Multi-camera Object Detection, to solve the problem. To incorporate multiview information as well as build upon previous efforts on monocular 3D object detection, the framework is built on sample-wise object proposals and designed to work in a twostage manner. First, we extract multi-scale features and generate the perspective object proposals on each monocular image. Second, the multi-view proposals are aggregated and then iteratively refined with multi-view and multi-scale visual features in the DETR3D-style. The refined proposals are endto-end decoded into the detection results. To further boost the performance, we incorporate the auxiliary branches alongside the proposal generation to enhance the feature learning. Also, we design the methods of target filtering and teacher forcing to promote the consistency of two-stage training. We conduct extensive experiments on the 3D object detection benchmark of nuScenes to demonstrate the effectiveness of SimMOD and achieve competitive performance. Code will be available at https://github.com/zhangyp15/SimMOD.

PDF Details DOI

AAAI Conference 2023 Conference Paper

Crafting Monocular Cues and Velocity Guidance for Self-Supervised Multi-Frame Depth Learning

Xiaofeng Wang
Zheng Zhu
Guan Huang
Xu Chi
Yun Ye
Ziwei Chen
Xingang Wang

Self-supervised monocular methods can efficiently learn depth information of weakly textured surfaces or reflective objects. However, the depth accuracy is limited due to the inherent ambiguity in monocular geometric modeling. In contrast, multi-frame depth estimation methods improve depth accuracy thanks to the success of Multi-View Stereo (MVS), which directly makes use of geometric constraints. Unfortunately, MVS often suffers from texture-less regions, non-Lambertian surfaces, and moving objects, especially in real-world video sequences without known camera motion and depth supervision. Therefore, we propose MOVEDepth, which exploits the MOnocular cues and VElocity guidance to improve multi-frame Depth learning. Unlike existing methods that enforce consistency between MVS depth and monocular depth, MOVEDepth boosts multi-frame depth learning by directly addressing the inherent problems of MVS. The key of our approach is to utilize monocular depth as a geometric priority to construct MVS cost volume, and adjust depth candidates of cost volume under the guidance of predicted camera velocity. We further fuse monocular depth and MVS depth by learning uncertainty in the cost volume, which results in a robust depth estimation against ambiguity in multi-view geometry. Extensive experiments show MOVEDepth achieves state-of-the-art performance: Compared with Monodepth2 and PackNet, our method relatively improves the depth accuracy by 20% and 19.8% on the KITTI benchmark. MOVEDepth also generalizes to the more challenging DDAD benchmark, relatively outperforming ManyDepth by 7.2%. The code is available at https://github.com/JeffWang987/MOVEDepth.

PDF Details DOI

ICLR Conference 2023 Conference Paper

Divide to Adapt: Mitigating Confirmation Bias for Domain Adaptation of Black-Box Predictors

Jianfei Yang
Xiangyu Peng
Kai Wang 0036
Zheng Zhu
Jiashi Feng
Lihua Xie 0001
Yang You 0001

Domain Adaptation of Black-box Predictors (DABP) aims to learn a model on an unlabeled target domain supervised by a black-box predictor trained on a source domain. It does not require access to both the source-domain data and the predictor parameters, thus addressing the data privacy and portability issues of standard domain adaptation methods. Existing DABP approaches mostly rely on knowledge distillation (KD) from the black-box predictor, i.e., training the model with its noisy target-domain predictions, which however inevitably introduces the confirmation bias accumulated from the prediction noises and leads to degrading performance. To mitigate such bias, we propose a new strategy, \textit{divide-to-adapt}, that purifies cross-domain knowledge distillation by proper domain division. This is inspired by an observation we make for the first time in domain adaptation: the target domain usually contains easy-to-adapt and hard-to-adapt samples that have different levels of domain discrepancy w.r.t. the source domain, and deep models tend to fit easy-to-adapt samples first. Leveraging easy-to-adapt samples with less noise can help KD alleviate the negative effect of prediction noises from black-box predictors. In this sense, the target domain can be divided into an easy-to-adapt subdomain with less noise and a hard-to-adapt subdomain at the early stage of training. Then the adaptation is achieved by semi-supervised learning. We further reduce distribution discrepancy between subdomains and develop weak-strong augmentation strategy to filter the predictor errors progressively. As such, our method is a simple yet effective solution to reduce error accumulation in cross-domain knowledge distillation for DABP. Moreover, we prove that the target error of DABP is bounded by the noise ratio of two subdomains, i.e., the confirmation bias, which provides the theoretical justifications for our method. Extensive experiments demonstrate our method achieves state of the art on all DABP benchmarks, outperforming the existing best approach by 7.0\% on VisDA-17, and is even comparable with the standard domain adaptation methods that use the source-domain data.

Details

ICRA Conference 2023 Conference Paper

Efficient and Hybrid Decoder for Local Map Construction in Bird'-Eye-View

Kun Tian
Yun Ye
Zheng Zhu
Peng Li
Guan Huang 0003

High-definition maps are crucial perception elements for autonomous robot navigation systems, which can provide accurate scene layout and environment information for downstream motion prediction and planning control tasks. Traditional methods based on manual annotation or SLAM algorithms require massive labor efforts and time costs, which hinders the deployment of practical applications. Online construction of local maps from on-board cameras offers an alternative solution. Aiming at the problems of unsatisfying precision and redundant computation of HDMapNet, we propose an efficient and hybrid decoder (EHD) that consists of a CNN-based segmentation (Seg) head and a query-based lane detection head (QLD). Specifically, the Seg head outputs pixel-level semantic maps, and QLD predicts instance mask for each lane object through learnable query embeddings. The designed decoding method eliminates the cumulative error caused by inaccurate semantic maps and does not require additional clustering algorithm for post-processing. Through combining with a variety of bird's-eye-view (BEV) encoders, the effectiveness and efficiency of our EHD is demonstrated by extensive experiments. For segmentation task, the mIoU scores of semantic map can be improved by 1. 3%∼2. 9%. Additionally, the accuracy of lane detection is also significantly increased (more than 10. 2% mAP) under all evaluation criteria. Since our method discards redundant post-processing, the inference speed is up to 22. 71 FPS, which is 32 times faster than HDMapNet.

Details

ICRA Conference 2023 Conference Paper

HFT: Lifting Perspective Representations via Hybrid Feature Transformation for BEV Perception

Jiayu Zou
Zheng Zhu
Junjie Huang 0005
Tian Yang
Guan Huang 0003
Xingang Wang 0003

Restoring an accurate Bird's Eye View (BEV) map plays a crucial role in the perception of autonomous driving. The existing works of lifting representations from frontal view to BEV can be classified into two categories, i. e. , Camera model-Based Feature Transformation (CBFT) and Camera model-Free Feature Transformation (CFFT). We empirically analyze the significant differences between CBFT and CFFT. The former method lift perspective features based on the flat- world assumption, which often causes distortion of regions lying above the ground plane. The latter method is limited in the perception performance due to the absence of geometric priors and time-consuming computing. In this paper, we propose a novel framework with a Hybrid Feature Transformation module (HFT) to lift perspective representations. Furthermore, we design a mutual learning scheme to augment hybrid transformation. The deformable attention mechanism enables the model to pay more attention to relevant regions and capture features with more semantics. We illustrate the effectiveness of HFT in BEV perception tasks, such as segmentation and object detection. Notably, in the task of semantic segmentation, extensive experiments demonstrate that HFT outperforms the previous state-of-the-art method by relatively 17. 9% on the Argoverse and 22. 0% on the KITTI 3D Object dataset. With negligible computing budget, HFT outperforms existing image- based methods on 3D object detection. The code will be released soon.

Details

NeurIPS Conference 2022 Conference Paper

OrdinalCLIP: Learning Rank Prompts for Language-Guided Ordinal Regression

Wanhua Li
Xiaoke Huang
Zheng Zhu
Yansong Tang
Xiu Li
Jie Zhou
Jiwen Lu

This paper presents a language-powered paradigm for ordinal regression. Existing methods usually treat each rank as a category and employ a set of weights to learn these concepts. These methods are easy to overfit and usually attain unsatisfactory performance as the learned concepts are mainly derived from the training set. Recent large pre-trained vision-language models like CLIP have shown impressive performance on various visual tasks. In this paper, we propose to learn the rank concepts from the rich semantic CLIP latent space. Specifically, we reformulate this task as an image-language matching problem with a contrastive objective, which regards labels as text and obtains a language prototype from a text encoder for each rank. While prompt engineering for CLIP is extremely time-consuming, we propose OrdinalCLIP, a differentiable prompting method for adapting CLIP for ordinal regression. OrdinalCLIP consists of learnable context tokens and learnable rank embeddings. The learnable rank embeddings are constructed by explicitly modeling numerical continuity, resulting in well-ordered, compact language prototypes in the CLIP space. Once learned, we can only save the language prototypes and discard the huge language model, resulting in zero additional computational overhead compared with the linear head counterpart. Experimental results show that our paradigm achieves competitive performance in general ordinal regression tasks, and gains improvements in few-shot and distribution shift settings for age estimation. The code is available at https: //github. com/xk-huang/OrdinalCLIP.

PDF Details

NeurIPS Conference 2021 Conference Paper

Global Filter Networks for Image Classification

Yongming Rao
Wenliang Zhao
Zheng Zhu
Jiwen Lu
Jie Zhou

Recent advances in self-attention and pure multi-layer perceptrons (MLP) models for vision have shown great potential in achieving promising performance with fewer inductive biases. These models are generally based on learning interaction among spatial locations from raw data. The complexity of self-attention and MLP grows quadratically as the image size increases, which makes these models hard to scale up when high-resolution features are required. In this paper, we present the Global Filter Network (GFNet), a conceptually simple yet computationally efficient architecture, that learns long-term spatial dependencies in the frequency domain with log-linear complexity. Our architecture replaces the self-attention layer in vision transformers with three key operations: a 2D discrete Fourier transform, an element-wise multiplication between frequency-domain features and learnable global filters, and a 2D inverse Fourier transform. We exhibit favorable accuracy/complexity trade-offs of our models on both ImageNet and downstream tasks. Our results demonstrate that GFNet can be a very competitive alternative to transformer-style models and CNNs in efficiency, generalization ability and robustness. Code is available at https: //github. com/raoyongming/GFNet

PDF Details

JBHI Journal 2021 Journal Article

Integrating Lung Parenchyma Segmentation and Nodule Detection With Deep Multi-Task Learning

Weihua Liu
Xiabi Liu
Huiyu Li
Mincan Li
Xinming Zhao
Zheng Zhu

Lung parenchyma segmentation is valuable for improving the performance of lung nodule detection in computed tomography (CT) images. Traditionally, the two tasks are performed separately. This paper proposes a deep multi-task learning (MTL) approach to integrate these tasks for better lung nodule detection. Three new ideas lead to our proposed approach. First, lung parenchyma segmentation is used as the attention module and is combined with nodule detection in a single deep network. Second, lung nodule detection is performed in an anchor-free manner by dividing it into two subtasks, nodule center identification and nodule size regression. Third, a novel pyramid dilated convolution block (PDCB) is proposed to utilize the advantage of dilated convolution and tackle its gridding problem for better lung parenchyma segmentation. Based on these ideas, we design our end-to-end deep network architecture and corresponding MTL method to achieve lung parenchyma segmentation and nodule detection simultaneously. We evaluate the proposed approach on the commonly used Lung Nodule Analysis 2016 (LUNA16) dataset. The experimental results show the value of our contributions and demonstrate that our approach can yield significant improvements compared with state-of-the-art counterparts.

Details DOI

AAAI Conference 2021 Conference Paper

SIMPLE: SIngle-network with Mimicking and Point Learning for Bottom-up Human Pose Estimation

Jiabin Zhang
Zheng Zhu
Jiwen Lu
Junjie Huang
Guan Huang
Jie Zhou

The practical application requests both accuracy and efficiency on multi-person pose estimation algorithms. But the high accuracy and fast inference speed are dominated by top-down methods and bottom-up methods respectively. To make a better trade-off between accuracy and efficiency, we propose a novel multi-person pose estimation framework, SIngle-network with Mimicking and Point Learning for Bottom-up Human Pose Estimation (SIMPLE). Specifically, in the training process, we enable SIMPLE to mimic the pose knowledge from the high-performance top-down pipeline, which significantly promotes SIMPLE’s accuracy while maintaining its high efficiency during inference. Besides, SIMPLE formulates human detection and pose estimation as a unified point learning framework to complement each other in single-network. This is quite different from previous works where the two tasks may interfere with each other. To the best of our knowledge, both mimicking strategy between different method types and unified point learning are firstly proposed in pose estimation. In experiments, our approach achieves the new state-of-the-art performance among bottom-up methods on the COCO, MPII and Pose- Track datasets. Compared with the top-down approaches, SIMPLE has comparable accuracy and faster inference speed.

PDF Details