Author name cluster

Ning Liu

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

40 papers

2 author rows

AAAI Conference 2026 Conference Paper

D2 Prune: Sparsifying Large Language Models via Dual Taylor Expansion and Attention Distribution Awareness

Lang Xiong
Ning Liu
Ao Ren
Yuheng Bai
Haining Fang
Binyan Zhang
Zhe Jiang
Yujuan Tan

Large language models (LLMs) face significant deployment challenges due to their massive computational demands. While pruning offers a promising compression solution, existing methods suffer from two critical limitations: (1) They neglect activation distribution shifts between calibration data and test data, resulting in inaccurate error estimations; (2) Overlooking the long-tail distribution characteristics of activations in the attention module. To address these limitations, this paper proposes a novel pruning method, D²Prune. First, we propose a dual Taylor expansion-based method that jointly models weight and activation perturbations for precise error estimation, leading to precise pruning mask selection and weight updating and facilitating error minimization during pruning. Second, we propose an attention-aware dynamic update strategy that preserves the long-tail attention pattern by jointly minimizing the KL divergence of attention distributions and the reconstruction error. Extensive experiments show that D²Prune consistently outperforms SOTA methods across various LLMs (e.g., OPT-125M, LLaMA2/3, Qwen3). Moreover, the dynamic attention update mechanism also generalizes well to ViT-based vision models like DeiT, achieving superior accuracy on ImageNet-1K.

PDF Details DOI

JBHI Journal 2026 Journal Article

Generation and Selection: A Self-Iterative Two-Stage Data Augmentation Method for Automated ECG Classification

Chaoying Jiang
Yujing Xin
Ning Liu
Yonghui Xu
Lei Liu
Lizhen Cui
Jianyong Wang

Automated electrocardiogram (ECG) classification tasks play a crucial role in clinical but face challenges due to the scarcity of accessible and well-labeled data. ECG data augmentation is an effective way to address these challenges, either by altering the characteristics of real ECG data or using statistical and generative models to generate labeled data. However, the generated data often suffer from noise in both the data and label, which can reduce the performance of classification models. To address this, we propose a novel self-iterative two-stage data augmentation method for automated ECG classification, called SiTs-ECG. In the generation stage, an unconditional diffusion model, guided by a Transformer encoder, is trained to capture the complex characteristics of long-term ECG signals, generating high-quality ECG-like samples. In the selection stage, the generated samples are assigned pseudo-labels by a well-trained base classification model, and those generated samples for which the model can confidently predict the pseudo-labels are selected. We then integrate these stages into a self-iterative training process to continually improve the performance of base classification model. Extensive experiments on three real-world datasets demonstrate the effectiveness of our method. Notably, on the Apnea-ECG dataset, using ECG-Transformer as the downstream classification model, Precision, Recall, F1, and Accuracy are improved by 7. 9, 9. 1, 9. 2, and 7. 3 percentage points, respectively. Furthermore, our method is versatile and compatible with various generative and downstream classification models, showing promising applications in automated ECG classification in the clinical field.

Details DOI

AAAI Conference 2026 Conference Paper

MedOmni-45°: A Safety–Performance Benchmark for Reasoning-Oriented LLMs in Medicine

Kaiyuan Ji
Yijin Guo
Zicheng Zhang
Xiangyang Zhu
Yuan Tian
Ning Liu

With the rapid integration of large language models (LLMs) into medical decision-support aids, ensuring reliability in reasoning steps—not just final answers—is increasingly critical. Two key safety dimensions are Chain-of-Thought (CoT) faithfulness, which assesses alignment of the model’s reasoning process with both its response and medical facts, and sycophancy, an emergent misalignment where models follow misleading cues instead of factual correctness. Yet existing benchmarks tend to prioritize performance evaluation, frequently collapsing nuanced safety vulnerabilities into a single accuracy score. To fill this gap, we introduce MedOmni-45°, a benchmark and evaluation workflow explicitly designed to quantify the safety–performance trade-off in LLMs under manipulative hint conditions. The benchmark contains 1,804 reasoning-focused medical questions across six clinical specialties and three task types, including 500 publicly comparable items from MedMCQA. Each question is systematically augmented with seven manipulative hint types, each embedding two distinct misleading cue variants, along with a No-Hint baseline, resulting in approximately 27,000 unique inputs. These inputs are then evaluated across seven LLMs spanning open- and closed-source, general-purpose and medical-specific, and base versus reasoning-enhanced variants, amounting to over 189K total inference instances. Three orthogonal metrics (Accuracy, CoT-Faithfulness, Anti-Sycophancy) are combined into a composite score visualized via a 45° safety–performance plot. Results reveal a universal trade-off, with no model surpassing the ideal diagonal. Open-source QwQ-32B approaches closest at 43.81°, demonstrating notable safety while not surpassing others in performance. MedOmni-45° thus highlights critical vulnerabilities of LLMs in reasoning oriented medical tasks, offering a robust benchmark for future alignment research.

PDF Details DOI

YNIMG Journal 2026 Journal Article

Neural-linguistic analysis for Alzheimer’s detection: A deep learning approach informed by cognitive neuroscience

Jianhui Lv
Shalli Rani
Keqin Li
Ning Liu

Alzheimer's disease (AD) is a progressive neurodegenerative disorder that disrupts cognitive function across multiple domains, particularly affecting language networks and speech production pathways in the brain. Patients demonstrate symptoms including aphasia, reduced syntactic complexity, and diminished verbal fluency that reflects underlying neural pathology in language-related cortical areas. Current detection methods rely on resource-intensive neuroimaging, invasive biomarker sampling, and extensive neuropsychological testing, creating substantial barriers to early diagnosis. While researchers have explored using acoustic features, paralinguistic markers, and text-based features for AD detection, existing approaches face fundamental limitations: traditional acoustic methods fail to capture semantic-cognitive content, text transcription is labor-intensive, and automatic speech recognition quality suffers due to pronunciation variations and cognitive impairments in elderly populations. This paper introduces cognitive acoustic symbolic transformation for ALzheimer's (COASTAL), a neurobiologically-inspired framework that models hierarchical speech processing pathways. COASTAL transforms acoustic patterns into discrete symbolic elements through a specialized transformation module before applying contextual analysis that mirrors prefrontal-temporal language networks. Evaluated on the ADReSSo corpus, COASTAL achieved 70.42% accuracy, outperforming established baselines by 5.63%. Integration with complementary self-supervised approaches through hierarchical fusion improved performance to 77.46%. Analysis revealed that preserving fine-grained temporal features through shallower transformation architecture significantly enhanced diagnostic accuracy, aligning with neuropsychological evidence that subtle timing patterns in speech provide sensitive markers of cognitive decline.

Details DOI

AAAI Conference 2026 Conference Paper

STEP-Nav: Spatial-Temporal Efficient Visual Token Pruning for Vision-and-Language Navigation with Large Language Models

Yantao Lu
Shiqi Sun
Ning Liu
Bo Jiang
Ying Zhang
Jinchao Chen
Chenglie Du

Vision-and-Language Navigation (VLN) plays a critical role in tasks of embodied AI, particularly in unseen environments following natural language instructions. Recent advancements leverage large language models (LLMs) to improve the accuracy and generalizability of VLN systems by encoding image sequences as dense token representations. However, this tokenization approach incurs substantial computational overhead due to two key inefficiencies: 1) ego-centric camera views often include navigation-irrelevant re- gions (e.g., sky or distant backgrounds), and 2) high-frame-rate image sequences introduce temporal redundancy. To address these challenges, we propose Spatial-Temporal Efficient Visual Token Pruning (STEP-Nav), a unified frame- work that simultaneously prunes redundant visual tokens and fine-tunes VLN models to preserve navigation performance. In particular, STEP-Nav incorporates a distance- and content-aware token evaluation mechanism to remove irrelevant tokens at the spatial level, along with temporal level similarity-based filtering to reduce redundancy across sequential frames. To ensure pruning does not harm task performance, we introduce a distortion-aware fine-tuning strategy that aligns pruned-token representations with their full-token counterparts while maintaining navigation accuracy. Experiments on the R2R and RxR benchmarks using Navid-CE and NavGPT-2 as base models demonstrate that STEP-Nav preserves over 95% of the performance while reducing 66.7% of tokens, outperforming existing token pruning baselines.

PDF Details DOI

AAAI Conference 2025 Conference Paper

A Comprehensive Overhaul of Multimodal Assistant with Small Language Models

Minjie Zhu
Yichen Zhu
Ning Liu
Xin Liu
Zhiyuan Xu
Chaomin Shen
Yaxin Peng

Multimodal Large Language Models (MLLMs) have showcased impressive skills in tasks related to visual understanding and reasoning. Yet, their widespread application faces obstacles due to the high computational demands during both the training and inference phases, restricting their use to a limited audience within the research and user communities. In this paper, we investigate the design aspects of Multimodal Small Language Models (MSLMs) and propose an efficient multimodal assistant named Mipha, which is designed to create synergy among various aspects: visual representation, language models, and optimization strategies. We show that without increasing the volume of training data, our Mipha-3B outperforms the state-of-the-art large MLLMs, especially LLaVA-1.5-13B, on multiple benchmarks. Through detailed discussion, we provide insights and guidelines for developing strong MSLMs that rival the capabilities of MLLMs.

PDF Details DOI

AAAI Conference 2025 Conference Paper

AUTE: Peer-Alignment and Self-Unlearning Boost Adversarial Robustness for Training Ensemble Models

Lifeng Huang
Tian Su
Chengying Gao
Ning Liu
Qiong Huang

Adversarial attacks poses a significant threat to the security of AI-based systems. To counteract these attacks, adversarial training (AT) and ensemble learning (EL) have emerged as widely adopted methods for enhancing model robustness. However, a counter-intuitive phenomenon arises where the simple combination of these approaches may potentially compromising adversarial robustness of ensemble models. In this paper, we propose a novel method called Alignment and Unlearning for Training Ensembles (AUTE), aiming to effectively integrate AT and EL to maximize their benefits. Specifically, AUTE incorporates two key components. Firstly, AUTE divides the ensemble into a big peer model and a single member in a loop manner, aligning their outputs for boosting robustness of each member. Secondly, AUTE introduces the concept of unlearning, actively forgetting specific data with over-confident properties to preserve model capacity to learn more robust features. Extensive experiments across various datasets and networks illustrate that AUTE achieves superior performance compared to baselines. For instance, a 5-member AUTE with ResNet-20 networks outperforms state-of-the-art method by 2.1% and 3.2% in classifying clean and adversarial data. Additionally, AUTE can easily extend to non-adversarial training paradigm, surpassing current standard ensemble learning methods by a large margin.

PDF Details DOI

JBHI Journal 2025 Journal Article

Bridging Brain and Machine: A Neurocognitive Framework for Early Alzheimer's Detection Through Hierarchical Speech Processing

Xiaohong Lyu
Jiayuan Bai
Pengfei Li
Fan Zhang
Xianglin Zhang
Yun Gu
Ning Liu

Alzheimer's disease progressively disrupts temporal-frontal language networks, manifesting through degraded speech production detectable years before clinical dementia diagnosis. Traditional automated detection systems rely on handcrafted acoustic features, including mel-frequency cepstral coefficients, that may discard diagnostically relevant temporal dynamics through fixed spectral transformations. Drawing inspiration from cognitive neuroscience models of hierarchical speech processing, we propose the neuro residual acoustic learning framework, an end-to-end system operating directly on raw waveforms while incorporating architectural principles mirroring biological auditory pathways. The framework implements acoustic-symbolic transformation bridging continuous signals and discrete phonetic-acoustic categories analogous to primary auditory cortex processing, hierarchical temporal encoding through residual blocks with dilated convolutions capturing multi-scale patterns across phoneme to utterance timescales, and squeeze-and-excitation attention mechanisms modeling prefrontal modulation of temporal representations. Evaluation on NCMMSC2021 Mandarin speech demonstrates 86. 55% accuracy on extended utterances and 81. 35% on brief segments, surpassing contemporary methods, including multimodal architectures and large language model hybrids, by 2. 88% and 2. 43% respectively. Cross-linguistic validation on INTERSPEECH2020 ADReSS English speech achieves 70. 83% accuracy, exceeding baselines by 2. 08%. Confusion matrix analysis reveals 93. 3% healthy control recall and asymmetric Alzheimer's disease versus mild cognitive impairment misclassification patterns consistent with prodromal acoustic biomarkers.

Details DOI

NeurIPS Conference 2025 Conference Paper

FreqPolicy: Efficient Flow-based Visuomotor Policy via Frequency Consistency

Yifei Su
Ning Liu
Dong Chen
Zhen Zhao
Kun Wu
Meng Li
Zhiyuan Xu
Zhengping Che

Generative modeling-based visuomotor policies have been widely adopted in robotic manipulation, attributed to their ability to model multimodal action distributions. However, the high inference cost of multi-step sampling limits its applicability in real-time robotic systems. Existing approaches accelerate sampling in generative modeling-based visuomotor policies by adapting techniques originally developed to speed up image generation. However, a major distinction exists: image generation typically produces independent samples without temporal dependencies, while robotic manipulation requires generating action trajectories with continuity and temporal coherence. To this end, we propose FreqPolicy, a novel approach that first imposes frequency consistency constraints on flow-based visuomotor policies. Our work enables the action model to capture temporal structure effectively while supporting efficient, high-quality one-step action generation. Concretely, we introduce a frequency consistency constraint objective that enforces alignment of frequency-domain action features across different timesteps along the flow, thereby promoting convergence of one-step action generation toward the target distribution. In addition, we design an adaptive consistency loss to capture structural temporal variations inherent in robotic manipulation tasks. We assess FreqPolicy on $53$ tasks across $3$ simulation benchmarks, proving its superiority over existing one-step action generators. We further integrate FreqPolicy into the vision-language-action (VLA) model and achieve acceleration without performance degradation on $40$ tasks of Libero. Besides, we show efficiency and effectiveness in real-world robotic scenarios with an inference frequency of $93. 5$ Hz.

PDF Details

TMLR Journal 2025 Journal Article

Graph Theory-Based Deep Graph Similarity Learning: A Unified Survey of Pipeline, Techniques, and Challenges

Zhouyang LIU
Ning Liu
Yixin Chen
Ziqing Wen
Jiezhong He
Dongsheng Li

Graph similarity computation, which measures the resemblance between graphs, is a crucial operation in fields such as graph search. Recent advances in graph neural networks have enabled the embedding of graphs into low-dimensional vector spaces, where the sim- ilarity or distance between graphs can be efficiently quantified. However, these methods are often tailored to specific tasks and function as black boxes, limiting both generalization and interpretability. To address these challenges, there is growing interest in incorporating domain-agnostic and interpretable concepts from graph theory—such as subgraph isomorphism, maximum common subgraph, and graph edit distance—into graph similarity learning as training objectives. This survey presents a comprehensive review of recent advancements in deep graph similarity learning, focusing on models that integrate these graph theory concepts. Despite the different training objectives of these approaches, they share significant commonalities in the training pipeline, techniques, and challenges. We analyze them within a unified lens referred to as graph theory-based deep similarity learning (GTDGSL) methods. We systematically compare existing GTDGSL methods alongside their common training pipeline, highlighting the technique trend and discussing key challenges, applications, and future research directions in this domain. We organize the papers included in this survey and their open-source implementations at https://github.com/liuzhouyang/Graph-Theory-Based-Deep-Graph-Similarity-Learning-Survey.

PDF Details

ICML Conference 2025 Conference Paper

Neural Interpretable PDEs: Harmonizing Fourier Insights with Attention for Scalable and Interpretable Physics Discovery

Ning Liu
Yue Yu

Attention mechanisms have emerged as transformative tools in core AI domains such as natural language processing and computer vision. Yet, their largely untapped potential for modeling intricate physical systems presents a compelling frontier. Learning such systems often entails discovering operators that map between functional spaces using limited instances of function pairs—a task commonly framed as a severely ill-posed inverse PDE problem. In this work, we introduce Neural Interpretable PDEs (NIPS), a novel neural operator architecture that builds upon and enhances Nonlocal Attention Operators (NAO) in both predictive accuracy and computational efficiency. NIPS employs a linear attention mechanism to enable scalable learning and integrates a learnable kernel network that acts as a channel-independent convolution in Fourier space. As a consequence, NIPS eliminates the need to explicitly compute and store large pairwise interactions, effectively amortizing the cost of handling spatial interactions into the Fourier transform. Empirical evaluations demonstrate that NIPS consistently surpasses NAO and other baselines across diverse benchmarks, heralding a substantial leap in scalable, interpretable, and efficient physics learning. Our code and data accompanying this paper are available at https: //github. com/fishmoon1234/Nonlocal-Attention-Operator.

Details

NeurIPS Conference 2025 Conference Paper

SEEA-R1: Tree-Structured Reinforcement Fine-Tuning for Self-Evolving Embodied Agents

Wanxin Tian
Shijie Zhang
Kevin Zhang
Xiaowei Chi
Chun-Kai Fan
Junyu Lu
Yulin Luo
Qiang Zhou

Self-evolution, the ability of agents to autonomously improve their reasoning and behavior, is essential for the embodied domain with long-horizon, real-world tasks. Despite current advancements in reinforcement fine-tuning (RFT) showing strong performance in enhancing reasoning in LLMs, its potential to enable self-evolving embodied intelligence with multi-modal interactions remains largely unexplored. Specifically, reinforcement fine-tuning faces two fundamental obstacles in embodied settings: (i) the lack of accessible intermediate rewards in multi-step reasoning tasks limits effective learning signals, and (ii) reliance on hand-crafted reward functions restricts generalization to novel tasks and environments. To address these challenges, we present Self-Evolving Embodied Agents-R1, SEEA-R1, the first RFT framework designed for enabling the self-evolving capabilities of embodied agents. Specifically, to convert sparse delayed rewards into denser intermediate signals that improve multi-step reasoning, we propose Tree-based group relative policy optimization ( Tree-GRPO ) integrates Monte Carlo Tree Search into GRPO. To generalize reward estimation across tasks and scenes, supporting autonomous adaptation and reward-driven self-evolution, we further introduce Multi-modal Generative Reward Model ( MGRM ). To holistically evaluate the effectiveness of SEEA-R1, we evaluate on the ALFWorld benchmark, surpassing state-of-the-art methods with scores of 85. 07\% (textual) and 46. 27\% (multi-modal), outperforming prior models including GPT-4o. SEEA-R1 also achieves scores of 80. 3\% (textual) and 44. 03\% (multi-modal) without ground truth reward, surpassing all open-source baselines and highlighting its scalability as a self-evolving embodied agent. Additional experiments and qualitative analysis further support the potential of SEEA-R1 for future research in scalable embodied intelligence. Project page is at https: //seea-r1. github. io/.

PDF Details

NeurIPS Conference 2024 Conference Paper

AlterMOMA: Fusion Redundancy Pruning for Camera-LiDAR Fusion Models with Alternative Modality Masking

Shiqi Sun
Yantao Lu
Ning Liu
Bo Jiang
Jinchao Chen
Ying Zhang

Camera-LiDAR fusion models significantly enhance perception performance in autonomous driving. The fusion mechanism leverages the strengths of each modality while minimizing their weaknesses. Moreover, in practice, camera-LiDAR fusion models utilize pre-trained backbones for efficient training. However, we argue that directly loading single-modal pre-trained camera and LiDAR backbones into camera-LiDAR fusion models introduces similar feature redundancy across modalities due to the nature of the fusion mechanism. Unfortunately, existing pruning methods are developed explicitly for single-modal models, and thus, they struggle to effectively identify these specific redundant parameters in camera-LiDAR fusion models. In this paper, to address the issue above on camera-LiDAR fusion models, we propose a novelty pruning framework Alternative Modality Masking Pruning (AlterMOMA), which employs alternative masking on each modality and identifies the redundant parameters. Specifically, when one modality parameters are masked (deactivated), the absence of features from the masked backbone compels the model to reactivate previous redundant features of the other modality backbone. Therefore, these redundant features and relevant redundant parameters can be identified via the reactivation process. The redundant parameters can be pruned by our proposed importance score evaluation function, Alternative Evaluation (AlterEva), which is based on the observation of the loss changes when certain modality parameters are activated and deactivated. Extensive experiments on the nuScene and KITTI datasets encompassing diverse tasks, baseline models, and pruning algorithms showcase that AlterMOMA outperforms existing pruning methods, attaining state-of-the-art performance.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

EDT: An Efficient Diffusion Transformer Framework Inspired by Human-like Sketching

Xinwang Chen
Ning Liu
Yichen Zhu
Feifei Feng
Jian Tang

Transformer-based Diffusion Probabilistic Models (DPMs) have shown more potential than CNN-based DPMs, yet their extensive computational requirements hinder widespread practical applications. To reduce the computation budget of transformer-based DPMs, this work proposes the Efficient Diffusion Transformer (EDT) framework. This framework includes a lightweight-design diffusion model architecture, and a training-free Attention Modulation Matrix and its alternation arrangement in EDT inspired by human-like sketching. Additionally, we propose a token relation-enhanced masking training strategy tailored explicitly for EDT to augment its token relation learning capability. Our extensive experiments demonstrate the efficacy of EDT. The EDT framework reduces training and inference costs and surpasses existing transformer-based diffusion models in image synthesis performance, thereby achieving a significant overall enhancement. With lower FID, EDT-S, EDT-B, and EDT-XL attained speed-ups of 3. 93x, 2. 84x, and 1. 92x respectively in the training phase, and 2. 29x, 2. 29x, and 2. 22x respectively in inference, compared to the corresponding sizes of MDTv2. Our code is available at https: //github. com/xinwangChen/EDT.

PDF Details DOI

AAAI Conference 2024 Conference Paper

EPSD: Early Pruning with Self-Distillation for Efficient Model Compression

Dong Chen
Ning Liu
Yichen Zhu
Zhengping Che
Rui Ma
Fachao Zhang
Xiaofeng Mou
Yi Chang

Neural network compression techniques, such as knowledge distillation (KD) and network pruning, have received increasing attention. Recent work `Prune, then Distill' reveals that a pruned student-friendly teacher network can benefit the performance of KD. However, the conventional teacher-student pipeline, which entails cumbersome pre-training of the teacher and complicated compression steps, makes pruning with KD less efficient. In addition to compressing models, recent compression techniques also emphasize the aspect of efficiency. Early pruning demands significantly less computational cost in comparison to the conventional pruning methods as it does not require a large pre-trained model. Likewise, a special case of KD, known as self-distillation (SD), is more efficient since it requires no pre-training or student-teacher pair selection. This inspires us to collaborate early pruning with SD for efficient model compression. In this work, we propose the framework named Early Pruning with Self-Distillation (EPSD), which identifies and preserves distillable weights in early pruning for a given SD task. EPSD efficiently combines early pruning and self-distillation in a two-step process, maintaining the pruned network's trainability for compression. Instead of a simple combination of pruning and SD, EPSD enables the pruned network to favor SD by keeping more distillable weights before training to ensure better distillation of the pruned network. We demonstrated that EPSD improves the training of pruned networks, supported by visual and quantitative analyses. Our evaluation covered diverse benchmarks (CIFAR-10/100, Tiny-ImageNet, full ImageNet, CUB-200-2011, and Pascal VOC), with EPSD outperforming advanced pruning and SD techniques.

PDF Details DOI

AAAI Conference 2024 Conference Paper

FlexKBQA: A Flexible LLM-Powered Framework for Few-Shot Knowledge Base Question Answering

Zhenyu Li
Sunqi Fan
Yu Gu
Xiuxing Li
Zhichao Duan
Bowen Dong
Ning Liu
Jianyong Wang

Knowledge base question answering (KBQA) is a critical yet challenging task due to the vast number of entities within knowledge bases and the diversity of natural language questions posed by users. Unfortunately, the performance of most KBQA models tends to decline significantly in real-world scenarios where high-quality annotated data is insufficient. To mitigate the burden associated with manual annotation, we introduce FlexKBQA by utilizing Large Language Models (LLMs) as program translators for addressing the challenges inherent in the few-shot KBQA task. Specifically, FlexKBQA leverages automated algorithms to sample diverse programs, such as SPARQL queries, from the knowledge base, which are subsequently converted into natural language questions via LLMs. This synthetic dataset facilitates training a specialized lightweight model for the KB. Additionally, to reduce the barriers of distribution shift between synthetic data and real user questions, FlexKBQA introduces an executionguided self-training method to iterative leverage unlabeled user questions. Furthermore, we explore harnessing the inherent reasoning capability of LLMs to enhance the entire framework. Consequently, FlexKBQA delivers substantial flexibility, encompassing data annotation, deployment, and being domain agnostic. Through extensive experiments on GrailQA, WebQSP, and KQA Pro, we observe that under the few-shot even the more challenging zero-shot scenarios, FlexKBQA achieves impressive results with a few annotations, surpassing all previous baselines and even approaching the performance of supervised models, achieving a remarkable 93% performance relative to the fully-supervised models. We posit that FlexKBQA represents a significant advancement towards exploring better integration of large and lightweight models. Code is available at https://github.com/leezythu/FlexKBQA.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

Nonlocal Attention Operator: Materializing Hidden Knowledge Towards Interpretable Physics Discovery

Yue Yu
Ning Liu
Fei Lu
Tian Gao
Siavash Jafarzadeh
Stewart Silling

Despite recent popularity of attention-based neural architectures in core AI fields like natural language processing (NLP) and computer vision (CV), their potential in modeling complex physical systems remains under-explored. Learning problems in physical systems are often characterized as discovering operators that map between function spaces based on a few instances of function pairs. This task frequently presents a severely ill-posed PDE inverse problem. In this work, we propose a novel neural operator architecture based on the attention mechanism, which we coin Nonlocal Attention Operator (NAO), and explore its capability towards developing a foundation physical model. In particular, we show that the attention mechanism is equivalent to a double integral operator that enables nonlocal interactions among spatial tokens, with a data-dependent kernel characterizing the inverse mapping from data to the hidden parameter field of the underlying operator. As such, the attention mechanism extracts global prior information from training data generated by multiple systems, and suggests the exploratory space in the form of a nonlinear kernel map. Consequently, NAO can address ill-posedness and rank deficiency in inverse PDE problems by encoding regularization and achieving generalizability. Lastly, we empirically demonstrate the advantages of NAO over baseline neural models in terms of the generalizability to unseen data resolutions and system states. Our work not only suggests a novel neural operator architecture for learning an interpretable foundation model of physical systems, but also offers a new perspective towards understanding the attention mechanism.

PDF Details DOI

ICRA Conference 2024 Conference Paper

Open X-Embodiment: Robotic Learning Datasets and RT-X Models: Open X-Embodiment Collaboration

Abby O'Neill
Abdul Rehman
Abhiram Maddukuri
Abhishek Gupta 0004
Abhishek Padalkar
Abraham Lee
Acorn Pooley
Agrim Gupta

Large, high-capacity models trained on diverse datasets have shown remarkable successes on efficiently tackling downstream applications. In domains from NLP to Computer Vision, this has led to a consolidation of pretrained models, with general pretrained backbones serving as a starting point for many applications. Can such a consolidation happen in robotics? Conventionally, robotic learning methods train a separate model for every application, every robot, and even every environment. Can we instead train "generalist" X-robot policy that can be adapted efficiently to new robots, tasks, and environments? In this paper, we provide datasets in standardized data formats and models to make it possible to explore this possibility in the context of robotic manipulation, alongside experimental results that provide an example of effective X-robot policies. We assemble a dataset from 22 different robots collected through a collaboration between 21 institutions, demonstrating 527 skills (160266 tasks). We show that a high-capacity model trained on this data, which we call RT-X, exhibits positive transfer and improves the capabilities of multiple robots by leveraging experience from other platforms. The project website is robotics-transformer-x. github.io.

Details

IJCAI Conference 2024 Conference Paper

Personalized Federated Learning for Cross-City Traffic Prediction

Yu Zhang
Hua Lu
Ning Liu
Yonghui Xu
Qingzhong Li
Lizhen Cui

Traffic prediction plays an important role in urban computing. However, many cities face data scarcity due to low levels of urban development. Although many approaches transfer knowledge from data-rich cities to data-scarce cities, the centralized training paradigm cannot uphold data privacy. For the sake of inter-city data privacy, Federated Learning has been used, which follows a decentralized training paradigm to enhance traffic knowledge of data-scarce cities. However, spatio-temporal data heterogeneity causes client drift, leading to unsatisfactory traffic prediction performance. In this work, we propose a novel personalized Federated learning method for Cross-city Traffic Prediction (pFedCTP). It learns traffic knowledge from multiple data-rich source cities and transfers the knowledge to a data-scarce target city while preserving inter-city data privacy. In the core of pFedCTP lies a Spatio-Temporal Neural Network (ST-Net) for clients to learn traffic representation. We decouple the ST-Net to learn space-independent traffic patterns to overcome cross-city spatial heterogeneity. Besides, pFedCTP adaptively interpolates the layer-wise global and local parameters to deal with temporal heterogeneity across cities. Extensive experiments on four real-world traffic datasets demonstrate significant advantages of pFedCTP over representative state-of-the-art methods.

PDF Details DOI

AAAI Conference 2024 Conference Paper

T-SciQ: Teaching Multimodal Chain-of-Thought Reasoning via Large Language Model Signals for Science Question Answering

Lei Wang
Yi Hu
Jiabang He
Xing Xu
Ning Liu
Hui Liu
Heng Tao Shen

Large Language Models (LLMs) have recently demonstrated exceptional performance in various Natural Language Processing (NLP) tasks. They have also shown the ability to perform chain-of-thought (CoT) reasoning to solve complex problems. Recent studies have explored CoT reasoning in complex multimodal scenarios, such as the science question answering task, by fine-tuning multimodal models with high-quality human-annotated CoT rationales. However, collecting high-quality COT rationales is usually time-consuming and costly. Besides, the annotated rationales are hardly accurate due to the external essential information missed. To address these issues, we propose a novel method termed T-SciQ that aims at teaching science question answering with LLM signals. The T-SciQ approach generates high-quality CoT rationales as teaching signals and is advanced to train much smaller models to perform CoT reasoning in complex modalities. Additionally, we introduce a novel data mixing strategy to produce more effective teaching data samples for simple and complex science question answer problems. Extensive experimental results show that our T-SciQ method achieves a new state-of-the-art performance on the ScienceQA benchmark, with an accuracy of 96.18%. Moreover, our approach outperforms the most powerful fine-tuned baseline by 4.5%. The code is publicly available at https://github.com/T-SciQ/T-SciQ.

PDF Details DOI

AAAI Conference 2023 Conference Paper

Alignment-Enriched Tuning for Patch-Level Pre-trained Document Image Models

Lei Wang
Jiabang He
Xing Xu
Ning Liu
Hui Liu

Alignment between image and text has shown promising improvements on patch-level pre-trained document image models. However, investigating more effective or finer-grained alignment techniques during pre-training requires a large amount of computation cost and time. Thus, a question naturally arises: Could we fine-tune the pre-trained models adaptive to downstream tasks with alignment objectives and achieve comparable or better performance? In this paper, we propose a new model architecture with alignment-enriched tuning (dubbed AETNet) upon pre-trained document image models, to adapt downstream tasks with the joint task-specific supervised and alignment-aware contrastive objective. Specifically, we introduce an extra visual transformer as the alignment-ware image encoder and an extra text transformer as the alignment-ware text encoder before multimodal fusion. We consider alignment in the following three aspects: 1) document-level alignment by leveraging the cross-modal and intra-modal contrastive loss; 2) global-local alignment for modeling localized and structural information in document images; and 3) local-level alignment for more accurate patch-level information. Experiments on various downstream tasks show that AETNet can achieve state-of-the-art performance on various downstream tasks. Notably, AETNet consistently outperforms state-of-the-art pre-trained models, such as LayoutLMv3 with fine-tuning techniques, on three different downstream tasks. Code is available at https://github.com/MAEHCM/AET.

PDF Details DOI

NeurIPS Conference 2023 Conference Paper

Domain Agnostic Fourier Neural Operators

Ning Liu
Siavash Jafarzadeh
Yue Yu

Fourier neural operators (FNOs) can learn highly nonlinear mappings between function spaces, and have recently become a popular tool for learning responses of complex physical systems. However, to achieve good accuracy and efficiency, FNOs rely on the Fast Fourier transform (FFT), which is restricted to modeling problems on rectangular domains. To lift such a restriction and permit FFT on irregular geometries as well as topology changes, we introduce domain agnostic Fourier neural operator (DAFNO), a novel neural operator architecture for learning surrogates with irregular geometries and evolving domains. The key idea is to incorporate a smoothed characteristic function in the integral layer architecture of FNOs, and leverage FFT to achieve rapid computations, in such a way that the geometric information is explicitly encoded in the architecture. In our empirical evaluation, DAFNO has achieved state-of-the-art accuracy as compared to baseline neural operator models on two benchmark datasets of material modeling and airfoil simulation. To further demonstrate the capability and generalizability of DAFNO in handling complex domains with topology changes, we consider a brittle material fracture evolution problem. With only one training crack simulation sample, DAFNO has achieved generalizability to unseen loading scenarios and substantially different crack patterns from the trained scenario. Our code and data accompanying this paper are available at https: //github. com/ningliu-iga/DAFNO.

PDF Details

YNIMG Journal 2023 Journal Article

Equivalent processing of facial expression and identity by macaque visual system and task-optimized neural network

Hui Zhang
Xuetong Ding
Ning Liu
Rachel Nolan
Leslie G Ungerleider
Shruti Japee

Both the primate visual system and artificial deep neural network (DNN) models show an extraordinary ability to simultaneously classify facial expression and identity. However, the neural computations underlying the two systems are unclear. Here, we developed a multi-task DNN model that optimally classified both monkey facial expressions and identities. By comparing the fMRI neural representations of the macaque visual cortex with the best-performing DNN model, we found that both systems: (1) share initial stages for processing low-level face features which segregate into separate branches at later stages for processing facial expression and identity respectively, and (2) gain more specificity for the processing of either facial expression or identity as one progresses along each branch towards higher stages. Correspondence analysis between the DNN and monkey visual areas revealed that the amygdala and anterior fundus face patch (AF) matched well with later layers of the DNN's facial expression branch, while the anterior medial face patch (AM) matched well with later layers of the DNN's facial identity branch. Our results highlight the anatomical and functional similarities between macaque visual system and DNN model, suggesting a common mechanism between the two systems.

Details DOI

EAAI Journal 2023 Journal Article

Improved LSTM-based deep learning model for COVID-19 prediction using optimized approach

Luyu Zhou
Chun Zhao
Ning Liu
Xingduo Yao
Zewei Cheng

Individuals in any country are badly impacted both economically and physically whenever an epidemic of infectious illnesses breaks out. A novel coronavirus strain was responsible for the outbreak of the coronavirus sickness in 2019. Corona Virus Disease 2019 (COVID-19) is the name that the World Health Organization (WHO) officially gave to the pneumonia that was caused by the novel coronavirus on February 11, 2020. The use of models that are informed by machine learning is currently a major focus of study in the field of improved forecasting. By displaying annual trends, forecasting models can be of use in performing impact assessments of potential outcomes. In this paper, proposed forecast models consisting of time series models such as long short-term memory (LSTM), bidirectional long short-term memory (Bi-LSTM), generalized regression unit (GRU), and dense-LSTM have been evaluated for time series prediction of confirmed cases, deaths, and recoveries in 12 major countries that have been affected by COVID-19. Tensorflow1. 0 was used for programming. Indices known as mean absolute error (MAE), root means square error (RMSE), Median Absolute Error (MEDAE) and r2 score are utilized in the process of evaluating the performance of models. We presented various ways to time-series forecasting by making use of LSTM models (LSTM, BiLSTM), and we compared these proposed methods to other machine learning models to evaluate the performance of the models. Our study suggests that LSTM based models are among the most advanced models to forecast time series data.

Details DOI

TCS Journal 2023 Journal Article

Trajectory optimization of laser-charged UAV to minimize the average age of information for wireless rechargeable sensor network

Chuanwen Luo
Ning Liu
Yunan Hou
Yi Hong
Zhibo Chen
Deying Li

This paper considers the laser-charged Unmanned Aerial Vehicle (UAV) assisted Wireless Rechargeable Sensor Network (WRSN), where rechargeable sensors are deployed in surveillance environment to monitor information, a UAV is not only used as aerial wireless mobile collector for gathering data from sensors but also used as mobile charger to replenish energy for sensors, Laser Beam Directors (LBDs) are uniformly deployed in the monitoring environment to charge UAV by emitting laser beams. In such network, we study the average Age of Information Optimization (AoIO) problem whose objective is to minimize the average AoI of data collected from sensors such that all data of the network are transported to the base station and the remaining energy of any sensor exceeds a certain threshold. We prove that the AoIO problem is NP-hard. To solve the AoIO problem, we first study the Total Flight Time Minimizing of UAV (TFTM) problem, which aims at finding an optimal charging solution of UAV to minimize the flight time of UAV based on the order of sensors visited by UAV. Then we prove that the TFTM problem is also NP-hard. Afterwards, we propose a heuristic algorithm to solve the TFTM problem by optimizing flight path, data collection, energy power transfer and laser charging of UAV. Based on the solution for the TFTM problem, we propose an approximation algorithm to solve the AoIO problem. Finally, we conduct extensive simulation experiments to verify the effectiveness of the proposed algorithm.

Details DOI

NeurIPS Conference 2022 Conference Paper

Teach Less, Learn More: On the Undistillable Classes in Knowledge Distillation

Yichen Zhu
Ning Liu
Zhiyuan Xu
Xin Liu
Weibin Meng
Louis Wang
Zhicai Ou
Jian Tang

Knowledge distillation (KD) can effectively compress neural networks by training a smaller network (student) to simulate the behavior of a larger one (teacher). A counter-intuitive observation is that a more expansive teacher does not make a better student, but the reasons for this phenomenon remain unclear. In this paper, we demonstrate that this is directly attributed to the presence of \textit{undistillable classes}: when trained with distillation, the teacher's knowledge of some classes is incomprehensible to the student model. We observe that while KD improves the overall accuracy, it is at the cost of the model becoming inaccurate in these undistillable classes. After establishing their widespread existence in state-of-the-art distillation methods, we illustrate their correlation with the capacity gap between teacher and student models. Finally, we present a simple Teach Less Learn More (TLLM) framework to identify and discard the undistillable classes during training. We validate the effectiveness of our approach on multiple datasets with varying network architectures. In all settings, our proposed method is able to exceed the performance of competitive state-of-the-art techniques.

PDF Details

JBHI Journal 2021 Journal Article

Gaining Insights Into Patient Satisfaction Through Interpretable Machine Learning

Ning Liu
Soundar Kumara
Eric Reich

Patient satisfaction is a key performance indicator of patient-centered care and hospital reimbursement. To discover the major factors that affect patient experiences is considered as an effective way to formulate corrective actions. A patient during his/her healthcare journey interacts with multiple health professionals across different service units. The health-related data generated at each step of the journey is a valuable resource for extracting actionable insights. In particular, self-reported satisfaction survey and the associated patient electronic health records play an important role in the hospital-patient interaction analysis. In this paper, we propose an interpretable machine learning framework to formulate the patient satisfaction problem as a supervised learning task and utilize a mixed-integer programming model to identify the most influential factors. The proposed framework transforms heterogeneous data into human-understandable features and integrates feature transformation, variable selection, and coefficient learning into the optimization process. Therefore, it can achieve desirable model performance while maintaining excellent model interpretability, which paves the way for successful real-world applications.

Details DOI

NeurIPS Conference 2021 Conference Paper

MEST: Accurate and Fast Memory-Economic Sparse Training Framework on the Edge

Geng Yuan
Xiaolong Ma
Wei Niu
Zhengang Li
Zhenglun Kong
Ning Liu
Yifan Gong
Zheng Zhan

Recently, a new trend of exploring sparsity for accelerating neural network training has emerged, embracing the paradigm of training on the edge. This paper proposes a novel Memory-Economic Sparse Training (MEST) framework targeting for accurate and fast execution on edge devices. The proposed MEST framework consists of enhancements by Elastic Mutation (EM) and Soft Memory Bound (&S) that ensure superior accuracy at high sparsity ratios. Different from the existing works for sparse training, this current work reveals the importance of sparsity schemes on the performance of sparse training in terms of accuracy as well as training speed on real edge devices. On top of that, the paper proposes to employ data efficiency for further acceleration of sparse training. Our results suggest that unforgettable examples can be identified in-situ even during the dynamic exploration of sparsity masks in the sparse training process, and therefore can be removed for further training speedup on edge devices. Comparing with state-of-the-art (SOTA) works on accuracy, our MEST increases Top-1 accuracy significantly on ImageNet when using the same unstructured sparsity scheme. Systematical evaluation on accuracy, training speed, and memory footprint are conducted, where the proposed MEST framework consistently outperforms representative SOTA works. A reviewer strongly against our work based on his false assumptions and misunderstandings. On top of the previous submission, we employ data efficiency for further acceleration of sparse training. And we explore the impact of model sparsity, sparsity schemes, and sparse training algorithms on the number of removable training examples. Our codes are publicly available at: https: //github. com/boone891214/MEST.

PDF Details

ICRA Conference 2021 Conference Paper

Robotic Electrospinning Actuated by Non-Circular Joint Continuum Manipulator for Endoluminal Therapy

Zicong Wu
Chuqian Lou
Zhu Jin
Shaoping Huang
Ning Liu
Yun Zou
Mirko Kovac
Anzhu Gao

Electrospinning has exhibited excellent benefits to treat the trauma for tissue engineering due to its produced micro/nano fibrous structure. It can effectively adhere to the tissue surface for long-term continuous therapy. This paper develops a robotic electrospinning platform for endoluminal therapy. The platform consists of a continuum manipulator, the electrospinning device, and the actuation unit. The continuum manipulator has two bending sections to facilitate the steering of the tip needle for a controllable spinning direction. Non-circular joint profile is carefully designed to enable a constant length of the centreline of a continuum manipulator for stable fluid transmission inside it. Experiments are performed on a bronchus phantom, and the steering ability and bending limitation in each direction are also investigated. The endoluminal electrospinning is also fulfilled by a trajectory following and points targeting experiments. The effective adhesive area of the produced fibre is also illustrated. The proposed robotic electrospinning shows its feasibility to precisely spread more therapeutic drug to construct fibrous structure for potential endoluminal treatments.

Details

NeurIPS Conference 2021 Conference Paper

Sanity Checks for Lottery Tickets: Does Your Winning Ticket Really Win the Jackpot?

Xiaolong Ma
Geng Yuan
Xuan Shen
Tianlong Chen
Xuxi Chen
Xiaohan Chen
Ning Liu
Minghai Qin

There have been long-standing controversies and inconsistencies over the experiment setup and criteria for identifying the "winning ticket" in literature. To reconcile such, we revisit the definition of lottery ticket hypothesis, with comprehensive and more rigorous conditions. Under our new definition, we show concrete evidence to clarify whether the winning ticket exists across the major DNN architectures and/or applications. Through extensive experiments, we perform quantitative analysis on the correlations between winning tickets and various experimental factors, and empirically study the patterns of our observations. We find that the key training hyperparameters, such as learning rate and training epochs, as well as the architecture characteristics such as capacities and residual connections, are all highly correlated with whether and when the winning tickets can be identified. Based on our analysis, we summarize a guideline for parameter settings in regards of specific architecture characteristics, which we hope to catalyze the research progress on the topic of lottery ticket hypothesis. Our codes are publicly available at: https: //github. com/boone891214/sanity-check-LTH.

PDF Details

NeurIPS Conference 2021 Conference Paper

Scalable Rule-Based Representation Learning for Interpretable Classification

Zhuo Wang
Wei Zhang
Ning Liu
Jianyong Wang

Rule-based models, e. g. , decision trees, are widely used in scenarios demanding high model interpretability for their transparent inner structures and good model expressivity. However, rule-based models are hard to optimize, especially on large data sets, due to their discrete parameters and structures. Ensemble methods and fuzzy/soft rules are commonly used to improve performance, but they sacrifice the model interpretability. To obtain both good scalability and interpretability, we propose a new classifier, named Rule-based Representation Learner (RRL), that automatically learns interpretable non-fuzzy rules for data representation and classification. To train the non-differentiable RRL effectively, we project it to a continuous space and propose a novel training method, called Gradient Grafting, that can directly optimize the discrete model using gradient descent. An improved design of logical activation functions is also devised to increase the scalability of RRL and enable it to discretize the continuous features end-to-end. Exhaustive experiments on nine small and four large data sets show that RRL outperforms the competitive interpretable approaches and can be easily adjusted to obtain a trade-off between classification accuracy and model complexity for different scenarios. Our code is available at: https: //github. com/12wang3/rrl.

PDF Details

AAAI Conference 2020 Conference Paper

AutoCompress: An Automatic DNN Structured Pruning Framework for Ultra-High Compression Rates

Ning Liu
Xiaolong Ma
Zhiyuan Xu
Yanzhi Wang
Jian Tang
Jieping Ye

Structured weight pruning is a representative model compression technique of DNNs to reduce the storage and computation requirements and accelerate inference. An automatic hyperparameter determination process is necessary due to the large number of ﬂexible hyperparameters. This work proposes AutoCompress, an automatic structured pruning framework with the following key performance improvements: (i) effectively incorporate the combination of structured pruning schemes in the automatic process; (ii) adopt the stateof-art ADMM-based structured weight pruning as the core algorithm, and propose an innovative additional puriﬁcation step for further weight reduction without accuracy loss; and (iii) develop effective heuristic search method enhanced by experience-based guided search, replacing the prior deep reinforcement learning technique which has underlying incompatibility with the target pruning problem. Extensive experiments on CIFAR-10 and ImageNet datasets demonstrate that AutoCompress is the key to achieve ultra-high pruning rates on the number of weights and FLOPs that cannot be achieved before. As an example, AutoCompress outperforms the prior work on automatic model compression by up to 33× in pruning rate (120× reduction in the actual parameter count) under the same accuracy. Signiﬁcant inference speedup has been observed from the AutoCompress framework on actual measurements on smartphone. We release models of this work at anonymous link: http: //bit. ly/2VZ63dS.

PDF Details

ICRA Conference 2020 Conference Paper

FBG-Based Triaxial Force Sensor Integrated with an Eccentrically Configured Imaging Probe for Endoluminal Optical Biopsy

Zicong Wu
Anzhu Gao
Ning Liu
Zhu Jin
Guang-Zhong Yang

Accurate force sensing is important for endoluminal intervention in terms of both safety and lesion targeting. This paper develops an FBG-based force sensor for robotic bronchoscopy by configuring three FBG sensors at the lateral side of a conical substrate. It allows a large and eccentric inner lumen for the interventional instrument, enabling a flexible imaging probe inside to perform optical biopsy. The force sensor is embodied with a laser-profiled continuum robot and thermo drift is fully compensated by three temperature sensors integrated on the circumference surface of the sensor substrate. Different decoupling approaches are investigated, and nonlinear decoupling is adopted based on the cross-validation SVM and a Gaussian kernel function, achieving an accuracy of 10. 58 mN, 14. 57 mN and 26. 32 mN along X, Y and Z axis, respectively. The tissue test is also investigated to further demonstrate the feasibility of the developed triaxial force sensor.

Details

JBHI Journal 2020 Journal Article

Semi-Supervised Learning Algorithm for Identifying High-Priority Drug–Drug Interactions Through Adverse Event Reports

Ning Liu
Cheng-Bang Chen
Soundar Kumara

Identifying drug-drug interactions (DDIs) is a critical enabler for reducing adverse drug events and improving patient safety. Generating proper DDI alerts during prescribing workflow has the potential to prevent DDIrelated adverse events. However, the implementation of DDI alerting system remains a challenge as users are experiencing alert overload which causes alert fatigue. One strategy to optimize the current system is to establish a list of high-priority DDIs for alerting purposes, though it is a resource-intensive task. In this study, we propose a machine learning framework to extract useful features from the FDA adverse event reports and then identify potential highpriority DDIs using an autoencoder-based semi-supervised learning algorithm. The experimental results demonstrate the effectiveness of using adverse event feature representations in differentiating highand low-priority DDIs. Additionally, the proposed algorithm utilizes stacked autoencoders and weighted support vector machine for boosting classification performance, which outperforms other competing methods in terms of F-measure and AUC score. This framework integrates multiple information sources, leverages domain knowledge and clinical evidence, and provides a practical approach for pre-screening high-priority DDI candidates for medication alerts.

Details DOI

AAAI Conference 2020 Conference Paper

Transparent Classification with Multilayer Logical Perceptrons and Random Binarization

Zhuo Wang
Wei Zhang
Ning Liu
Jianyong Wang

Models with transparent inner structure and high classiﬁcation performance are required to reduce potential risk and provide trust for users in domains like health care, ﬁnance, security, etc. However, existing models are hard to simultaneously satisfy the above two properties. In this paper, we propose a new hierarchical rule-based model for classi- ﬁcation tasks, named Concept Rule Sets (CRS), which has both a strong expressive ability and a transparent inner structure. To address the challenge of efﬁciently learning the nondifferentiable CRS model, we propose a novel neural network architecture, Multilayer Logical Perceptron (MLLP), which is a continuous version of CRS. Using MLLP and the Random Binarization (RB) method we proposed, we can search the discrete solution of CRS in continuous space using gradient descent and ensure the discrete CRS acts almost the same as the corresponding continuous MLLP. Experiments on 12 public data sets show that CRS outperforms the state-of-theart approaches and the complexity of the learned CRS is close to the simple decision tree.

PDF Details

ICRA Conference 2018 Conference Paper

Design and Kinematics Characterization of a Laser-Profiled Continuum Manipulator for the Guidance of Bronchoscopic Instruments * This work was supported by Engineering and Physical Sciences Research Council (EPSRC), United Kingdom (EP/N019318/1). Ning Liu and Mali Shen are also supported by Chinese Scholarship Council (CSC)

Ning Liu
Mohamed E. M. K. Abdelaziz
Mali Shen
Guang-Zhong Yang

Bronchoscopic intervention, as a minimally invasive method for the diagnosis and treatment of lung diseases, has attracted more and more attention in recent years. However, existing endobronchial instruments lack the steerability accessing the peripheral airways with difficult bifurcations. This paper presents a novel wire-driven dexterous manipulator for the guidance of such instruments. Precision laser profiling is used to cut a stainless steel tube into multiple interlocked segments with revolute joints. The outer diameter of the manipulator is 2. 20 mm which is small enough to be inserted into the working channels of most commercial bronchoscopes and distal airways, while keeping a large inner lumen with a diameter of 1. 44 mm for passing various bronchoscopic instruments. The small bending radius provides enough flexibility to navigate inside the complex bronchial tree. Two kinematic models are proposed to predict the manipulator configuration from the translation of actuation wires. The former model is geometrically derived with the assumption of constant curvature bending and the latter one is statistically driven by capturing the motion trajectories of manipulator joints. A prototype of our low-cost add-on instrument guidance robot for bronchoscopic intervention is presented which can be easily integrated into current clinical routine.

Details

ICRA Conference 2016 Conference Paper

Design and analysis of a wire-driven flexible manipulator for bronchoscopic interventions

Ning Liu
Christos Bergeles
Guang-Zhong Yang

Bronchoscopic interventions are widely performed for the diagnosis and treatment of lung diseases. However, for most endobronchial devices, the lack of a bendable tip restricts their access ability to get into distal bronchi with complex bifurcations. This paper presents the design of a new wire-driven continuum manipulator to help guide these devices. The proposed manipulator is built by assembling miniaturized blocks that are featured with interlocking circular joints. It has the capability of maintaining its integrity when the lengths of actuation wires change due to the shaft flex. It allows the existence of a relatively large central cavity to pass through other instruments and enables two rotational degrees of freedom. All these features make it suitable for procedures where tubular anatomies are involved and the flexible shafts have to be considerably bent in usage, just like bronchoscopic interventions. A kinematic model is built to estimate the relationship between the translations of actuation wires and the manipulator tip position. A scale-up model is produced for evaluation experiments and the results validate the performance of the proposed mechanism.

Details

YNIMG Journal 2016 Journal Article

Face-selective regions differ in their ability to classify facial expressions

Hui Zhang
Shruti Japee
Rachel Nolan
Carlton Chu
Ning Liu
Leslie G. Ungerleider

Recognition of facial expressions is crucial for effective social interactions. Yet, the extent to which the various face-selective regions in the human brain classify different facial expressions remains unclear. We used functional magnetic resonance imaging (fMRI) and support vector machine pattern classification analysis to determine how well face-selective brain regions are able to decode different categories of facial expression. Subjects participated in a slow event-related fMRI experiment in which they were shown 32 face pictures, portraying four different expressions: neutral, fearful, angry, and happy and belonging to eight different identities. Our results showed that only the amygdala and the posterior superior temporal sulcus (STS) were able to accurately discriminate between these expressions, albeit in different ways: the amygdala discriminated fearful faces from non-fearful faces, whereas STS discriminated neutral from emotional (fearful, angry and happy) faces. In contrast to these findings on the classification of emotional expression, only the fusiform face area (FFA) and anterior inferior temporal cortex (aIT) could discriminate among the various facial identities. Further, the amygdala and STS were better than FFA and aIT at classifying expression, while FFA and aIT were better than the amygdala and STS at classifying identity. Taken together, our findings indicate that the decoding of facial emotion and facial identity occurs in different neural substrates: the amygdala and STS for the former and FFA and aIT for the latter.

Details DOI

AAAI Conference 2011 Conference Paper

Collaborative Users’ Brand Preference Mining across Multiple Domains from Implicit Feedbacks

Jian Tang
Jun Yan
Lei Ji
Ming Zhang
Shaodan Guo
Ning Liu
Xianfang Wang
Zheng Chen

Advanced e-applications require comprehensive knowledge about their users’ preferences in order to provide accurate personalized services. In this paper, we propose to learn users’ preferences to product brands from their implicit feedbacks such as their searching and browsing behaviors in user Web browsing log data. The user brand preference learning problem is challenge since (1) the users’ implicit feedbacks are extremely sparse in various product domains; and (2) we can only observe positive feedbacks from users’ behaviors. In this paper, we propose a latent factor model to collaboratively mine users’ brand preferences across multiple domains simultaneously. By collective learning, the learning processes in all the domains are mutually enhanced and hence the problem of data scarcity in each single domain can be effectively addressed. On the other hand, we learn our model with an adaption of the Bayesian personalized ranking (BPR) optimization criterion which is a general learning framework for collaborative filtering from implicit feedbacks. Experiments with both synthetic and real world datasets show that our proposed model significantly outperforms the baselines.

PDF Details

YNIMG Journal 2010 Journal Article

Alterations in the processing of non-drug-related affective stimuli in abstinent heroin addicts

Zhao-Xin Wang
John X. Zhang
Qiu-Lin Wu
Ning Liu
Xiao-Ping Hu
Raymond C.K. Chan
Zhuang-Wei Xiao

Long-term exposure to drug may alter the neural system associated with affective processing, as evidenced by both clinical observations and behavioral data documenting dysfunctions in emotional experiences and processing in drug addicts. Although many imaging studies examined neural responses to drug or drug-related cues in addicts, there have been few studies explicitly designed to reveal their neural abnormalities in processing non-drug-related natural affective materials. The present study asked abstinent heroin addicts and normal controls to passively view standardized affective pictures of positive, negative, or neutral valence and compared their brain activities with functional MRI. Compared to normal controls, addicts showed reduced activation in right amygdala in response to the affective pictures, consistent with previous reports of blunted subjective experience for affective stimuli in addicts. Furthermore, in two visual cortical areas BA 19 and 37, while the controls showed greater responses to positive pictures than to negative ones replicating literature findings, the addicts showed the opposite pattern. The results reveal a complex pattern of altered processing of non-drug-related affective materials in addicts showing both heightened and blunted neural responses in different brain regions and for different stimulus valence. The present study highlights the importance of brain imaging research on drug addicts' processing of affective stimuli in understanding disruptions in their emotion circuitry.

Details DOI