Arrow Research search

Author name cluster

Chen Liu

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

37 papers
2 author rows

Possible papers

37

AAAI Conference 2026 Conference Paper

RCP-LO: A Relative Coordinate Prediction Framework for Generalizable Deep LiDAR Odometry

  • Chen Liu
  • Wen Li
  • Yongshu Huang
  • Minghang Zhu
  • Yuyang Yang
  • Dunqiang Liu
  • Sheng Ao
  • Cheng Wang

LiDAR odometry is a critical component of SLAM in autonomous driving and robotics. Learning-based methods have shown remarkable performance by regressing relative poses in an end-to-end manner. However, when applying these trained models, originally developed on the widely used KITTI dataset, to other scenes, performance often drops significantly. In other words, existing methods struggle to generalize well to new environments. To address this challenge, we propose RCP-LO, a simple yet effective LiDAR odometry framework. We introduce a novel representation for relative poses, reformulating them as relative coordinates, which can then be solved using geometrical verification. This approach avoids overly simplified pose representations and makes better use of scene geometry, thereby improving generalization. Moreover, to capture the inherent uncertainties in relative pose estimation from occluded LiDAR point clouds from dynamic environments, we adapt our framework to learn a denoising diffusion model, allowing for sampling plausible relative coordinates while improving robustness. We also introduce a differentiable geometric weighted singular value decomposition module, enabling efficient pose estimation through a single forward pass. Extensive experiments demonstrate that RCP-LO, trained exclusively on the KITTI dataset, achieves competitive performance compared to SOTA learning-based methods and generalizes effectively to the KITTI-360, Ford, and Oxford datasets.

JBHI Journal 2026 Journal Article

SASG-DA: Sparse-Aware Semantic-Guided Diffusion Augmentation For Myoelectric Gesture Recognition

  • Chen Liu
  • Can Han
  • Weishi Xu
  • Yaqi Wang
  • Dahong Qian

Surface electromyography (sEMG)-based gesture recognition plays a critical role in human-machine interaction (HMI), particularly for rehabilitation and prosthetic control. However, sEMG-based systems often suffer from the scarcity of informative training data, leading to overfitting and poor generalization in deep learning models. Data augmentation offers a promising approach to increasing the size and diversity of training data, where faithfulness and diversity are two critical factors to effectiveness. However, promoting untargeted diversity can result in redundant samples with limited utility. To address these challenges, we propose a novel diffusion-based data augmentation approach, Sparse-Aware Semantic-Guided Diffusion Augmentation (SASG-DA). To enhance generation faithfulness, we introduce the Semantic Representation Guidance (SRG) mechanism by leveraging fine-grained, task-aware semantic representations as generation conditions. To enable flexible and diverse sample generation, we propose a Gaussian Modeling Semantic Sampling (GMSS) strategy, which models the semantic representation distribution and allows stochastic sampling to produce both faithful and diverse samples. To enhance targeted diversity, we further introduce a Sparse-Aware Semantic Sampling strategy to explicitly explore underrepresented regions, improving distribution coverage and sample utility. Extensive experiments on benchmark sEMG datasets, Ninapro DB2, DB4, and DB7, demonstrate that SASG-DA significantly outperforms existing augmentation methods. Overall, our proposed data augmentation approach effectively mitigates overfitting and improves recognition performance and generalization by offering both faithful and diverse samples.

JBHI Journal 2026 Journal Article

Spatiospectral Representation and Neural Decoding of Somatic Perception of Acupuncture Stimulations

  • Haitao Yu
  • Zaidong Lin
  • Fan Li
  • Jialin Liu
  • Chen Liu
  • Jiang Wang

Characterizing the neural representations underlying somatic perception is crucial for neural decoding of external stimulations. Acupuncture is an important therapeutic method of traditional Chinese medicine and can effectively modulate brain activity for the treatment of neural diseases. In this work, we investigate the neural representations based on the power spectral density (PSD) estimated from electroencephalogram (EEG) across the whole brain with deep learning. Frequency and spatial characteristics of PSD can reliably represent the dynamical brain responses to acupuncture with different manipulations, manifesting enhanced alpha power in parietal lobe. By removing aperiodic components, periodic spatial spectrum shows a higher representation ability of different brain states during acupuncture stimulations, and twiring-rotating (TR) manipulation have a more pronounced modulatory effect than lifting-thrusting (LT) manipulation. Moreover, we further infer the low-dimensional feature-disentangled representations with generative adversarial network (GAN), i. e. , w -latents of StyleGAN, which can capture the latent features of periodic spatial spectrum and strike a balance between separability and generalizability. The effectiveness of feature-disentangled representations is evaluated by decoding the acupuncture states, which can achieve a highest accuracy of 95. 71% with Transformer classifier. Compared with high-dimensional spatial spectrum, low-dimensional latent features can best characterize different brain states, indicating a precise representation of somatic perception of acupuncture stimulations. Taken together, our results highlight the significant role of spatial spectral representation underlying somatic perception and serve as an important benchmark for the evaluation of acupuncture effect on human brain.

NeurIPS Conference 2025 Conference Paper

Creativity or Brute Force? Using Brainteasers as a Window into the Problem-Solving Abilities of Large Language Models

  • Sophia Han
  • Howard Dai
  • Stephen Xia
  • Grant Zhang
  • Chen Liu
  • Lichang Chen
  • Hoang H Nguyen
  • Hongyuan Mei

Accuracy remains a standard metric for evaluating AI systems, but it offers limited insight into how models arrive at their solutions. In this work, we introduce a benchmark based on brainteasers written in long narrative form to probe more deeply into the types of reasoning strategies that models use. Brainteasers are well-suited for this goal because they can be solved with multiple approaches, such as a few-step solution that uses a creative insight or a longer solution that uses more brute force. We investigate large language models (LLMs) across multiple layers of reasoning, focusing not only on correctness but also on the quality and creativity of their solutions. We investigate many aspects of the reasoning process: (1) semantic parsing of the brainteasers into precise mathematical competition style formats; (2) self-correcting solutions based on gold solutions; (3) producing step-by-step sketches of solutions; and (4) making use of hints. We find that LLMs are in many cases able to find creative, insightful solutions to brainteasers, suggesting that they capture some of the capacities needed to solve novel problems in creative ways. Nonetheless, there also remain situations where they rely on brute force despite the availability of more efficient, creative solutions, highlighting a potential direction for improvement in the reasoning abilities of LLMs.

NeurIPS Conference 2025 Conference Paper

Data Selection Matters: Towards Robust Instruction Tuning of Large Multimodal Models

  • Xu Yang
  • Chen Liu
  • Ying Wei

Selecting a compact subset of visual instruction–following data has emerged as an effective way to align large multimodal models with human intentions while avoiding the high cost of full-dataset training. Yet we observe that both full-data training and existing state-of-the-art data selection methods tend to inherit underlying dataset biases such as position bias and spurious correlations, leading to biased model behaviors. To address this issue, we introduce ARDS, a robustness-aware targeted visual instruction-selection framework that explicitly mitigates these weaknesses, sidestepping the need for access to downstream data or time-consuming gradient computation. Specifically, we first identify the worst-case evaluation subgroups through visual and textual task-specific perturbations. The robust training mixture is then constructed by prioritizing samples that are semantically closer to these subgroups in a rich multimodal embedding space. Extensive experiments demonstrate that ARDS substantially boosts both robustness and data efficiency for visual instruction tuning. We also showcase that the robust mixtures produced with a smaller model transfer effectively to larger architectures. Our code and selected datasets that have been demonstrated transferable across models are available at https: //github. com/xyang583/ARDS.

NeurIPS Conference 2025 Conference Paper

DualOptim: Enhancing Efficacy and Stability in Machine Unlearning with Dual Optimizers

  • Xuyang Zhong
  • Haochen Luo
  • Chen Liu

Existing machine unlearning (MU) approaches exhibit significant sensitivity to hyperparameters, requiring meticulous tuning that limits practical deployment. In this work, we first empirically demonstrate the instability and suboptimal performance of existing popular MU methods when deployed in different scenarios. To address this issue, we propose Dual Optimizer (DualOptim), which incorporates adaptive learning rate and decoupled momentum factors. Empirical and theoretical evidence demonstrates that DualOptim contributes to effective and stable unlearning. Through extensive experiments, we show that DualOptim can significantly boost MU efficacy and stability across diverse tasks, including image classification, image generation, and large language models, making it a versatile approach to empower existing MU algorithms.

NeurIPS Conference 2025 Conference Paper

Greed is Good: A Unifying Perspective on Guided Generation

  • Zander Blasingame
  • Chen Liu

Training-free guided generation is a widely used and powerful technique that allows the end user to exert further control over the generative process of flow/diffusion models. Generally speaking, two families of techniques have emerged for solving this problem for gradient-based guidance: namely, posterior guidance ( i. e. , guidance via projecting the current sample to the target distribution via the target prediction model) and end-to-end guidance ( i. e. , guidance by performing backpropagation throughout the entire ODE solve). In this work, we show that these two seemingly separate families can actually be unified by looking at posterior guidance as a greedy strategy of end-to-end guidance. We explore the theoretical connections between these two families and provide an in-depth theoretical of these two techniques relative to the continuous ideal gradients. Motivated by this analysis we then show a method for interpolating between these two families enabling a trade-off between compute and accuracy of the guidance gradients. We then validate this work on several inverse image problems and property-guided molecular generation.

NeurIPS Conference 2025 Conference Paper

Multimodal Disease Progression Modeling via Spatiotemporal Disentanglement and Multiscale Alignment

  • Chen Liu
  • Wenfang Yao
  • Kejing Yin
  • William K. Cheung
  • Jing Qin

Longitudinal multimodal data, including electronic health records (EHR) and sequential chest X-rays (CXRs), is critical for modeling disease progression, yet remains underutilized due to two key challenges: (1) redundancy in consecutive CXR sequences, where static anatomical regions dominate over clinically-meaningful dynamics, and (2) temporal misalignment between sparse, irregular imaging and continuous EHR data. We introduce $\texttt{DiPro}$, a novel framework that addresses these challenges through region-aware disentanglement and multi-timescale alignment. First, we disentangle static (anatomy) and dynamic (pathology progression) features in sequential CXRs, prioritizing disease-relevant changes. Second, we hierarchically align these static and dynamic CXR features with asynchronous EHR data via local (pairwise interval-level) and global (full-sequence) synchronization to model coherent progression pathways. Extensive experiments on the MIMIC dataset demonstrate that $\texttt{DiPro}$ could effectively extract temporal clinical dynamics and achieve state-of-the-art performance on both disease progression identification and general ICU prediction tasks.

JMLR Journal 2025 Journal Article

Optimal Sample Selection Through Uncertainty Estimation and Its Application in Deep Learning

  • Yong Lin
  • Chen Liu
  • Chenlu Ye
  • Qing Lian
  • Yuan Yao
  • Tong Zhang

Modern deep learning heavily relies on large labeled datasets, which often comse with high costs in terms of both manual labeling and computational resources. To mitigate these challenges, researchers have explored the use of informative subset selection techniques. In this study, we present a theoretically optimal solution for addressing both sampling with and without labels within the context of linear softmax regression. Our proposed method, COPS (unCertainty based OPtimal Sub-sampling), is designed to minimize the expected loss of a model trained on subsampled data. Unlike existing approaches that rely on explicit calculations of the inverse covariance matrix, which are not easily applicable to deep learning scenarios, COPS leverages the model's logits to estimate the sampling ratio. This sampling ratio is closely associated with model uncertainty and can be effectively applied to deep learning tasks. Furthermore, we address the challenge of model sensitivity to misspecification by incorporating a down-weighting approach for low-density samples, drawing inspiration from previous works. To assess the effectiveness of our proposed method, we conducted extensive empirical experiments using deep neural networks on benchmark datasets. The results consistently showcase the superior performance of COPS compared to baseline methods, reaffirming its efficacy. [abs] [ pdf ][ bib ] [ code ] &copy JMLR 2025. ( edit, beta )

NeurIPS Conference 2025 Conference Paper

Towards Reliable and Holistic Visual In-Context Learning Prompt Selection

  • Wenxiao Wu
  • Jing-Hao Xue
  • Chengming Xu
  • Chen Liu
  • Xinwei Sun
  • Changxin Gao
  • Nong Sang
  • Yanwei Fu

Visual In-Context Learning (VICL) has emerged as a prominent approach for adapting visual foundation models to novel tasks, by effectively exploiting contextual information embedded in in-context examples, which can be formulated as a global ranking problem of potential candidates. Current VICL methods, such as Partial2Global and VPR, are grounded in the similarity-priority assumption that images more visually similar to a query image serve as better in-context examples. This foundational assumption, while intuitive, lacks sufficient justification for its efficacy in selecting optimal in-context examples. Furthermore, Partial2Global constructs its global ranking from a series of randomly sampled pairwise preference predictions. Such a reliance on random sampling can lead to incomplete coverage and redundant samplings of comparisons, thus further adversely impacting the final global ranking. To address these issues, this paper introduces an enhanced variant of Partial2Global designed for reliable and holistic selection of in-context examples in VICL. Our proposed method, dubbed RH-Partial2Global, leverages a jackknife conformal prediction-guided strategy to construct reliable alternative sets and a covering design-based sampling approach to ensure comprehensive and uniform coverage of pairwise preferences. Extensive experiments demonstrate that RH-Partial2Global achieves excellent performance and outperforms Partial2Global across diverse visual tasks.

NeurIPS Conference 2025 Conference Paper

Understanding and Improving Fast Adversarial Training against $l_0$ Bounded Perturbations

  • Xuyang Zhong
  • Yixiao Huang
  • Chen Liu

This work studies fast adversarial training against sparse adversarial perturbations bounded by $l_0$ norm. We first demonstrate the unique challenges of employing $1$-step attacks on $l_0$ bounded perturbations, especially catastrophic overfitting (CO) that cannnot be properly addressed by existing fast adversarial training method for other $l_p$ norms ($p \geq 1$). We highlight that CO in $l_0$ adversarial training arises from sub-optimal perturbation locations of $1$-step attack. Some strategies like multi-$\epsilon$ can mitigate this sub-optimality to some extent, they lead to unstable training in turn. Theoretical and numerical analyses also reveal that the loss landscape of $l_0$ adversarial training is more craggy than its $l_\infty$, $l_2$ and $l_1$ counterparts, which exaggerates CO. To address this issue, we adopt soft labels and the trade-off loss function to smooth the adversarial loss landscape. Extensive experiments demonstrate our method can overcome the challenge of CO, achieve state-of-the-art performance, and narrow the performance gap between $1$-step and multi-step adversarial training against sparse attacks.

NeurIPS Conference 2024 Conference Paper

Addressing Asynchronicity in Clinical Multimodal Fusion via Individualized Chest X-ray Generation

  • Wenfang Yao
  • Chen Liu
  • Kejing Yin
  • William K. Cheung
  • Jing Qin

Integrating multi-modal clinical data, such as electronic health records (EHR) and chest X-ray images (CXR), is particularly beneficial for clinical prediction tasks. However, in a temporal setting, multi-modal data are often inherently asynchronous. EHR can be continuously collected but CXR is generally taken with a much longer interval due to its high cost and radiation dose. When clinical prediction is needed, the last available CXR image might have been outdated, leading to suboptimal predictions. To address this challenge, we propose DDL-CXR, a method that dynamically generates an up-to-date latent representation of the individualized CXR images. Our approach leverages latent diffusion models for patient-specific generation strategically conditioned on a previous CXR image and EHR time series, providing information regarding anatomical structures and disease progressions, respectively. In this way, the interaction across modalities could be better captured by the latent CXR generation process, ultimately improving the prediction performance. Experiments using MIMIC datasets show that the proposed model could effectively address asynchronicity in multimodal fusion and consistently outperform existing methods.

NeurIPS Conference 2024 Conference Paper

AdjointDEIS: Efficient Gradients for Diffusion Models

  • Zander W. Blasingame
  • Chen Liu

The optimization of the latents and parameters of diffusion models with respect to some differentiable metric defined on the output of the model is a challenging and complex problem. The sampling for diffusion models is done by solving either the probability flow ODE or diffusion SDE wherein a neural network approximates the score function allowing a numerical ODE/SDE solver to be used. However, naive backpropagation techniques are memory intensive, requiring the storage of all intermediate states, and face additional complexity in handling the injected noise from the diffusion term of the diffusion SDE. We propose a novel family of bespoke ODE solvers to the continuous adjoint equations for diffusion models, which we call AdjointDEIS. We exploit the unique construction of diffusion SDEs to further simplify the formulation of the continuous adjoint equations using exponential integrators. Moreover, we provide convergence order guarantees for our bespoke solvers. Significantly, we show that continuous adjoint equations for diffusion SDEs actually simplify to a simple ODE. Lastly, we demonstrate the effectiveness of AdjointDEIS for guided generation with an adversarial attack in the form of the face morphing problem. Our code will be released on our project page https: //zblasingame. github. io/AdjointDEIS/

NeurIPS Conference 2024 Conference Paper

Concentrate Attention: Towards Domain-Generalizable Prompt Optimization for Language Models

  • Chengzhengxu Li
  • Xiaoming Liu
  • Zhaohan Zhang
  • Yichen Wang
  • Chen Liu
  • Yu Lan
  • Chao Shen

Recent advances in prompt optimization have notably enhanced the performance of pre-trained language models (PLMs) on downstream tasks. However, the potential of optimized prompts on domain generalization has been under-explored. To explore the nature of prompt generalization on unknown domains, we conduct pilot experiments and find that (i) Prompts gaining more attention weight from PLMs’ deep layers are more generalizable and (ii) Prompts with more stable attention distributions in PLMs’ deep layers are more generalizable. Thus, we offer a fresh objective towards domain-generalizable prompts optimization named ''Concentration'', which represents the ''lookback'' attention from the current decoding token to the prompt tokens, to increase the attention strength on prompts and reduce the fluctuation of attention distribution. We adapt this new objective to popular soft prompt and hard prompt optimization methods, respectively. Extensive experiments demonstrate that our idea improves comparison prompt optimization methods by 1. 42% for soft prompt generalization and 2. 16% for hard prompt generalization in accuracy on the multi-source domain generalization setting, while maintaining satisfying in-domain performance. The promising results validate the effectiveness of our proposed prompt optimization objective and provide key insights into domain-generalizable prompts.

TMLR Journal 2024 Journal Article

Controlling the Fidelity and Diversity of Deep Generative Models via Pseudo Density

  • Shuangqi Li
  • Chen Liu
  • Tong Zhang
  • Hieu Le
  • Sabine Susstrunk
  • Mathieu Salzmann

We introduce an approach to bias deep generative models, such as GANs and diffusion models, towards generating data with either enhanced fidelity or increased diversity. Our approach involves manipulating the distribution of training and generated data through a novel metric for individual samples, named pseudo density, which is based on the nearest-neighbor information from real samples. Our approach offers three distinct techniques to adjust the fidelity and diversity of deep generative models: 1) Per-sample perturbation, enabling precise adjustments for individual samples towards either more common or more unique characteristics; 2) Importance sampling during model inference to enhance either fidelity or diversity in the generated data; 3) Fine-tuning with importance sampling, which guides the generative model to learn an adjusted distribution, thus controlling fidelity and diversity. Furthermore, our fine-tuning method demonstrates the ability to improve the Frechet Inception Distance (FID) for pre-trained generative models with minimal iterations.

IJCAI Conference 2024 Conference Paper

Large Language Model Guided Knowledge Distillation for Time Series Anomaly Detection

  • Chen Liu
  • Shibo He
  • Qihang Zhou
  • Shizhong Li
  • Wenchao Meng

Self-supervised methods have gained prominence in time series anomaly detection due to the scarcity of available annotations. Nevertheless, they typically demand extensive training data to acquire a generalizable representation map, which conflicts with scenarios of a few available samples, thereby limiting their performance. To overcome the limitation, we propose AnomalyLLM, a knowledge distillation-based time series anomaly detection approach where the student network is trained to mimic the features of the large language model (LLM)-based teacher network that is pretrained on large-scale datasets. During the testing phase, anomalies are detected when the discrepancy between the features of the teacher and student networks is large. To circumvent the student network from learning the teacher network’s feature of anomalous samples, we devise two key strategies. 1) Prototypical signals are incorporated into the student network to consolidate the normal feature extraction. 2) We use synthetic anomalies to enlarge the representation gap between the two networks. AnomalyLLM demonstrates state-of-the-art performance on 15 datasets, improving accuracy by at least 14. 5% in the UCR dataset.

NeurIPS Conference 2024 Conference Paper

Mixture of Adversarial LoRAs: Boosting Robust Generalization in Meta-Tuning

  • Xu Yang
  • Chen Liu
  • Ying Wei

This paper introduces AMT, an \textbf{A}dversarial \textbf{M}eta-\textbf{T}uning methodology, to boost the robust generalization of pre-trained models in the out-of-domain (OOD) few-shot learning. To address the challenge of transferring knowledge from source domains to unseen target domains, we construct the robust LoRAPool by meta-tuning LoRAs with dual perturbations applied to not only the inputs but also singular values and vectors of the weight matrices at various robustness levels. On top of that, we introduce a simple yet effective test-time merging mechanism to dynamically merge discriminative LoRAs for test-time task customization. Extensive evaluations demonstrate that AMT yields significant improvements, up to 12. 92\% in clean generalization and up to 49. 72\% in adversarial generalization, over previous state-of-the-art methods across a diverse range of OOD few-shot image classification tasks on three benchmarks, confirming the effectiveness of our approach to boost the robust generalization of pre-trained models. Our code is available at \href{https: //github. com/xyang583/AMT}{https: //github. com/xyang583/AMT}.

JMLR Journal 2024 Journal Article

On the Impact of Hard Adversarial Instances on Overfitting in Adversarial Training

  • Chen Liu
  • Zhichao Huang
  • Mathieu Salzmann
  • Tong Zhang
  • Sabine Süsstrunk

Adversarial training is a popular method to robustify models against adversarial attacks. However, it exhibits much more severe overfitting than training on clean inputs. In this work, we investigate this phenomenon from the perspective of training instances, i.e., training input-target pairs. Based on a quantitative metric measuring the relative difficulty of an instance in the training set, we analyze the model's behavior on training instances of different difficulty levels. This lets us demonstrate that the decay in generalization performance of adversarial training is a result of fitting hard adversarial instances. We theoretically verify our observations for both linear and general nonlinear models, proving that models trained on hard instances have worse generalization performance than ones trained on easy instances, and that this generalization gap increases with the size of the adversarial budget. Finally, we investigate solutions to mitigate adversarial overfitting in several scenarios, including fast adversarial training and fine-tuning a pretrained model with additional data. Our results demonstrate that using training data adaptively improves the model's robustness. [abs] [ pdf ][ bib ] [ code ] &copy JMLR 2024. ( edit, beta )

NeurIPS Conference 2024 Conference Paper

Towards Global Optimal Visual In-Context Learning Prompt Selection

  • Chengming Xu
  • Chen Liu
  • Yikai Wang
  • Yuan Yao
  • Yanwei Fu

Visual In-Context Learning (VICL) is a prevailing way to transfer visual foundation models to new tasks by leveraging contextual information contained in in-context examples to enhance learning and prediction of query sample. The fundamental problem in VICL is how to select the best prompt to activate its power as much as possible, which is equivalent to the ranking problem to test the in-context behavior of each candidate in the alternative set and select the best one. To utilize more appropriate ranking metric and leverage more comprehensive information among the alternative set, we propose a novel in-context example selection framework to approximately identify the global optimal prompt, i. e. choosing the best performing in-context examples from all alternatives for each query sample. Our method, dubbed Partial2Global, adopts a transformer-based list-wise ranker to provide a more comprehensive comparison within several alternatives, and a consistency-aware ranking aggregator to generate globally consistent ranking. The effectiveness of Partial2Global is validated through experiments on foreground segmentation, single object detection and image colorization, demonstrating that Partial2Global selects consistently better in-context examples compared with other methods, and thus establish the new state-of-the-arts.

NeurIPS Conference 2024 Conference Paper

USCILab3D: A Large-scale, Long-term, Semantically Annotated Outdoor Dataset

  • Kiran Lekkala
  • Henghui Bao
  • Peixu Cai
  • Wei Z. Lim
  • Chen Liu
  • Laurent Itti

In this paper, we introduce the \textbf{USCILab3D dataset}, a large-scale, annotated outdoor dataset designed for versatile applications across multiple domains, including computer vision, robotics, and machine learning. The dataset was acquired using a mobile robot equipped with 5 cameras and a 32-beam, $360^{\circ}$ scanning LIDAR. The robot was teleoperated, over the course of a year and under a variety of weather and lighting conditions, through a rich variety of paths within the USC campus (229 acres = $\sim 92. 7$ hectares). The raw data was annotated using state-of-the-art large foundation models, and processed to provide multi-view imagery, 3D reconstructions, semantically-annotated images and point clouds (267 semantic categories), and text descriptions of images and objects within. The dataset also offers a diverse array of complex analyses using pose-stamping and trajectory data. In sum, the dataset offers 1. 4M point clouds and 10M images ($\sim 6$TB of data). Despite covering a narrower geographical scope compared to a whole-city dataset, our dataset prioritizes intricate intersections along with denser multi-view scene images and semantic point clouds, enabling more precise 3D labelling and facilitating a broader spectrum of 3D vision tasks. For data, code and more details, please visit our website.

JBHI Journal 2023 Journal Article

An Enhanced EEG Microstate Recognition Framework Based on Deep Neural Networks: An Application to Parkinson's Disease

  • Chunguang Chu
  • Zhen Zhang
  • Zhenxi Song
  • Zifan Xu
  • Jiang Wang
  • Fei Wang
  • Wei Liu
  • Liying Lu

Variations in brain activity patterns reveal impairments of motor and cognitive functions in the human brain. Electroencephalogram (EEG) microstates embody brain activity patterns at a microscopic time scale. However, current microstate analysis method can only recognize less than 90% of EEG signals per subject, which severely limits the characterization of dynamic brain activity. As an application to early Parkinson's disease (PD), we propose an enhanced EEG microstate recognition framework based on deep neural networks, which yields recognition rates from 90% to 99%, as accompanied by a strong anti-artifact property. Additionally, gradient-weighted class activation mapping, as a visualization technique, is employed to locate the activated functional brain regions of each microstate class. We find that each microstate class corresponds to a particular activated brain region. Finally, based on the improved identification of microstate sequences, we explore the EEG microstate characteristics and their clinical associations. We show that the decreased occurrences of a particular microstate class reflect the degree of cognitive decline in early PD, and reduced transitions between certain microstates suggest injury in motor-related brain regions. The novel EEG microstate recognition framework paves the way to revealing more effective biomarkers for early PD.

ICLR Conference 2023 Conference Paper

Behavior Prior Representation learning for Offline Reinforcement Learning

  • Hongyu Zang
  • Xin Li 0033
  • Jie Yu
  • Chen Liu
  • Riashat Islam
  • Remi Tachet des Combes
  • Romain Laroche

Offline reinforcement learning (RL) struggles in environments with rich and noisy inputs, where the agent only has access to a fixed dataset without environment interactions. Past works have proposed common workarounds based on the pre-training of state representations, followed by policy training. In this work, we introduce a simple, yet effective approach for learning state representations. Our method, Behavior Prior Representation (BPR), learns state representations with an easy-to-integrate objective based on behavior cloning of the dataset: we first learn a state representation by mimicking actions from the dataset, and then train a policy on top of the fixed representation, using any off-the-shelf Offline RL algorithm. Theoretically, we prove that BPR carries out performance guarantees when integrated into algorithms that have either policy improvement guarantees (conservative algorithms) or produce lower bounds of the policy values (pessimistic algorithms). Empirically, we show that BPR combined with existing state-of-the-art Offline RL algorithms leads to significant improvements across several offline control benchmarks. The code is available at \url{https://github.com/bit1029public/offline_bpr}

NeurIPS Conference 2022 Conference Paper

Robust Binary Models by Pruning Randomly-initialized Networks

  • Chen Liu
  • Ziqi Zhao
  • Sabine Süsstrunk
  • Mathieu Salzmann

Robustness to adversarial attacks was shown to require a larger model capacity, and thus a larger memory footprint. In this paper, we introduce an approach to obtain robust yet compact models by pruning randomly-initialized binary networks. Unlike adversarial training, which learns the model parameters, we initialize the model parameters as either +1 or −1, keep them fixed, and find a subnetwork structure that is robust to attacks. Our method confirms the Strong Lottery Ticket Hypothesis in the presence of adversarial attacks, and extends this to binary networks. Furthermore, it yields more compact networks with competitive performance than existing works by 1) adaptively pruning different network layers; 2) exploiting an effective binary initialization scheme; 3) incorporating a last batch normalization layer to improve training stability. Our experiments demonstrate that our approach not only always outperforms the state-of-the-art robust binary networks, but also can achieve accuracy better than full-precision ones on some datasets. Finally, we show the structured patterns of our pruned binary networks.

AAAI Conference 2021 Conference Paper

Learning a Few-shot Embedding Model with Contrastive Learning

  • Chen Liu
  • Yanwei Fu
  • Chengming Xu
  • Siqian Yang
  • Jilin Li
  • Chengjie Wang
  • Li Zhang

Few-shot learning (FSL) aims to recognize target classes by adapting the prior knowledge learned from source classes. Such knowledge usually resides in a deep embedding model for a general matching purpose of the support and query image pairs. The objective of this paper is to repurpose the contrastive learning for such matching to learn a few-shot embedding model. We make the following contributions: (i) We investigate the contrastive learning with Noise Contrastive Estimation (NCE) in a supervised manner for training a fewshot embedding model; (ii) We propose a novel contrastive training scheme dubbed infoPatch, exploiting the patch-wise relationship to substantially improve the popular infoNCE; (iii) We show that the embedding learned by the proposed infoPatch is more effective; (iv) Our model is thoroughly evaluated on few-shot recognition task; and demonstrates state-ofthe-art results on miniImageNet and appealing performance on tieredImageNet, Fewshot-CIFAR100 (FC-100).

IJCAI Conference 2021 Conference Paper

MG-DVD: A Real-time Framework for Malware Variant Detection Based on Dynamic Heterogeneous Graph Learning

  • Chen Liu
  • Bo Li
  • Jun Zhao
  • Ming Su
  • Xu-Dong Liu

Detecting the newly emerging malware variants in real time is crucial for mitigating cyber risks and proactively blocking intrusions. In this paper, we propose MG-DVD, a novel detection framework based on dynamic heterogeneous graph learning, to detect malware variants in real time. Particularly, MG-DVD first models the fine-grained execution event streams of malware variants into dynamic heterogeneous graphs and investigates real-world meta-graphs between malware objects, which can effectively characterize more discriminative malicious evolutionary patterns between malware and their variants. Then, MG-DVD presents two dynamic walk-based heterogeneous graph learning methods to learn more comprehensive representations of malware variants, which significantly reduces the cost of the entire graph retraining. As a result, MG-DVD is equipped with the ability to detect malware variants in real time, and it presents better interpretability by introducing meaningful meta-graphs. Comprehensive experiments on large-scale samples prove that our proposed MG-DVD outperforms state-of-the-art methods in detecting malware variants in terms of effectiveness and efficiency.

NeurIPS Conference 2020 Conference Paper

On the Loss Landscape of Adversarial Training: Identifying Challenges and How to Overcome Them

  • Chen Liu
  • Mathieu Salzmann
  • Tao Lin
  • Ryota Tomioka
  • Sabine Süsstrunk

We analyze the influence of adversarial training on the loss landscape of machine learning models. To this end, we first provide analytical studies of the properties of adversarial loss functions under different adversarial budgets. We then demonstrate that the adversarial loss landscape is less favorable to optimization, due to increased curvature and more scattered gradients. Our conclusions are validated by numerical analyses, which show that training under large adversarial budgets impede the escape from suboptimal random initialization, cause non-vanishing gradients and make the models' minima found sharper. Based on these observations, we show that a periodic adversarial scheduling (PAS) strategy can effectively overcome these challenges, yielding better results than vanilla adversarial training while being much less sensitive to the choice of learning rate.

IJCAI Conference 2018 Conference Paper

Integrating Demand Response and Renewable Energy In Wholesale Market

  • Chaojie Li
  • Chen Liu
  • Xinghuo Yu
  • Ke Deng
  • Tingwen Huang
  • Liangchen Liu

Demand response (DR) can provide a cost-effect approach for reducing peak loads while renewable energy sources (RES) can result in an environmental-friendly solution for solving the problem of power shortage. The increasingly integration of DR and renewable energy bring challenging issues for energy policy makers, and electricity market regulators in the main power grid. In this paper, a new two-stage stochastic game model is introduced to operate the electricity market, where Stochastic Stackelberg-Cournot-Nash (SSCN) equilibrium is applied to characterize the optimal energy bidding strategy of the forward market and the optimal energy trading strategy of the spot market. To obtain a SSCN equilibrium, sampling average approximation (SAA) technique is harnessed to address the stochastic game model in a distributed way. By this game model, the participation ratio of demand response can be significantly increased while the unreliability of power system caused by renewable energy resources can be considerably reduced. The effectiveness of proposed model is illustrated by extensive simulations.