Arrow Research search

Author name cluster

Yang Song

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

70 papers
2 author rows

Possible papers

70

JBHI Journal 2026 Journal Article

Explainable Deep Learning for Cyber Threat in IoMT: A Synchronization-Enhanced Sparse Autoencoder Approach

  • Yang Song
  • Amel Ksibi
  • Kadambri Agarwal
  • Baoze Qu

The interconnected medical devices and networks face escalating cyber threats that demand intelligent detection mechanisms. Traditional deep learning approaches for cyber threat intelligence (CTI) in healthcare networks suffer from the black-box problem, limiting their practical deployment in critical medical environments where explainability is paramount. This paper presents a novel explainable deep learning (XDL) framework that integrates sparse autoencoders with neural synchronization mechanisms for transparent cyber threat detection in Internet of Medical Things (IoMT) networks. Our approach, termed XDL-CTI-MedNet, employs neuron-level local activation consistency constraints and synchronization-based functional module construction to achieve high detection accuracy and interpretable decision-making processes. The framework incorporates a multi-dimensional interpretability evaluation system that assesses explanation accuracy, stability, purity, and diversity from various analytical perspectives. Experiments on CIC IoMT 2024 and IoT healthcare security datasets demonstrate superior performance with 98. 4-98. 8% accuracy while achieving interpretability scores of 0. 935-0. 947, outperforming six baseline methods across all evaluation dimensions. Statistical validation through ten independent runs with different random initializations confirms robust performance with standard deviations below 0. 8%, and the Friedman test analysis establishes statistical significance at the 0. 05 confidence level.

Details DOI

AAAI Conference 2026 System Paper

Principles2Plan: LLM-Guided System for Operationalising Ethical Principles into Plans

  • Tammy Zhong
  • Yang Song
  • Maurice Pagnucco

Ethical awareness is critical for robots operating in human environments, yet existing automated planning tools provide little support. Manually specifying ethical rules is labour-intensive and highly context-specific. We present Principles2Plan, an interactive research prototype demonstrating how a human and a Large Language Model (LLM) can collaborate to produce context-sensitive ethical rules and guide automated planning. A domain expert provides the planning domain, problem details, and relevant high-level principles such as beneficence and privacy. The system generates operationalisable ethical rules consistent with these principles, which the user can review, prioritise, and supply to a planner to produce ethically-informed plans. To our knowledge, no prior system supports users in generating principle-grounded rules for classical planning contexts. Principles2Plan showcases the potential of human-LLM collaboration for making ethical automated planning more practical and feasible.

PDF Details DOI

AAAI Conference 2026 Conference Paper

Relative Advantage Debiasing for Watch-Time Prediction in Short-Video Recommendation

  • Emily Liu
  • Kuan Han
  • Minfeng Zhan
  • Bocheng Zhao
  • Guanyu Mu
  • Yang Song

Watch time is widely used as a proxy for user satisfaction in video recommendation platforms. However, raw watch times are influenced by confounding factors such as video duration, popularity, and individual user behaviors, potentially distorting preference signals and resulting in biased recommendation models. We propose a novel relative advantage debiasing framework that corrects watch time by comparing it to empirically derived reference distributions conditioned on user and item groups. This approach yields a quantile-based preference signal and introduces a two-stage architecture that explicitly separates distribution estimation from preference learning. Additionally, we present distributional embeddings to efficiently parameterize watch-time quantiles without requiring online sampling or storage of historical data. Both offline and online experiments demonstrate significant improvements in recommendation accuracy and robustness compared to existing baseline methods.

PDF Details DOI

YNICL Journal 2026 Journal Article

Study of sex differences in the whole brain white matter using diffusion MRI tractography and suprathreshold fiber cluster statistics

  • Fan Zhang
  • Jarrett Rushmore
  • Yijie Li
  • Suheyla Cetin-Karayumak
  • Yang Song
  • Weidong Cai
  • Carl-Fredrik Westin
  • James J. Levitt

Sex-specific characteristics demonstrate a substantial influence on the brain white matter, suggesting distinct structural connectivity patterns between females and males. Diffusion MRI tractography is an important tool in assessing white matter connectivity and brain tissue microstructure across different populations. Whole brain tractography analysis for group statistical comparison is a challenging task due to the large number of white matter connections. This work studies whole-brain white matter connectivity differences between females and males using dMRI tractography. We study a large cohort of 707 subjects from the Human Connectome Project Young Adult dataset. By applying a well-established fiber clustering pipeline and a suprathreshold fiber cluster statistical method, we analyze tracts in the cerebral cortex and understudied pathways like those connecting to the cerebellum. We identify several tracts with significant sex differences in terms of their fractional anisotropy and/or mean diffusivity. These include deep tracts like the arcuate fasciculus, corticospinal tract, and corpus callosum, superficial tracts in the frontal lobe, and cerebellar tracts. Finally, a canonical correlation analysis (CCA) that identifies covariance patterns between white matter and behavior measures reveals that these white matter differences are associated with a range of neurobehavioral measures, with the strongest and most consistent associations observed for motor function, suggesting motor circuits as a potential key focus for future research.

Details DOI

JBHI Journal 2026 Journal Article

Transformer-Enhanced Joint Information Extraction Framework for Bladder Cancer Medical Dialogue Analysis

  • Yang Song
  • Yanlong Yang
  • Jiaojiao Qin
  • Qiang Zhang

Bladder cancer diagnosis and treatment monitoring through medical dialogues present unique information extraction requirements that traditional single-task approaches struggle to address effectively. Extracting ac curate clinical information from patient-physician conversations about bladder cancer involves recognizing subtle symptom descriptions, understanding treatment responses, and capturing disease progression indicators that may be scattered across extended dialogue sequences. Bladder cancer transformer for joint information extraction (BCT-JIE) addresses these requirements through a collaborative transformer architecture that simultaneously performs entity recognition, intent classification, and clinical state assessment within bladder cancer medical dialogues. The framework employs a sliding window mechanism combined with bidirectional long short-term memory networks to capture historical dialogue context, while utilizing a symptom-aware module to identify bladder cancer specific clinical indicators. Global attention mechanisms enhance the model's ability to understand relationships between urological symptoms and their clinical manifestations across extended conversation sequences. Interactive guidance modules explicitly model the interdependencies between symptom extraction, treatment intent recognition, and disease progression assessment tasks, enabling more accurate information capture for bladder cancer management. Experimental validation on MedDialog and Multi MedQA datasets demonstrates superior performance compared to baseline approaches, with notable improvements in bladder cancer-specific entity extraction and clinical state classification accuracy, achieving 91. 76% F1-score in symptom entity extraction and 83. 42% accuracy in staging classification. Ablation studies with five-fold repetition con firm that both the sliding window mechanism and global attention module contribute reliably to extraction performance across all tasks.

Details DOI

ICLR Conference 2025 Conference Paper

Can a Large Language Model be a Gaslighter?

  • Wei Li 0076
  • Luyao Zhu
  • Yang Song
  • Ruixi Lin
  • Rui Mao 0010
  • Yang You

Large language models (LLMs) have gained human trust due to their capabilities and helpfulness. However, this in turn may allow LLMs to affect users' mindsets by manipulating language. It is termed as gaslighting, a psychological effect. In this work, we aim to investigate the vulnerability of LLMs under prompt-based and fine-tuning-based gaslighting attacks. Therefore, we propose a two-stage framework DeepCoG designed to: 1) elicit gaslighting plans from LLMs with the proposed DeepGaslighting prompting template, and 2) acquire gaslighting conversations from LLMs through our Chain-of-Gaslighting method. The gaslighting conversation dataset along with a corresponding safe dataset is applied to fine-tuning-based attacks on open-source LLMs and anti-gaslighting safety alignment on these LLMs. Experiments demonstrate that both prompt-based and fine-tuning-based attacks transform three open-source LLMs into gaslighters. In contrast, we advanced three safety alignment strategies to strengthen~(by $12.05\%$) the safety guardrail of LLMs. Our safety alignment strategies have minimal impacts on the utility of LLMs. Empirical studies indicate that an LLM may be a potential gaslighter, even if it passed the harmfulness test on general dangerous queries.

Details

JAIR Journal 2025 Journal Article

Computational Machine Ethics: A Survey

  • Tammy Zhong
  • Yang Song
  • Raynaldio Limarga
  • Maurice Pagnucco

Computational Machine Ethics (CME) is an interdisciplinary field that integrates moral philosophy into an agent’s decision-making process, contributing to the broader domain of Artificial Intelligence Ethics. Technological advancements have transformed the world, where technology has become an integral part of society, progressively given more autonomy in making judgments within various domains in our lives. Inevitably, issues of ethics come into play in these judgments, making ethical decision-making in machines an increasingly critical problem to solve. This survey provides an overview of CME, highlighting the breadth of directions and the use of techniques within the field. We also provide some background on the ethical dimension before introducing our taxonomy used to categorise and detail the variety of existing approaches from a more technical perspective. Finally, we identify limitations in the research and suggest potential open challenges for future work.

PDF Details DOI

AAAI Conference 2025 Conference Paper

Granularity-Adaptive Spatial Evidence Tokenization for Video Question Answering

  • Hao Jiang
  • Yang Jin
  • Zhicheng Sun
  • Kun Xu
  • Liwei Chen
  • Yang Song
  • Kun Gai
  • Yadong Mu

Video question answering plays a vital role in computer vision, and recent advances in large language models have further propelled the development of this field. However, existing video question answering techniques often face limitations in grasping fine-grained video content in spatial dimensions. It mainly stems from the fixed and low-resolution input of video frames. While some approaches using high-resolution inputs partially alleviate this problem, they introduce excessive computational burdens by encoding the entire high-resolution image. In this work, we propose a granularity-adaptive spatial evidence tokenization model for video question answering. Our method introduces multi-granular visual tokenization in the spatial dimension to produce video tokens at various granularities based on the question. It highlights spatially activated patches at low resolutions through a granularity weighting module and then adaptively encodes these activated patches at high resolution for detail supplementation. To mitigate the computational overhead associated with high-resolution frame encoding, a masking and acceleration module is developed for efficient visual tokenization. Moreover, a granularity compression module is designed to dynamically select and compress visual tokens of varying granularities based on questions. We conduct extensive experiments on 11 mainstream video question answering datasets and the experimental results demonstrate the effectiveness of our proposed method.

PDF Details DOI

AAAI Conference 2025 Conference Paper

GRPose: Learning Graph Relations for Human Image Generation with Pose Priors

  • Xiangchen Yin
  • Donglin Di
  • Lei Fan
  • Hao Li
  • Wei Chen
  • Gouxiaofei
  • Yang Song
  • Xiao Sun

Recent methods using diffusion models have made significant progress in human image generation with various control signals such as pose priors. However, existing efforts are still struggling to generate high-quality images with consistent pose alignment, resulting in unsatisfactory output. In this paper, we propose a framework that delves into the graph relations of pose priors to provide control information for human image generation. The main idea is to establish a graph topological structure between the pose priors and latent representation of diffusion models to capture the intrinsic associations between different pose parts. A Progressive Graph Integrator (PGI) is designed to learn the spatial relationships of the pose priors with the graph structure, adopting a hierarchical strategy within an Adapter to gradually propagate information across different pose parts. Besides, a pose perception loss is introduced based on a pretrained pose estimation network to minimize the pose differences. Extensive qualitative and quantitative experiments conducted on the Human-Art and LAION-Human datasets clearly demonstrate that our model can achieve significant performance improvement over the latest benchmark models.

PDF Details DOI

NeurIPS Conference 2025 Conference Paper

GVPO: Group Variance Policy Optimization for Large Language Model Post-Training

  • Kaichen Zhang
  • Yuzhong Hong
  • Junwei Bao
  • Hongfei Jiang
  • Yang Song
  • Hong Dingqian
  • Hui Xiong

Post-training plays a crucial role in refining and aligning large language models to meet specific tasks and human preferences. While recent advancements in post-training techniques, such as Group Relative Policy Optimization (GRPO), leverage increased sampling with relative reward scoring to achieve superior performance, these methods often suffer from training instability that limits their practical adoption. As a next step, we present Group Variance Policy Optimization (GVPO). GVPO incorporates the analytical solution to KL-constrained reward maximization directly into its gradient weights, ensuring alignment with the optimal policy. The method provides intuitive physical interpretations: its gradient mirrors the mean squared error between the central distance of implicit rewards and that of actual rewards. GVPO offers two key advantages: (1) it guarantees a unique optimal solution, exactly the KL-constrained reward maximization objective, (2) it supports flexible sampling distributions that avoids on-policy and importance sampling limitations. By unifying theoretical guarantees with practical adaptability, GVPO establishes a new paradigm for reliable and versatile LLM post-training.

PDF Details

ECAI Conference 2025 Conference Paper

IPBA: Imperceptible Perturbation Backdoor Attack in Federated Self-Supervised Learning

  • Jiayao Wang 0004
  • Yang Song
  • Zhendong Zhao
  • Jiale Zhang
  • Junwu Zhu
  • Qilin Wu
  • Dongfang Zhao 0001

Federated Self-Supervised Learning (FSSL) combines the advantages of decentralized modeling and unlabeled representation learning, serving as a cutting-edge paradigm with strong potential for scalability and privacy preservation. Although FSSL has garnered increasing attention, research indicates that it remains vulnerable to backdoor attacks. Existing methods generally rely on visually obvious triggers, which makes it difficult to meet the requirements for stealth and practicality in real-world deployment. In this paper, we propose an imperceptible and effective backdoor attack method against FSSL, called IPBA. Our empirical study reveals that existing imperceptible triggers face a series of challenges in FSSL, particularly limited transferability, feature entanglement with augmented samples, and out-of-distribution properties. These issues collectively undermine the effectiveness and stealthiness of traditional backdoor attacks in FSSL. To overcome these challenges, IPBA decouples the feature distributions of backdoor and augmented samples, and introduces Sliced-Wasserstein distance to mitigate the out-of-distribution properties of backdoor samples, thereby optimizing the trigger generation process. Our experimental results on several FSSL scenarios and datasets show that IPBA significantly outperforms existing backdoor attack methods in performance and exhibits strong robustness under various defense mechanisms.

Details

NeurIPS Conference 2025 Conference Paper

LocDiff: Identifying Locations on Earth by Diffusing in the Hilbert Space

  • Zhangyu Wang
  • Zeping Liu
  • Jielu Zhang
  • Zhongliang Zhou
  • Qian Cao
  • Nemin Wu
  • Lan Mu
  • Yang Song

Image geolocalization is a fundamental yet challenging task, aiming at inferring the geolocation on Earth where an image is taken. State-of-the-art methods employ either grid-based classification or gallery-based image-location retrieval, whose spatial generalizability significantly suffers if the spatial distribution of test images does not align with the choices of grids and galleries. Recently emerging generative approaches, while getting rid of grids and galleries, use raw geographical coordinates and suffer quality losses due to their lack of multi-scale information. To address these limitations, we propose a multi-scale latent diffusion model called LocDiff for image geolocalization. We developed a novel positional encoding-decoding framework called Spherical Harmonics Dirac Delta (SHDD) Representations, which encodes points on a spherical surface (e. g. , geolocations on Earth) into a Hilbert space of Spherical Harmonics coefficients and decodes points (geolocations) by mode-seeking on spherical probability distributions. We also propose a novel SirenNet-based architecture (CS-UNet) to learn an image-based conditional backward process in the latent SHDD space by minimizing a latent KL-divergence loss. To the best of our knowledge, LocDiff is the first image geolocalization model that performs latent diffusion in a multi-scale location encoding space and generates geolocations under the guidance of images. Experimental results show that LocDiff can outperform all state-of-the-art grid-based, retrieval-based, and diffusion-based baselines across 5 challenging global-scale image geolocalization datasets, and demonstrates significantly stronger generalizability to unseen geolocations.

PDF Details

IJCAI Conference 2025 Conference Paper

Multimodal Cancer Survival Analysis via Hypergraph Learning with Cross-Modality Rebalance

  • Mingcheng Qu
  • Guang Yang
  • Donglin Di
  • Tonghua Su
  • Yue Gao
  • Yang Song
  • Lei Fan

Multimodal pathology-genomic analysis has become increasingly prominent in cancer survival prediction. However, existing studies mainly utilize multi-instance learning to aggregate patch-level features, neglecting the information loss of contextual and hierarchical details within pathology images. Furthermore, the disparity in data granularity and dimensionality between pathology and genomics leads to a significant modality imbalance. The high spatial resolution inherent in pathology data renders it a dominant role while overshadowing genomics in multimodal integration. In this paper, we propose a multimodal survival prediction framework that incorporates hypergraph learning to effectively capture both contextual and hierarchical details from pathology images. Moreover, it employs a modality rebalance mechanism and an interactive alignment fusion strategy to dynamically reweight the contributions of the two modalities, thereby mitigating the pathology-genomics imbalance. Quantitative and qualitative experiments are conducted on five TCGA datasets, demonstrating that our model outperforms advanced methods by over 3. 4% in C-Index performance. Code: https: //github. com/MCPathology/MRePath.

PDF Details DOI

AAAI Conference 2025 Conference Paper

Preference-Oriented Supervised Fine-Tuning: Favoring Target Model over Aligned Large Language Models

  • Yuchen Fan
  • Yuzhong Hong
  • Qiushi Wang
  • Junwei Bao
  • Hongfei Jiang
  • Yang Song

Alignment, endowing a pre-trained Large language model (LLM) with the ability to follow instructions, is crucial for its real-world applications. Conventional supervised fine-tuning (SFT) methods formalize it as causal language modeling typically with a cross-entropy objective, requiring a large amount of high-quality instruction-response pairs. However, the quality of widely used SFT datasets can not be guaranteed due to the high cost and intensive labor for the creation and maintenance in practice. To overcome the limitations associated with the quality of SFT datasets, we introduce a novel preference-oriented supervised fine-tuning approach, namely PoFT. The intuition is to boost SFT by imposing a particular preference: favoring the target model over aligned LLMs on the same SFT data. This preference encourages the target model to predict a higher likelihood than that predicted by the aligned LLMs, incorporating assessment information on data quality (i.e., predicted likelihood by the aligned LLMs) into the training process. Extensive experiments are conducted, and the results validate the effectiveness of the proposed method. PoFT achieves stable and consistent improvements over the SFT baselines across different training datasets and base models. Moreover, we prove that PoFT can be integrated with existing SFT data filtering methods to achieve better performance, and further improved by following preference optimization procedures, such as DPO.

PDF Details DOI

ICLR Conference 2025 Conference Paper

Simplifying, Stabilizing and Scaling Continuous-time Consistency Models

  • Cheng Lu
  • Yang Song

Consistency models (CMs) are a powerful class of diffusion-based generative models optimized for fast sampling. Most existing CMs are trained using discretized timesteps, which introduce additional hyperparameters and are prone to discretization errors. While continuous-time formulations can mitigate these issues, their success has been limited by training instability. To address this, we propose a simplified theoretical framework that unifies previous parameterizations of diffusion models and CMs, identifying the root causes of instability. Based on this analysis, we introduce key improvements in diffusion process parameterization, network architecture, and training objectives. These changes enable us to train continuous-time CMs at an unprecedented scale, reaching 1.5B parameters on ImageNet 512×512. Our proposed training algorithm, using only two sampling steps, achieves FID scores of 2.06 on CIFAR-10, 1.48 on ImageNet 64×64, and 1.88 on ImageNet 512×512, narrowing the gap in FID scores with the best existing diffusion models to within 10\%.

Details

NeurIPS Conference 2025 Conference Paper

Split Gibbs Discrete Diffusion Posterior Sampling

  • Wenda Chu
  • Zihui Wu
  • Yifan Chen
  • Yang Song
  • Yisong Yue

We study the problem of posterior sampling in discrete-state spaces using discrete diffusion models. While posterior sampling methods for continuous diffusion models have achieved remarkable progress, analogous methods for discrete diffusion models remain challenging. In this work, we introduce a principled plug-and-play discrete diffusion posterior sampling algorithm based on split Gibbs sampling, which we call SGDD. Our algorithm enables reward-guided generation and solving inverse problems in discrete-state spaces. We demonstrate the convergence of SGDD to the target posterior distribution and verify this through controlled experiments on synthetic benchmarks. Our method enjoys state-of-the-art posterior sampling performance on a range of benchmarks for discrete data, including DNA sequence design, discrete image inverse problems, and music infilling, achieving more than 30% improved performance compared to existing baselines.

PDF Details

AAAI Conference 2025 Conference Paper

Trigger3:Refining Query Correction via Adaptive Model Selector

  • Kepu Zhang
  • Zhongxiang Sun
  • Xiao Zhang
  • Xiaoxue Zang
  • Kai Zheng
  • Yang Song
  • Jun Xu

In search scenarios, user experience can be hindered by erroneous queries due to typos, voice errors, or knowledge gaps. Therefore, query correction is crucial for search engines. Current correction models, usually small models trained on specific data, often struggle with queries beyond their training scope or those requiring contextual understanding. While the advent of Large Language Models (LLMs) offers a potential solution, they are still limited by their pre-training data and inference cost, particularly for complex queries, making them not always effective for query correction. To tackle these, we propose Trigger3, a large-small model collaboration framework that integrates the traditional correction model and LLM for query correction, capable of adaptively choosing the appropriate correction method based on the query and the correction results from the traditional correction model and LLM. Trigger3 first employs a correction trigger to filter out correct queries. Incorrect queries are then corrected by the traditional correction model. If this fails, an LLM trigger is activated to call the LLM for correction. Finally, for queries that no model can correct, a fallback trigger decides to return the original query. Extensive experiments demonstrate Trigger3 outperforms correction baselines while maintaining efficiency.

PDF Details DOI

AAAI Conference 2024 Conference Paper

Coupling Graph Neural Networks with Fractional Order Continuous Dynamics: A Robustness Study

  • Qiyu Kang
  • Kai Zhao
  • Yang Song
  • Yihang Xie
  • Yanan Zhao
  • Sijie Wang
  • Rui She
  • Wee Peng Tay

In this work, we rigorously investigate the robustness of graph neural fractional-order differential equation (FDE) models. This framework extends beyond traditional graph neural (integer-order) ordinary differential equation (ODE) models by implementing the time-fractional Caputo derivative. Utilizing fractional calculus allows our model to consider long-term memory during the feature updating process, diverging from the memoryless Markovian updates seen in traditional graph neural ODE models. The superiority of graph neural FDE models over graph neural ODE models has been established in environments free from attacks or perturbations. While traditional graph neural ODE models have been verified to possess a degree of stability and resilience in the presence of adversarial attacks in existing literature, the robustness of graph neural FDE models, especially under adversarial conditions, remains largely unexplored. This paper undertakes a detailed assessment of the robustness of graph neural FDE models. We establish a theoretical foundation outlining the robustness characteristics of graph neural FDE models, highlighting that they maintain more stringent output perturbation bounds in the face of input and graph topology disturbances, compared to their integer-order counterparts. Our empirical evaluations further confirm the enhanced robustness of graph neural FDE models, highlighting their potential in adversarially robust applications.

PDF Details DOI

AAAI Conference 2024 Conference Paper

Decoupled Optimisation for Long-Tailed Visual Recognition

  • Cong Cong
  • Shiyu Xuan
  • Sidong Liu
  • Shiliang Zhang
  • Maurice Pagnucco
  • Yang Song

When training on a long-tailed dataset, conventional learning algorithms tend to exhibit a bias towards classes with a larger sample size. Our investigation has revealed that this biased learning tendency originates from the model parameters, which are trained to disproportionately contribute to the classes characterised by their sample size (e.g., many, medium, and few classes). To balance the overall parameter contribution across all classes, we investigate the importance of each model parameter to the learning of different class groups, and propose a multistage parameter Decouple and Optimisation (DO) framework that decouples parameters into different groups with each group learning a specific portion of classes. To optimise the parameter learning, we apply different training objectives with a collaborative optimisation step to learn complementary information about each class group. Extensive experiments on long-tailed datasets, including CIFAR100, Places-LT, ImageNet-LT, and iNaturaList 2018, show that our framework achieves competitive performance compared to the state-of-the-art.

PDF Details DOI

ICLR Conference 2024 Conference Paper

Diffusion Posterior Sampling for Linear Inverse Problem Solving: A Filtering Perspective

  • Zehao Dou
  • Yang Song

Diffusion models have achieved tremendous success in generating high-dimensional data like images, videos and audio. These models provide powerful data priors that can solve linear inverse problems in zero shot through Bayesian posterior sampling. However, exact posterior sampling for diffusion models is intractable. Current solutions often hinge on approximations that are either computationally expensive or lack strong theoretical guarantees. In this work, we introduce an efficient diffusion sampling algorithm for linear inverse problems that is guaranteed to be asymptotically accurate. We reveal a link between Bayesian posterior sampling and Bayesian filtering in diffusion models, proving the former as a specific instance of the latter. Our method, termed filtering posterior sampling, leverages sequential Monte Carlo methods to solve the corresponding filtering problem. It seamlessly integrates with all Markovian diffusion samplers, requires no model re-training, and guarantees accurate samples from the Bayesian posterior as particle counts rise. Empirical tests demonstrate that our method generates better or comparable results than leading zero-shot diffusion posterior samplers on tasks like image inpainting, super-resolution, and deblurring.

Details

AAAI Conference 2024 Conference Paper

DistilVPR: Cross-Modal Knowledge Distillation for Visual Place Recognition

  • Sijie Wang
  • Rui She
  • Qiyu Kang
  • Xingchao Jian
  • Kai Zhao
  • Yang Song
  • Wee Peng Tay

The utilization of multi-modal sensor data in visual place recognition (VPR) has demonstrated enhanced performance compared to single-modal counterparts. Nonetheless, integrating additional sensors comes with elevated costs and may not be feasible for systems that demand lightweight operation, thereby impacting the practical deployment of VPR. To address this issue, we resort to knowledge distillation, which empowers single-modal students to learn from cross-modal teachers without introducing additional sensors during inference. Despite the notable advancements achieved by current distillation approaches, the exploration of feature relationships remains an under-explored area. In order to tackle the challenge of cross-modal distillation in VPR, we present DistilVPR, a novel distillation pipeline for VPR. We propose leveraging feature relationships from multiple agents, including self-agents and cross-agents for teacher and student neural networks. Furthermore, we integrate various manifolds, characterized by different space curvatures for exploring feature relationships. This approach enhances the diversity of feature relationships, including Euclidean, spherical, and hyperbolic relationship modules, thereby enhancing the overall representational capacity. The experiments demonstrate that our proposed pipeline achieves state-of-the-art performance compared to other distillation baselines. We also conduct necessary ablation studies to show design effectiveness. The code is released at: https://github.com/sijieaaa/DistilVPR

PDF Details DOI

AAAI Conference 2024 Conference Paper

Enhancing Job Recommendation through LLM-Based Generative Adversarial Networks

  • Yingpeng Du
  • Di Luo
  • Rui Yan
  • Xiaopei Wang
  • Hongzhi Liu
  • Hengshu Zhu
  • Yang Song
  • Jie Zhang

Recommending suitable jobs to users is a critical task in online recruitment platforms. While existing job recommendation methods encounter challenges such as the low quality of users' resumes, which hampers their accuracy and practical effectiveness.With the rapid development of large language models (LLMs), utilizing the rich external knowledge encapsulated within them, as well as their powerful reasoning capabilities, is a promising way to complete users' resumes for more accurate recommendations. However, directly leveraging LLMs to enhance recommendation results is not a one-size-fits-all solution, as LLMs may suffer from fabricated generation and few-shot problems, which degrade the quality of resume completion. In this paper, we propose a novel LLM-based approach for job recommendation. To alleviate the limitation of fabricated generation for LLMs, we extract accurate and valuable information beyond users' self-description, which helps the LLMs better profile users for resume completion. Specifically, we not only extract users' explicit properties (e.g., skills, interests) from their self-description but also infer users' implicit characteristics from their behaviors for more accurate and meaningful resume completion. Nevertheless, some users still suffer from few-shot problems, which arise due to scarce interaction records, leading to limited guidance for high-quality resume generation. To address this issue, we propose aligning unpaired low-quality with high-quality generated resumes by Generative Adversarial Networks (GANs), which can refine the resume representations for better recommendation results. Extensive experiments on three large real-world recruitment datasets demonstrate the effectiveness of our proposed method.

PDF Details DOI

IJCAI Conference 2024 Conference Paper

Federated Adaptation for Foundation Model-based Recommendations

  • Chunxu Zhang
  • Guodong Long
  • Hongkuan Guo
  • Xiao Fang
  • Yang Song
  • Zhaojie Liu
  • Guorui Zhou
  • Zijian Zhang

With the recent success of large language models, particularly foundation models with generalization abilities, applying foundation models for recommendations becomes a new paradigm to improve existing recommendation systems. It becomes a new open challenge to enable the foundation model to capture user preference changes in a timely manner with reasonable communication and computation costs while preserving privacy. This paper proposes a novel federated adaptation mechanism to enhance the foundation model-based recommendation system in a privacy-preserving manner. Specifically, each client will learn a lightweight personalized adapter using its private data. The adapter then collaborates with pre-trained foundation models to provide recommendation service efficiently with fine-grained manners. Importantly, users' private behavioral data remains secure as it is not shared with the server. This data localization-based privacy preservation is embodied via the federated learning framework. The model can ensure that shared knowledge is incorporated into all adapters while simultaneously preserving each user's personal preferences. Experimental results on four benchmark datasets demonstrate our method's superior performance. The code is available.

PDF Details DOI

IJCAI Conference 2024 Conference Paper

Formalisation and Evaluation of Properties for Consequentialist Machine Ethics

  • Raynaldio Limarga
  • Yang Song
  • Abhaya Nayak
  • David Rajaratnam
  • Maurice Pagnucco

As artificial intelligence (AI) technologies continue to influence our daily lives, there has been a growing need to ensure that AI enabled decision making systems adhere to principles expected of human decision makers. This need has given rise to the area of Machine Ethics. We formalise several ethical principles from the philosophical literature in the situation calculus framework to verify the ethical permissibility of a plan. Moreover, we propose several important properties, including some of our own that are intuitively appealing, and a number derived from the social choice literature that would appear to be relevant in evaluating the various approaches. Finally we provide an assessment of how our various situation calculus models of Machine Ethics that we examine satisfy the important properties we have identified.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

Fully Distributed, Flexible Compositional Visual Representations via Soft Tensor Products

  • Bethia Sun
  • Maurice Pagnucco
  • Yang Song

Since the inception of the classicalist vs. connectionist debate, it has been argued that the ability to systematically combine symbol-like entities into compositional representations is crucial for human intelligence. In connectionist systems, the field of disentanglement has gained prominence for its ability to produce explicitly compositional representations; however, it relies on a fundamentally symbolic, concatenative representation of compositional structure that clashes with the continuous, distributed foundations of deep learning. To resolve this tension, we extend Smolensky's Tensor Product Representation (TPR) and introduce Soft TPR, a representational form that encodes compositional structure in an inherently distributed, flexible manner, along with Soft TPR Autoencoder, a theoretically-principled architecture designed specifically to learn Soft TPRs. Comprehensive evaluations in the visual representation learning domain demonstrate that the Soft TPR framework consistently outperforms conventional disentanglement alternatives -- achieving state-of-the-art disentanglement, boosting representation learner convergence, and delivering superior sample efficiency and low-sample regime performance in downstream tasks. These findings highlight the promise of a distributed and flexible approach to representing compositional structure by potentially enhancing alignment with the core principles of deep learning over the conventional symbolic approach.

PDF Details DOI

ICLR Conference 2024 Conference Paper

Improved Techniques for Training Consistency Models

  • Yang Song
  • Prafulla Dhariwal

Consistency models are a nascent family of generative models that can sample high quality data in one step without the need for adversarial training. Current consistency models achieve optimal sample quality by distilling from pre-trained diffusion models and employing learned metrics such as LPIPS. However, distillation limits the quality of consistency models to that of the pre-trained diffusion model, and LPIPS causes undesirable bias in evaluation. To tackle these challenges, we present improved techniques for consistency training, where consistency models learn directly from data without distillation. We delve into the theory behind consistency training and identify a previously overlooked flaw, which we address by eliminating Exponential Moving Average from the teacher consistency model. To replace learned metrics like LPIPS, we adopt Pseudo-Huber losses from robust statistics. Additionally, we introduce a lognormal noise schedule for the consistency training objective, and propose to double total discretization steps every set number of training iterations. Combined with better hyperparameter tuning, these modifications enable consistency models to achieve FID scores of 2.51 and 3.25 on CIFAR-10 and ImageNet $64\times 64$ respectively in a single sampling step. These scores mark a 3.5$\times$ and 4$\times$ improvement compared to prior consistency training approaches. Through two-step sampling, we further reduce FID scores to 2.24 and 2.77 on these two datasets, surpassing those obtained via distillation in both one-step and two-step settings, while narrowing the gap between consistency models and other state-of-the-art generative models.

Details

NeurIPS Conference 2024 Conference Paper

Mixture of In-Context Experts Enhance LLMs' Long Context Awareness

  • Hongzhan Lin
  • Ang Lv
  • Yuhan Chen
  • Chen Zhu
  • Yang Song
  • Hengshu Zhu
  • Rui Yan

Many studies have revealed that large language models (LLMs) exhibit uneven awareness of different contextual positions. Their limited context awareness can lead to overlooking critical information and subsequent task failures. While several approaches have been proposed to enhance LLMs' context awareness, achieving both effectiveness and efficiency remains challenging. In this paper, for LLMs utilizing RoPE as position embeddings, we introduce a novel method called "Mixture of In-Context Experts" (MoICE) to address this challenge. MoICE comprises two key components: a router integrated into each attention head within LLMs and a lightweight router-only training optimization strategy: (1) MoICE views each RoPE angle as an 'in-context' expert, demonstrated to be capable of directing the attention of a head to specific contextual positions. Consequently, each attention head flexibly processes tokens using multiple RoPE angles dynamically selected by the router to attend to the needed positions. This approach mitigates the risk of overlooking essential contextual information. (2) The router-only training strategy entails freezing LLM parameters and exclusively updating routers for only a few steps. When applied to open-source LLMs including Llama and Mistral, MoICE surpasses prior methods across multiple tasks on long context understanding and generation, all while maintaining commendable inference efficiency.

PDF Details DOI

AAAI Conference 2024 Conference Paper

PosDiffNet: Positional Neural Diffusion for Point Cloud Registration in a Large Field of View with Perturbations

  • Rui She
  • Sijie Wang
  • Qiyu Kang
  • Kai Zhao
  • Yang Song
  • Wee Peng Tay
  • Tianyu Geng
  • Xingchao Jian

Point cloud registration is a crucial technique in 3D computer vision with a wide range of applications. However, this task can be challenging, particularly in large fields of view with dynamic objects, environmental noise, or other perturbations. To address this challenge, we propose a model called PosDiffNet. Our approach performs hierarchical registration based on window-level, patch-level, and point-level correspondence. We leverage a graph neural partial differential equation (PDE) based on Beltrami flow to obtain high-dimensional features and position embeddings for point clouds. We incorporate position embeddings into a Transformer module based on a neural ordinary differential equation (ODE) to efficiently represent patches within points. We employ the multi-level correspondence derived from the high feature similarity scores to facilitate alignment between point clouds. Subsequently, we use registration methods such as SVD-based algorithms to predict the transformation using corresponding point pairs. We evaluate PosDiffNet on several 3D point cloud datasets, verifying that it achieves state-of-the-art (SOTA) performance for point cloud registration in large fields of view with perturbations. The implementation code of experiments is available at https://github.com/AI-IT-AVs/PosDiffNet.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

RectifID: Personalizing Rectified Flow with Anchored Classifier Guidance

  • Zhicheng Sun
  • Zhenhao Yang
  • Yang Jin
  • Haozhe Chi
  • Kun Xu
  • Liwei Chen
  • Hao Jiang
  • Yang Song

Customizing diffusion models to generate identity-preserving images from user-provided reference images is an intriguing new problem. The prevalent approaches typically require training on extensive domain-specific images to achieve identity preservation, which lacks flexibility across different use cases. To address this issue, we exploit classifier guidance, a training-free technique that steers diffusion models using an existing classifier, for personalized image generation. Our study shows that based on a recent rectified flow framework, the major limitation of vanilla classifier guidance in requiring a special classifier can be resolved with a simple fixed-point solution, allowing flexible personalization with off-the-shelf image discriminators. Moreover, its solving procedure proves to be stable when anchored to a reference flow trajectory, with a convergence guarantee. The derived method is implemented on rectified flow with different off-the-shelf image discriminators, delivering advantageous personalization results for human faces, live subjects, and certain objects. Code is available at https: //github. com/feifeiobama/RectifID.

PDF Details DOI

TMLR Journal 2024 Journal Article

SwinGNN: Rethinking Permutation Invariance in Diffusion Models for Graph Generation

  • Qi Yan
  • Zhengyang Liang
  • Yang Song
  • Renjie Liao
  • Lele Wang

Permutation-invariant diffusion models of graphs achieve the invariant sampling and invariant loss functions by restricting architecture designs, which often sacrifice empirical performances. In this work, we first show that the performance degradation may also be contributed by the increasing modes of target distributions brought by invariant architectures since 1) the optimal one-step denoising scores are score functions of Gaussian mixtures models (GMMs) whose components center on these modes and 2) learning the scores of GMMs with more components is often harder. Motivated by the analysis, we propose SwinGNN along with a simple yet provable trick that enables permutation-invariant sampling. It benefits from more flexible (non-invariant) architecture designs and permutation-invariant sampling. We further design an efficient 2-WL message passing network using the shifted-window self-attention. Extensive experiments on synthetic and real-world protein and molecule datasets show that SwinGNN outperforms existing methods by a substantial margin on most metrics. Our code is released at https://github.com/qiyan98/SwinGNN.

PDF Details

AAAI Conference 2024 Conference Paper

Your Career Path Matters in Person-Job Fit

  • Zhuocheng Gong
  • Yang Song
  • Tao Zhang
  • Ji-Rong Wen
  • Dongyan Zhao
  • Rui Yan

We are again confronted with one of the most vexing aspects of the advancement of technology: automation and AI technology cause the devaluation of human labor, resulting in unemployment. With this background, automatic person-job fit systems are promising solutions to promote the employment rate. The purpose of person-job fit is to calculate a matching score between the job seeker's resume and the job posting, determining whether the job seeker is suitable for the position. In this paper, we propose a new approach to person-job fit that characterizes the hidden preference derived from the job seeker's career path. We categorize and utilize three types of preferences in the career path: consistency, likeness, and continuity. We prove that understanding the career path enables us to provide more appropriate career suggestions to job seekers. To demonstrate the practical value of our proposed model, we conduct extensive experiments on real-world data extracted from an online recruitment platform and then present detailed cases to show how the career path matter in person-job fit.

PDF Details DOI

NeurIPS Conference 2023 Conference Paper

Adversarial Robustness in Graph Neural Networks: A Hamiltonian Approach

  • Kai Zhao
  • Qiyu Kang
  • Yang Song
  • Rui She
  • Sijie Wang
  • Wee Peng Tay

Graph neural networks (GNNs) are vulnerable to adversarial perturbations, including those that affect both node features and graph topology. This paper investigates GNNs derived from diverse neural flows, concentrating on their connection to various stability notions such as BIBO stability, Lyapunov stability, structural stability, and conservative stability. We argue that Lyapunov stability, despite its common use, does not necessarily ensure adversarial robustness. Inspired by physics principles, we advocate for the use of conservative Hamiltonian neural flows to construct GNNs that are robust to adversarial attacks. The adversarial robustness of different neural flow GNNs is empirically compared on several benchmark datasets under a variety of adversarial attacks. Extensive numerical experiments demonstrate that GNNs leveraging conservative Hamiltonian flows with Lyapunov stability substantially improve robustness against adversarial perturbations. The implementation code of experiments is available at \url{https: //github. com/zknus/NeurIPS-2023-HANG-Robustness}.

PDF Details

ICML Conference 2023 Conference Paper

Consistency Models

  • Yang Song
  • Prafulla Dhariwal
  • Mark Chen 0003
  • Ilya Sutskever

Diffusion models have significantly advanced the fields of image, audio, and video generation, but they depend on an iterative sampling process that causes slow generation. To overcome this limitation, we propose consistency models, a new family of models that generate high quality samples by directly mapping noise to data. They support fast one-step generation by design, while still allowing multistep sampling to trade compute for sample quality. They also support zero-shot data editing, such as image inpainting, colorization, and super-resolution, without requiring explicit training on these tasks. Consistency models can be trained either by distilling pre-trained diffusion models, or as standalone generative models altogether. Through extensive experiments, we demonstrate that they outperform existing distillation techniques for diffusion models in one- and few-step sampling, achieving the new state-of-the-art FID of 3. 55 on CIFAR-10 and 6. 20 on ImageNet 64x64 for one-step generation. When trained in isolation, consistency models become a new family of generative models that can outperform existing one-step, non-adversarial generative models on standard benchmarks such as CIFAR-10, ImageNet 64x64 and LSUN 256x256.

Details

YNIMG Journal 2023 Journal Article

Deep fiber clustering: Anatomically informed fiber clustering with self-supervised deep learning for fast and effective tractography parcellation

  • Yuqian Chen
  • Chaoyi Zhang
  • Tengfei Xue
  • Yang Song
  • Nikos Makris
  • Yogesh Rathi
  • Weidong Cai
  • Fan Zhang

White matter fiber clustering is an important strategy for white matter parcellation, which enables quantitative analysis of brain connections in health and disease. In combination with expert neuroanatomical labeling, data-driven white matter fiber clustering is a powerful tool for creating atlases that can model white matter anatomy across individuals. While widely used fiber clustering approaches have shown good performance using classical unsupervised machine learning techniques, recent advances in deep learning reveal a promising direction toward fast and effective fiber clustering. In this work, we propose a novel deep learning framework for white matter fiber clustering, Deep Fiber Clustering (DFC), which solves the unsupervised clustering problem as a self-supervised learning task with a domain-specific pretext task to predict pairwise fiber distances. This process learns a high-dimensional embedding feature representation for each fiber, regardless of the order of fiber points reconstructed during tractography. We design a novel network architecture that represents input fibers as point clouds and allows the incorporation of additional sources of input information from gray matter parcellation. Thus, DFC makes use of combined information about white matter fiber geometry and gray matter anatomy to improve the anatomical coherence of fiber clusters. In addition, DFC conducts outlier removal naturally by rejecting fibers with low cluster assignment probability. We evaluate DFC on three independently acquired cohorts, including data from 220 individuals across genders, ages (young and elderly adults), and different health conditions (healthy control and multiple neuropsychiatric disorders). We compare DFC to several state-of-the-art white matter fiber clustering algorithms. Experimental results demonstrate superior performance of DFC in terms of cluster compactness, generalization ability, anatomical coherence, and computational efficiency.

Details DOI

IJCAI Conference 2023 Conference Paper

Graph Neural Convection-Diffusion with Heterophily

  • Kai Zhao
  • Qiyu Kang
  • Yang Song
  • Rui She
  • Sijie Wang
  • Wee Peng Tay

Graph neural networks (GNNs) have shown promising results across various graph learning tasks, but they often assume homophily, which can result in poor performance on heterophilic graphs. The connected nodes are likely to be from different classes or have dissimilar features on heterophilic graphs. In this paper, we propose a novel GNN that incorporates the principle of heterophily by modeling the flow of information on nodes using the convection-diffusion equation (CDE). This allows the CDE to take into account both the diffusion of information due to homophily and the ``convection'' of information due to heterophily. We conduct extensive experiments, which suggest that our framework can achieve competitive performance on node classification tasks for heterophilic graphs, compared to the state-of-the-art methods. The code is available at https: //github. com/zknus/Graph-Diffusion-CDE.

PDF Details DOI

YNIMG Journal 2022 Journal Article

Computer-aided extraction of select MRI markers of cerebral small vessel disease: A systematic review

  • Jiyang Jiang
  • Dadong Wang
  • Yang Song
  • Perminder S. Sachdev
  • Wei Wen

Cerebral small vessel disease (CSVD) is a major vascular contributor to cognitive impairment in ageing, including dementias. Imaging remains the most promising method for in vivo studies of CSVD. To replace the subjective and laborious visual rating approaches, emerging studies have applied state-of-the-art artificial intelligence to extract imaging biomarkers of CSVD from MRI scans. We aimed to summarise published computer-aided methods for the examination of three imaging biomarkers of CSVD, namely cerebral microbleeds (CMB), dilated perivascular spaces (PVS), and lacunes of presumed vascular origin. Seventy classical image processing, classical machine learning, and deep learning studies were identified. Transfer learning and weak supervision techniques have been applied to accommodate the limitations in the training data. While good performance metrics were achieved in local datasets, there have not been generalisable pipelines validated in different research and/or clinical cohorts. Future studies could consider pooling data from multiple sources to increase data size and diversity, and evaluating performance using both image processing metrics and associations with clinical measures.

Details DOI

JBHI Journal 2022 Journal Article

Multiple Sclerosis Lesion Analysis in Brain Magnetic Resonance Images: Techniques and Clinical Applications

  • Yang Ma
  • Chaoyi Zhang
  • Mariano Cabezas
  • Yang Song
  • Zihao Tang
  • Dongnan Liu
  • Weidong Cai
  • Michael Barnett

Multiple sclerosis (MS) is a chronic inflammatory and degenerative disease of the central nervous system, characterized by the appearance of focal lesions in the white and gray matter that topographically correlate with an individual patient’s neurological symptoms and signs. Magnetic resonance imaging (MRI) provides detailed in-vivo structural information, permitting the quantification and categorization of MS lesions that critically inform disease management. Traditionally, MS lesions have been manually annotated on 2D MRI slices, a process that is inefficient and prone to inter-/intra-observer errors. Recently, automated statistical imaging analysis techniques have been proposed to detect and segment MS lesions based on MRI voxel intensity. However, their effectiveness is limited by the heterogeneity of both MRI data acquisition techniques and the appearance of MS lesions. By learning complex lesion representations directly from images, deep learning techniques have achieved remarkable breakthroughs in the MS lesion segmentation task. Here, we provide a comprehensive review of state-of-the-art automatic statistical and deep-learning MS segmentation methods and discuss current and future clinical applications. Further, we review technical strategies, such as domain adaptation, to enhance MS lesion segmentation in real-world clinical settings.

Details DOI

NeurIPS Conference 2022 Conference Paper

On the Robustness of Graph Neural Diffusion to Topology Perturbations

  • Yang Song
  • Qiyu Kang
  • Sijie Wang
  • Kai Zhao
  • Wee Peng Tay

Neural diffusion on graphs is a novel class of graph neural networks that has attracted increasing attention recently. The capability of graph neural partial differential equations (PDEs) in addressing common hurdles of graph neural networks (GNNs), such as the problems of over-smoothing and bottlenecks, has been investigated but not their robustness to adversarial attacks. In this work, we explore the robustness properties of graph neural PDEs. We empirically demonstrate that graph neural PDEs are intrinsically more robust against topology perturbation as compared to other GNNs. We provide insights into this phenomenon by exploiting the stability of the heat semigroup under graph topology perturbations. We discuss various graph diffusion operators and relate them to existing graph neural PDEs. Furthermore, we propose a general graph neural PDE framework based on which a new class of robust GNNs can be defined. We verify that the new model achieves comparable state-of-the-art performance on several benchmark datasets.

PDF Details

EAAI Journal 2021 Journal Article

Computer vision detection of foreign objects in coal processing using attention CNN

  • Kanghui Zhang
  • Weidong Wang
  • Ziqi Lv
  • Yuhan Fan
  • Yang Song

Foreign objects in coal seriously affect the efficiency and safety of clean coal production. Currently, the removal of foreign objects in coal preparation plant mainly depends on manual picking, which has disadvantages of high labor intensity and low efficiency. Therefore, there is an urgent need for rapid detection and removal of foreign objects. However, due to the inference of the background and surround objects, it is a challenge for the accurate detection of foreign objects. In this study, a convolutional neural network (CNN) with attention modules was designed to accurately segment foreign objects from a complex background in real-time. The proposed network consists of an encoder and a decoder, and the attention mechanism was introduced into the decoder to capture rich semantic information. The visualization results proved that the attention modules could focus on the features of the salient region and inhibit the irrelevant background, which significantly improved the accuracy of the detection The results showed that the proposed model correctly recognized 97% of the foreign objects in the 1871 sets of test images. The mean intersection over union (MIOU) of the optimal model was 91. 24%, and the inference speed was greater than 15 fps/s, which satisfied the real-time requirement.

Details DOI

NeurIPS Conference 2021 Conference Paper

CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputation

  • Yusuke Tashiro
  • Jiaming Song
  • Yang Song
  • Stefano Ermon

The imputation of missing values in time series has many applications in healthcare and finance. While autoregressive models are natural candidates for time series imputation, score-based diffusion models have recently outperformed existing counterparts including autoregressive models in many tasks such as image generation and audio synthesis, and would be promising for time series imputation. In this paper, we propose Conditional Score-based Diffusion model (CSDI), a novel time series imputation method that utilizes score-based diffusion models conditioned on observed data. Unlike existing score-based approaches, the conditional diffusion model is explicitly trained for imputation and can exploit correlations between observed values. On healthcare and environmental data, CSDI improves by 40-65% over existing probabilistic imputation methods on popular performance metrics. In addition, deterministic imputation by CSDI reduces the error by 5-20% compared to the state-of-the-art deterministic imputation methods. Furthermore, CSDI can also be applied to time series interpolation and probabilistic forecasting, and is competitive with existing baselines. The code is available at https: //github. com/ermongroup/CSDI.

PDF Details

AAAI Conference 2021 Conference Paper

Error-Correcting Output Codes with Ensemble Diversity for Robust Learning in Neural Networks

  • Yang Song
  • Qiyu Kang
  • Wee Peng Tay

Though deep learning has been applied successfully in many scenarios, malicious inputs with human-imperceptible perturbations can make it vulnerable in real applications. This paper proposes an error-correcting neural network (ECNN) that combines a set of binary classifiers to combat adversarial examples in the multi-class classification problem. To build an ECNN, we propose to design a code matrix so that the minimum Hamming distance between any two rows (i. e. , two codewords) and the minimum shared information distance between any two columns (i. e. , two partitions of class labels) are simultaneously maximized. Maximizing row distances can increase the system fault tolerance while maximizing column distances helps increase the diversity between binary classifiers. We propose an end-to-end training method for our ECNN, which allows further improvement of the diversity between binary classifiers. The end-to-end training renders our proposed ECNN different from the traditional error-correcting output code (ECOC) based methods that train binary classifiers independently. ECNN is complementary to other existing defense approaches such as adversarial training and can be applied in conjunction with them. We empirically demonstrate that our proposed ECNN is effective against the state-of-the-art white-box and black-box attacks on several datasets while maintaining good classification accuracy on normal examples.

PDF Details

NeurIPS Conference 2021 Conference Paper

Estimating High Order Gradients of the Data Distribution by Denoising

  • Chenlin Meng
  • Yang Song
  • Wenzhe Li
  • Stefano Ermon

The first order derivative of a data density can be estimated efficiently by denoising score matching, and has become an important component in many applications, such as image generation and audio synthesis. Higher order derivatives provide additional local information about the data distribution and enable new applications. Although they can be estimated via automatic differentiation of a learned density model, this can amplify estimation errors and is expensive in high dimensional settings. To overcome these limitations, we propose a method to directly estimate high order derivatives (scores) of a data density from samples. We first show that denoising score matching can be interpreted as a particular case of Tweedie’s formula. By leveraging Tweedie’s formula on higher order moments, we generalize denoising score matching to estimate higher order derivatives. We demonstrate empirically that models trained with the proposed method can approximate second order derivatives more efficiently and accurately than via automatic differentiation. We show that our models can be used to quantify uncertainty in denoising and to improve the mixing speed of Langevin dynamics via Ozaki discretization for sampling synthetic data and natural images.

PDF Details

NeurIPS Conference 2021 Conference Paper

Imitation with Neural Density Models

  • Kuno Kim
  • Akshat Jindal
  • Yang Song
  • Jiaming Song
  • Yanan Sui
  • Stefano Ermon

We propose a new framework for Imitation Learning (IL) via density estimation of the expert's occupancy measure followed by Maximum Occupancy Entropy Reinforcement Learning (RL) using the density as a reward. Our approach maximizes a non-adversarial model-free RL objective that provably lower bounds reverse Kullback–Leibler divergence between occupancy measures of the expert and imitator. We present a practical IL algorithm, Neural Density Imitation (NDI), which obtains state-of-the-art demonstration efficiency on benchmark control tasks.

PDF Details

NeurIPS Conference 2021 Conference Paper

Maximum Likelihood Training of Score-Based Diffusion Models

  • Yang Song
  • Conor Durkan
  • Iain Murray
  • Stefano Ermon

Score-based diffusion models synthesize samples by reversing a stochastic process that diffuses data to noise, and are trained by minimizing a weighted combination of score matching losses. The log-likelihood of score-based diffusion models can be tractably computed through a connection to continuous normalizing flows, but log-likelihood is not directly optimized by the weighted combination of score matching losses. We show that for a specific weighting scheme, the objective upper bounds the negative log-likelihood, thus enabling approximate maximum likelihood training of score-based diffusion models. We empirically observe that maximum likelihood training consistently improves the likelihood of score-based diffusion models across multiple datasets, stochastic processes, and model architectures. Our best models achieve negative log-likelihoods of 2. 83 and 3. 76 bits/dim on CIFAR-10 and ImageNet $32\times 32$ without any data augmentation, on a par with state-of-the-art autoregressive models on these tasks.

PDF Details

NeurIPS Conference 2021 Conference Paper

Pseudo-Spherical Contrastive Divergence

  • Lantao Yu
  • Jiaming Song
  • Yang Song
  • Stefano Ermon

Energy-based models (EBMs) offer flexible distribution parametrization. However, due to the intractable partition function, they are typically trained via contrastive divergence for maximum likelihood estimation. In this paper, we propose pseudo-spherical contrastive divergence (PS-CD) to generalize maximum likelihood learning of EBMs. PS-CD is derived from the maximization of a family of strictly proper homogeneous scoring rules, which avoids the computation of the intractable partition function and provides a generalized family of learning objectives that include contrastive divergence as a special case. Moreover, PS-CD allows us to flexibly choose various learning objectives to train EBMs without additional computational cost or variational minimax optimization. Theoretical analysis on the proposed method and extensive experiments on both synthetic data and commonly used image datasets demonstrate the effectiveness and modeling flexibility of PS-CD, as well as its robustness to data contamination, thus showing its superiority over maximum likelihood and $f$-EBMs.

PDF Details

NeurIPS Conference 2021 Conference Paper

Stable Neural ODE with Lyapunov-Stable Equilibrium Points for Defending Against Adversarial Attacks

  • Qiyu Kang
  • Yang Song
  • Qinxu Ding
  • Wee Peng Tay

Deep neural networks (DNNs) are well-known to be vulnerable to adversarial attacks, where malicious human-imperceptible perturbations are included in the input to the deep network to fool it into making a wrong classification. Recent studies have demonstrated that neural Ordinary Differential Equations (ODEs) are intrinsically more robust against adversarial attacks compared to vanilla DNNs. In this work, we propose a neural ODE with Lyapunov-stable equilibrium points for defending against adversarial attacks (SODEF). By ensuring that the equilibrium points of the ODE solution used as part of SODEF are Lyapunov-stable, the ODE solution for an input with a small perturbation converges to the same solution as the unperturbed input. We provide theoretical results that give insights into the stability of SODEF as well as the choice of regularizers to ensure its stability. Our analysis suggests that our proposed regularizers force the extracted feature points to be within a neighborhood of the Lyapunov-stable equilibrium points of the SODEF ODE. SODEF is compatible with many defense methods and can be applied to any neural network's final regressor layer to enhance its stability against adversarial attacks.

PDF Details

NeurIPS Conference 2020 Conference Paper

Autoregressive Score Matching

  • Chenlin Meng
  • Lantao Yu
  • Yang Song
  • Jiaming Song
  • Stefano Ermon

Autoregressive models use chain rule to define a joint probability distribution as a product of conditionals. These conditionals need to be normalized, imposing constraints on the functional families that can be used. To increase flexibility, we propose autoregressive conditional score models (AR-CSM) where we parameterize the joint distribution in terms of the derivatives of univariate log-conditionals (scores), which need not be normalized. To train AR-CSM, we introduce a new divergence between distributions named Composite Score Matching (CSM). For AR-CSM models, this divergence between data and model distributions can be computed and optimized efficiently, requiring no expensive sampling or adversarial training. Compared to previous score matching algorithms, our method is more scalable to high dimensional data and more stable to optimize. We show with extensive experimental results that it can be applied to density estimation on synthetic data, image generation, image denoising, and training latent variable models with implicit encoders.

PDF Details

NeurIPS Conference 2020 Conference Paper

Diversity can be Transferred: Output Diversification for White- and Black-box Attacks

  • Yusuke Tashiro
  • Yang Song
  • Stefano Ermon

Adversarial attacks often involve random perturbations of the inputs drawn from uniform or Gaussian distributions, e. g. to initialize optimization-based white-box attacks or generate update directions in black-box attacks. These simple perturbations, however, could be sub-optimal as they are agnostic to the model being attacked. To improve the efficiency of these attacks, we propose Output Diversified Sampling (ODS), a novel sampling strategy that attempts to maximize diversity in the target model's outputs among the generated samples. While ODS is a gradient-based strategy, the diversity offered by ODS is transferable and can be helpful for both white-box and black-box attacks via surrogate models. Empirically, we demonstrate that ODS significantly improves the performance of existing white-box and black-box attacks. In particular, ODS reduces the number of queries needed for state-of-the-art black-box attacks on ImageNet by a factor of two.

PDF Details

NeurIPS Conference 2020 Conference Paper

Efficient Learning of Generative Models via Finite-Difference Score Matching

  • Tianyu Pang
  • Kun Xu
  • Chongxuan Li
  • Yang Song
  • Stefano Ermon
  • Jun Zhu

Several machine learning applications involve the optimization of higher-order derivatives (e. g. , gradients of gradients) during training, which can be expensive with respect to memory and computation even with automatic differentiation. As a typical example in generative modeling, score matching~(SM) involves the optimization of the trace of a Hessian. To improve computing efficiency, we rewrite the SM objective and its variants in terms of directional derivatives, and present a generic strategy to efficiently approximate any-order directional derivative with finite difference~(FD). Our approximation only involves function evaluations, which can be executed in parallel, and no gradient computations. Thus, it reduces the total computational cost while also improving numerical stability. We provide two instantiations by reformulating variants of SM objectives into the FD forms. Empirically, we demonstrate that our methods produce results comparable to the gradient-based counterparts while being much more computationally efficient.

PDF Details

NeurIPS Conference 2020 Conference Paper

Improved Techniques for Training Score-Based Generative Models

  • Yang Song
  • Stefano Ermon

Score-based generative models can produce high quality image samples comparable to GANs, without requiring adversarial optimization. However, existing training procedures are limited to images of low resolution (typically below 32 x 32), and can be unstable under some settings. We provide a new theoretical analysis of learning and sampling from score models in high dimensional spaces, explaining existing failure modes and motivating new solutions that generalize across datasets. To enhance stability, we also propose to maintain an exponential moving average of model weights. With these improvements, we can effortlessly scale score-based generative models to images with unprecedented resolutions ranging from 64 x 64 to 256 x 256. Our score-based models can generate high-fidelity samples that rival best-in-class GANs on various image datasets, including CelebA, FFHQ, and multiple LSUN categories.

PDF Details

AAAI Conference 2020 Conference Paper

Shape-Oriented Convolution Neural Network for Point Cloud Analysis

  • Chaoyi Zhang
  • Yang Song
  • Lina Yao
  • Weidong Cai

Point cloud is a principal data structure adopted for 3D geometric information encoding. Unlike other conventional visual data, such as images and videos, these irregular points describe the complex shape features of 3D objects, which makes shape feature learning an essential component of point cloud analysis. To this end, a shape-oriented message passing scheme dubbed ShapeConv is proposed to focus on the representation learning of the underlying shape formed by each local neighboring point. Despite this intra-shape relationship learning, ShapeConv is also designed to incorporate the contextual effects from the inter-shape relationship through capturing the long-ranged dependencies between local underlying shapes. This shape-oriented operator is stacked into our hierarchical learning architecture, namely Shape-Oriented Convolutional Neural Network (SOCNN), developed for point cloud analysis. Extensive experiments have been performed to evaluate its significance in the tasks of point cloud classification and part segmentation.

PDF Details

NeurIPS Conference 2019 Conference Paper

Efficient Graph Generation with Graph Recurrent Attention Networks

  • Renjie Liao
  • Yujia Li
  • Yang Song
  • Shenlong Wang
  • Will Hamilton
  • David Duvenaud
  • Raquel Urtasun
  • Richard Zemel

We propose a new family of efficient and expressive deep generative models of graphs, called Graph Recurrent Attention Networks (GRANs). Our model generates graphs one block of nodes and associated edges at a time. The block size and sampling stride allow us to trade off sample quality for efficiency. Compared to previous RNN-based graph generative models, our framework better captures the auto-regressive conditioning between the already-generated and to-be-generated parts of the graph using Graph Neural Networks (GNNs) with attention. This not only reduces the dependency on node ordering but also bypasses the long-term bottleneck caused by the sequential nature of RNNs. Moreover, we parameterize the output distribution per block using a mixture of Bernoulli, which captures the correlations among generated edges within the block. Finally, we propose to handle node orderings in generation by marginalizing over a family of canonical orderings. On standard benchmarks, we achieve state-of-the-art time efficiency and sample quality compared to previous models. Additionally, we show our model is capable of generating large graphs of up to 5K nodes with good quality. Our code is released at: \url{https: //github. com/lrjconan/GRAN}.

PDF Details

NeurIPS Conference 2019 Conference Paper

Generative Modeling by Estimating Gradients of the Data Distribution

  • Yang Song
  • Stefano Ermon

We introduce a new generative model where samples are produced via Langevin dynamics using gradients of the data distribution estimated with score matching. Because gradients can be ill-defined and hard to estimate when the data resides on low-dimensional manifolds, we perturb the data with different levels of Gaussian noise, and jointly estimate the corresponding scores, i. e. , the vector fields of gradients of the perturbed data distribution for all noise levels. For sampling, we propose an annealed Langevin dynamics where we use gradients corresponding to gradually decreasing noise levels as the sampling process gets closer to the data manifold. Our framework allows flexible model architectures, requires no sampling during training or the use of adversarial methods, and provides a learning objective that can be used for principled model comparisons. Our models produce samples comparable to GANs on MNIST, CelebA and CIFAR-10 datasets, achieving a new state-of-the-art inception score of 8. 87 on CIFAR-10. Additionally, we demonstrate that our models learn effective representations via image inpainting experiments.

PDF Details

NeurIPS Conference 2019 Conference Paper

MintNet: Building Invertible Neural Networks with Masked Convolutions

  • Yang Song
  • Chenlin Meng
  • Stefano Ermon

We propose a new way of constructing invertible neural networks by combining simple building blocks with a novel set of composition rules. This leads to a rich set of invertible architectures, including those similar to ResNets. Inversion is achieved with a locally convergent iterative procedure that is parallelizable and very fast in practice. Additionally, the determinant of the Jacobian can be computed analytically and efficiently, enabling their generative use as flow models. To demonstrate their flexibility, we show that our invertible neural networks are competitive with ResNets on MNIST and CIFAR-10 classification. When trained as generative models, our invertible networks achieve competitive likelihoods on MNIST, CIFAR-10 and ImageNet 32x32, with bits per dimension of 0. 98, 3. 32 and 4. 06 respectively.

PDF Details

IJCAI Conference 2019 Conference Paper

Nuclei Segmentation via a Deep Panoptic Model with Semantic Feature Fusion

  • Dongnan Liu
  • Donghao Zhang
  • Yang Song
  • Chaoyi Zhang
  • Fan Zhang
  • Lauren O'Donnell
  • Weidong Cai

Automated detection and segmentation of individual nuclei in histopathology images is important for cancer diagnosis and prognosis. Due to the high variability of nuclei appearances and numerous overlapping objects, this task still remains challenging. Deep learning based semantic and instance segmentation models have been proposed to address the challenges, but these methods tend to concentrate on either the global or local features and hence still suffer from information loss. In this work, we propose a panoptic segmentation model which incorporates an auxiliary semantic segmentation branch with the instance branch to integrate global and local features. Furthermore, we design a feature map fusion mechanism in the instance branch and a new mask generator to prevent information loss. Experimental results on three different histopathology datasets demonstrate that our method outperforms the state-of-the-art nuclei segmentation methods and popular semantic and instance segmentation models by a large margin.

PDF Details

IJCAI Conference 2019 Conference Paper

Talking Face Generation by Conditional Recurrent Adversarial Network

  • Yang Song
  • Jingwen Zhu
  • Dawei Li
  • Andy Wang
  • Hairong Qi

Given an arbitrary face image and an arbitrary speech clip, the proposed work attempts to generate the talking face video with accurate lip synchronization. Existing works either do not consider temporal dependency across video frames thus yielding abrupt facial and lip movement or are limited to the generation of talking face video for a specific person thus lacking generalization capacity. We propose a novel conditional recurrent generation network that incorporates both image and audio features in the recurrent unit for temporal dependency. To achieve both image- and video-realism, a pair of spatial-temporal discriminators are included in the network for better image/video quality. Since accurate lip synchronization is essential to the success of talking face video generation, we also construct a lip-reading discriminator to boost the accuracy of lip synchronization. We also extend the network to model the natural pose and expression of talking face on the Obama Dataset. Extensive experimental results demonstrate the superiority of our framework over the state-of-the-arts in terms of visual quality, lip sync accuracy, and smooth transition pertaining to both lip and facial movement.

PDF Details

NeurIPS Conference 2018 Conference Paper

Constructing Unrestricted Adversarial Examples with Generative Models

  • Yang Song
  • Rui Shu
  • Nate Kushman
  • Stefano Ermon

Adversarial examples are typically constructed by perturbing an existing data point within a small matrix norm, and current defense methods are focused on guarding against this type of attack. In this paper, we propose a new class of adversarial examples that are synthesized entirely from scratch using a conditional generative model, without being restricted to norm-bounded perturbations. We first train an Auxiliary Classifier Generative Adversarial Network (AC-GAN) to model the class-conditional distribution over data samples. Then, conditioned on a desired class, we search over the AC-GAN latent space to find images that are likely under the generative model and are misclassified by a target classifier. We demonstrate through human evaluation that these new kind of adversarial images, which we call Generative Adversarial Examples, are legitimate and belong to the desired class. Our empirical results on the MNIST, SVHN, and CelebA datasets show that generative adversarial examples can bypass strong adversarial training and certified defense methods designed for traditional adversarial attacks.

PDF Details

AAAI Conference 2018 Conference Paper

r-BTN: Cross-Domain Face Composite and Synthesis From Limited Facial Patches

  • Yang Song
  • Zhifei Zhang
  • Hairong Qi

Recent face composite and synthesis related works have shown promising results in generating realistic face images from deep convolutional networks. However, these works either do not generate consistent results when the constituent patches contain large domain variations (i. e. , from face and sketch domains) or cannot generate high-resolution images with limited facial patches (e. g. , the inpainting approach tends to blur the generated region when the missing area is more than 50%). Motivated by the mental imagery and simulation in human cognition, we exploit the potential of deep learning networks in filling large missing region (e. g. , as high as 95% missing) and generating realistic faces with high- fidelity in cross domains. We propose the recursive generation by bidirectional transformation networks (r-BTN) that recursively generates a whole face/sketch from a small sketch/face patch. The large missing area and domain variations make it difficult to generate satisfactory results using a unidirectional cross-domain learning structure. We explore that the bidirectional transformation network can lead to the consistent result by minimizing the forward and backward errors in the crossdomain scenario. On the other hand, a forward and backward bidirectional learning between the face and sketch domains would enable recursive estimation of the missing region in an incremental manner to yield appealing results. r-BTN also adopts an adversarial constraint to encourage the generation of realistic faces/sketches. Extensive experiments have been conducted to demonstrate the superior performance from r- BTN as compared to existing potential solutions.

PDF Details

YNIMG Journal 2018 Journal Article

Suprathreshold fiber cluster statistics: Leveraging white matter geometry to enhance tractography statistical analysis

  • Fan Zhang
  • Weining Wu
  • Lipeng Ning
  • Gloria McAnulty
  • Deborah Waber
  • Borjan Gagoski
  • Kiera Sarill
  • Hesham M. Hamoda

This work presents a suprathreshold fiber cluster (STFC) method that leverages the whole brain fiber geometry to enhance statistical group difference analyses. The proposed method consists of 1) a well-established study-specific data-driven tractography parcellation to obtain white matter tract parcels and 2) a newly proposed nonparametric, permutation-test-based STFC method to identify significant differences between study populations. The basic idea of our method is that a white matter parcel's neighborhood (nearby parcels with similar white matter anatomy) can support the parcel's statistical significance when correcting for multiple comparisons. We propose an adaptive parcel neighborhood strategy to allow suprathreshold fiber cluster formation that is robust to anatomically varying inter-parcel distances. The method is demonstrated by application to a multi-shell diffusion MRI dataset from 59 individuals, including 30 attention deficit hyperactivity disorder patients and 29 healthy controls. Evaluations are conducted using both synthetic and in-vivo data. The results indicate that the STFC method gives greater sensitivity in finding group differences in white matter tract parcels compared to several traditional multiple comparison correction methods.

Details DOI

YNIMG Journal 2018 Journal Article

Whole brain white matter connectivity analysis using machine learning: An application to autism

  • Fan Zhang
  • Peter Savadjiev
  • Weidong Cai
  • Yang Song
  • Yogesh Rathi
  • Birkan Tunç
  • Drew Parker
  • Tina Kapur

In this paper, we propose an automated white matter connectivity analysis method for machine learning classification and characterization of white matter abnormality via identification of discriminative fiber tracts. The proposed method uses diffusion MRI tractography and a data-driven approach to find fiber clusters corresponding to subdivisions of the white matter anatomy. Features extracted from each fiber cluster describe its diffusion properties and are used for machine learning. The method is demonstrated by application to a pediatric neuroimaging dataset from 149 individuals, including 70 children with autism spectrum disorder (ASD) and 79 typically developing controls (TDC). A classification accuracy of 78. 33% is achieved in this cross-validation study. We investigate the discriminative diffusion features based on a two-tensor fiber tracking model. We observe that the mean fractional anisotropy from the second tensor (associated with crossing fibers) is most affected in ASD. We also find that local along-tract (central cores and endpoint regions) differences between ASD and TDC are helpful in differentiating the two groups. These altered diffusion properties in ASD are associated with multiple robustly discriminative fiber clusters, which belong to several major white matter tracts including the corpus callosum, arcuate fasciculus, uncinate fasciculus and aslant tract; and the white matter structures related to the cerebellum, brain stem, and ventral diencephalon. These discriminative fiber clusters, a small part of the whole brain tractography, represent the white matter connections that could be most affected in ASD. Our results indicate the potential of a machine learning pipeline based on white matter fiber clustering.

Details DOI

AAAI Conference 2016 Conference Paper

Bayesian Matrix Completion via Adaptive Relaxed Spectral Regularization

  • Yang Song
  • Jun Zhu

Bayesian matrix completion has been studied based on a lowrank matrix factorization formulation with promising results. However, little work has been done on Bayesian matrix completion based on the more direct spectral regularization formulation. We fill this gap by presenting a novel Bayesian matrix completion method based on spectral regularization. In order to circumvent the difficulties of dealing with the orthonormality constraints of singular vectors, we derive a new equivalent form with relaxed constraints, which then leads us to design an adaptive version of spectral regularization feasible for Bayesian inference. Our Bayesian method requires no parameter tuning and can infer the number of latent factors automatically. Experiments on synthetic and real datasets demonstrate encouraging results on rank recovery and collaborative filtering, with notably good results for very sparse matrices.

PDF Details

AAAI Conference 2016 Conference Paper

Exploring Multiple Feature Spaces for Novel Entity Discovery

  • Zhaohui Wu
  • Yang Song
  • C. Giles

Continuously discovering novel entities in news and Web data is important for Knowledge Base (KB) maintenance. One of the key challenges is to decide whether an entity mention refers to an in-KB or out-of-KB entity. We propose a principled approach that learns a novel entity classifier by modeling mention and entity representation into multiple feature spaces, including contextual, topical, lexical, neural embedding and query spaces. Different from most previous studies that address novel entity discovery as a submodule of entity linking systems, our model is more a generalized approach and can be applied as a pre-filtering step of novel entities for any entity linking systems. Experiments on three real-world datasets show that our method significantly outperforms existing methods on identifying novel entities.

PDF Details

IROS Conference 2016 Conference Paper

Forming repeating patterns of mobile robots: A provably correct decentralized algorithm

  • Yang Song
  • Jason M. O'Kane

We describe a new decentralized algorithm for multi-robot systems to form arbitrary repeated lattice patterns. Prior work showed how to represent a desired pattern using a directed graph in which each edge is labeled with a rigid body transformation, and proposed an algorithm that accepts this graph as input and computes destinations for each robot using only local information. In this paper, we improve upon that result by describing a new algorithm, substantially different both in message passing procedure and in movement strategy, to resolve several limitations of the existing algorithm. We prove that, by executing this algorithm, the robots will form the desired lattice pattern in a bounded amount of time. We further show that, if the robots' communication graph is connected at the start of the algorithm, it will remain connected throughout the algorithm's execution. Using a simulation, we demonstrate that this algorithm works correctly for systems with dozens of autonomous robots to form various lattice patterns. Moreover, the experiments show a significant improvement in solution quality for our new algorithm compared to the previous approach.

Details

NeurIPS Conference 2016 Conference Paper

Kernel Bayesian Inference with Posterior Regularization

  • Yang Song
  • Jun Zhu
  • Yong Ren

We propose a vector-valued regression problem whose solution is equivalent to the reproducing kernel Hilbert space (RKHS) embedding of the Bayesian posterior distribution. This equivalence provides a new understanding of kernel Bayesian inference. Moreover, the optimization problem induces a new regularization for the posterior embedding estimator, which is faster and has comparable performance to the squared regularization in kernel Bayes' rule. This regularization coincides with a former thresholding approach used in kernel POMDPs whose consistency remains to be established. Our theoretical work solves this open problem and provides consistency analysis in regression settings. Based on our optimizational formulation, we propose a flexible Bayesian posterior regularization framework which for the first time enables us to put regularization at the distribution level. We apply this method to nonparametric state-space filtering tasks with extremely nonlinear dynamics and show performance gains over all other baselines.

PDF Details

YNIMG Journal 2016 Journal Article

Sample-poor estimation of order and common signal subspace with application to fusion of medical imaging data

  • Yuri Levin-Schwartz
  • Yang Song
  • Peter J. Schreier
  • Vince D. Calhoun
  • Tülay Adalı

Due to their data-driven nature, multivariate methods such as canonical correlation analysis (CCA) have proven very useful for fusion of multimodal neurological data. However, being able to determine the degree of similarity between datasets and appropriate order selection are crucial to the success of such techniques. The standard methods for calculating the order of multimodal data focus only on sources with the greatest individual energy and ignore relations across datasets. Additionally, these techniques as well as the most widely-used methods for determining the degree of similarity between datasets assume sufficient sample support and are not effective in the sample-poor regime. In this paper, we propose to jointly estimate the degree of similarity between datasets and their order when few samples are present using principal component analysis and canonical correlation analysis (PCA-CCA). By considering these two problems simultaneously, we are able to minimize the assumptions placed on the data and achieve superior performance in the sample-poor regime compared to traditional techniques. We apply PCA-CCA to the pairwise combinations of functional magnetic resonance imaging (fMRI), structural magnetic resonance imaging (sMRI), and electroencephalogram (EEG) data drawn from patients with schizophrenia and healthy controls while performing an auditory oddball task. The PCA-CCA results indicate that the fMRI and sMRI datasets are the most similar, whereas the sMRI and EEG datasets share the least similarity. We also demonstrate that the degree of similarity obtained by PCA-CCA is highly predictive of the degree of significance found for components generated using CCA.

Details DOI

NeurIPS Conference 2016 Conference Paper

Stochastic Gradient Geodesic MCMC Methods

  • Chang Liu
  • Jun Zhu
  • Yang Song

We propose two stochastic gradient MCMC methods for sampling from Bayesian posterior distributions defined on Riemann manifolds with a known geodesic flow, e. g. hyperspheres. Our methods are the first scalable sampling methods on these manifolds, with the aid of stochastic gradients. Novel dynamics are conceived and 2nd-order integrators are developed. By adopting embedding techniques and the geodesic integrator, the methods do not require a global coordinate system of the manifold and do not involve inner iterations. Synthetic experiments show the validity of the method, and its application to the challenging inference for spherical topic models indicate practical usability and efficiency.

PDF Details

ICRA Conference 2014 Conference Paper

Decentralized formation of arbitrary multi-robot lattices

  • Yang Song
  • Jason M. O'Kane

In this paper, we propose a decentralized algorithm to form arbitrary repeating formations of multiple robots. Methods are known to form specific kinds of repeating structures such as squares, triangles, and hexagons by modeling each robot as a particle that responds to attractive and repulsive forces generated by nearby robots. However, such methods are generally designed by hand for one specific type of lattice. Our approach is more general, in the sense that we present a single algorithm, for which a description of the desired repeating pattern is part of the input. We represent this pattern as a directed graph, in which edges show the desired rigid body transformations between the local frames of pairs of neighbor robots. The robots autonomously organize themselves into a family of rooted trees, and use these trees to perform task assignments locally and without conflicts. We show, via our simulated implementation, that our algorithm works for robot systems with hundreds of robots to form various lattice patterns. Our experiments also show that the approach can recover rapidly from robot failures, even if those failures impact a large fraction of the robot population.

Details

ICRA Conference 2012 Conference Paper

Comparison of constrained geometric approximation strategies for planar information states

  • Yang Song
  • Jason M. O'Kane

This paper describes and analyzes a new technique for reasoning about uncertainty called constrained geometric approximation (CGA). We build upon recent work that has developed methods to explicitly represent a robot's knowledge as an element, called an information state, in an appropriately defined information space. The intuition of our new approach is to constrain the I-state to remain in a structured subset of the I-space, and to enforce that constraint using appropriate over-approximation methods. The result is a collection of algorithms that enable mobile robots with extreme limitations in both sensing and computation to maintain simple but provably mean-ingful representations of the incomplete information available to them. We present a simulated implementation of this technique for a sensor-based navigation task, along with experimental results for this task showing that CGA, compared to a high-fidelity representation of the un-approximated I-state, achieves a similar success rate at a small fraction of the computational cost.

Details

YNIMG Journal 2011 Journal Article

Surface-based TBM boosts power to detect disease effects on the brain: An N=804 ADNI study

  • Yalin Wang
  • Yang Song
  • Priya Rajagopalan
  • Tuo An
  • Krystal Liu
  • Yi-Yu Chou
  • Boris Gutman
  • Arthur W. Toga

Computational anatomy methods are now widely used in clinical neuroimaging to map the profile of disease effects on the brain and its clinical correlates. In Alzheimer's disease (AD), many research groups have modeled localized changes in hippocampal and lateral ventricular surfaces, to provide candidate biomarkers of disease progression for drug trials. We combined the power of parametric surface modeling and tensor-based morphometry to study hippocampal differences associated with AD and mild cognitive impairment (MCI) in 490 subjects (97 AD, 245 MCI, 148 controls) and ventricular differences in 804 subjects scanned as part of the Alzheimer's Disease Neuroimaging Initiative (ADNI; 184 AD, 391 MCI, 229 controls). We aimed to show that a new multivariate surface statistic based on multivariate tensor-based morphometry (mTBM) and radial distance provides a more powerful way to detect localized anatomical differences than conventional surface-based analysis. In our experiments, we studied correlations between hippocampal atrophy and ventricular enlargement and clinical measures and cerebrospinal fluid biomarkers. The new multivariate statistics gave better effect sizes for detecting morphometric differences, relative to other statistics including radial distance, analysis of the surface tensor and the Jacobian determinant. In empirical tests using false discovery rate curves, smaller sample sizes were needed to detect associations with diagnosis. The analysis pipeline is generic and automated. It may be applied to analyze other brain subcortical structures including the caudate nucleus and putamen. This publically available software may boost power for morphometric studies of subcortical structures in the brain.

Details DOI

NeurIPS Conference 2001 Conference Paper

Unsupervised Learning of Human Motion Models

  • Yang Song
  • Luis Goncalves
  • Pietro Perona

This paper presents an unsupervised learning algorithm that can derive the probabilistic dependence structure of parts of an object (a moving hu- man body in our examples) automatically from unlabeled data. The dis- tinguished part of this work is that it is based on unlabeled data, i. e. , the training features include both useful foreground parts and background clutter and the correspondence between the parts and detected features are unknown. We use decomposable triangulated graphs to depict the probabilistic independence of parts, but the unsupervised technique is not limited to this type of graph. In the new approach, labeling of the data (part assignments) is taken as hidden variables and the EM algo- rithm is applied. A greedy algorithm is developed to select parts and to search for the optimal structure based on the differential entropy of these variables. The success of our algorithm is demonstrated by applying it to generate models of human motion automatically from unlabeled real image sequences.

PDF Details
v2026.09.13