Author name cluster

Tianyu Wang

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

24 papers

2 author rows

EAAI Journal 2026 Journal Article

Multimodal knowledge-enhanced language model with online test-time adaptation for cross-domain industrial tabular prediction

Tianyu Wang
Maite Zhang
Jingbo Qu
Zhihao Liu
Mian Li
Xi Vincent Wang

Reliable prediction of key variables in industrial scenarios requires effective models for structured tabular data. Traditional data-driven methods often lack generalization across diverse data distributions, while existing transfer learning techniques face difficulties in incorporating multimodal and heterogeneous industrial features. Although Language Models (LMs) have shown strong capability in semantic tasks, their systematic adaptation to structured numerical prediction remains limited, and non-stationary factors such as sensor degradation and condition shifts further pose challenges. To address these issues, this paper proposes a cross-domain modeling method that integrates specialized LM pre-training, parameter-efficient fine-tuning, and lightweight gradient-free test-time adaptation. Numerical, categorical, and textual variables are embedded into unified multimodal representations and processed by a transformer-based LM architecture. Pre-training leverages both general text corpora and heterogeneous industrial tabular datasets, followed by low-rank adaptation for efficient domain-specific fine-tuning. During online inference, model representations are further refined through prompt optimization with covariance matrix adaptation evolution strategy to handle deployment-time distribution shifts. Extensive experiments on 20 industrial datasets across multiple sectors demonstrate that the proposed method achieves consistently strong performance compared with state-of-the-art tabular prediction and transfer learning baselines, validating its effectiveness and robustness for real-world industrial prediction tasks

Details DOI

ICLR Conference 2025 Conference Paper

Mutual Effort for Efficiency: A Similarity-based Token Pruning for Vision Transformers in Self-Supervised Learning

Sheng Li 0019
Qitao Tan
Yue Dai 0005
Zhenglun Kong
Tianyu Wang
Jun Liu 0075
Ao Li 0004
Ninghao Liu 0001

Self-supervised learning (SSL) offers a compelling solution to the challenge of extensive labeled data requirements in traditional supervised learning. With the proven success of Vision Transformers (ViTs) in supervised tasks, there is increasing interest in adapting them for SSL frameworks. However, the high computational demands of SSL pose substantial challenges, particularly on resource-limited platforms like edge devices, despite its ability to achieve high accuracy without labeled data. Recent studies in supervised learning have shown that token pruning can reduce training costs by removing less informative tokens without compromising accuracy. However, SSL’s dual-branch encoders make traditional single-branch pruning strategies less effective, as they fail to account for the critical cross-branch similarity information, leading to reduced accuracy in SSL. To this end, we introduce SimPrune, a novel token pruning strategy designed for ViTs in SSL. SimPrune leverages cross-branch similarity information to efficiently prune tokens, retaining essential semantic information across dual branches. Additionally, we incorporate a difficulty-aware pruning strategy to further enhance SimPrune's effectiveness. Experimental results show that our proposed approach effectively reduces training computation while maintaining accuracy. Specifically, our approach offers 24\% savings in training costs compared to SSL baseline, without sacrificing accuracy.

Details

NeurIPS Conference 2025 Conference Paper

OmniVCus: Feedforward Subject-driven Video Customization with Multimodal Control Conditions

Yuanhao Cai
He Zhang
Xi Chen
Jinbo Xing
Yiwei Hu
Yuqian Zhou
Kai Zhang
Zhifei Zhang

Existing feedforward subject-driven video customization methods mainly study single-subject scenarios due to the difficulty of constructing multi-subject training data pairs. Another challenging problem that how to use the signals such as depth, mask, camera, and text prompts to control and edit the subject in the customized video is still less explored. In this paper, we first propose a data construction pipeline, VideoCus-Factory, to produce training data pairs for multi-subject customization from raw videos without labels and control signals such as depth-to-video and mask-to-video pairs. Based on our constructed data, we develop an Image-Video Transfer Mixed (IVTM) training with image editing data to enable instructive editing for the subject in the customized video. Then we propose a diffusion Transformer framework, OmniVCus, with two embedding mechanisms, Lottery Embedding (LE) and Temporally Aligned Embedding (TAE). LE enables inference with more subjects by using the training subjects to activate more frame embeddings. TAE encourages the generation process to extract guidance from temporally aligned control signals by assigning the same frame embeddings to the control and noise tokens. Experiments demonstrate that our method significantly surpasses state-of-the-art methods in both quantitative and qualitative evaluations. Project page is at https: //caiyuanhao1998. github. io/project/OmniVCus/

PDF Details

IROS Conference 2025 Conference Paper

RAG-6DPose: Retrieval-Augmented 6D Pose Estimation via Leveraging CAD as Knowledge Base

Kuanning Wang
Yuqian Fu
Tianyu Wang
Yanwei Fu 0001
Longfei Liang
Yu-Gang Jiang 0001
Xiangyang Xue 0001

Accurate 6D pose estimation is key for robotic manipulation, enabling precise object localization for tasks like grasping. We present RAG-6DPose, a retrieval-augmented approach that leverages 3D CAD models as a knowledge base by integrating both visual and geometric cues. Our RAG-6DPose roughly contains three stages: 1) Building a Multi-Modal CAD Knowledge Base by extracting 2D visual features from multi-view CAD rendered images and also attaching 3D points; 2) Retrieving relevant CAD features from the knowledge base based on the current query image via our ReSPC module; and 3) Incorporating retrieved CAD information to refine pose predictions via retrieval-augmented decoding. Experimental results on standard benchmarks and real-world robotic tasks demonstrate the effectiveness and robustness of our approach, particularly in handling occlusions and novel viewpoints. Supplementary material is available on our project website: https://sressers.github.io/RAG-6DPose.

Details

NeurIPS Conference 2025 Conference Paper

SmartCache: Context-aware Semantic Cache for Efficient Multi-turn LLM Inference

Chengye Yu
Tianyu Wang
Zili Shao
Song Jiang

Large Language Models (LLMs) for multi-turn conversations suffer from inefficiency: semantically similar queries across different user sessions trigger redundant computation and duplicate memory-intensive Key-Value (KV) caches. Existing optimizations such as prefix caching overlook semantic similarities, while typical semantic caches either ignore conversational context or are not integrated with low-level KV cache management. We propose SmartCache, a system-algorithm co-design framework that tackles this inefficiency by exploiting semantic query similarity across sessions. SmartCache leverages a Semantic Forest structure to hierarchically index conversational turns, enabling efficient retrieval and reuse of responses only when both the semantic query and conversational context match. To maintain accuracy during topic shifts, it leverages internal LLM attention scores—computed during standard prefill—to dynamically detect context changes with minimal computational overhead. Importantly, this semantic understanding is co-designed alongside the memory system: a novel two-level mapping enables transparent cross-session KV cache sharing for semantically equivalent states, complemented by a semantics-aware eviction policy that significantly improves memory utilization. This holistic approach significantly reduces redundant computations and optimizes GPU memory utilization. The evaluation demonstrates SmartCache's effectiveness across multiple benchmarks. On the CoQA and SQuAD datasets, SmartCache reduces KV cache memory usage by up to $59. 1\%$ compared to prefix caching and $56. 0\%$ over semantic caching, while cutting Time-to-First-Token (TTFT) by $78. 0\%$ and $71. 7\%$, respectively. It improves answer quality metrics, achieving $39. 9\%$ higher F1 and $39. 1\%$ higher ROUGE-L for Qwen-2. 5-1. 5B on CoQA. The Semantic-aware Tiered Eviction Policy (STEP) outperforms LRU/LFU by $29. 9\%$ in reuse distance under skewed workloads.

PDF Details

ICRA Conference 2025 Conference Paper

You Only Estimate Once: Unified, One-stage, Real-Time Category-Level Articulated Object 6D Pose Estimation for Robotic Grasping

Jingshun Huang
Haitao Lin
Tianyu Wang
Yanwei Fu 0001
Yu-Gang Jiang 0001
Xiangyang Xue 0001

This paper addresses the problem of category-level pose estimation for articulated objects in robotic manipulation tasks. Recent works have shown promising results in estimating part pose and size at the category level. However, these approaches primarily follow a complex multi-stage pipeline that first segments part instances in the point cloud and then estimates the Normalized Part Coordinate Space (NPCS) representation for 6D poses. These approaches suffer from high computational costs and low performance in real-time robotic tasks. To address these limitations, we propose YOEO, a single-stage method that simultaneously outputs instance segmentation and NPCS representations in an end-to-end manner. We use a unified network to generate point-wise semantic labels and centroid offsets, allowing points from the same part instance to vote for the same centroid. We further utilize a clustering algorithm to distinguish points based on their estimated centroid distances. Finally, we first separate the NPCS region of each instance. Then, we align the separated regions with the real point cloud to recover the final pose and size. Experimental results on the GAPart dataset demonstrate the pose estimation capabilities of our proposed single-shot method. We also deploy our synthetically-trained model in a real-world setting, providing real-time visual feedback at 200Hz, enabling a physical Kinova robot to interact with unseen articulated objects. This showcases the utility and effectiveness of our proposed method 2.

Details

JBHI Journal 2024 Journal Article

A Feature Fusion Model Based on Temporal Convolutional Network for Automatic Sleep Staging Using Single-Channel EEG

Jiameng Bao
Guangming Wang
Tianyu Wang
Ning Wu
Shimin Hu
Won Hee Lee
Sio-Long Lo
Xiangguo Yan

Sleep staging is a crucial task in sleep monitoring and diagnosis, but clinical sleep staging is both time-consuming and subjective. In this study, we proposed a novel deep learning algorithm named feature fusion temporal convolutional network (FFTCN) for automatic sleep staging using single-channel EEG data. This algorithm employed a one-dimensional convolutional neural network (1D-CNN) to extract temporal features from raw EEG, and a two-dimensional CNN (2D-CNN) to extract time-frequency features from spectrograms generated through continuous wavelet transform (CWT) at the epoch level. These features were subsequently fused and further fed into a temporal convolutional network (TCN) to classify sleep stages at the sequence level. Moreover, a two-step training strategy was used to enhance the model's performance on an imbalanced dataset. Our proposed method exhibits superior performance in the 5-class classification task for healthy subjects, as evaluated on the SHHS-1, Sleep-EDF-153, and ISRUC-S1 datasets. This work provided a straightforward and promising method for improving the accuracy of automatic sleep staging using only single-channel EEG, and the proposed method exhibited great potential for future applications in professional sleep monitoring, which could effectively alleviate the workload of sleep technicians.

Details DOI

EAAI Journal 2024 Journal Article

Comparative study of a newly proposed machine learning classification to detect damage occurrence in structures

Vahid Ahmadian
S. Bahram Beheshti Aval
Mohammad Noori
Tianyu Wang
Wael A. Altabey

Over the past two decades, an increasing number of large-scale structures have been built around the world. Constructing these structures has been a time consuming and highly expensive process. Thus, providing a structural health monitoring system to guarantee their proper functionality is important. In recent years, the advancement of technology and artificial intelligence methods based on signal processing and machine learning has attracted the attention of researchers. The challenges currently exist in the field of structural health monitoring to identify and classify damages to achieve high accuracy in a health-monitoring program. The presence of noise in measurement, various exciting load types, and varying environmental conditions cause difficulty in the practical identification and classification of damage in structures. Recent studies have employed finite element modeling to test the effectiveness of proposed methods for identifying damages in structures. However, detecting damage in real-world structures as mentioned above, presents unique difficulties, and the effectiveness of the proposed methods for damage detection in real-world structures remains uncertain. In order to improve the performance of damage detection methods and increase the accuracy of these methods as much as possible, the most important action is to identify damage sensitive data in the structure. The next challenge is to choose a high performance algorithm for damage identification and classification. One of the advanced algorithms, which has a very high ability to extract the desired features from the measured data, is the XGBoost algorithm. This algorithm has recently attracted the attention of researchers and has been used in different fields. So far, the ability of this algorithm has not been examined in the field of damage detection in order to extract desirable features. This article deals with the identification, classification, and severity of damages in the SMC benchmark bridge, which is an existing megastructure in the real world, as well as the IASC-ASCE benchmark structure, whose responses were taken under applied loads in the laboratory environment. First, using the XGBoost algorithm, the importance of the features extracted from the sensors' data is evaluated, and then the features, which are effective in the damage detection process, are selected. The results of this algorithm indicate that only by selecting 6 features from a large volume of data, the best performance can be achieved and selecting more does not help increase efficiency. In the next step, the Stacking method, which is a hybrid machine learning algorithm for damage classification, is evaluated and compared with some conventional machine learning algorithms that have been used in previous studies. The Stacking method stands out as the top performer with an average accuracy rate of 93. 1%, leading to the conclusion that it is the most effective approach. Finally, by applying the presented algorithm to the two mentioned structures, its validation is appraised.

Details DOI

NeurIPS Conference 2024 Conference Paper

Evaluation of Text-to-Video Generation Models: A Dynamics Perspective

Mingxiang Liao
Hannan Lu
Xinyu Zhang
Fang Wan
Tianyu Wang
Yuzhong Zhao
Wangmeng Zuo
Qixiang Ye

Comprehensive and constructive evaluation protocols play an important role when developing sophisticated text-to-video (T2V) generation models. Existing evaluation protocols primarily focus on temporal consistency and content continuity, yet largely ignore dynamics of video content. Such dynamics is an essential dimension measuring the visual vividness and the honesty of video content to text prompts. In this study, we propose an effective evaluation protocol, termed DEVIL, which centers on the dynamics dimension to evaluate T2V generation models, as well as improving existing evaluation metrics. In practice, we define a set of dynamics scores corresponding to multiple temporal granularities, and a new benchmark of text prompts under multiple dynamics grades. Upon the text prompt benchmark, we assess the generation capacity of T2V models, characterized by metrics of dynamics ranges and T2V alignment. Moreover, we analyze the relevance of existing metrics to dynamics metrics, improving them from the perspective of dynamics. Experiments show that DEVIL evaluation metrics enjoy up to about 90\% consistency with human ratings, demonstrating the potential to advance T2V generation models.

PDF Details DOI

ICML Conference 2024 Conference Paper

Geometry-Calibrated DRO: Combating Over-Pessimism with Free Energy Implications

Jiashuo Liu
Jiayun Wu
Tianyu Wang
Hao Zou 0001
Bo Li 0064
Peng Cui 0001

Machine learning algorithms minimizing average risk are susceptible to distributional shifts. Distributionally Robust Optimization (DRO) addresses this issue by optimizing the worst-case risk within an uncertainty set. However, DRO suffers from over-pessimism, leading to low-confidence predictions, poor parameter estimations as well as poor generalization. In this work, we conduct a theoretical analysis of a probable root cause of over-pessimism: excessive focus on noisy samples. To alleviate the impact of noise, we incorporate data geometry into calibration terms in DRO, resulting in our novel Geometry-Calibrated DRO (GCDRO) for regression. We establish the connection between our risk objective and the Helmholtz free energy in statistical physics, and this free-energy-based risk can extend to standard DRO methods. Leveraging gradient flow in Wasserstein space, we develop an approximate minimax optimization algorithm with a bounded error ratio and elucidate how our approach mitigates noisy sample effects. Comprehensive experiments confirm GCDRO’s superiority over conventional DRO methods.

Details

NeurIPS Conference 2024 Conference Paper

Is Cross-validation the Gold Standard to Estimate Out-of-sample Model Performance?

Garud Iyengar
Henry Lam
Tianyu Wang

Cross-Validation (CV) is the default choice for estimate the out-of-sample performance of machine learning models. Despite its wide usage, their statistical benefits have remained half-understood, especially in challenging nonparametric regimes. In this paper we fill in this gap and show that, in terms of estimating the out-of-sample performances, for a wide spectrum of models, CV does not statistically outperform the simple ``plug-in'' approach where one reuses training data for testing evaluation. Specifically, in terms of both the asymptotic bias and coverage accuracy of the associated interval for out-of-sample evaluation, $K$-fold CV provably cannot outperform plug-in regardless of the rate at which the parametric or nonparametric models converge. Leave-one-out CV can have a smaller bias as compared to plug-in; however, this bias improvement is negligible compared to the variability of the evaluation, and in some important cases leave-one-out again does not outperform plug-in once this variability is taken into account. We obtain our theoretical comparisons via a novel higher-order Taylor analysis that dissects the limit theorems of testing evaluations, which applies to model classes that are not amenable to previously known sufficient conditions. Our numerical results demonstrate that plug-in performs indeed no worse than CV in estimating model performance across a wide range of examples.

PDF Details DOI

TMLR Journal 2024 Journal Article

Optical Transformers

Maxwell Anderson
Shi-Yuan Ma
Tianyu Wang
Logan Wright
Peter McMahon

The rapidly increasing size of deep-learning models has renewed interest in alternatives to digital-electronic computers as a means to dramatically reduce the energy cost of running state-of-the-art neural networks. Optical matrix-vector multipliers are best suited to performing computations with very large operands, which suggests that large Transformer models could be a good target for them. In this paper, we investigate---through a combination of simulations and experiments on prototype optical hardware---the feasibility and potential energy benefits of running Transformer models on future optical accelerators that perform matrix-vector multiplication. We use simulations, with noise models validated by small-scale optical experiments, to show that optical accelerators for matrix-vector multiplication should be able to accurately run a typical Transformer architecture model for language processing. We demonstrate that optical accelerators can achieve the same (or better) perplexity as digital-electronic processors at 8-bit precision, provided that the optical hardware uses sufficiently many photons per inference, which translates directly to a requirement on optical energy per inference. We studied numerically how the requirement on optical energy per inference changes as a function of the Transformer width $d$ and found that the optical energy per multiply--accumulate (MAC) scales approximately as $\frac{1}{d}$, giving an asymptotic advantage over digital systems. We also analyze the total system energy costs for optical accelerators running Transformers, including both optical and electronic costs, as a function of model size. We predict that well-engineered, large-scale optical hardware should be able to achieve a $100 \times$ energy-efficiency advantage over current digital-electronic processors in running some of the largest current Transformer models, and if both the models and the optical hardware are scaled to the quadrillion-parameter regime, optical accelerators could have a $>8,000\times$ energy-efficiency advantage. Under plausible assumptions about future improvements to electronics and Transformer quantization techniques (5× cheaper memory access, double the digital--analog conversion efficiency, and 4-bit precision), we estimate that the energy advantage for optical processors versus electronic processors operating at 300~fJ/MAC could grow to $>100,000\times$.

PDF Details

IROS Conference 2024 Conference Paper

Polaris: Open-ended Interactive Robotic Manipulation via Syn2Real Visual Grounding and Large Language Models

Tianyu Wang
Haitao Lin
Junqiu Yu
Yanwei Fu 0001

This paper investigates the task of the open-ended interactive robotic manipulation on table-top scenarios. While recent Large Language Models (LLMs) enhance robots' comprehension of user instructions, their lack of visual grounding constrains their ability to physically interact with the environment. This is because the robot needs to locate the target object for manipulation within the physical workspace. To this end, we introduce an interactive robotic manipulation framework called Polaris, which integrates perception and interaction by utilizing GPT-4 alongside grounded vision models. For precise manipulation, it is essential that such grounded vision models produce detailed object pose for the target object, rather than merely identifying pixels belonging to them in the image. Consequently, we propose a novel Synthetic-to-Real (Syn2Real) pose estimation pipeline. This pipeline utilizes rendered synthetic data for training and is then transferred to real-world manipulation tasks. The real-world performance demonstrates the efficacy of our proposed pipeline and underscores its potential for extension to more general categories. Moreover, real-robot experiments have showcased the impressive performance of our framework in grasping and executing multiple manipulation tasks. This indicates its potential to generalize to scenarios beyond the tabletop. More information and video results are available here: https://star-uu-wang.github.io/Polaris/.

Details

NeurIPS Conference 2024 Conference Paper

UrbanDataLayer: A Unified Data Pipeline for Urban Science

Yiheng Wang
Tianyu Wang
Yuying Zhang
Hongji Zhang
Haoyu Zheng
Guanjie Zheng
Linghe Kong

The rapid progression of urbanization has generated a diverse array of urban data, facilitating significant advancements in urban science and urban computing. Current studies often work on separate problems case by case using diverse data, e. g. , air quality prediction, and built-up areas classification. This fragmented approach hinders the urban research field from advancing at the pace observed in Computer Vision and Natural Language Processing, due to two primary reasons. On the one hand, the diverse data processing steps lead to the lack of large-scale benchmarks and therefore decelerate iterative methodology improvement on a single problem. On the other hand, the disparity in multi-modal data formats hinders the combination of the related modal data to stimulate more research findings. To address these challenges, we propose UrbanDataLayer (UDL), a suite of standardized data structures and pipelines for city data engineering, providing a unified data format for researchers. This allows researchers to easily build up large-scale benchmarks and combine multi-modal data, thus expediting the development of multi-modal urban foundation models. To verify the effectiveness of our work, we present four distinct urban problem tasks utilizing the proposed data layer. UrbanDataLayer aims to enhance standardization and operational efficiency within the urban science research community. The examples and source code are available at https: //github. com/SJTU-CILAB/udl.

PDF Details DOI

AAAI Conference 2024 Conference Paper

Winnie: Task-Oriented Dialog System with Structure-Aware Contrastive Learning and Enhanced Policy Planning

Kaizhi Gao
Tianyu Wang
Zhongjing Ma
Suli Zou

Pre-trained encoder-decoder models are widely applied in Task-Oriented Dialog (TOD) systems on the session level, mainly focusing on modeling the dialog semantic information. Dialogs imply structural information indicating the interaction among user utterances, belief states, database search results, system acts and responses, which is also crucial for TOD systems. In addition, for the system acts, additional pre-training and datasets are considered to improve their accuracies, undoubtedly introducing a burden. Therefore, a novel end-to-end TOD system named Winnie is proposed in this paper to improve the TOD performance. First, to make full use of the intrinsic structural information, supervised contrastive learning is adopted to narrow the gap in the representation space between text representations of the same category and enlarge the overall continuous representation margin between text representations of different categories in dialog context. Then, a system act classification task is introduced for policy optimization during fine-tuning. Empirical results show that Winnie substantially improves the performance of the TOD system. By introducing the supervised contrastive and system act classification losses, Winnie achieves state-of-the-art results on benchmark datasets, including MultiWOZ2.2, In-Car, and Camrest676. Their end-to-end combined scores are improved by 3.2, 1.9, and 1.1 points, respectively.

PDF Details DOI

EAAI Journal 2023 Journal Article

A novel intelligent monitoring method for the closing time of the taphole of blast furnace based on two-stage classification

Zhaohui Jiang
Jinzong Dong
Dong Pan
Tianyu Wang
Weihua Gui

Determining the taphole closing time is an essential task in the blast furnace ironmaking process because the closing time directly affects the efficiency of iron production and the stability of the blast furnace. However, at present, the taphole closing time in most ironmaking plants is judged by on-site workers based on experience, which lacks scientific guidance. To determine the taphole closing time intelligently and accurately, a novel monitoring method is proposed, which innovatively simplifies the monitoring problem of the absolute taphole closing time into a two-stage classification problem of relative tapping state. In the first stage, a classification algorithm SE-ResNeXt, which only takes the molten iron flow image data as the input data, is used to preliminarily determine the current molten iron flow state in the time dimension during tapping. When it is recognized that the molten iron flow is in the last tapping state in the first stage, the second stage is carried out. In the second stage, a novel multimodal data fusion network SENeXt-Decoder consisting of a novel image feature extraction module, a novel fusion module and a multi-head attention decoder is proposed to obtain the exact taphole closing time, which fuses the molten iron flow image data and blast furnace operating state data. The comparison experiment with the actual taphole closing time on site shows that the absolute monitoring error of this method is within 120 s, and the relative monitoring error is within 1. 2%, which better meets the factory’s demand for error accuracy.

Details DOI

NeurIPS Conference 2023 Conference Paper

On the Need for a Language Describing Distribution Shifts: Illustrations on Tabular Datasets

Jiashuo Liu
Tianyu Wang
Peng Cui
Hongseok Namkoong

Different distribution shifts require different algorithmic and operational interventions. Methodological research must be grounded by the specific shifts they address. Although nascent benchmarks provide a promising empirical foundation, they \emph{implicitly} focus on covariate shifts, and the validity of empirical findings depends on the type of shift, e. g. , previous observations on algorithmic performance can fail to be valid when the $Y|X$ distribution changes. We conduct a thorough investigation of natural shifts in 5 tabular datasets over 86, 000 model configurations, and find that $Y|X$-shifts are most prevalent. To encourage researchers to develop a refined language for distribution shifts, we build ``WhyShift``, an empirical testbed of curated real-world shifts where we characterize the type of shift we benchmark performance over. Since $Y|X$-shifts are prevalent in tabular settings, we \emph{identify covariate regions} that suffer the biggest $Y|X$-shifts and discuss implications for algorithmic and data-based interventions. Our testbed highlights the importance of future research that builds an understanding of why distributions differ.

PDF Details

NeurIPS Conference 2022 Conference Paper

A Direct Approximation of AIXI Using Logical State Abstractions

Samuel Yang-Zhao
Tianyu Wang
Kee Siong Ng

We propose a practical integration of logical state abstraction with AIXI, a Bayesian optimality notion for reinforcement learning agents, to significantly expand the model class that AIXI agents can be approximated over to complex history-dependent and structured environments. The state representation and reasoning framework is based on higher-order logic, which can be used to define and enumerate complex features on non-Markovian and structured environments. We address the problem of selecting the right subset of features to form state abstractions by adapting the $\Phi$-MDP optimisation criterion from state abstraction theory. Exact Bayesian model learning is then achieved using a suitable generalisation of Context Tree Weighting over abstract state sequences. The resultant architecture can be integrated with different planning algorithms. Experimental results on controlling epidemics on large-scale contact networks validates the agent's performance.

PDF Details

NeurIPS Conference 2022 Conference Paper

Lipschitz Bandits with Batched Feedback

Yasong Feng
zengfeng Huang
Tianyu Wang

In this paper, we study Lipschitz bandit problems with batched feedback, where the expected reward is Lipschitz and the reward observations are communicated to the player in batches. We introduce a novel landscape-aware algorithm, called Batched Lipschitz Narrowing (BLiN), that optimally solves this problem. Specifically, we show that for a $T$-step problem with Lipschitz reward of zooming dimension $d_z$, our algorithm achieves theoretically optimal (up to logarithmic factors) regret rate $\widetilde{\mathcal{O}}\left(T^{\frac{d_z+1}{d_z+2}}\right)$ using only $ \mathcal{O} \left( \log\log T\right) $ batches. We also provide complexity analysis for this problem. Our theoretical lower bound implies that $\Omega(\log\log T)$ batches are necessary for any algorithm to achieve the optimal regret. Thus, BLiN achieves optimal regret rate using minimal communication.

PDF Details

NeurIPS Conference 2022 Conference Paper

Sparse2Dense: Learning to Densify 3D Features for 3D Object Detection

Tianyu Wang
Xiaowei Hu
Zhengzhe Liu
Chi-Wing Fu

LiDAR-produced point clouds are the major source for most state-of-the-art 3D object detectors. Yet, small, distant, and incomplete objects with sparse or few points are often hard to detect. We present Sparse2Dense, a new framework to efficiently boost 3D detection performance by learning to densify point clouds in latent space. Specifically, we first train a dense point 3D detector (DDet) with a dense point cloud as input and design a sparse point 3D detector (SDet) with a regular point cloud as input. Importantly, we formulate the lightweight plug-in S2D module and the point cloud reconstruction module in SDet to densify 3D features and train SDet to produce 3D features, following the dense 3D features in DDet. So, in inference, SDet can simulate dense 3D features from regular (sparse) point cloud inputs without requiring dense inputs. We evaluate our method on the large-scale Waymo Open Dataset and the Waymo Domain Adaptation Dataset, showing its high performance and efficiency over the state of the arts.

PDF Details

IROS Conference 2022 Conference Paper

WFA-IRL: Inverse Reinforcement Learning of Autonomous Behaviors Encoded as Weighted Finite Automata

Tianyu Wang
Nikolay Atanasov 0001

This paper presents a method for learning logical task specifications and cost functions from demonstrations. Constructing specifications by hand is challenging for complex objectives and constraints in autonomous systems. Instead, we consider demonstrated task executions, whose logic structure and transition costs need to be inferred by an autonomous agent. We employ a spectral learning approach to extract a weighted finite automaton (WFA), approximating the unknown task logic. Thereafter, we define a product between the WFA for high-level task guidance and a labeled Markov decision process for low-level control. An inverse reinforcement learning (IRL) problem is considered to learn a cost function by backpropagating the loss between agent and expert behaviors through the planning algorithm. Our proposed model, termed WFA-IRL, is capable of generalizing the execution of the inferred task specification in a suite of MiniGrid environments.

Details

JMLR Journal 2021 Journal Article

FLAME: A Fast Large-scale Almost Matching Exactly Approach to Causal Inference

Tianyu Wang
Marco Morucci
M. Usaid Awan
Yameng Liu
Sudeepa Roy
Cynthia Rudin
Alexander Volfovsky

A classical problem in causal inference is that of matching, where treatment units need to be matched to control units based on covariate information. In this work, we propose a method that computes high quality almost-exact matches for high-dimensional categorical datasets. This method, called FLAME (Fast Large-scale Almost Matching Exactly), learns a distance metric for matching using a hold-out training data set. In order to perform matching efficiently for large datasets, FLAME leverages techniques that are natural for query processing in the area of database management, and two implementations of FLAME are provided: the first uses SQL queries and the second uses bit-vector techniques. The algorithm starts by constructing matches of the highest quality (exact matches on all covariates), and successively eliminates variables in order to match exactly on as many variables as possible, while still maintaining interpretable high-quality matches and balance between treatment and control groups. We leverage these high quality matches to estimate conditional average treatment effects (CATEs). Our experiments show that FLAME scales to huge datasets with millions of observations where existing state-of-the-art methods fail, and that it achieves significantly better performance than other matching methods. [abs] [ pdf ][ bib ] [ website ] &copy JMLR 2021. ( edit, beta )

PDF Details

NeurIPS Conference 2020 Conference Paper

Adaptive Discretization for Model-Based Reinforcement Learning

Sean Sinclair
Tianyu Wang
Gauri Jain
Siddhartha Banerjee
Christina Yu

We introduce the technique of adaptive discretization to design an efficient model-based episodic reinforcement learning algorithm in large (potentially continuous) state-action spaces. Our algorithm is based on optimistic one-step value iteration extended to maintain an adaptive discretization of the space. From a theoretical perspective we provide worst-case regret bounds for our algorithm which are competitive compared to the state-of-the-art model-based algorithms. Moreover, our bounds are obtained via a modular proof technique which can potentially extend to incorporate additional structure on the problem. From an implementation standpoint, our algorithm has much lower storage and computational requirements due to maintaining a more efficient partition of the state and action spaces. We illustrate this via experiments on several canonical control problems, which shows that our algorithm empirically performs significantly better than fixed discretization in terms of both faster convergence and lower memory usage. Interestingly, we observe empirically that while fixed discretization model-based algorithms vastly outperform their model-free counterparts, the two achieve comparable performance with adaptive discretization.

PDF Details

ICRA Conference 2020 Conference Paper

Learning Navigation Costs from Demonstration in Partially Observable Environments

Tianyu Wang
Vikas Dhiman
Nikolay Atanasov 0001

This paper focuses on inverse reinforcement learning (IRL) to enable safe and efficient autonomous navigation in unknown partially observable environments. The objective is to infer a cost function that explains expert-demonstrated navigation behavior while relying only on the observations and state-control trajectory used by the expert. We develop a cost function representation composed of two parts: a probabilistic occupancy encoder, with recurrent dependence on the observation sequence, and a cost encoder, defined over the occupancy features. The representation parameters are optimized by differentiating the error between demonstrated controls and a control policy computed from the cost encoder. Such differentiation is typically computed by dynamic programming through the value function over the whole state space. We observe that this is inefficient in large partially observable environments because most states are unexplored. Instead, we rely on a closed-form subgradient of the cost-to-go obtained only over a subset of promising states via an efficient motion-planning algorithm such as A* or RRT. Our experiments show that our model exceeds the accuracy of baseline IRL algorithms in robot navigation tasks, while substantially improving the efficiency of training and test-time inference.

Details