Author name cluster

Zhao Chen

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

8 papers

2 author rows

EAAI Journal 2025 Journal Article

Enhancing the hydraulic engineering circular-18 pier scour equation with knowledge-guided symbolic regression and field data augmentation

Wen Xiong
Qilin Yang
Zhao Chen
Rongzhao Zhang

Bridge scour is a top cause of bridge failures. However, many mainstream pier scour design equations still perform less satisfactorily, i. e. , excessive overestimation or even risky underestimation can occur during practice. To improve the robustness of the widely used Hydraulic Engineering Circular-18 (HEC-18) equation, this work proposes a computational framework empowered by expressive symbolic regression and assorted datasets. In particular, a knowledge-guided genetic algorithm compensates for a sedimentary gap between lab and field conditions, and a multi-objective optimization advocates a balance between safety and budget. Furthermore, the authors modify the field data to alleviate its uncertain maturity and employ the K-means clustering to select statistically representative training data based on hydraulic, sedimentary and structural input parameters. Evaluated by multiple independent lab and field datasets, the enhanced equation can offer more economical and consistent design suggestions compared to multiple popular benchmarks, if not equivalently competitively. In particular, when evaluated by unseen lab data, its relative error is 23. 14% compared with 21. 70 ∼ 45. 24% of benchmarks. While as safe as the other equations, the proposed equation is at least 5 ∼ 37% closer to field measurements, alleviating over-estimation issues. When predicting new field data, the standard deviation of the proposed equation’s errors is about 0. 07, indicating a competitively consistent prediction capability. A post-analysis also investigates the enhanced equation’s applicability and the value of embedded knowledge.

Details DOI

ICML Conference 2025 Conference Paper

Generative Data Mining with Longtail-Guided Diffusion

David S. Hayden
Mao Ye 0006
Timur Garipov
Gregory P. Meyer
Carl Vondrick
Zhao Chen
Yuning Chai
Eric M. Wolff

It is difficult to anticipate the myriad challenges that a predictive model will encounter once deployed. Common practice entails a reactive, cyclical approach: model deployment, data mining, and retraining. We instead develop a proactive longtail discovery process by imagining additional data during training. In particular, we develop general model-based longtail signals, including a differentiable, single forward pass formulation of epistemic uncertainty that does not impact model parameters or predictive performance but can flag rare or hard inputs. We leverage these signals as guidance to generate additional training data from a latent diffusion model in a process we call Longtail Guidance (LTG). Crucially, we can perform LTG without retraining the diffusion model or the predictive model, and we do not need to expose the predictive model to intermediate diffusion states. Data generated by LTG exhibit semantically meaningful variation, yield significant generalization improvements on numerous image classification benchmarks, and can be analyzed by a VLM to proactively discover, textually explain, and address conceptual gaps in a deployed predictive model.

Details

NeurIPS Conference 2025 Conference Paper

PROFIT: A Specialized Optimizer for Deep Fine Tuning

Anirudh Chakravarthy
Shuai Zheng
Xin Huang
Sachithra Hemachandra
Xiao Zhang
Yuning Chai
Zhao Chen

The fine-tuning of pre-trained models has become ubiquitous in generative AI, computer vision, and robotics. Although much attention has been paid to improving the efficiency of fine-tuning model, there has been less scholarship around fine-tuning specifically for improved model performance. To remedy this gap, we present PROFIT, one of the first optimizers designed to incrementally fine-tune converged models on new tasks and/or datasets. Unlike traditional optimizers such as SGD or Adam, which make minimal assumptions due to random initializations, PROFIT takes the properties of a converged model into account explicitly to regularize the optimization process. Employing a temporal gradient-orthogonalization process, PROFIT outperforms fine-tuning methods in various tasks, from image classification to multimodal language model training to large-scale motion prediction. Moreover, PROFIT is encapsulated as a modular optimizer, which makes it easy to integrate directly into any training pipeline with minimal engineering effort.

PDF Details

IJCAI Conference 2023 Conference Paper

Sancus: Staleness-Aware Communication-Avoiding Full-Graph Decentralized Training in Large-Scale Graph Neural Networks (Extended Abstract)

Jingshu Peng
Zhao Chen
Yingxia Shao
Yanyan Shen
Lei Chen
Jiannong Cao

Graph neural networks (GNNs) have emerged due to their success at modeling graph data. Yet, it is challenging for GNNs to efficiently scale to large graphs. Thus, distributed GNNs come into play. To avoid communication caused by expensive data movement between workers, we propose SANCUS, a staleness-aware communication-avoiding decentralized GNN system. By introducing a set of novel bounded embedding staleness metrics and adaptively skipping broadcasts, SANCUS abstracts decentralized GNN processing as sequential matrix multiplication and uses historical embeddings via cache. Theoretically, we show bounded approximation errors of embeddings and gradients with convergence guarantee. Empirically, we evaluate SANCUS with common GNN models via different system setups on large-scale benchmark datasets. Compared to SOTA works, SANCUS can avoid up to 74% communication with at least 1: 86_ faster throughput on average without accuracy loss.

PDF Details DOI

ICML Conference 2022 Conference Paper

HyperPrompt: Prompt-based Task-Conditioning of Transformers

Yun He
Huaixiu Steven Zheng
Yi Tay
Jai Prakash Gupta 0001
Yu Du
Vamsi Aribandi
Zhe Zhao 0001
YaGuang Li

Prompt-Tuning is a new paradigm for finetuning pre-trained language models in a parameter efficient way. Here, we explore the use of HyperNetworks to generate hyper-prompts: we propose HyperPrompt, a novel architecture for prompt-based task-conditioning of self-attention in Transformers. The hyper-prompts are end-to-end learnable via generation by a HyperNetwork. HyperPrompt allows the network to learn task-specific feature maps where the hyper-prompts serve as task global memories for the queries to attend to, at the same time enabling flexible information sharing among tasks. We show that HyperPrompt is competitive against strong multi-task learning baselines with as few as 0. 14% of additional task-conditioning parameters, achieving great parameter and computational efficiency. Through extensive empirical experiments, we demonstrate that HyperPrompt can achieve superior performances over strong T5 multi-task learning baselines and parameter-efficient adapter variants including Prompt-Tuning and HyperFormer++ on Natural Language Understanding benchmarks of GLUE and SuperGLUE across many model sizes.

Details

JBHI Journal 2021 Journal Article

Weakly Supervised Histopathology Image Segmentation With Sparse Point Annotations

Zhe Chen
Zhao Chen
Jingxin Liu
Qiang Zheng
Yuang Zhu
Yanfei Zuo
Zhaoyu Wang
Xiaosong Guan

Digital histopathology image segmentation can facilitate computer-assisted cancer diagnostics. Given the difficulty of obtaining manual annotations, weak supervision is more suitable for the task than full supervision is. However, most weakly supervised models are not ideal for handling severe intra-class heterogeneity and inter-class homogeneity in histopathology images. Therefore, we propose a novel end-to-end weakly supervised learning framework named WESUP. With only sparse point annotations, it performs accurate segmentation and exhibits good generalizability. The training phase comprises two major parts, hierarchical feature representation and deep dynamic label propagation. The former uses superpixels to capture local details and global context from the convolutional feature maps obtained via transfer learning. The latter recognizes the manifold structure of the hierarchical features and identifies potential targets with the sparse annotations. Moreover, these two parts are trained jointly to improve the performance of the whole framework. To further boost test performance, pixel-wise inference is adopted for finer prediction. As demonstrated by experimental results, WESUP is able to largely resolve the confusion between histological foreground and background. It outperforms several state-of-the-art weakly supervised methods on a variety of histopathology datasets with minimal annotation efforts. Trained by very sparse point annotations, WESUP can even beat an advanced fully supervised segmentation network.

Details DOI

NeurIPS Conference 2020 Conference Paper

Just Pick a Sign: Optimizing Deep Multitask Models with Gradient Sign Dropout

Zhao Chen
Jiquan Ngiam
Yanping Huang
Thang Luong
Henrik Kretzschmar
Yuning Chai
Dragomir Anguelov

The vast majority of deep models use multiple gradient signals, typically corresponding to a sum of multiple loss terms, to update a shared set of trainable weights. However, these multiple updates can impede optimal training by pulling the model in conflicting directions. We present Gradient Sign Dropout (GradDrop), a probabilistic masking procedure which samples gradients at an activation layer based on their level of consistency. GradDrop is implemented as a simple deep layer that can be used in any deep net and synergizes with other gradient balancing approaches. We show that GradDrop outperforms the state-of-the-art multiloss methods within traditional multitask and transfer learning settings, and we discuss how GradDrop reveals links between optimal multiloss training and gradient stochasticity.

PDF Details

AAAI Conference 2018 Conference Paper

Ranking Users in Social Networks With Higher-Order Structures

Huan Zhao
Xiaogang Xu
Yangqiu Song
Dik Lun Lee
Zhao Chen
Han Gao

PageRank has been widely used to measure the authority or the inﬂuence of a user in social networks. However, conventional PageRank only makes use of edge-based relations, ignoring higher-order structures captured by motifs, subgraphs consisting of a small number of nodes in complex networks. In this paper, we propose a novel framework, motif-based PageRank (MPR), to incorporate higher-order structures into conventional PageRank computation. We conduct extensive experiments in three real-world networks, i. e. , DBLP, Epinions, and Ciao, to show that MPR can signiﬁcantly improve the effectiveness of PageRank for ranking users in social networks. In addition to numerical results, we also provide detailed analysis for MPR to show how and why incorporating higher-order information works better than PageRank in ranking users in social networks. 1

PDF Details