Author name cluster

Shihao Zhang

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

4 papers

2 author rows

ICLR Conference 2025 Conference Paper

Improving Deep Regression with Tightness

Shihao Zhang
Yuguang Yan
Angela Yao

For deep regression, preserving the ordinality of the targets with respect to the feature representation improves performance across various tasks. However, a theoretical explanation for the benefits of ordinality is still lacking. This work reveals that preserving ordinality reduces the conditional entropy $H(Z|Y)$ of representation $Z$ conditional on the target $Y$. However, our findings reveal that typical regression losses fail to sufficiently reduce $H(Z|Y)$, despite its crucial role in generalization performance. With this motivation, we introduce an optimal transport-based regularizer to preserve the similarity relationships of targets in the feature space to reduce $H(Z|Y)$. Additionally, we introduce a simple yet efficient strategy of duplicating the regressor targets, also with the aim of reducing $H(Z|Y)$. Experiments on three real-world regression tasks verify the effectiveness of our strategies to improve deep regression. Code: https://github.com/needylove/Regression_tightness

Details

ICML Conference 2024 Conference Paper

Deep Regression Representation Learning with Topology

Shihao Zhang
Kenji Kawaguchi
Angela Yao

Most works studying representation learning focus only on classification and neglect regression. Yet, the learning objectives and, therefore, the representation topologies of the two tasks are fundamentally different: classification targets class separation, leading to disconnected representations, whereas regression requires ordinality with respect to the target, leading to continuous representations. We thus wonder how the effectiveness of a regression representation is influenced by its topology, with evaluation based on the Information Bottleneck (IB) principle. The IB principle is an important framework that provides principles for learning effective representations. We establish two connections between it and the topology of regression representations. The first connection reveals that a lower intrinsic dimension of the feature space implies a reduced complexity of the representation $Z$. This complexity can be quantified as the conditional entropy of $Z$ on the target $Y$, and serves as an upper bound on the generalization error. The second connection suggests a feature space that is topologically similar to the target space will better align with the IB principle. Based on these two connections, we introduce PH-Reg, a regularizer specific to regression that matches the intrinsic dimension and topology of the feature space with the target space. Experiments on synthetic and real-world regression tasks demonstrate the benefits of PH-Reg. Code: https: //github. com/needylove/PH-Reg.

Details

TCS Journal 2024 Journal Article

Quantum algorithms for learning hidden strings with applications to matroid problems

Xiaowei Huang
Shihao Zhang
Lvzhou Li

In this paper, we explore quantum speedups for the problem, inspired by matroid theory, of learning a pair of n-bit binary strings that are promised to have the same number of 1s and differ in exactly two bits, by using the max inner product oracle and the sub-set oracle. More specifically, given two string s, s ′ ∈ { 0, 1 } n satisfying the above constraints, for any x ∈ { 0, 1 } n the max inner product oracle O m a x ( x ) returns the max value between s ⋅ x and s ′ ⋅ x, and the sub-set oracle O s u b ( x ) indicates whether the index set of the 1s in x is a subset of that in s or s ′. We present an exact quantum algorithm consuming O ( 1 ) queries to the max inner product oracle for identifying the pair { s, s ′ }, and prove that any randomized algorithm requires Ω ( n / log 2 ⁡ n ) queries. Also, we present a quantum algorithm consuming n 2 + O ( n ) queries to the subset oracle, and prove that any randomized algorithm requires at least n − 3 + Ω ( log 2 ⁡ n ) queries. Therefore, quantum speedups are revealed in the two oracle models. Furthermore, the above results are applied to the problem in matroid theory of finding all the bases of a 2-bases matroid, where a matroid is called k-bases if it has k bases.

Details DOI

ICLR Conference 2023 Conference Paper

Improving Deep Regression with Ordinal Entropy

Shihao Zhang
Linlin Yang
Michael Bi Mi
Xiaoxu Zheng
Angela Yao

In computer vision, it is often observed that formulating regression problems as a classification task yields better performance. We investigate this curious phenomenon and provide a derivation to show that classification, with the cross-entropy loss, outperforms regression with a mean squared error loss in its ability to learn high-entropy feature representations. Based on the analysis, we propose an ordinal entropy loss to encourage higher-entropy feature spaces while maintaining ordinal relationships to improve the performance of regression tasks. Experiments on synthetic and real-world regression tasks demonstrate the importance and benefits of increasing entropy for regression.

Details