Author name cluster

Zhiren Wang

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

5 papers

2 author rows

ICML Conference 2024 Conference Paper

CW Complex Hypothesis for Image Data

Yi Wang 0075
Zhiren Wang

We examine both the manifold hypothesis (Bengio et al. , 2013) and the union of manifold hypothesis (Brown et al. , 2023), and argue that, in contrast to these hypotheses, the local intrinsic dimension varies from point to point even in the same connected component. We propose an alternative CW complex hypothesis that image data is distributed in “manifolds with skeletons". We support the hypothesis through visualization of distributions of image data of random geometric objects, as well as by introducing and testing a criterion on natural image datasets. One motivation of our work is to explain why diffusion models have difficulty generating accurate higher dimensional details such as human hands. Under the CW complex hypothesis and with both theoretical and empirical evidences, we provide an interpretation that the mixture of higher and lower dimensional components in data obstructs diffusion models from efficient learning.

Details

ICLR Conference 2024 Conference Paper

Fast Equilibrium of SGD in Generic Situations

Zhiyuan Liu
Yi Wang 0075
Zhiren Wang

Normalization layers are ubiquitous in deep learning, greatly accelerating optimization. However, they also introduce many unexpected phenomena during training, for example, the Fast Equilibrium conjecture proposed by (Li et al.,2020), which states that the scale-invariant normalized network, when trained by SGD with $\eta$ learning rate and $\lambda$ weight decay, mixes to an equilibrium in $\tilde{O}(1/\eta\lambda)$ steps, as opposed to classical $e^{O(\eta^{-1})}$ mixing time. Recent works by Wang & Wang (2022); Li et al. (2022c) proved this conjecture under different sets of assumptions. This paper aims to answer the fast equilibrium conjecture in full generality by removing the non-generic assumptions of Wang & Wang (2022); Li et al. (2022c) that the minima are isolated, that the region near minima forms a unique basin, and that the set of minima is an analytic set. Our main technical contribution is to show that with probability close to 1, in exponential time trajectories will not escape the attracting basin containing its initial position.

Details

ICML Conference 2022 Conference Paper

Three-stage Evolution and Fast Equilibrium for SGD with Non-degerate Critical Points

Yi Wang 0075
Zhiren Wang

We justify the fast equilibrium conjecture on stochastic gradient descent from (Li et al. 2020) under the assumptions that critical points are non-degenerate and the stochastic noise is a standard Gaussian. In this case, we prove an SGD with constant effective learning rate consists of three stages: descent, diffusion and tunneling, and explicitly identify temporary equilibrium states in the normalized parameter space that can be observed within practical training time. This interprets the gap between the mixing time in the fast equilibrium conjecture and the previously known upper bound. While our assumptions do not represent typical implementations of SGD of neural networks in practice, this is the first description of the three-stage mechanism in any case. The main finding in this mechanism is that a temporary equilibrium of local nature is quickly achieved after polynomial time (in term of the reciprocal of the intrinsic learning rate) and then stabilizes within observable time scales; and that the temporary equilibrium is in general different from the global Gibbs equilibrium, which will only appear after an exponentially long period beyond typical training limits. Our experiments support that this mechanism may extend to the general case.

Details

YNICL Journal 2020 Journal Article

Abnormal amygdala subregional-sensorimotor connectivity correlates with positive symptom in schizophrenia

Meng Zhang
Fude Yang
Fengmei Fan
Zhiren Wang
Xiang Hong
Yunlong Tan
Shuping Tan
L. Elliot Hong

Altered resting-state functional connectivity (rsFC) of the amygdala has been demonstrated to be implicated in schizophrenia neuronal pathophysiology. However, whether rsFC of amygdala subregions is differentially affected in schizophrenia remains unclear. This study compared the functional networks of each amygdala subdivision between healthy controls (HC) and patients with first-episode schizophrenia (FES). In total, 47 HC and 78 patients with FES underwent resting-state functional magnetic resonance imaging. The amygdala was divided into the following three subregions using the Juelich histological atlas: basolateral amygdala (BLA), centromedial amygdala (CMA), and superficial amygdala (SFA). The rsFC of the three amygdala subdivisions was computed and compared between the two groups. Significantly increased rsFC of the right CMA with the right postcentral gyrus and decreased rsFC of the right BLA with the left precentral gyrus were observed in the FES group compared with the HC group. Notably, the right BLA-left precentral gyrus connectivity was negatively correlated with positive symptoms and conceptual disorganization in patients with FES. In conclusion, this study found that patients with FES had abnormal functional connectivity in the amygdala subregions, and the altered rsFC was associated with positive symptoms. The present findings demonstrate the disruptive rsFC patterns of amygdala subregional-sensorimotor networks in FES and may provide new insights into the neuronal pathophysiology of FES.

Details DOI

YNIMG Journal 2020 Journal Article

Increased power by harmonizing structural MRI site differences with the ComBat batch adjustment method in ENIGMA

Joaquim Radua
Eduard Vieta
Russell Shinohara
Peter Kochunov
Yann Quidé
Melissa J. Green
Cynthia S. Weickert
Thomas Weickert

A common limitation of neuroimaging studies is their small sample sizes. To overcome this hurdle, the Enhancing Neuro Imaging Genetics through Meta-Analysis (ENIGMA) Consortium combines neuroimaging data from many institutions worldwide. However, this introduces heterogeneity due to different scanning devices and sequences. ENIGMA projects commonly address this heterogeneity with random-effects meta-analysis or mixed-effects mega-analysis. Here we tested whether the batch adjustment method, ComBat, can further reduce site-related heterogeneity and thus increase statistical power. We conducted random-effects meta-analyses, mixed-effects mega-analyses and ComBat mega-analyses to compare cortical thickness, surface area and subcortical volumes between 2897 individuals with a diagnosis of schizophrenia and 3141 healthy controls from 33 sites. Specifically, we compared the imaging data between individuals with schizophrenia and healthy controls, covarying for age and sex. The use of ComBat substantially increased the statistical significance of the findings as compared to random-effects meta-analyses. The findings were more similar when comparing ComBat with mixed-effects mega-analysis, although ComBat still slightly increased the statistical significance. ComBat also showed increased statistical power when we repeated the analyses with fewer sites. Results were nearly identical when we applied the ComBat harmonization separately for cortical thickness, cortical surface area and subcortical volumes. Therefore, we recommend applying the ComBat function to attenuate potential effects of site in ENIGMA projects and other multi-site structural imaging work. We provide easy-to-use functions in R that work even if imaging data are partially missing in some brain regions, and they can be trained with one data set and then applied to another (a requirement for some analyses such as machine learning).

Details DOI