Author name cluster

Koen Helwegen

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

3 papers

2 author rows

YNIMG Journal 2023 Journal Article

Structural and functional connectivity reconstruction with CATO - A Connectivity Analysis TOolbox

Siemon C. de Lange
Koen Helwegen
Martijn P. van den Heuvel

We describe a Connectivity Analysis TOolbox (CATO) for the reconstruction of structural and functional brain connectivity based on diffusion weighted imaging and resting-state functional MRI data. CATO is a multimodal software package that enables researchers to run end-to-end reconstructions from MRI data to structural and functional connectome maps, customize their analyses and utilize various software packages to preprocess data. Structural and functional connectome maps can be reconstructed with respect to user-defined (sub)cortical atlases providing aligned connectivity matrices for integrative multimodal analyses. We outline the implementation and usage of the structural and functional processing pipelines in CATO. Performance was calibrated with respect to simulated diffusion weighted imaging data from the ITC2015 challenge and test-retest diffusion weighted imaging data and resting-state functional MRI data from the Human Connectome Project. CATO is open-source software distributed under the MIT License and available as a MATLAB toolbox and as a stand-alone application at www.dutchconnectomelab.nl/CATO.

Details DOI

ICML Conference 2021 Conference Paper

How Do Adam and Training Strategies Help BNNs Optimization

Zechun Liu
Zhiqiang Shen
Shichao Li 0002
Koen Helwegen
Dong Huang 0007
Kwang-Ting (Tim) Cheng

The best performing Binary Neural Networks (BNNs) are usually attained using Adam optimization and its multi-step training variants. However, to the best of our knowledge, few studies explore the fundamental reasons why Adam is superior to other optimizers like SGD for BNN optimization or provide analytical explanations that support specific training strategies. To address this, in this paper we first investigate the trajectories of gradients and weights in BNNs during the training process. We show the regularization effect of second-order momentum in Adam is crucial to revitalize the weights that are dead due to the activation saturation in BNNs. We find that Adam, through its adaptive learning rate strategy, is better equipped to handle the rugged loss surface of BNNs and reaches a better optimum with higher generalization ability. Furthermore, we inspect the intriguing role of the real-valued weights in binary networks, and reveal the effect of weight decay on the stability and sluggishness of BNN optimization. Through extensive experiments and analysis, we derive a simple training scheme, building on existing Adam-based optimization, which achieves 70. 5% top-1 accuracy on the ImageNet dataset using the same architecture as the state-of-the-art ReActNet while achieving 1. 1% higher accuracy. Code and models are available at https: //github. com/liuzechun/AdamBNN.

Details

NeurIPS Conference 2019 Conference Paper

Latent Weights Do Not Exist: Rethinking Binarized Neural Network Optimization

Koen Helwegen
James Widdicombe
Lukas Geiger
Zechun Liu
Kwang-Ting Cheng
Roeland Nusselder

Optimization of Binarized Neural Networks (BNNs) currently relies on real-valued latent weights to accumulate small update steps. In this paper, we argue that these latent weights cannot be treated analogously to weights in real-valued networks. Instead their main role is to provide inertia during training. We interpret current methods in terms of inertia and provide novel insights into the optimization of BNNs. We subsequently introduce the first optimizer specifically designed for BNNs, Binary Optimizer (Bop), and demonstrate its performance on CIFAR-10 and ImageNet. Together, the redefinition of latent weights as inertia and the introduction of Bop enable a better understanding of BNN optimization and open up the way for further improvements in training methodologies for BNNs.

PDF Details