Author name cluster

Ling-Yu Duan

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

9 papers

1 author row

NeurIPS Conference 2024 Conference Paper

DenseFusion-1M: Merging Vision Experts for Comprehensive Multimodal Perception

Xiaotong Li
Fan Zhang
Haiwen Diao
Yueze Wang
Xinlong Wang
Ling-Yu Duan

Existing Multimodal Large Language Models (MLLMs) increasingly emphasize complex understanding of various visual elements, including multiple objects, text information, spatial relations. Their development for comprehensive visual perception hinges on the availability of high-quality image-text datasets that offer diverse visual elements and throughout image descriptions. However, the scarcity of such hyper-detailed datasets currently hinders progress within the MLLM community. The bottleneck stems from the limited perceptual capabilities of current caption engines, which fall short in providing complete and accurate annotations. To facilitate the cutting-edge research of MLLMs on comprehensive vision perception, we thereby propose Perceptual Fusion, using a low-budget but highly effective caption engine for complete and accurate image descriptions. Specifically, Perceptual Fusion integrates diverse perception experts as image priors to provide explicit information on visual elements and adopts an efficient MLLM as a centric pivot to mimic advanced MLLMs' perception abilities. We carefully select 1M highly representative images from uncurated LAION dataset and generate dense descriptions using our engine, dubbed DenseFusion-1M. Extensive experiments validate that our engine outperforms its counterparts, where the resulting dataset significantly improves the perception and cognition abilities of existing MLLMs across diverse vision-language benchmarks, especially with high-resolution images as inputs. The code and dataset are available at https: //huggingface. co/datasets/BAAI/DenseFusion-1M.

PDF Details DOI

TIST Journal 2022 Journal Article

Intrinsic Performance Influence-based Participant Contribution Estimation for Horizontal Federated Learning

Lin Zhang
Lixin Fan
Yong Luo
Ling-Yu Duan

The rapid development of modern artificial intelligence technique is mainly attributed to sufficient and high-quality data. However, in the data collection, personal privacy is at risk of being leaked. This issue can be addressed by federated learning, which is proposed to achieve efficient model training among multiple data providers without direct data access and aggregation. To encourage more parties owning high-quality data to participate in the federated learning, it is important to evaluate and reward the participant contribution in a reasonable, robust, and efficient manner. To achieve this goal, we propose a novel contribution estimation method: Intrinsic Performance Influence-based Contribution Estimation (IPICE). In particular, the class-level intrinsic performance influence is adopted as the contribution estimation criteria in IPICE, and a neural network is employed to exploit the non-linear relationship between the performance change and estimated contribution. Extensive experiments are conducted on various datasets, and the results demonstrate that IPICE is more accurate and stable than the counterpart in various data distribution settings. The computational complexity is significantly reduced in our IPICE, especially when a new party joins the federation. IPICE assigns small contributions to bad/garbage data and thus prevent them from participating and deteriorating the learning ecosystem.

Details DOI

AAAI Conference 2022 Conference Paper

Neighborhood Consensus Contrastive Learning for Backward-Compatible Representation

Shengsen Wu
Liang Chen
Yihang Lou
Yan Bai
Tao Bai
Minghua Deng
Ling-Yu Duan

In object re-identification (ReID), the development of deep learning techniques often involves model updates and deployment. It is unbearable to re-embedding and re-index with the system suspended when deploying new models. Therefore, backward-compatible representation is proposed to enable “new” features to be compared with “old” features directly, which means that the database is active when there are both “new” and “old” features in it. Thus we can scroll-refresh the database or even do nothing on the database to update. The existing backward-compatible methods either require a strong overlap between old and new training data or simply conduct constraints at the instance level. Thus they are difficult in handling complicated cluster structures and are limited in eliminating the impact of outliers in old embeddings, resulting in a risk of damaging the discriminative capability of new features. In this work, we propose a Neighborhood Consensus Contrastive Learning (NCCL) method. With no assumptions about the new training data, we estimate the subcluster structures of old embeddings. A new embedding is constrained with multiple old embeddings in both embedding space and discrimination space at the sub-class level. The effect of outliers diminished, as the multiple samples serve as “mean teachers”. Besides, we propose a scheme to filter the old embeddings with low credibility, further improving the compatibility robustness. Our method ensures the compatibility without impairing the accuracy of the new model. It can even improve the new model’s accuracy in most scenarios.

PDF Details

IJCAI Conference 2020 Conference Paper

Disentangled Feature Learning Network for Vehicle Re-Identification

Yan Bai
Yihang Lou
Yongxing Dai
Jun Liu
Ziqian Chen
Ling-Yu Duan

Vehicle Re-Identification (ReID) has attracted lots of research efforts due to its great significance to the public security. In vehicle ReID, we aim to learn features that are powerful in discriminating subtle differences between vehicles which are visually similar, and also robust against different orientations of the same vehicle. However, these two characteristics are hard to be encapsulated into a single feature representation simultaneously with unified supervision. Here we propose a Disentangled Feature Learning Network (DFLNet) to learn orientation specific and common features concurrently, which are discriminative at details and invariant to orientations, respectively. Moreover, to effectively use these two types of features for ReID, we further design a feature metric alignment scheme to ensure the consistency of the metric scales. The experiments show the effectiveness of our method that achieves state-of-the-art performance on three challenging datasets.

PDF Details DOI

AAAI Conference 2016 Conference Paper

Affinity Preserving Quantization for Hashing: A Vector Quantization Approach to Learning Compact Binary Codes

Zhe Wang
Ling-Yu Duan
Tiejun Huang
Gao Wen

Hashing techniques are powerful for approximate nearest neighbour (ANN) search. Existing quantization methods in hashing are all focused on scalar quantization (SQ) which is inferior in utilizing the inherent data distribution. In this paper, we propose a novel vector quantization (VQ) method named afﬁnity preserving quantization (APQ) to improve the quantization quality of projection values, which has significantly boosted the performance of state-of-the-art hashing techniques. In particular, our method incorporates the neighbourhood structure in the pre- and post-projection data space into vector quantization. APQ minimizes the quantization errors of projection values as well as the loss of afﬁnity property of original space. An effective algorithm has been proposed to solve the joint optimization problem in APQ, and the extension to larger binary codes has been resolved by applying product quantization to APQ. Extensive experiments have shown that APQ consistently outperforms the state-ofthe-art quantization methods, and has signiﬁcantly improved the performance of various hashing techniques.

PDF Details

IJCAI Conference 2016 Conference Paper

To Project More or to Quantize More: Minimize Reconstruction Bias for Learning Compact Binary Codes

Zhe Wang
Ling-Yu Duan
Junsong Yuan
Tiejun Huang
Wen Gao

We present a novel approach called Minimal Reconstruction Bias Hashing (MRH) to learn similarity preserving binary codes that jointly optimize both projection and quantization stages. Our work tackles an important problem of how to elegantly connect optimizing projection with optimizing quantization, and to maximize the complementary effects of two stages. Distinct from previous works, MRH can adaptively adjust the projection dimensionality to balance the information loss between projection and quantization. It is formulated as a problem of minimizing reconstruction bias of compressed signals. Extensive experiment results have shown the proposed MRH significantly outperforms a variety of state-of-the-art methods over several widely used benchmarks.

PDF Details

IJCAI Conference 2015 Conference Paper

Hamming Compatible Quantization for Hashing

Zhe Wang
Ling-Yu Duan
Jie Lin
Xiaofang Wang
Tiejun Huang
Wen Gao

Hashing is one of the effective techniques for fast Approximate Nearest Neighbour (ANN) search. Traditional single-bit quantization (SBQ) in most hashing methods incurs lots of quantization error which seriously degrades the search performance. To address the limitation of SBQ, researchers have proposed promising multi-bit quantization (MBQ) methods to quantize each projection dimension with multiple bits. However, some MBQ methods need to adopt specific distance for binary code matching instead of the original Hamming distance, which would significantly decrease the retrieval speed. Two typical MBQ methods Hierarchical Quantization and Double Bit Quantization retain the Hamming distance, but both of them only consider the projection dimensions during quantization, ignoring the neighborhood structure of raw data inherent in Euclidean space. In this paper, we propose a multi-bit quantization method named Hamming Compatible Quantization (HCQ) to preserve the capability of similarity metric between Euclidean space and Hamming space by utilizing the neighborhood structure of raw data. Extensive experiment results have shown our approach significantly improves the performance of various stateof-the-art hashing methods while maintaining fast retrieval speed.

PDF Details

TIST Journal 2012 Journal Article

A Generic Approach for Systematic Analysis of Sports Videos

Ning Zhang
Ling-Yu Duan
Lingfang Li
Qingming Huang
Jun Du
Wen Gao
Ling Guan

Various innovative and original works have been applied and proposed in the field of sports video analysis. However, individual works have focused on sophisticated methodologies with particular sport types and there has been a lack of scalable and holistic frameworks in this field. This article proposes a solution and presents a systematic and generic approach which is experimented on a relatively large-scale sports consortia. The system aims at the event detection scenario of an input video with an orderly sequential process. Initially, domain knowledge-independent local descriptors are extracted homogeneously from the input video sequence. Then the video representation is created by adopting a bag-of-visual-words (BoW) model. The video’s genre is first identified by applying the k-nearest neighbor (k-NN) classifiers on the initially obtained video representation, and various dissimilarity measures are assessed and evaluated analytically. Subsequently, an unsupervised probabilistic latent semantic analysis (PLSA)-based approach is employed at the same histogram-based video representation, characterizing each frame of video sequence into one of four view groups, namely closed-up-view, mid-view, long-view, and outer-field-view. Finally, a hidden conditional random field (HCRF) structured prediction model is utilized for interesting event detection. From experimental results, k-NN classifier using KL-divergence measurement demonstrates the best accuracy at 82.16% for genre categorization. Supervised SVM and unsupervised PLSA have average classification accuracies at 82.86% and 68.13%, respectively. The HCRF model achieves 92.31% accuracy using the unsupervised PLSA based label input, which is comparable with the supervised SVM based input at an accuracy of 93.08%. In general, such a systematic approach can be widely applied in processing massive videos generically.

Details DOI

IJCAI Conference 2011 Conference Paper

Learning Compact Visual Descriptor for Low Bit Rate Mobile Landmark Search

Rongrong Ji
Ling-Yu Duan
Jie Chen
Hongxun Yao
Tiejun Huang
Wen Gao

In this paper, we propose to extract a compact yet discriminative visual descriptor directly on the mobile device, which tackles the wireless query transmission latency in mobile landmark search. This descriptor is offline learnt from the location contexts of geo-tagged Web photos from both Flickr and Panoramio with two phrases: First, we segment the landmark photo collections into discrete geographical regions using a Gaussian Mixture Model [Stauffer et al. , 2000]. Second, a ranking sensitive vocabulary boosting is introduced to learn a compact codebook within each region. To tackle the locally optimal descriptor learning caused by imprecise geographical segmentation, we further iterate above phrases by feedback an "entropy" based descriptor compactness into a prior distribution to constrain the Gaussian mixture modeling. Consequently, when entering a specific geographical region, the codebook in the mobile device is downstream adapted, which ensures efficient extraction of compact descriptor, its low bit rate transmission, as well as promising discrimination ability. We deploy our descriptor within both HTC and iPhone mobile phones, testing landmark search in typical areas included Beijing, New York, and Barcelona containing one million images. Our learning descriptor outperforms alternative compact descriptors [Chen et al. , 2009][Chen et al. , 2010][Chandrasekhar et al. , 2009a][Chandrasekhar et al. , 2009b] with a large margin.

PDF Details DOI