Dou Shen

IJCAI Conference 2018 Conference Paper

Automatic Gating of Attributes in Deep Structure

• Xiaoming Jin
• Tao He
• Cheng Wan
• Lan Yi
• Guiguang Ding
• Dou Shen

Deep structure has been widely applied in a large variety of fields for its excellence of representing data. Attributes are a unique type of data descriptions that have been successfully utilized in numerous tasks to enhance performance. However, to introduce attributes into deep structure is complicated and challenging, because different layers in deep structure accommodate features of different abstraction levels, while different attributes may naturally represent the data in different abstraction levels. This demands adaptively and jointly modeling of attributes and deep structure by carefully examining their relationship. Different from existing works that treat attributes straightforwardly as the same level without considering their abstraction levels, we can make better use of attributes in deep structure by properly connecting them. In this paper, we move forward along this new direction by proposing a deep structure named Attribute Gated Deep Belief Network (AG-DBN) that includes a tunable attribute-layer gating mechanism and automatically learns the best way of connecting attributes to appropriate hidden layers. Experimental results on a manually-labeled subset of ImageNet, a-Yahoo and a-Pascal data set justify the superiority of AG-DBN against several baselines including CNN model and other AG-DBN variants. Specifically, it outperforms the CNN model, VGG19, by significantly reducing the classification error from 26. 70% to 13. 56% on a-Pascal.

AAAI Conference 2015 Conference Paper

Gaussian Cardinality Restricted Boltzmann Machines

• Cheng Wan
• Xiaoming Jin
• Guiguang Ding
• Dou Shen

Restricted Boltzmann Machine (RBM) has been applied to a wide variety of tasks due to its advantage in feature extraction. Implementing sparsity constraint in the activated hidden units is an important improvement on RBM. The sparsity constraints in the existing methods are usually specified by users and are independent of the input data. However, the input data could be heterogeneous in content and thus naturally demand elastic and adaptive settings of the sparsity constraints. To solve this problem, we proposed a generalized model with adaptive sparsity constraint, named Gaussian Cardinality Restricted Boltzmann Machines (GC-RBM). In this model, the thresholds of hidden unit activations are decided by the input data and a given Gaussian distribution in the pre-training phase. We provide a principled method to train the GC-RBM with Gaussian prior. Experimental results on two real world data sets justify the effectiveness of the proposed method and its superiority over CaRBM in terms of classification accuracy.

AAAI Conference 2012 Conference Paper

Transfer Learning with Graph Co-Regularization

• Mingsheng Long
• Jianmin Wang
• Guiguang Ding
• Dou Shen
• Qiang Yang

Transfer learning proves to be effective for leveraging labeled data in the source domain to build an accurate classifier in the target domain. The basic assumption behind transfer learning is that the involved domains share some common latent factors. Previous methods usually explore these latent factors by optimizing two separate objective functions, i. e. , either maximizing the empirical likelihood, or preserving the geometric structure. Actually, these two objective functions are complementary to each other and optimizing them simultaneously can make the solution smoother and further improve the accuracy of the final model. In this paper, we propose a novel approach called Graph co-regularized Transfer Learning (GTL) for this purpose, which integrates the two objective functions seamlessly into one unified optimization problem. Thereafter, we present an iterative algorithm for the optimization problem with rigorous analysis on convergence and complexity. Our empirical study on two open data sets validates that GTL can consistently improve the classification accuracy compared to the state-of-the-art transfer learning methods.

IJCAI Conference 2011 Conference Paper

Short Text Classification Improved by Learning Multi-Granularity Topics

• Mengen Chen
• Xiaoming Jin
• Dou Shen

Understanding the rapidly growing short text is very important. Short text is different from traditional documents in its shortness and sparsity, which hinders the application of conventional machine learning and text mining algorithms. Two major approaches have been exploited to enrich the representation of short text. One is to fetch contextual information of a short text to directly add more text; the other is to derive latent topics from existing large corpus, which are used as features to enrich the representation of short text. The latter approach is elegant and efficient in most cases. The major trend along this direction is to derive latent topics of certain granularity through well-known topic models such as latent Dirichlet allocation (LDA). However, topics of certain granularity are usually not sufficient to set up effective feature spaces. In this paper, we move forward along this direction by proposing an method to leverage topics at multiple granularity, which can model the short text more precisely. Taking short text classification as an example, we compared our proposed method with the state-of-the-art baseline over one open data set. Our method reduced the classification error by 20. 25% and 16. 68%respectively on two classifiers.

AAAI Conference 2010 Conference Paper

Clickthrough Log Analysis by Collaborative Ranking

• Bin Cao
• Dou Shen
• Kuansan Wang
• Qiang Yang

Analyzing clickthrough log data is important for improving search performance as well as understanding user behaviors. In this paper, we propose a novel collaborative ranking model to tackle two difficulties in analyzing clickthrough log. First, previous studies have shown that users tend to click topranked results even they are less relevant. Therefore, we use pairwise ranking relation to avoid the position bias in clicks. Second, since click data are extremely sparse with respect to each query or user, we construct a collaboration model to eliminate the sparseness problem. We also find that the proposed model and previous popular used click-based models address different aspects of clickthrough log data. We further propose a hybrid model that can achieve significant improvement compared to the baselines on a large-scale real world dataset.

IJCAI Conference 2007 Conference Paper

• Bin Cao
• Dou Shen
• Jian-Tao Sun
• Xuanhui Wang
• Qiang Yang
• Zheng Chen

Detecting and tracking latent factors from temporal data is an important task. Most existing algorithms for latent topic detection such as Nonnegative Matrix Factorization (NMF) have been designed for static data. These algorithms are unable to capture the dynamic nature of temporally changing data streams. In this paper, we put forward an online NMF (ONMF) algorithm to detect latent factors and track their evolution while the data evolve. By leveraging the already detected latent factors and the newly arriving data, the latent factors are automatically and incrementally updated to reflect the change of factors. Furthermore, by imposing orthogonality on the detected latent factors, we can not only guarantee the unique solution of NMF but also alleviate the partial-data problem, which may cause NMF to fail when the data are scarce or the distribution is incomplete. Experiments on both synthesized data and real data validate the efficiency and effectiveness of our ONMF algorithm.

IJCAI Conference 2007 Conference Paper

• Dou Shen
• Jian-Tao Sun
• Hua Li
• Qiang Yang
• Zheng Chen

Many methods, including supervised and unsupervised algorithms, have been developed for extractive document summarization. Most supervised methods consider the summarization task as a two-class classification problem and classify each sentence individually without leveraging the relationship among sentences. The unsupervised methods use heuristic rules to select the most informative sentences into a summary directly, which are hard to generalize. In this paper, we present a Conditional Random Fields (CRF) based framework to keep the merits of the above two kinds of approaches while avoiding their disadvantages. What is more, the proposed framework can take the outcomes of previous methods as features and seamlessly integrate them. The key idea of our approach is to treat the summarization task as a sequence labeling problem. In this view, each document is a sequence of sentences and the summarization procedure labels the sentences by 1 and 0. The label of a sentence depends on the assignment of labels of others. We compared our proposed approach with eight existing methods on an open benchmark data set. The results show that our approach can improve the performance by more than 7. 1% and 12. 1% over the best supervised baseline and unsupervised baseline respectively in terms of two popular metrics F1 and ROUGE-2. Detailed analysis of the improvement is presented as well.

ICML Conference 2007 Conference Paper

Feature selection in a kernel space

• Bin Cao 0001
• Dou Shen
• Jian-Tao Sun
• Qiang Yang 0001
• Zheng Chen 0001

We address the problem of feature selection in a kernel space to select the most discriminative and informative features for classification and data analysis. This is a difficult problem because the dimension of a kernel space may be infinite. In the past, little work has been done on feature selection in a kernel space. To solve this problem, we derive a basis set in the kernel space as a first step for feature selection. Using the basis set, we then extend the margin-based feature selection algorithms that are proven effective even when many features are dependent. The selected features form a subspace of the kernel space, in which different state-of-the-art classification algorithms can be applied for classification. We conduct extensive experiments over real and simulated data to compare our proposed method with four baseline algorithms. Both theoretical analysis and experimental results validate the effectiveness of our proposed method.

AAAI Conference 2007 Conference Paper

Mining Web Query Hierarchies from Clickthrough Data

• Dou Shen
• Weizhu Chen

In this paper, we propose to mine query hierarchies from clickthrough data, which is within the larger area of automatic acquisition of knowledge from the Web. When a user submits a query to a search engine and clicks on the returned Web pages, the user’s understanding of the query as well as its relation to the Web pages is encoded in the clickthrough data. With millions of queries being submitted to search engines every day, it is both important and beneficial to mine the knowledge hidden in the queries and their intended Web pages. We can use this information in various ways, such as providing query suggestions and organizing the queries. In this paper, we plan to exploit the knowledge hidden in clickthrough logs by constructing query hierarchies, which can re- flect the relationship among queries. Our proposed method consists of two stages: generating candidate queries and determining “generalization/specialization” relations between these queries in a hierarchy. We test our method on some labeled data sets and illustrate the effectiveness of our proposed solution empirically.