Reinforced co-learning for semi-supervised ranking

Abstract

Learning to rank is vital to information retrieval and recommendation systems. Directly optimizing the listwise evaluation measure such as normalized discounted cumulative gain (NDCG) is an advanced way to learn a ranking model. However, this is only suited for training data with effective labels. In real applications, we are more often faced with the semi-supervised setting that only a partial set of data has labels. In this paper, we propose a co-learning strategy for the semi-supervised ranking problem. Our model has two modules: the classifier module and the reinforcement ranker module. Given a query, the classifier module is trained to classify whether a document is relevant or not. The reinforcement ranker module is trained to give relevance scores on the basis of treating ranking problems as Markov decision processes (MDP). We name our approach "reinforced co-learning" because the two modules are iteratively optimized and affect each other while training. When training the classifier module, we use the reinforcement module to give every candidate a relevance score and sample lower scored documents as irrelevant samples (negative samples). Likewise, in order to train the reinforcement ranker module, we use the classifier module to predict labels in the sequence in order to calculate the combined rewards. The linkage between the two modules is also reflected in the network structure. We add the feature sharing layer, which enables the classifier to distill its intermediate representations to the learning of reinforcement ranker module. Extensive experiments and ablation studies show that both our co-learning strategy and feature sharing can improve semi-supervised ranking problems.