Author name cluster

Longxiang Shi

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

2 papers

1 author row

AAMAS Conference 2023 Conference Paper

Efficient Interactive Recommendation via Huffman Tree-based Policy Learning

Longxiang Shi
Zilin Zhang
Shoujin Wang
Binbin Zhou
Minghui Wu
Cheng Yang
Shijian Li

Interactive recommender systems (IRSs) are an essential part of our daily life, as they can suggest items to persistently satisfy our demands. Due to the interactive nature, conventional static recommendation methods such as matrix factorization, and content-based filtering are ineffective to capture the dynamic preferences of users. Recently, reinforcement learning (RL) has shown great potential in addressing the challenges in IRSs, since it can capture users’ dynamic preferences and model the long-term profit of user-item interactions. However, millions of items in real-world IRSs lead to a large discrete action space in the RL setting, rendering RL-based IRSs inefficient and hindering their widespread application. Such an inefficiency issue has not been well addressed in the literature. In order to address this issue, we propose a novel Huffman Tree Policy Recommendation (HTPR) framework. Specifically, a novel policy learning network based on a newly designed Huffman tree is proposed for policy representation learning, which effectively improves the learning efficiency. Moreover, a novel parameter-sharing scheme is devised to further reduce unnecessary computations. Extensive experiments on two real-world benchmark datasets demonstrate the superiority of HTPR over the state-of-the-art IRS methods in terms of both recommendation accuracy and efficiency.

PDF

AAMAS Conference 2019 Conference Paper

TBQ(σ): Improving Efficiency of Trace Utilization for Off-Policy Reinforcement Learning

Longxiang Shi
Shijian Li
Longbing Cao
Long Yang
Gang Pan

Off-policy reinforcement learning with eligibility traces faces is challenging because of the discrepancy between target policy and behavior policy. One common approach is to measure the difference between two policies in a probabilistic way, such as importance sampling and tree-backup. However, existing off-policy learning methods based on probabilistic policy measurement are inefficient when utilizing traces under a greedy target policy, which is ineffective for control problems. The traces are cut immediately when a non-greedy action is taken, which may lose the advantage of eligibility traces and slow down the learning process. Alternatively, some non-probabilistic measurement methods such as General Q(λ) and Naive Q(λ) never cut traces, but face convergence problems in practice. To address the above issues, this paper introduces a new method named TBQ(σ), which effectively unifies the tree-backup algorithm and Naive Q(λ). By introducing a new parameter σ to illustrate the degree of utilizing traces, TBQ(σ) creates an effective integration of TB(λ) and Naive Q(λ) and continuous role shift between them. The contraction property of TB(σ) is theoretically analyzed for both policy evaluation and control settings. We also derive the online version of TBQ(σ) and give the convergence proof. We empirically show that, for ϵ ∈ (0, 1] in ϵ-greedy policies, there exists some degree of utilizing traces for λ ∈ [0, 1], which can improve the efficiency in trace utilization for off-policy reinforcement learning, to both accelerate the learning process and improve the performance.

PDF