Zhimin Li

EAAI Journal 2024 Journal Article

A benchmarking framework for eye-tracking-based vigilance prediction of vessel traffic controllers

• Zhimin Li
• Ruilin Li
• Liqiang Yuan
• Jian Cui
• Fan Li

IJCAI Conference 2024 Conference Paper

Digital Avatars: Framework Development and Their Evaluation

• Timothy Rupprecht
• Sung-En Chang
• Yushu Wu
• Lei Lu
• Enfu Nan
• Chih-hsiang Li
• Caiyue Lai
• Zhimin Li

We present a novel prompting strategy for artificial intelligence driven digital avatars. To better quantify how our prompting strategy affects anthropomorphic features like humor, authenticity, and favorability we present Crowd Vote - an adaptation of Crowd Score that allows for judges to elect a large language model (LLM) candidate over competitors answering the same or similar prompts. To visualize the responses of our LLM, and the effectiveness of our prompting strategy we propose an end-to-end framework for creating high-fidelity artificial intelligence (AI) driven digital avatars. This pipeline effectively captures an individual's essence for interaction and our streaming algorithm delivers a high-quality digital avatar with real-time audio-video streaming from server to mobile device. Both our visualization tool, and our Crowd Vote metrics demonstrate our AI driven digital avatars have state-of-the-art humor, authenticity, and favorability outperforming all competitors and baselines. In the case of our Donald Trump and Joe Biden avatars, their authenticity and favorability are rated higher than even their real-world equivalents.

AAAI Conference 2022 Conference Paper

Improving Human-Object Interaction Detection via Phrase Learning and Label Composition

• Zhimin Li
• Cheng Zou
• Yu Zhao
• Boxun Li
• Sheng Zhong

Human-Object Interaction (HOI) detection is a fundamental task in high-level human-centric scene understanding. We propose PhraseHOI, containing a HOI branch and a novel phrase branch, to leverage language prior and improve relation expression. Specifically, the phrase branch is supervised by semantic embeddings, whose ground truths are automatically converted from the original HOI annotations without extra human efforts. Meanwhile, a novel label composition method is proposed to deal with the long-tailed problem in HOI, which composites novel phrase labels by semantic neighbors. Further, to optimize the phrase branch, a loss composed of a distilling loss and a balanced triplet loss is proposed. Extensive experiments are conducted to prove the effectiveness of the proposed PhraseHOI, which achieves significant improvement over the baseline and surpasses previous state-of-the-art methods on Full and NonRare on the challenging HICO-DET benchmark.