PKR-QA: A Benchmark for Procedural Knowledge Reasoning with Knowledge Module Learning

We introduce PKR-QA (Procedural Knowledge Reasoning Question Answering), a new benchmark for question answering over procedural tasks that require structured reasoning. PKR-QA is constructed semi-automatically using a procedural knowledge graph (PKG), which encodes task-specific knowledge across diverse domains. The PKG is built by curating and linking information from the COIN instructional video dataset and the ontology, enriched with commonsense knowledge from ConceptNet and structured outputs from Large Language Models (LLMs), followed by manual verification. To generate question-answer pairs, we design graph traversal templates where each template is applied systematically over PKG. To enable interpretable reasoning, we propose a neurosymbolic approach called Knowledge Module Learning (KML), which learns procedural relations via neural modules and composes them for structured reasoning with LLMs. Experiments demonstrate that this paradigm improves reasoning performance on PKR-QA and enables step-by-step reasoning traces that facilitate interpretability.

Authors

• Thanh-Son Nguyen Institute of High Performance Computing, Agency for Science, Technology and Research, Singapore
• Hong Yang
• Tzeh Yuan Neoh Agency for Science, Technology and Research, Singapore
• Hao Zhang vivo Mobile Communication Co., Ltd
• Ee Yeo Keat Centre for Frontier AI Research, Agency for Science, Technology and Research, Singapore Institute of High Performance Computing, Agency for Science, Technology and Research, Singapore
• Basura Fernando

Keywords

No keywords are indexed for this paper.