Arrow Research search

Author name cluster

Fangyu Liu

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

4 papers
1 author row

Possible papers

4

EAAI Journal 2025 Journal Article

Physics-informed graph neural network for 3D spatiotemporal structural response modeling of flexible pavements

  • Fangyu Liu
  • Imad L. Al-Qadi

Quantifying pavement damage is crucial for roadway agencies' maintenance planning. This study proposed a Physics-informed Graph Neural Network-based Pavement Simulator (PhyGPS) to predict three-dimensional (3D) asphalt concrete pavement responses, building upon an established data-driven Graph Neural Network-based Pavement Simulator (GPS) model. The key innovation lies in integrating knowledge graphs and mechanics equations to create a physics loss function, distinguishing it from its data-driven counterpart. The physics loss function comprises strain-displacement and stress loss components derived from 3D strain-displacement relations and stress equilibrium principles. A thorough 3D finite element (FE) pavement database supported the model development. The 3D FE pavement data was transformed into graph format where nodes and edges represent 3D FE pavement models’ nodes and node connections, respectively. Performance evaluation employed two case studies: “OneStep” for assessing short-term predictive capabilities and “Rollout” for examining long-term prediction accuracy under practical conditions. Results demonstrated that the physics-informed GPS model showed superior long-term predictive capability and robustness while maintaining excellent short-term accuracy compared to the data-driven model. Both models achieve rollout time under 8 s per FE simulation case, a dramatic improvement over the 12-h runtime of traditional 3D FE pavement models. The PhyGPS model successfully integrates physics principles, spatial relationships between structural components, temporal correlations in structural data, and complex material properties, offering an accurate, robust, and computationally efficient solution for predicting 3D pavement responses.

NeurIPS Conference 2024 Conference Paper

ReMI: A Dataset for Reasoning with Multiple Images

  • Mehran Kazemi
  • Nishanth Dikkala
  • Ankit Anand
  • Petar Devic
  • Ishita Dasgupta
  • Fangyu Liu
  • Bahare Fatemi
  • Pranjal Awasthi

With the continuous advancement of large language models (LLMs), it is essential to create new benchmarks to evaluate their expanding capabilities and identify areas for improvement. This work focuses on multi-image reasoning, an emerging capability in state-of-the-art LLMs. We introduce ReMI, a dataset designed to assess LLMs' ability to reason with multiple images. This dataset encompasses a diverse range of tasks, spanning various reasoning domains such as math, physics, logic, code, table/chart understanding, and spatial and temporal reasoning. It also covers a broad spectrum of characteristics found in multi-image reasoning scenarios. We have benchmarked several cutting-edge LLMs using ReMI and found a substantial gap between their performance and human-level proficiency. This highlights the challenges in multi-image reasoning and the need for further research. Our analysis also reveals the strengths and weaknesses of different models, shedding light on the types of reasoning that are currently attainable and areas where future models require improvement. We anticipate that ReMI will be a valuable resource for developing and evaluating more sophisticated LLMs capable of handling real-world multi-image understanding tasks.

AAAI Conference 2021 Conference Paper

Visual Pivoting for (Unsupervised) Entity Alignment

  • Fangyu Liu
  • Muhao Chen
  • Dan Roth
  • Nigel Collier

This work studies the use of visual semantic representations to align entities in heterogeneous knowledge graphs (KGs). Images are natural components of many existing KGs. By combining visual knowledge with other auxiliary information, we show that the proposed new approach, EVA, creates a holistic entity representation that provides strong signals for cross-graph entity alignment. Besides, previous entity alignment methods require human labelled seed alignment, restricting availability. EVA provides a completely unsupervised solution by leveraging the visual similarity of entities to create an initial seed dictionary (visual pivots). Experiments on benchmark data sets DBP15k and DWY15k show that EVA offers state-of-the-art performance on both monolingual and cross-lingual entity alignment tasks. Furthermore, we discover that images are particularly useful to align long-tail KG entities, which inherently lack the structural contexts that are necessary for capturing the correspondences. Code release: https: //github. com/cambridgeltl/eva; project page: http: //cogcomp. org/page/publication view/927.

AAAI Conference 2020 Conference Paper

HAL: Improved Text-Image Matching by Mitigating Visual Semantic Hubs

  • Fangyu Liu
  • Rongtian Ye
  • Xun Wang
  • Shuaipeng Li

The hubness problem widely exists in high-dimensional embedding space and is a fundamental source of error for crossmodal matching tasks. In this work, we study the emergence of hubs in Visual Semantic Embeddings (VSE) with application to text-image matching. We analyze the pros and cons of two widely adopted optimization objectives for training VSE and propose a novel hubness-aware loss function (HAL) that addresses previous methods’ defects. Unlike (Faghri et al. 2018) which simply takes the hardest sample within a minibatch, HAL takes all samples into account, using both local and global statistics to scale up the weights of “hubs”. We experiment our method with various configurations of model architectures and datasets. The method exhibits exceptionally good robustness and brings consistent improvement on the task of text-image matching across all settings. Specifically, under the same model architectures as (Faghri et al. 2018) and (Lee et al. 2018), by switching only the learning objective, we report a maximum R@1 improvement of 7. 4% on MS-COCO and 8. 3% on Flickr30k. 1