Generating Biomedical Hypothesis With Spatiotemporal Transformers

Generating biomedical hypotheses is a difficult task as it requires uncovering the implicit associations between massive scientific terms from a large body of published literature. A recent line of Hypothesis Generation (HG) approaches - temporal graph-based approaches - have shown great success in modeling temporal evolution of term-pair relationships. However, these approaches model the temporal evolution of each term or term-pair with Recurrent Neural Network (RNN) independently, which neglects the rich covariation among all terms or term-pairs while ignoring direct dependencies between any two timesteps in a temporal sequence. To address this problem, we propose a Spatiotemporal Transformer-based Hypothesis Generation (STHG) method to interleave spatial covariation and temporal progression in a unified framework for constructing direct connections between any two term-pairs while modeling the temporal relevance between any two timesteps. Experiments on three biomedical relationship datasets show that STHG outperforms the state-of-the-art methods.

Authors

• Huiwei Zhou
• Lanlan Wang
• Weihong Yao
• Wenchu Li
• Hao Zhou Naval University of Engineering
• Hongyun Zeng

Keywords

• Transformers
• Spatiotemporal phenomena
• Biological system modeling
• Task analysis
• Semantics
• Recurrent neural networks
• Diseases
• Temporal Evolution
• Recurrent Neural Network
• Development Of Relationships
• Spatial Coverage
• Hypothesis Generation
• Temporal Sequence
• Direct Dependence
• Scientific Terms
• Temporal Approach
• Semantic
• Temporal Dimension
• Spatial Dimensions
• Temporal Information
• Spatial Dependence
• Spatial Attention
• Static Model
• Blood Viscosity
• Temporal Dependencies
• Embedding Dimension
• Graph Neural Networks
• Temporal Attention
• Temporal Graph
• ABC Model
• Attention Block
• Sequence Embedding
• Butyrylcholinesterase
• Association Rules
• Positional Encoding
• Node Embeddings
• Migraine
• biomedical term relation prediction
• spatiotemporal transformers
• temporal relational graph
• Humans
• Neural Networks, Computer
• Algorithms
• Biomedical Research
• Computational Biology