ETA: Learning Optical Flow with Efficient Temporal Attention

Considering the potential of using multi-frame information to solve the occlusion problem, we introduce a novel idea of multi-frame information integration, which uses the attention mechanism to fuse the temporal information from the previous frame. The idea can effectively improve the estimation accuracy in occluded regions and optimize the inference speed under multi-frame settings. Meanwhile, we suggest the concept of attention confidence to provide an explicit value criterion for the model to utilize useful attention information more efficiently. Furthermore, we propose an Efficient Temporal Attention network (ETA), which achieves promising results on Sintel and KITTI benchmarks, especially with a 9. 4% error reduction compared to the baseline method GMA on Sintel (test) Clean.

Authors

• Bo Wang 0144
• Zhenping Sun
• Yang Yu 0014
• Li Liu 0002
• Jian Li 0003
• Dewen Hu

Keywords

• Attention mechanisms
• Accuracy
• Fuses
• Aggregates
• Estimation
• Benchmark testing
• Time-domain analysis
• Optical flow
• Intelligent robots
• Temporal Attention
• Attention Mechanism
• Temporal Information
• Baseline Methods
• Previous Frame
• Explicit Criteria
• Inference Speed
• Idea Of Integration
• Accuracy In Regions
• Occlusion Problem
• Occluded Regions
• Image Features
• Time Domain
• Projection Matrix
• Extensive Evaluation
• Motion Features
• Consecutive Frames
• Flow Estimation
• Current Frame
• Optical Flow Estimation
• Adjacent Frames
• Temporal Aggregation
• Cost Volume
• Forward Flow
• Accurate Flow
• Continuous Transmission
• Temporal Modulation
• Optical Networks
• Backward Flow