Infinite-Dimensional Feature Interaction

The past neural network design has largely focused on feature \textit{representation space} dimension and its capacity scaling (e. g. , width, depth), but overlooked the feature \textit{interaction space} scaling. Recent advancements have shown shifted focus towards element-wise multiplication to facilitate higher-dimensional feature interaction space for better information transformation. Despite this progress, multiplications predominantly capture low-order interactions, thus remaining confined to a finite-dimensional interaction space. To transcend this limitation, classic kernel methods emerge as a promising solution to engage features in an infinite-dimensional space. We introduce InfiNet, a model architecture that enables feature interaction within an infinite-dimensional space created by RBF kernel. Our experiments reveal that InfiNet achieves new state-of-the-art, owing to its capability to leverage infinite-dimensional interactions, significantly enhancing model performance.

Authors

• Chenhui Xu
• Fuxun Yu
• Maoliang Li
• Zihao Zheng Harbin Institute of Technology, Shenzhen
• Zirui Xu Shanghai University of Electric Power
• Jinjun Xiong
• Xiang Chen Zhejiang University

Keywords

No keywords are indexed for this paper.