Date of Publication :5th April 2025

Abstract: In recent years, machine learning models have shown great success in tasks involving text data or structured data. However, most existing models are designed to handle one type of data at a time. When both text (like research paper titles and abstracts) and structured features (like numerical or categorical metadata) are present, current models either ignore one type or combine them in a basic way, which leads to low performance and slow training. Additionally, models that do try to combine both types, such as hybrids of transformers and graph neural networks, often require heavy computational resources and are difficult to train. To solve these challenges, we propose a new model called LFGFormer (Lightweight Fusion GraphFormer). This model is designed to handle both text and structured data efficiently. It uses a lightweight transformer to process text information and a shallow graph structure to represent the relationships between structured features. These two representations are then merged through a fusion layer that allows the model to learn important connections between the two data types. Our model is designed to work well even on CPUs, with faster training time and reduced memory usage. It can be used for various tasks such as classifi cation and regression, especially in fields like scientific paper analysis, financial modeling, and healthcare data processing. The proposed model opens up new possibilities for creating efficient and powerful machine learning systems for real-world applications..

Reference :