Li Luo

ICLR Conference 2025 Conference Paper

On the Performance Analysis of Momentum Method: A Frequency Domain Perspective

• Xianliang Li
• Jun Luo
• Zhiwei Zheng
• Hanxiao Wang
• Li Luo
• Lingkun Wen
• Linlong Wu
• Sheng Xu 0004

Momentum-based optimizers are widely adopted for training neural networks. However, the optimal selection of momentum coefficients remains elusive. This uncertainty impedes a clear understanding of the role of momentum in stochastic gradient methods. In this paper, we present a frequency domain analysis framework that interprets the momentum method as a time-variant filter for gradients, where adjustments to momentum coefficients modify the filter characteristics. Our experiments support this perspective and provide a deeper understanding of the mechanism involved. Moreover, our analysis reveals the following significant findings: high-frequency gradient components are undesired in the late stages of training; preserving the original gradient in the early stages, and gradually amplifying low-frequency gradient components during training both enhance performance. Based on these insights, we propose Frequency Stochastic Gradient Descent with Momentum (FSGDM), a heuristic optimizer that dynamically adjusts the momentum filtering characteristic with an empirically effective dynamic magnitude response. Experimental results demonstrate the superiority of FSGDM over conventional momentum optimizers.

NeurIPS Conference 2025 Conference Paper

S-Crescendo: A Nested Transformer Weaving Framework for Scalable Nonlinear System in S-Domain Representation

• Junlang Huang
• Chen Hao
• Li Luo
• Yong Cai
• Lexin Zhang
• Tianhao Ma
• Yitian Zhang
• Zhong Guan

Simulation of high-order nonlinear system requires extensive computational resources, especially in modern VLSI backend design where bifurcation-induced instability and chaos-like transient behaviors pose challenges. We present S-Crescendo - a nested transformer weaving framework that synergizes S-domain with neural operators for scalable time-domain prediction in high-order nonlinear networks, alleviating the computational bottlenecks of conventional solvers via Newton-Raphson method. By leveraging the partial-fraction decomposition of an n-th order transfer function into first-order modal terms with repeated poles and residues, our method bypasses the conventional Jacobian matrix-based iterations and efficiently reduces computational complexity from cubic $O(n^3)$ to linear $O(n)$. The proposed architecture seamlessly integrates an S-domain encoder with an attention-based correction operator to simultaneously isolate dominant response and adaptively capture higher-order non-linearities. Validated on order-1 to order-10 networks, our method achieves up to 0. 99 test-set \(R^2\) accuracy against HSPICE golden waveforms and accelerates simulation by up to 18\(\times\), providing a scalable, physics-aware framework for high-dimensional nonlinear modeling.

ECAI Conference 2023 Conference Paper

Multi-Modal Fusion with Semantic Supervision for Radiology Report Generation

• Xing Jia
• Yun Xiong
• Yao Zhang 0009
• Li Luo

Radiology report generation, one way of analyzing radiology images, is to generate a textual report automatically for the given image, and it is of great significance to assist diagnosis and alleviate the workload of radiologists. Some report generation methods have been therefore proposed. However, these methods suffer from the problem of low-quality generation, because of the visual and textual bias and training with text similarity oriented objective. To solve this problem, we propose a novel radiology report generation model with multi-modal fusion and semantic supervision, namely MS-Gen. MS-Gen consists of two main components, i. e. , the semantic-visual fusion module and the semantic weighted contrastive loss. Specifically, the main idea of the semantic-visual fusion module is to make use of the domain-specific prior knowledge contained in a large pre-trained visual-language model and also the complementary nature between the image and text modalities. Moreover, a novel optimization term, i. e. , the semantic weighted contrastive loss, is proposed to guide the optimization process with semantic similarity objective, and further enforce the generated reports with higher clinical accuracy. Extensive experiments conducted on two real datasets of IU X-Ray and MIMIC-CXR demonstrate the effectiveness of MS-Gen.

JBHI Journal 2022 Journal Article

Bayesian Comorbidity Network and Cost Analysis for Asthma

• Zhilin Yong
• Li Luo
• Yonghong Gu
• Chunyang Li

The evolving disease spectrum poses significant challenges to the asthma management, thus worsening health quality and increased financial burden on patients. However, potential dependency pattern in comorbidity spectrum remains unclear. We built comorbidity networks based on Bayesian networks utilizing 19604 asthma-patient hospitalization data to investigate dependency patterns among asthma comorbidities. We analyze static properties and trajectory behaviors of gender- and age-stratified asthmatic comorbidity networks. Results suggest that chronic obstructive pulmonary disease, respiratory failure, hypertension, atherosclerosis, and gastritis and duodenitis are the hubs of the asthma comorbidity network. They have a strong dependency pattern, while most of the associations among other comorbidities are sparse and weak. The strength of association between comorbidities is higher in female asthmatics than in males. Although the comorbidity network in children with asthma is simple and stable, the onset of common comorbidities as they age will enhance the association between comorbidities and thus increase the risk of developing other comorbidities. Furthermore, the more attributes of comorbidities, the stronger association with each other, and the greater risk of causing high treatment costs. Our study will help to dissect the asthma co-morbidity network and provide a basis for improving asthma management and cost control.

JBHI Journal 2021 Journal Article

Design Comorbidity Portfolios to Improve Treatment Cost Prediction of Asthma Using Machine Learning

• Li Luo
• Xinzhu Yu
• Zhilin Yong
• Chunyang Li
• Yonghong Gu

Comorbidity is an important factor to consider when trying to predict the cost of treating asthma patients. When an asthmatic patient suffered from comorbidity, the cost of treating such a patient becomes dependent on the nature of the comorbidity. Therefore, lack of recognition of comorbidity on asthmatic patient poses a challenge in predicting the cost of treatment. In this study, we proposed a comorbidity portfolio design that improves the prediction cost of treating asthmatic patients by regrouping frequently occurred comorbidities in different cost groups. In the experiment, predictive models, including logistic regression, random forest, support vector machine, classification regression tree, and backpropagation neural network were trained with real-world data of asthmatic patients from 2012 to 2014 in a large city of China. The 10-fold cross validation and random search algorithm were employed to optimize the hyper-parameters. We recorded significant improvements using our model, which are attributed to comorbidity portfolios in area under curve (AUC) and sensitivity increase of 46. 89% (standard deviation: 4. 45%) and 101. 07% (standard deviation: 44. 94%), respectively. In risk analysis of comorbidity on cost, respiratory diseases with a cumulative proportion in the adjusted odds ratio of 36. 38% (95%CI: 27. 61%, 47. 86%) and circulatory diseases with a cumulative proportion in the adjusted odds ratio of 23. 83% (95%CI: 15. 95%, 35. 22%) are the dominant risks of asthmatic patients that affects the treatment cost. It is found that the comorbidity portfolio is robust, and provides a better prediction of the high-cost of treating asthmatic patients. The preliminary characterization of the joint risk of multiple comorbidities posed on cost are also reported. This study will be of great help in improving cost prediction and comorbidity management.

JBHI Journal 2017 Journal Article

Modeling the Length of Stay of Respiratory Patients in Emergency Department Using Coxian Phase-Type Distributions with Covariates

• Ting Zhu
• Li Luo
• Xinli Zhang
• Wenwu Shen

Variability and unpredictability are typical features of emergency departments (EDs) where patients randomly arrive with diverse conditions. Patient length of stay (LOS) represents the consumption level of hospital resources, and it is positively skewed and heterogeneous. Both accurate modeling of patient ED LOS and analysis of potential blocking causes are especially useful for patient scheduling and resource management. To tackle the uncertainty of ED LOS, this paper introduces two methods: statistical modeling and distribution fitting. The models are applied to 894 respiratory diseases patients data in the year 2014 from ED of a Chinese public tertiary hospital. Covariates recorded include patient region, gender, age, arrival time, arrival mode, triage category, and treatment area. A Coxian phase-type (PH) distribution model with covariates is proposed as an alternative method for modeling ED LOS. The expectation-maximization (EM) algorithm is used to implement parameter estimation. The results show that ED LOS data can be modeled well by the proposed models. Distributions of ED LOS differ significantly with respect to patients' gender, arrival mode, and treatment area. Using the fitted Coxian PH model will assist ED managers in identifying patients who are most likely to have an extreme ED LOS and in predicting the forthcoming workload for resources.

JBHI Journal 2017 Journal Article

Time-Series Approaches for Forecasting the Number of Hospital Daily Discharged Inpatients

• Ting Zhu
• Li Luo
• Xinli Zhang
• Yingkang Shi
• Wenwu Shen

For hospitals where decisions regarding acceptable rates of elective admissions are made in advance based on expected available bed capacity and emergency requests, accurate predictions of inpatient bed capacity are especially useful for capacity reservation purposes. As given, the remaining unoccupied beds at the end of each day, bed capacity of the next day can be obtained by examining the forecasts of the number of discharged patients during the next day. The features of fluctuations in daily discharges like trend, seasonal cycles, special-day effects, and autocorrelation complicate decision optimizing, while time-series models can capture these features well. This research compares three models: a model combining seasonal regression and ARIMA, a multiplicative seasonal ARIMA (MSARIMA) model, and a combinatorial model based on MSARIMA and weighted Markov Chain models in generating forecasts of daily discharges. The models are applied to three years of discharge data of an entire hospital. Several performance measures like the direction of the symmetry value, normalized mean squared error, and mean absolute percentage error are utilized to capture the under- and overprediction in model selection. The findings indicate that daily discharges can be forecast by using the proposed models. A number of important practical implications are discussed, such as the use of accurate forecasts in discharge planning, admission scheduling, and capacity reservation.

TCS Journal 2014 Journal Article

Semi-online scheduling with two GoS levels and unit processing time

• Taibo Luo
• Yinfeng Xu
• Li Luo
• Changzheng He