Research on Stock Price Prediction Based on a Hybrid LSTM-Transformer Model

Abstract

The goal of this paper is to establish a hybrid Long Short-Term Memory (LSTM)-Transformer model to achieve stock price prediction. By comparing the baseline LSTM model to the hybrid model, it is evident that the hybrid model outperforms the baseline in stock price prediction. It is vital for achieving optimal capital allocation, effective risk management, and successful execution. This model is based on next-day log return regression with return to price reconstruction, trained on daily OHLCV (data of stock about OPEN, HIGH, LOW, CLOSE, and VOLUME) features from 2021 to 2023 and evaluated in the first quarter of 2024 (Q1 2024). During the testing period, the mean absolute error (MAE) drops from 10.43 to 1.58, the root mean square error (RMSE) from 12.34 to 2.33, and the Price  rises from 0.25 to 0.97. Directional Accuracy improves from 60.66% to 65.57%, and on strong move days, it improves from 59.57% to 65.96%. The above results demonstrate that the hybrid LSTM-Transformer model outperforms the LSTM baseline model in terms of stock price prediction. Residual diagnostics reveal a mean close to zero and heavy tails with occasional spikes, which is consistent with event-driven jumps that cannot be anticipated using end-of-day data alone. These results support the use of the hybrid model for daily equity prediction and point out directions for future improvement, such as adding the process of volatility normalization, considering explicit event or gap features, and adopting tail robust or quantile objectives.