Analyzing Hyperparameter Impact on TimeMixer Accuracy for Short-Term Load Forecasting

This paper examines the sensitivity of the TimeMixer forecasting model to key hyperparameters in short-term load forecasting (STLF). Experiments are conducted on the New South Wales (NSW) half-hourly load dataset under a fixed forecasting protocol with a 7-day input window (input_size = 336) and a 1-day-ahead horizon (h = 48). Models are trained using a recent-window retraining setup and evaluated on a time-ordered holdout set. The research contribution is twofold: To quantify the effects of four influential hyperparameters (learning_rate, dropout, d_model, and e_layers) on TimeMixer accuracy using trial-level error distributions and summary statistics, and to compare three hyperparameter optimization strategies, Random Search (RS), Tree-structured Parzen Estimator (TPE), and a Genetic Algorithm (GA) under an identical evaluation budget. Forecasting performance is assessed using mean absolute percentage error (MAPE), and hyperparameter effects are characterized through boxplots across trials and median performance across discrete hyperparameter levels. Results show that hyperparameter optimization consistently improves TimeMixer over the default configuration, reducing the best MAPE to 2.154% (RS), 2.119% (TPE), and 1.895% (GA), with GA achieving the most significant improvement. These findings provide practical guidance on selecting both an optimization strategy and robust hyperparameter settings when deploying TimeMixer for STLF.