This paper proposes a time-varying robust weighted least squares (TRWLS) approach to forecast excess returns to S&P 500 index. TRWLS combines time-dependent weights and robustness weights, which allows parameter to vary over time and to be robust to noises. The hyperparameters are selected using a machine learning method. Empirical results show that the forecast combinations for TRWLS models can reveal significant return predictability in both statistical and economic evaluation frameworks. Moreover, the TRWLS method dominates the ordinary least squares (OLS) method and the state-of-the-art methods dealing with parameter instability. The predictive power of the TRWLS method comes from the complementary effect of two dimensions of the weights and hyperparameters learning. The forecasting performance is robust to the alternative multivariate information methods, kernel functions, and validation sizes.