We proposed a time-varying higher-order co-moment estimate based on a single factor time-varying semi-nonparametric (SF-TVSNP) model. The model specification, model estimation and model selection approaches are given in this paper. The single factor model can efficiently reduce “the curse of dimensionality” problem in the time-varying higher-order co-moments estimation, and the semi-parametric structure can improve the robustness of the SF-TVSNP model. The empirical studies show that the SF-TVSNP model can effectively capture the time-varying structure of higher-order co-moments of asset returns, and it is more suitable for the latent structure of asset returns. High-dimensional dynamic portfolio based on the SF-TVSNP model can generate higher and stable economic value, which is further confirmed by robust analysis.