We propose new parameterizations for neural networks in order to estimate out-of-sample Expected Shortfall, {and even more generally, out-of-sample conditional tail moments}, in heavy-tailed settings as functions of confidence levels. The proposed neural network estimator is able to extrapolate in the distribution tails thanks to an extension of the usual extreme-value second-order condition to an arbitrary order. The convergence rate of the uniform error between the log-{conditional tail moment} and its neural network approximation is established. The finite sample performance of the neural network estimator is compared to bias-reduced extreme-value competitors on simulated data. It is shown that our method outperforms them in difficult heavy-tailed situations where other estimators almost all fail. Finally, the neural network estimator is tested on real data to investigate the behavior of cryptocurrency extreme loss returns.