Loss landscapes are sharp for deep networks:

This shows that resnet offers an inductive bias where close-by policies have close rewards during training.

Art project: https://losslandscape.com/

Paper citation:

@article{DBLP:journals/corr/abs-1712-09913,
  author    = {Hao Li and
               Zheng Xu and
               Gavin Taylor and
               Tom Goldstein},
  title     = {Visualizing the Loss Landscape of Neural Nets},
  journal   = {CoRR},
  volume    = {abs/1712.09913},
  year      = {2017},
  url       = {http://arxiv.org/abs/1712.09913},
  eprinttype = {arXiv},
  eprint    = {1712.09913},
  timestamp = {Thu, 16 May 2019 13:19:49 +0200},
  biburl    = {https://dblp.org/rec/journals/corr/abs-1712-09913.bib},
  bibsource = {dblp computer science bibliography, https://dblp.org}
}