Future galaxy surveys such as Euclid, LSST and SKA will cover larger and larger scales where general relativistic effects become important. On the other hand, our study of large scale structure still relies on Newtonian N-body simulations. I show how standard Newtonian N-body simulations can be interpreted in terms of the weak-field limit of general relativity. Our framework allows the inclusion of radiation perturbations and the non-linear evolution of matter. I show how to construct the weak-field metric by combining Newtonian simulations with results from Einstein-Boltzmann codes. I discuss observational effects on weak lensing and ray tracing, identifying important relativistic corrections. Finally, I briefly discuss recent developments of GR simulations in cosmology.