Gravitational waves (GW) can be used to probe various epochs in the early Universe. In this talk I will discuss about the production of Gravitational waves in a particular model of inflationary magnetogenesis. In this model, we require a low energy scale for inflation and reheating (reheating temperature, TR < 104 GeV) and have a blue spectrum of electromagnetic (EM) field which peaks around the horizon scale of reheating. The anisotropic stress associated with these EM fields naturally source the production of a stochastic background of GW with frequencies in the range of tens of nano Hertz to milli Hertz. These two extremes of the range can be probed respectively by pulsar timing arrays (PTA) experiments and the upcoming Laser Interferometric Space Array (LISA). If detected in future these GW will provide a unique probe of such models of inflationary magnetogenesis.