Ising superconductors have attracted quite some attention recently, due to their resilience against magnetic fields way beyond the Pauli-paramagnetic limit. Their protection against external magnetic field relies on strong Ising spin-orbit coupling, which originates from in-plane inversion symmetry breaking. Due to the heavy atom nature of Ising SCs, a smaller but sizable Rashba SOC could be present through gating or interfacial effects. Here, we consider the effect of Rashba SOC in a two valley model of Ising superconductors with an attractive s-wave interaction. We show that Rashba SOC gives a critical magnetic field, above which the superconducting order parameter collapses at low temperatures. This effect, however, disappears at high temperatures. Our findings demonstrate that the low- and high temperature physics of Ising SCs is quantitatively and qualitatively different in our two-valley model, and may lead to new ways to determine the strength of the Rashba SOC in Ising SCs.