Recently, coexisting ferroelectricity and superconductivity were reported in bilayer Td-MoTe2 and twisted bilayer graphene. Importantly, it was observed that an applied displacement field switches the superconductivity with a ferroelectric hysteresis. Such direct coupling between the ferroelectricity and superconductivity offers promising pathways for developing low-power, non-volatile memory devices. However, the coupling mechanism between the ferroelectricity and superconductivity remains poorly understood. In this work, we demonstrate that in a superconducting bilayer, the hysteretic switching of superconductivity can arise from an interlayer pairing. By deriving the Landau Ginzburg free energy expansion for the interlayer pairing, we show that along the ferroelectric hysteresis loop, the hysteretic exceeding of the critical polarization P_c that destroys the interlayer pairing leads to the hysteretic switching of superconductivity. The condition to have a ferroelectric hysteretic superconducting state is established to be P_r being smaller than P_c and P_c being smaller than P_s, where P_r and P_s denote the remanent and saturated polarization, respectively. Crucially, our scenario of interlayer pairing yields two predictions: (1) an enhancement of the upper critical displacement field with stronger interlayer coupling and (2) a pronounced, gate-tunable interlayer crossed Andreev reflection, both of which provide clear pathways for experimental verification.