Second-order electrical responses, such as the nonlinear Hall effect, probe inversion-symmetry-breaking terms invisible to linear transport, offering a direct window into the quantum geometry of Bloch electrons. Due to the strong spin-orbit interaction of its tantalum 5d orbitals and interfacial inversion symmetry breaking, KTaO3 serves as an ideal platform to investigate these effects. Here, we report that second-harmonic resistivities in KTaO3 exhibit large, reproducible mesoscopic fluctuations that persist in macroscopically large 200 microb devices. While macroscopic conductivity adheres to underlying crystal symmetries, these fluctuations are ubiquitous across all studied surface orientations. We propose these fluctuations arise from local inversion symmetry breaking driven by ferroelectric polarization pinned to the impurity landscape. This mechanism is supported by the signal's suppression above 10 K due to phase decoherence, and a complete loss of mesoscopic memory upon thermal cycling above 40 K. By revealing robust, magnetic-field-driven interference patterns that are highly sensitive to electrostatic gating, our findings establish that second-order responses are governed by the mesoscopic details of the Berry curvature. More broadly, we demonstrate that quantum geometry serves as a powerful tool to study materials, utilizing nonlinear transport to reveal local ordering tendencies that remain fundamentally hidden to other probes.