In the last decade, the Nitrogen-Vacancy (NV) defect in diamond has emerged as an ideal quantum sensor to probe magnetic stray fields. By using a diamond nanopillar, containing a single NV center, as the tip of an atomic force microscope (AFM), it becomes possible to perform Scanning NV magnetometry (SNVM). This configuration not only enables us to measure with high spatial resolution and sensitivity, it also makes a broad temperature range accessible. While room temperature NV systems are widely used already, cryogenic NV setups still remain sparse. At the MPI CPfS, we have successfully built such a cryogenic SNVM setup, capable of measuring at variable temperatures down to 1.8 K, which we demonstrate by presenting first measurements, conducted at low temperatures. This paves the way to study a multitude of interesting phenomena, like magnetic phase transitions and superconductivity. In particular we want to focus on investigating magnetic structures of unconventional superconductors, like the recently emerged PtBi2, reported to have superconducting surface states as seen from ARPES and STM.