A defining characteristic of superconductors is their tendency to expel magnetic fields, yet above a critical threshold, magnetic flux penetrates in discrete quanta carried by Abrikosov vortices. The superconducting gap is completely suppressed at the vortex core, rendering them dissipative, semi-classical entities that impact applications from high-current-density wires to quantum devices. Material disorder can drive a crossover to vortices that preserve an energy gap at the core, due to intrinsic or emergent granularity on the scale of the coherence length. Although in this effective tunnel-junction regime quantum vortex behavior could emerge, and signatures have been observed in diverse systems, coherent manipulation of vortex states has remained elusive. Here we present evidence that vortices trapped in granular superconducting films can behave as two-level systems, exhibiting microsecond-range quantum coherence and energy relaxation times that reach fractions of a millisecond. Using the tools of circuit quantum electrodynamics, we perform coherent manipulation and quantum non-demolition readout of vortex states in granular aluminum microwave resonators, heralding new directions for quantum information processing, materials characterization, and sensing.