We experimentally investigate the effects of noise on the adiabatic and cyclic geometric phase, also termed the Berry phase. By introducing artificial fluctuations in the path of the control field, we measure the geometric contribution to dephasing of an effective two-level system for a variety of noise powers and different paths. Our results, measured using a microwave-driven superconducting qubit, clearly show that only fluctuations which distort the path lead to geometric dephasing. In a direct comparison with the dynamic phase, which is path independent, we observe that the Berry phase is less affected by noise-induced dephasing. This observation directly points towards the potential of geometric phases for quantum gates or metrological applications.