We derive the duality-invariant quantum phase that accumulates a dyon upon encircling a solenoid enclosing electric and magnetic fluxes. The phase is topological because it depends on the number of windings the dyon carries out around the solenoid and is independent of the shape of the dyon trajectory... The phase is nonlocal because there is no generalised Lorentz force locally acting on the dyon and then the electric and magnetic fluxes have no local consequences at any point of the dyon trajectory. We also obtain the energy levels of the proposed dyon-solenoid configuration and envision a quantum two-slit interference experiment associated to the new phase. We show that the duality symmetry of the new phase unifies the Aharonov-Bohm phase and the Dowling-Williams-Franson phase. This duality also suggests two new alternative interpretations of the Aharonov-Bohm phase. (read more)