If you take a closer look at how to connect relays, solenoids and other inductive loads, you will notice that they contain an extra diode.

This freewheel diode is there to make sure that the magnetic energy stored in the coil will not be converted into a huge voltage when the coil current is cut.

You do need it - you even face the risk of broken electronics if you don't. The coil want to maintain the same current. But since the resistance in the coil circuit changes from very low to very high, the voltage has to be stepped up tremendously. So you get a huge voltage spike - unless there is a reversed diode over the coil, that maintains a low-impedance loop for the coil current to take.