The higher voltage a processor uses, the higher voltages all chip capacitors have to charged to.

So reducing the voltage means less amount of energy to charge/discharge all capacitors with.

And lower voltages also means that the distance between features on the chip can be made smaller since the isolation required is lower.

So manufacturers really do want to drop the voltages. Some chips does it by instead having an internal DC-DC converter, allowing just the core to run at lower voltages, while still keeping the large input/output transistors that both supports 5V (but possibly with 3.3V input logic levels) and can source/sink much current and stand a lot of abuse. Few people wants an external DC/DC for smaller microcontrollers because of the extra components, complexities, noise, ...

Desktop processors draws too much current for internal DC/DC, so they just have multiple sets of VCC pins for core, I/O pins, memory interface, ...