You put the decoupling capacitors as close to the ICs as possible, trying to minimize the wire length to VCC and to GND.

And you normally have one cap/chip if the chip has only one VCC pin. For larger processors with multiple VCC and GND pins, you normally try to fit multiple decoupling capacitors.

For DIL sockets, you can normally always see a capacitor on one short end of every chip if you look at existing electronics. For surface-mounted boards, it isn't uncommon that the cap may be on the other side of the board, directly below each chip. When you know what to look for, you can take a look at the motherboard of your PC and you will see a huge number of decoupling capacitors. Especially for power-hungry chips like the processor.