Except that I would like to be able to twinkle (or read back) maybe 20 pins concurrently at up to 5MHz, and it would take a huge number of I2C or SPI clocks to issue these commands. Assuming a 16-bit output buffer with a 8-bit command and same for readback, it would take 24 SPI clocks to drive 16 outputs and 24 SPI clocks to read 16 inputs. That would mean 48*5 = 240MHz SPI clock speed. I2C would broadcast TV transmissions ;)

So a programmable buffer that had a SPI-like interface would need be more like the SD interface, i.e. one clock and multiple data lines. If not having 1-to-1 between input pin and output pin, I would need 16-bit or 32-bit wide transfers for each clock cycle - potentially created by multiple 8-bit wide interfaces run concurrently. But it would be quite close to the previous suggestion - multiple AVR chips that copies data from one 8-bit port to another, while performing a lookup to map 8 received bits into correct output pins and then a corresponding mapping for data in the reverse direction. But the poor AVR chips would have to glow from the effort.