Sorry, I don't have time to read all code and comment on it - or write examples but a couple of things.

I think we should wait with discussing any ring buffer for now. You don't have any need of a ring buffer before you have the basic steps working ok.

1) As laready mentioned, your current-limiting resistors are mounted incorrecty. You should have one 47 ohm resistor for each column, not for each row. Why? Because each column will only light one LED at a time, and you want a constant number of lit LED to share the resistor. Your row drive on the other hand will have different number of LED lit depending on what you display. Before you make this change, the light intensity of the individual LED in a scanline will vary depending on the displayed pattern. So start by fixing your electronics.

2) Always think in full bytes when you think about algorithsm (or for larger processors think of 16-bit or 32-bit or 64-bit block sizes for your arrays). Your processor don't think 5 bits are fun to play with. It either works on a single (1) bit, or it works on a full byte of eight (8) bits.

3) Your processor is not a fast general-purpose processor. But it is a very fast bit-processing processor. Don't use one byte/pixel for your character sets or for your bitmaps. One LED = one bit. Then you can store 8 pixels in a single byte. But you have to spend time thinking about if one byte should store one column of 8 LED, or if it should store 8 LED from a scanline. Storing one byte / scanline makes scrolling easier. Storing 8 pixels from same scanline in a byte makes output to the matrix faster.

A scanline is 80 pixels long. That is 10 bytes.
A frame buffer - bitmap for the full message - must then be 8*10 = 80 bytes. Since you have two text lines, you will finally end up with 160 bytes for the data to display.

As I wrote in an earlier post in this thread, I think you should start by focusing on writing code that will emit a pre-built bitmap as fast as you can. That will give you an indication of how much time your processor will need to drive a 80x16 matrix. If that figure gets above 50% of your total CPU capacity, it is time to sit down and think about using a different processor - either one running at a higher clock frequency, or running with fewer clock cycles/instruction.

In another thread (remember that you have several concurrent threads active) where you had problems with your interrupts I suggested that you should use a loop-unrolled sequence to emit 8 bits at a time to the shift register.

For this step, create two 80x8 bitmaps (one for each text line) designed with the data stored in scanlines, i.e. let the first 10 bytes store the data for the top scanline. The next 10 bytes store the next scanline.

Your output loop would then do:
- Set pointer p1 to point at the first byte of your first 80 byte large bitmap buffer.
- Set pointer p2 to point at the first byte of your second 80 byte large bitmap buffer.
- Repeat the following 10 times (10*8 = 80 pixels in a scanline)
* Pick up one byte from first array and store in bitmapped byte 1 and increment pointer p1.
* Pick up one byte from second array and store in bitmapped byte 2 and increment pointer p2.
* Assign bit0 of the bitmapped byte 1 to the top-line shift register data line.
* Assign bit0 of the bitmapped byte 2 to the bottom-line shift register data line.
* Activate clock of shift registers.
* Deactivate clock of shift registers.
* Assign bit1 of the bitmapped byte 1 to the top-line shift register data line.
* Assign bit1 of the bitmapped byte 2 to the bottom-line shift register data line.
* Activate clock of shift registers.
* Deactivate clock of shift registers.
* ...
* Assign bit7 of the bitmapped byte 1 to the top-line shift register data line.
* Assign bit7 of the bitmapped byte 2 to the bottom-line shift register data line.
* Activate clock of shift registers.
* Deactivate clock of shift registers.
- Wait until your timer signal the time to emit next scan line.
- Deactivate all row-drive transistors.
- Activate latch bit of shift registers.
- Deactivate latch bit of shift registers.
- Activate topmost row-drive transistors for both top and bottom text lines.
- Your p1 and p2 have now both stepped 10 bytes (a full scanline) in your two bitmapps.
- Repeat the following 10 times (10*8 = 80 pixels in a scanline)
* Pick up one byte from first array and store in bitmapped byte 1 and increment pointer p1.
* Pick up one byte from second array and store in bitmapped byte 2 and increment pointer p2.
* Assign bit0 of the bitmapped byte 1 to the top-line shift register data line.
* Assign bit0 of the bitmapped byte 2 to the bottom-line shift register data line.
* Activate clock of shift registers.
* Deactivate clock of shift registers.
* ...
- Wait until your timer signal the time to emit next scan line.
- Deactivate all row-drive transistors.
- Activate latch bit of shift registers.
- Deactivate latch bit of shift registers.
- Activate second row-drive transistor for for both top and bottom text lines.
- Repet the above steps until all 8 scan-lines have been emitted and displayed.
- Restore p1 and p2 to point at the start of the two bitmaps.

The above represent a fast way to drive your shift registers. The majority of this code can be done in your main application - it is enough that your scanline interrupt does the switching part, i.e deactivate row-drive, latch the shift register, activate row-drive for next scanline.

Your main loop can do the bit-shifting, just as long as it makes sure that it has shifted out all data before the next scan-line interrupt.

If the above works well, then you can spend time thinking about how to build the bitmap instead of running with a fixed bitmap. You might possibly move all of the output code to an interrupt if you do not have other interrupts that are critical to serve. But you may also implement a state machine in the main loop, where it checks if a line switch has been made. If so, it shifts out a new scanline before doing work building a new bitmap (to a second set of 2*80 byte bitmap buffers). In this case, the main loop can't build a full bitmap in one pass, but must process a few characters at a time before checking a scanline interrupt flag to emit the next scanlines. All in all, the percentage of CPU time left from the output code is what you have to build new bitmaps.

The above output code was for data stored in scanline order. You may also test yourself to rewrite the code and output data stored in columns. The output code will be slower, but horisontal scrolls will be way easier. When you learn how to use a circular output buffer, you will not need to rebuild the output buffer between each scroll step. With a circular output buffer, it will be enough to rebuild the bitmap when you have received new data from the serial port.