It sounds as if the processor has some bits which would be helpful in processing the video, in fact I think I looked at it for a projectsome time ago and decided that it wasn't quite man enough to do what I needed.
The first step you need to do is work out how you are going to do things such as the Bayer filtering which is what allows you to convert the raw data from the sensor into individual colour and luminance components.The standard Bayer filter implementation has all sorts of exceptions built into it for when its at the edge of the image data but a little thought about moving windows allows you to simplify it greatly.