Hi,
I have experience in video processing and image manipulation (classification, object detection etc). I am happy to tackle your problem.
Firstly are you running the software on a system with a GPU? If so we can delegate that decoding/transcoding to the GPU to give some breathing space to the CPU.
Also, what Protocol are you using (RTSP, MJEG etc)? Do you have control over the stream resolutions?
There are many ways I can try to improve the software, but I will need more details.
Ping me to get started.