OpenCV GPU ORB issues

So I was noticing some issues with ORB on the GPU. The descriptors obtained at (in https://github.com/arcanon/raspbot/blob/master/video_reader.cpp):

                orbGpu(finalFrame, maskFrame, gpuKeyPoints, gpuDescriptors);
                orbGpu.downloadKeyPoints(gpuKeyPoints, keyPoints);

These were somewhat "unreliable" because there was no Gaussian blur before calculating the descriptor values. You could see this easily if you paused the video, so that the key points were always calculated on the same image. In this case, for the GPU the points jumped around while for the CPU they remain constant. This was fixed with
              orbGpu.blurForDescriptor = true;

This issue is well described at http://answers.opencv.org/question/10835/orb_gpu-not-as-good-as-orbcpu/.

The next step will be a stereo camera setup, so that I can filter out the background, which will help a lot filtering out false positives.

The other issue is that the camera has an exposure that is too long, this is a problem for the speed at which the detection happens. You can't move the robot to fast (or the objects can't move past the robot) because then they start to blur and then nothing is recognizable.

Raspbot helps find my cofee

Ever lost your coffee and needed some help finding it? I have and it can be a stressful situation, as you don't have the coffee to help calm your nerves. That is where the raspbot jumps in to save the day. Just make sure you have a distinct enough mug, so that it does not get lost in the background. This makes use of OpenCV and object detection. It streams the video from a raspberry pi to a server, which is running OpenCV and has a GPU. Then GPU is then used to decode the H264 stream. Then either the GPU or the CPU is used to analyze a frame for interesting information (in this case, where is the mug). OpenCV has GPU and CPU support, so you can choose.

System overview

Here is the code for the server https://github.com/arcanon/raspbot/blob/master/video_reader.cpp. The other core code is also in that repository. You obviously need to OpenCV. I was running on windows, you have to fight a bit to get it compiled on windows, I will give some tips at some point on that, maybe its also easier with newer versions.

Capture Thread:
1.) Winsock buffer connects to netcat client on PI
2.) Buffers data and passes to H264StreamSource
3.) Data then Passed to GPU for decoding

Detection Thread: 
1.) Frame from capture thread recieved
2.) Converts to grey scale
3.) uses ORB/SURF for feature extraction. On CPU or GPU.
4.) Compares against detection object.
5.) Sets up commands to send to PI

Command Thread: 
1.) Connects to Python client on PI
2.) When Detection thread has commands, these are sent to the PI.

Server Threads

Raspberry PI threads/processes