A step towards SLAM - compiling ScaViSLAM for windows

So after much fighting and nail biting I have managed to get ScaViSLAM to compile on windows (why not just use linux? I don't really have a linux system right now and generally have fewer issues with windows). I had to fix a lot of cmake files and figure out how libraries should be compiled with the correct runtime libraries (need to match for all dependencies). I have tested it with the new college data. The main code base is at:
   https://github.com/strasdat/ScaViSLAM
and my fork is at
    https://github.com/arcanon/ScaViSLAM

I have also made a fork of the vision tools at https://github.com/arcanon/VisionTools to update some things to support windows. I guess other packages also need some special changes and those are not all there yet.

Runs at 2-3 fps with a debug build (all dependent libraries as well). Very, very fast :). on the default width = 512 and height = 384.

Proof that ScaViSLAM runs on windows

So the next step is to plug the previous stuff I did with the stereo camera into this system from http://subgroup-ash.blogspot.de/2013/11/stereo-camera-with-block-matching.html. The UI of ScaViSLAM is also really nice, all GL and has perf and nice input fields. So the final system I create would change to this system probably.

To plug in my stereo camera, I have to work out the relation from the stereo matrices calculated in the stereo calibration because ScaViSLAM uses a slightly different start point. I think there the stereo camera is always calibrated and the camera "baseline" is well defined. Should work, but need to figure out the maths that gets the right values at the right places.

In general the next steps could be:
   1.) Compiling a complete ROS (robot operating system). This could be linux or windows, mostly just important to have access to the rich environment that ROS offers. such as the PTAM implementation.
   2.) Think about porting ScaViSLAM to that environment.
   3.) Think about using PTAM (http://wiki.ros.org/ethzasl_ptam/ptam)

Stereo camera with block matching

So I managed to start using my stereo camera to seeing some depth and got the first steps working to generating a point cloud of the scene.

Top row: Left and right images (rectified). and disparity image is lower left using the block matching algorithm.

The new camera uses adjustable screws, see image below:

New camera setting with adjustable screws

I used the code/app from http://blog.martinperis.com/2011/01/opencv-stereo-camera-calibration.html to calibrate the cameras and work out there matrices/ rectifying values for each feed. Once the feeds are rectified (another way to describe this would be to normalize the images, so that they can be interpreted correctly) then the block matching algorithm can be used to determine the disparity map. Next would be to calculate the actual distance of each point from the camera and then we can generate a point cloud. The algorithm only runs at about 5 fps right now on the HD frame on my NVS 4200. I checked with NVIDIA nsight profiling and the block matching kernel takes 188ms, which is the reason for the poor perf. See image below

Profile using NVIDIA nsight of stereo block matching

 As always, the code is up at https://github.com/arcanon/raspbot. I know, it won't really compile, which I hope to fix. Next I want to generate the point cloud and then I will make everything easily compile-able. well as easy as you can get, because its using a whole bunch of complicated libraries...

OpenCV GPU ORB issues

So I was noticing some issues with ORB on the GPU. The descriptors obtained at (in https://github.com/arcanon/raspbot/blob/master/video_reader.cpp):

                orbGpu(finalFrame, maskFrame, gpuKeyPoints, gpuDescriptors);
                orbGpu.downloadKeyPoints(gpuKeyPoints, keyPoints);

These were somewhat "unreliable" because there was no Gaussian blur before calculating the descriptor values. You could see this easily if you paused the video, so that the key points were always calculated on the same image. In this case, for the GPU the points jumped around while for the CPU they remain constant. This was fixed with
              orbGpu.blurForDescriptor = true;

This issue is well described at http://answers.opencv.org/question/10835/orb_gpu-not-as-good-as-orbcpu/.

The next step will be a stereo camera setup, so that I can filter out the background, which will help a lot filtering out false positives.

The other issue is that the camera has an exposure that is too long, this is a problem for the speed at which the detection happens. You can't move the robot to fast (or the objects can't move past the robot) because then they start to blur and then nothing is recognizable.