Marker vs markerless (bundle adjustment)
#augmentedreality
Here is a sample of image registration with fiduciary marker (actually the marker I used in my games) vs registration with bundle adjustment. Blue lines are points heights (relatively to marker plane) calculated using marker registration and triangulation. White lines are the same using bundle adjustment(modified). Points extracted with multiscale FAST and fitted with log-polar Fourier descriptors for correspondence (actually SURF descriptor produce the same correspondence).
As you can see markerless is in no way worse then markers, at least on this example ))).
Tracking planes in the city
In relation to tracking cityscape I did some planar segmentation test. Segmented FAST generated corners with simple 5-points projective invariant.
In some cases 5-point give some rough approximation:
In some cases outliers are quite bad – some point have very close projective invariant but still are in diffferent planes.
So simple method not quite work…
Tracking cityscape
One of the big problem in image registration/structure from motion/3d tracking is using global information of the image. Feature/blob extraction, like SIFT, SURF or FAST etc using only local information around the point. Region detector like MSER using area information, but MSER is not good at tracking textures, and not quite stable at complex scenes. Edge detection provide some non-local information, but require processing edges. That could be computationally heavy, but looks promising anyway. There are a lot of methods which use global information – all kind of texture segmentation, epitome, snakes/appearance models, but those are computationally heavy and not suitable for mobiles. The question is how to incorporate global information from the image into tracker, and make it with minimal amount of operations. One way is to optimise tracker for specific environment – for example use the property of cityscape, a lot of planar structures and straight lines. Such multiplanar tracker wouldn’t work in the forest or park, but could be a working compromise.
Markerless tracking with FAST
Testing outdoor markerless tracking with FAST/SURF feature detector.
The plane of the camera is not parallel to the earth, that make difficult for eye to estimate precision.
FAST with SURF descriptor
Feature detected with multistage FAST and fitted with SURF descriptors
Less strict threshold give a lot more correspondences, but also some false positives
Multiscale FAST detector
Experimenting with multiscale FAST detector with images from cell phone camera.
so far so good…
Testin FAST feature detector
Testing FAST feature detector on the Mikolajczyk ’s dataset. Here scale space seems actually useful. With “brick wall” dataset repeatability goes form .3 to .7 with scale from 0 t 2^^3, and threshold/barrier lowering from 40 to 20.
From SURF to FAST
I did some research on the SURF optimization. While it still possible to make it significantly faster with lazy evaluation, the problem of the scale remain. Fine-scale features are not detectedable on the bigger scale, so it doesn’t look like there is an easy way to reduce search area using only upper scale. If scale-space is not helping to reduce search area it become liability for mobile tracking – range can’t change too fast for a mobile pone, so scale of the feature will be about the same between frames.
Will try plain, not scale-space corner detectors now, starting with FAST.
