Mirror Image

Mostly AR and Stuff

10,000 Year Clock construction is moving forward

Via slashdot
model
model
And if you are reading this blog and havn’t read Anathem yet, you should give it try :)

16, April, 2009 Posted by | Uncategorized | , , , , , | Comments Off on 10,000 Year Clock construction is moving forward

Why 3d markerless tracking is difficult for mobile augmented reality

I often hear sentiments from users that they don’t like markers, and they are wondering, why there are so relatively few markerless AR around. First I want to say that there is no excuse for using markers in the static scene with immobile camera, or if desktop computer is used. Brute force methods for tracking like bundle adjustment and fundamental matrix are well developed and used for years and years in the computer vision and photogrammetry. However those methods in their original form could hardly produce acceptable frame rate on the mobile devices. From the other hand marker trackers on mobile devices could be made fast, stable and robust.
So why markers are easy and markerless are not ?
The problem is the structure , or “shape” of the points cloud generated by feature detector of the markerless tracker. The problem with structure is that depth coordinate of the points is not easily calculated. That is even more difficult because camera frame taken from mobile device have narrow baseline – frames taken form position close one to another, so “stereo” depth perception is quite rough. It is called structure from motion problem.
In the case of the marker tracker all feature points of the markers are on the same plane, and that allow to calculate position of the camera (up to constant scale factor) from the single frame. Essentially, if all the points produced by detector are on the same plane, like for example from the pictures lying on the table, the problem of structure from motion goes away. Planar cloud of point is essentially the same as the set of markers – for example any four points could be considered as marker and the same algorithm could apply. Structure from motion problem is why there is no easy step from “planar only” tracker to real 3d markerless tracker.
However not everything is so bad for mobile markerless tracker. If tracking environment is indoor, or cityscape there is a lot of rectangles, parallel lines and other planar structures around. Those could be used as initial approximation for one the of structure from motion algorithm, or/and as substitutes for markers.
Another approach of cause is to find some variation of structure from motion method which is fast and works for mobile. Some variation of bundle adjustment algorithm looks most promising to me.
PS PTAM tracker, which is ported to iPhone, use yet another approach – instead of using bundle adjustment for each frame, bundle adjustment is running in the separate thread asynchronously, and more simple method used for frame to frame tracking.
PPS And the last thing, from 2011:

30, March, 2009 Posted by | Coding AR | , , , , , , , , | 4 Comments

Tracking cityscape

One of the big problem in image registration/structure from motion/3d tracking is using global information of the image. Feature/blob extraction, like SIFT, SURF or FAST etc using only local information around the point. Region detector like MSER using area information, but MSER is not good at tracking textures, and not quite stable at complex scenes. Edge detection provide some non-local information, but require processing edges. That could be computationally heavy, but looks promising anyway. There are a lot of methods which use global information – all kind of texture segmentation, epitome, snakes/appearance models, but those are computationally heavy and not suitable for mobiles. The question is how to incorporate global information from the image into tracker, and make it with minimal amount of operations. One way is to optimise tracker for specific environment – for example use the property of cityscape, a lot of planar structures and straight lines. Such multiplanar tracker wouldn’t work in the forest or park, but could be a working compromise.

12, March, 2009 Posted by | Coding AR | , , , , , , , , , , , , | Comments Off on Tracking cityscape

More on the battery life – solar powered phone

I had already written in this blog, the main factor slowing mobile AR development is the battery life. Faster CPU require more energy, and eating device battery very fast.
One way to work around the problem is to use different, task specific architecture for CPU units/coprocessors, thus to get more processing power with the same power consumption.
Another – get more energy. That’s what Samsung did with “”Blue Earth” phone. The phone has full solar panel on its back.
blue erath phone

PS. Some tricks to increase battery life.

14, February, 2009 Posted by | Augmented Reality, Mobile, Nokia N95 | , , | Comments Off on More on the battery life – solar powered phone

“Probabilistic” CMOS

I was intrigued by reports of ultra-efficient chips based on the probabilistic logicPCMOS. After some googling I found this pdf, which clear the subject somehow. It seems probabilistic logic is not went into equation. Instead this architecture suggest normal, deterministic CPU with probabilistic coprocessor. Coprocessor use noise as source for random number generator (essentially analog random number generator), and can use this random number generator in different Monte-Carlo algorithms, like random neural networks, probabilistic cellular automata and likes. It seems to me the gain could be achived only for specific applications which use random number generators. In this PCMOS is not different from GPU, DSP and other task-specific accelerators.

12, February, 2009 Posted by | Uncategorized | , | 1 Comment

My take on the next gen mobile augmented reality device specs

Everyone talking about what near-future AR device should look like, so I’d like too.

First possibility – videoglasses with camera + lightweight PC.
Dedicated wearable PC is out IMO, it’s a too hardcore staff.
Next close thing is a netbook. Netbook could be used both in its main capacity and as an AR platform. However the problem here is the weight. Somehow I don’t think average user would want to carry around 1kg netbook in the backpack, while using AR. Anything more light wouldn’t have enough processing power to track mid-resolution stereo cameras in real time. Nevertheless weight could be reduced.
Make display and keyboard detachable, use carbon fiber case and easily replaceable dual (or may be triple) batteries. If all this would reduce weight below 400g while keeping CPU above 1.5Mhz with 2.5 hours of full load battery life (5 hours with dual batteries) such netbook could be viable platform for AR with videoglasses.

Second possibility – handheld/smartphone. Here we have severe limitations on the battery/CPU/GPU. That’s why I don’t think high resolution display would be beneficial for such AR device. To process high-resolution images require a lot of CPU power. And with small size of display hi-res wouldn’t look much better then low/mid res. 320×240 is good enough , 400×320 is optimal probably. For the same reason 1 Mb camera is enough too, but it should be fast, preferably 60 fps, camera with high quality sensors, without any distortions and good for low-light conditions. I’m not sure about auto-focus. Slow auto-focus could be a problem. Accelerometers and compass would be good. GPS is a must. CPU – 600 Mhz at least, with hardware floating point. Lightweight GPU. Most important thing is API. Complete access to Image Processor (if present), Digital Signal Processor and raw camera data. That kind of API is not easily accessible on the most of the modern smartphones.
Actually if existing smartphones had such an API opened right now there would be breakthrough in the mobile AR already. Access to DSP could make image processing a lot faster.

Now would any such device allow real time AR like in the Coca-Cola avatar ad? Definitely no.
The bottleneck here is the battery IMO. We have to wait for some new future tech like carbon nanotubes
supercapacitors to see complete coverage AR.

1, February, 2009 Posted by | Augmented Reality | , , , , , , , | 5 Comments

Polynesian stick charts were mapping wave patterns

Polynesian Stick Charts were completely different way of navigation, they were mapping not only locations, but also oceanic swells, patterns of waves.
stick carts
Specific map encoding was closely guarded secret, known only to group of navigators who own them.
chart
Navigating by the wave pattern navigator “would crouch in the bow of his canoe and literally feel every motion of the vessel.”They “concentrated on refraction of swells as they came in contact with undersea slopes of islands and the bending of swells around islands as they interacted with swells coming from opposite directions.”
Fascinating staff, kind of technology which could have been developed by alien, or in alternate history line.

22, January, 2009 Posted by | Uncategorized | , , , , | Comments Off on Polynesian stick charts were mapping wave patterns

Five Centuries of Board Games

Great post by bibliodyssey about board games from 16 to 20 centuries.
photo

6, January, 2009 Posted by | Games | , , | Comments Off on Five Centuries of Board Games

Quantum dots in the eye

New Scientist report a new method of improving vision was patented. The idea is to amplify the light hitting the retina for vision impaired. Nanoscale specs of semiconductor = quantum dots are injected into the eye. They fluoresce when hit by photons and thus make light hitting retina cell brighter. Tests on rats shown that rats with quantum dot injected have more retina electrical activity. No words if improvement in the sight of the rats was actually observed.

25, December, 2008 Posted by | Augmented Reality | , , , | Comments Off on Quantum dots in the eye

No camera – no fun

As everyone twittering about new Vuzix Wrap 920AV glasses it’s not clear for me from the photo
new glasses
if they have camera or not.
Old Vuzix SightMate have clearly visible camera.
old glasses
I don’t see anything like that on the new glasses. Vuzix promise “augmented reality features”, but no camera – no AR. What would be point of stylish videoglasses with ad-hock attached camera ?

PS. A new announcement from Vuzix saying that glasses will have attachable stereo camera add-on. We well see how good will it looks.

23, December, 2008 Posted by | Augmented Reality | , , , | 2 Comments