Mirror Image

Mostly AR and Stuff

Samsung SARI 1.5 Augmented Reality SDK is out in the wild

Something I did for Samsung (kernel of tracker). Biggest improvement in SARI 1.5 is the sensors fusion, which allow for a lot more robust tracking.
Here is example of run-time localization and mapping with SARI 1.5:

This is the AR EdiBear game (free in Samsung apps store)


14, November, 2011 Posted by | Augmented Reality, Coding AR, computer vision, Demo, Games, mobile games | , , , , , , , , , , | 3 Comments

TI demoed tablet with stereocamera

In relation to this post TI demoed OMAP3 tablet with dual camera capable of recording 3d images.
TI promise dual core OMAP4 will be even better at this.

18, February, 2010 Posted by | Augmented Reality, Mobile | , , , , , | Comments Off on TI demoed tablet with stereocamera

Augmented reality: from Tangible Space to Intelligent Space

Threr is such thing as Milgram’s Reality-Virtuality Continuum
Milgrams continuum
Milgram’s continuum shows progress of interface from raw environment to completely synthetic environment.
It looks like it’s possible to add another dimension to this picture. There exists a concept of “Tangible Space” in AR. “Tangible Space” basically mean that user can interact with real-world objects and those actions affect virtual environment. For example AR game which use real world objects as part of gameplay, track positions and any changes of state of those objects. Essentially “Tangible Space” is virtual wrapping around real-world interaction.
However that line of thought could be stretched beyond augmented reality. In the “Tangible Space” real-world interaction affect virtual environment. What if virtual interaction affect real-world environment? In that case we would have “Intelligent Space”, or iSpace.
Based on DIND – Distributed Intelligent Networked Device. iSpace is an augmented(or virtual) reality environment “augmented” with mobile robots and/or other actuators. Intelligent network now not only track physical environment, but also actively interact with it using physical agents. If Augmented Reality is an extension of eye, Intelligent Space is an extension of both eye and hands. Not only real environment is a part of interface now(as in “Tangible Space”) , it now actively help human to perform some task, and also should guess how to do it. Human and robots are now integrated system, something like distributed exoskeleton.
Now we have a new dimension for Milgram’s Continuum:
Passive View of Real Environment->Augmented Reality->Tangible Space->Intelligent Space
If you remember Vernor Vinge’s “Rainbow Ends”, the enviroment in it not just an Augmented Reality – it’s an Intelligent Space

8, February, 2010 Posted by | Augmented Reality | , , , , | Comments Off on Augmented reality: from Tangible Space to Intelligent Space

I want smartphone with stereocamera

Smartphone with stereocamera is not exactly a new concept
Motorola stereo camera phone
But 3d registration, rangefinding, augmented reality would be a lot more robust and efficient with stereocamera.
Of cause it should be implemented properly, distance between lenses should be as big as possible. Preferably with lenses near opposite ends of the phone, to increase baseline, which would increase 3d precision.
Special geek model could have second camera on the retractable extender for even more precision.
Stereocamera would make AR markerless tracking trivial. 3d structure of the scene could be triangulated in one step form the single stereoframe.

12, January, 2010 Posted by | Augmented Reality | , , , , , | 2 Comments

Vizux intoduce some serious Augmented Reality eyewear.

Via Marketwire.Here it is, Wrap 920AR:
Wrap 930AR
* 1/3-inch wide VGA Digital Image Sensor
* Resolution: 752H x 480W per lens
* Frame rate: 60 fps
* High-speed USB 2.0
* some kind of 6DoF tracker (probably 3-axis accelerometer and/or e-compass, I don’t have hopes for gyroscope)
* Supported by Vuzix Software Developer Program
$799.99, expected availability is 2nd quarter of 2010.
The Wrap 920AR’s stereo camera assembly and 6-DoF Tracker will also be available separately for upgrading existing Wrap video eyewear. Here is Wrap 920AR at vizux homepage

7, January, 2010 Posted by | Augmented Reality | , , , | 1 Comment

Availability of technology

From xkcd

“A technology that is ’20 years away’ will be 20 years away indefinitely.”

21, December, 2009 Posted by | Augmented Reality | , | Comments Off on Availability of technology

Google Googles, QR codes, localization

There are two recent developments related to Augmented Reality and Google – Google Goggles and Google integrating QR codes into Google Maps. While I was talking on twitter with @noazark the question arise about Google Google not doing real-time localization of the user, thus not being a “real” AR.
Here come QR codes. QR code are extremely easy to recognize in the camera image, and their square shape allow for fast calculation of camera position relatively to QR. In fact each QR code include three fiduciary markers:

And well known marker-tracking technique easily applied to them. Marker tracking could be augmented (pan intended:) by planar tracking of the corners of the pattern itself. That allow for attaching virtual 3d objects/animations to QR codes, but there is more in it. As QR code contain more than 4k of data, exact GPS coordinate, pattern orientation and its’ size could be encoded in the pattern. That way mobile phone seeing the code can easily calculate it’s exact 3d coordinate and orientation, not only relatively to QR, but absolute.
More of it – QR code can have coordinates of nearby QR codes, creating kind of localization grid, which can point user to any location covered by that grid with arrow on the screen of the phone.
Now to markerless tracking – QR code can be used to jump-start markerless tracker and assist it with error-correction(drift compensation), especially mentioned grid of the codes. That is especially relevant to markereless trackers which use planar structures and straight edges.
Now there is one problem here – white QR code is easy to segment out of dark background. But on white background it not so easy to recognize, and embedded fiduciary markers will not be seen form afar. Here is suggestion – make thick black frame around the QR, and make it part of the extended standard. This square shape would be easy to recognize, even if it’s only couple of dozen of pixel in diameter. With incremental tracking phone will be able to track it(after initial close up) even if moved quite far from the QR. If this square frame is part of the standard, always having the same relative size, it could be used for distance estimation.
Now combine it with Google Goggles real time and you have functional AR with 3d registration.

9, December, 2009 Posted by | Augmented Reality | , , , , , | 2 Comments

What I would say to Nokia about mobile AR (if it would listen)

I have been struck off the list of the Nokia Augmented Reality co-creation session, so here is a gist of what I was intending to say about AR-friendly mobile devices.
I will not repeat obvious here (requirements for CPU, FPU, RAM etc.) but concentrate on things which are often missed.
I. Hardware side
1. Battery life is the most important thing here. AR applications are eating battery extremely fast – full CPU load, memory access, working camera and on top of it wireless data access, GPS and e-compass.
It’s not realistic to expect dramatic improvement in the battery life in near future, though fuel cells and air-fueled batteries give some hope. If one think short term the dual battery is the most realistic solution. AR-capable devices tend to be quite heavy and not quite slim anyway, so second battery will not make dramatic difference (iPhone could be exception here).
Now how to make maximum out of it? Make batteries hot-swappable with separate slots and provide separate battery charger. If user indoor he/she can remove empty battery and put it on charge while device is running on the second.
2. Heating. Up until now no one was paying attention to the heating of mobile devices, mostly because CPU-heavy apps are very few now (may be only 3d games). AR application produce even more heat than 3d game and device could become quite hot. So heatsinks and heatpumps are on the agenda.
3. Camera. For AR the speed of the camera is more important than the resolution. Speed is the most important factor, slow camera produce blurred images which are extremely hard to process (extract features, edges etc)
Position of the camera. Most of the users are holding device horizontally while using AR. Specific of the mobile AR is that simultaneously user is getting input from the peripheral vision. To produce picture consistent with peripheral vision camera should be in the center of the device, not on the extreme edge like in N900.
Lack of skewing, off-center, radial and rolling shutter distortions of the camera is another factor. In this respect Nokia phone cameras are quite good for now, unlike iPhone.
4. Buttons. Touchscreen is not very helpful to AR, all screen real estate should be dedicated to the environment representation. While it’s quite possible to make completely gesture-driven AR interface buttons are still helpful. There should be at least one easily accessible button on the front panel. N95 with slider out to the right is the almost perfect setup – one big button on front panel and some on the slider on the opposite side. N900 with buttons only on the slider, slider sliding only down and no buttons on the front panel is the example of unhelpful buttons placement.

II. Software side
1. Fragmentation.
Platform fragmentation is the bane of mobile developers. Especially if several new models launched every quarter. One of the reasons of the phenomenal success of iPhone application platform is that there is no fragmentation whatsoever. Whit the huge zoo of models it practically impossible support all that are in the suitable hardware range. That is especially difficult with AR apps, which are closely coupled with camera technical specification, display size and ratio etc. If manufacturers want to make it easy for devs they should concentrate on one AR-friendly line of devices, with binary, or at least source code compatibility between models.
2. Easy access to DSP in API. It would effectively give developer a second CPU.
3. Access to raw data from camera. Why row data from camera are not accessible from ordinary API and only available to selected elite developer houses is a mistery to me. Right now, for example for Symbain OS camera viewfinder convert data to YUV422, from YUV422 to BMP and ordinary viewfinder API have access to BMP only. Quite overhead.
4. API to access internal camera parameters – focus distance etc. Otherwise every device have to be calibrated by developer.

10, November, 2009 Posted by | Augmented Reality, Mobile | , , , , , , | 9 Comments

(augmented) reality imitate art

predicted it:

16, July, 2009 Posted by | Augmented Reality | , , | 6 Comments

Computer vision accelerator in FPGA for smartphone

Tony Chun form Intel integrated platform research group talk about “methodology” of putting computer vision algorithm(or speech recognition) into hardware. He specifically mention smartphone and mobile augmented reality. Tony suggest that this accelerator should be programmable, with some software language to make it flexible. It’s not clear if he is talking about FPGA prototype, or putting FPGA into smartphone. Idea to use FPGA chip for mobile CV task is not new, for example in this LinkedIn discussion Stanislav Sinyagin suggested some specific hardware to play with.

Thanks artimes.rouli.net for pointing this one.

7, July, 2009 Posted by | Augmented Reality | , , , | 2 Comments