Computer vision accelerator in FPGA for smartphone

#augmentedreality
Tony Chun form Intel integrated platform research group talk about “methodology” of putting computer vision algorithm(or speech recognition) into hardware. He specifically mention smartphone and mobile augmented reality. Tony suggest that this accelerator should be programmable, with some software language to make it flexible. It’s not clear if he is talking about FPGA prototype, or putting FPGA into smartphone. Idea to use FPGA chip for mobile CV task is not new, for example in this LinkedIn discussion Stanislav Sinyagin suggested some specific hardware to play with.

Thanks artimes.rouli.net for pointing this one.

Zen of debugging

Shenxiu produce gatha:

The project is a source tree,
the code a standing mirror bright.
At all times polish it diligently,
and let no bugs crawl.

When a fra passed the rice mill chanting Shenxiu’s gatha, Huineng immediately knew this verse lacked true insight. He went to the wall, and asked a avout there to write a gatha of his own for him. The avout was surprised, “How extraordinary! You can not write assembly code, and you want to compose a gatha?” Whereupon Huineng said, “If you seek supreme enlightenment, do not slight anyone. Lowly java programmers may have great insights, and assembly coders may commit foolish acts.” In veneration, the avout wrote Huineng’s gatha on the wall for him, next to Shenxiu’s, which stated:

Project has no tree,
nor is the the code a standing mirror bright.
Since all is originally empty,
where does the bugs appear?

Huineng then went back to rice pounding. However, this gatha created a bigger stir; everyone was saying, “Amazing! You can’t judge a person by his looks! Maybe he will become a Living Saunt soon!” However, when the alarmed Hongren came out, he just casually said, “This hasn’t seen the essential nature either,” and proceeded to wipe the gatha off with his shoe.

Augmented Reality on Android – now with NDK

With release of native code kit Android now looks more like a functional AR platform. NDK allow for native C/C++ libraries, and complete application seems need java wrapper still. It’s not clear to me still how accessible are video and OpenGL API from NDK – have to look into it.
On related note – there are rumors about pretty powerful 1Ghz phone for Android 2.0

Air-fueled batteries

As I’d already written I think the battery life is the key to adoption of high-performance mobile devices, strong enough for advanced image processing and real-time augmented reality.
Here are some news – Technologyreview report it seems there are some advances in lithium-air batteries. Air-fueled batteries is something similar to fuel-air explosives Like FAE air-fueled batteries are not storing oxidizer in themselves, but use oxidizer from the air. AFB should allow ten times energy density of the common batteries.
Lithium AFB are developed by IBM, Hitachi and could use not only lithium but zinc and aluminium

Nokia consider Maemo Linux as alternative to Symbian ?

As cnet point out Symbian is not mentioned in the joint Intel-Nokia press release about 3G and Open Source Software collaboration. Only Maemo and Moblin are mentioned. Symbian, though also open sourced is left out. It could be that Nokia is less enthusiastic about Symbain OS now. Existing Symbain OS UIs are inferior to iPhone UI, Symbian OS third party applications are not getting enough traction and most of Symbian users are not even aware they exist. Symbian Signed restrictions are not helping either. BTW most of Symbian users are not even aware they are Symbian users.
So Nokia seems hedging its bets with Maemo linux. Cnet think Nokia could switch to Maemo for high-end devices and leave Symbian for mid-range.

Sand Animation

Just discovered for myself another new media – sand animation
Here is a great example – Russian artist performance about memories of WWII

Bing vs Google for augmented reality and computer vision

I’m using Google a lot for my work, looking for articles, unknown to me definitions and techniques and so on. So I’ve decided to check Microsoft Bing too.
First test – augmented reality
Google – definition in the first line, links give pretty comprehensive coverage for beginner
Bing – four obscure links with job and phd references

Second test – MSER definition
Google – give definition in the first line
Bing – unrelated garbage

Third test – preserving symmetry in cholesky decomposition
Google result
Bing result
Similar results. Both engines relay on the wikipedia heavily

Forth test: “multiscale segmentation”
Google result
Bing result
Surprisingly I like Bing results better.

Conclusion:
Google engine seems have more “common sense” and more useful for introduction into subject. Could be because of bigger indexed base.
Bing could be actually useful in specific searches.

Open Source programmable camera for image processing

Interesting product – camera for computer vision applications, with open sourced DSP

From sci.image.processing:
“The entire camera (hardware as well as software) is open source. It features a 752×480 pixel CMOS sensor, 64MB of SDRAM and 4MB of flash, Ethernet and div. IOs.
The camera runs a uClinux and comes with an image processing framework.”
Datasheet is here

Augmented reality, enforced locality, geometric hashing

I had discussion with Lester Madden at linkedin MAR group. The thing we discussed was the concept of the locality in the AR. That is, each AR object should be attached to specific location and accessible only from that location.
I’ll try explain it more in depth here.
Augmented graffiti, augmented reality mail/drop boxes and billboards, user-built reality overlays – all of those should be attached to specific location. This locality could be enforced – only local data would be available (filtered into) in the specific location. This locality of data prevent user from sinking in the augmented noise, generated all other the world, and reduce possibility of spam.
For example you can have neighborhood billboard, leave note for the friends in the park and so on. All those AR objects data could be accessed only locally for both read and write – to read billboard and to post a message on it you would have to go to it.
The user should get the data/content only if he is physically present at the specific location. The same way poster/producer of the data or AR object should physically visit each location where it placed.
If locality is enforced, to place note for your friend in the park you have to visit park, and there is no way around it.
Locality could be enforced with location-based encryption. I think this encryption could be made with use of geometric hashing. User scan environment and make 3d registration with his mobile or wearable device. Encryption key is generated by mobile device from the scanned 3d model of the environment.
If user want to get data attached to the location, he access the server, retrieve local data and decrypt them with that key.
In the opposite direction, if user want to attach some object or data to location, mobile device encrypt data with part of the hash key and send other part of the key to server. Before storing data the server do uniqueness check. Nearby data already stored on the server are checked, and the new data allowed in only if there is some distance from new key to keys of all the other stored data. After that new data encrypted with the second part of the key by server and stored.
Each object encrypted by two keys, one of which is server side. Server have no access to content of the data, but have access to the part of the location hash key. That way no two objects or data attached to exactly the same location. Clattering of AR objects could be reduced. More importantly if poster have to physically visit location where he want to place AR object, he should have at least some relation to that location, and he is not some spammer from the other end of the world.
If spammer forge location key without actually visiting the place, that will most probably be non-existing location, and no one will be hit by his data.
That all is of cause is a rough outline of how could enforced locality works. Building robust algorithm for extracting geometric hash could be non-trivial.

10,000 Year Clock construction is moving forward

Via slashdot


And if you are reading this blog and havn’t read Anathem yet, you should give it try