What I would say to Nokia about mobile AR (if it would listen)

#augmentedreality
I have been struck off the list of the Nokia Augmented Reality co-creation session, so here is a gist of what I was intending to say about AR-friendly mobile devices.
I will not repeat obvious here (requirements for CPU, FPU, RAM etc.) but concentrate on things which are often missed.
I. Hardware side
1. Battery life is the most important thing here. AR applications are eating battery extremely fast – full CPU load, memory access, working camera and on top of it wireless data access, GPS and e-compass.
It’s not realistic to expect dramatic improvement in the battery life in near future, though fuel cells and air-fueled batteries give some hope. If one think short term the dual battery is the most realistic solution. AR-capable devices tend to be quite heavy and not quite slim anyway, so second battery will not make dramatic difference (iPhone could be exception here).
Now how to make maximum out of it? Make batteries hot-swappable with separate slots and provide separate battery charger. If user indoor he/she can remove empty battery and put it on charge while device is running on the second.
2. Heating. Up until now no one was paying attention to the heating of mobile devices, mostly because CPU-heavy apps are very few now (may be only 3d games). AR application produce even more heat than 3d game and device could become quite hot. So heatsinks and heatpumps are on the agenda.
3. Camera. For AR the speed of the camera is more important than the resolution. Speed is the most important factor, slow camera produce blurred images which are extremely hard to process (extract features, edges etc)
Position of the camera. Most of the users are holding device horizontally while using AR. Specific of the mobile AR is that simultaneously user is getting input from the peripheral vision. To produce picture consistent with peripheral vision camera should be in the center of the device, not on the extreme edge like in N900.
Lack of skewing, off-center, radial and rolling shutter distortions of the camera is another factor. In this respect Nokia phone cameras are quite good for now, unlike iPhone.
4. Buttons. Touchscreen is not very helpful to AR, all screen real estate should be dedicated to the environment representation. While it’s quite possible to make completely gesture-driven AR interface buttons are still helpful. There should be at least one easily accessible button on the front panel. N95 with slider out to the right is the almost perfect setup – one big button on front panel and some on the slider on the opposite side. N900 with buttons only on the slider, slider sliding only down and no buttons on the front panel is the example of unhelpful buttons placement.

II. Software side
1. Fragmentation.
Platform fragmentation is the bane of mobile developers. Especially if several new models launched every quarter. One of the reasons of the phenomenal success of iPhone application platform is that there is no fragmentation whatsoever. Whit the huge zoo of models it practically impossible support all that are in the suitable hardware range. That is especially difficult with AR apps, which are closely coupled with camera technical specification, display size and ratio etc. If manufacturers want to make it easy for devs they should concentrate on one AR-friendly line of devices, with binary, or at least source code compatibility between models.
2. Easy access to DSP in API. It would effectively give developer a second CPU.
3. Access to raw data from camera. Why row data from camera are not accessible from ordinary API and only available to selected elite developer houses is a mistery to me. Right now, for example for Symbain OS camera viewfinder convert data to YUV422, from YUV422 to BMP and ordinary viewfinder API have access to BMP only. Quite overhead.
4. API to access internal camera parameters – focus distance etc. Otherwise every device have to be calibrated by developer.

Symbian foundation is considering options for improving Symbian Signed

David Wood from Symbian Foundation answered a question about Symbian Signed (mandatory digital signature for all Symbian OS app with advanced capabilities) in comments to his blog post about Symbian Release Plan :

“>What about Symbian Signed? Are there any plans to drop it, or at least relax it…?
A number of options for improving the operation of Symbian Signed are under active consideration.”

So it seems Symbian foundation is hearing to developers and end users lamentation and things could be better soon.

Rumors about Nokia Symbian OS application store.

Engadget report Nokia may open a software portal for its Symbian OS applications, with a formal announcement to come at the Mobile World Congress. Rumors about Nokia Symbian application store were floating at Nokia developer forum couple of months ago. Actually I also have written about it in this blog post – “What iPhone can teach Nokia” :)

Nokia released cleaner for the “Curse of Silence”

Nokia released tool for FP1 phones, cleaning the “Curse of Silence” exploit form the phones.

Carbide c++ is free now!

Symbian c++ development environment Carbide.c++ 2.0 form Nokia

What iPhone can teach Nokia

Developers flock to iPhone, iPhone software sales booming. The banking crisis doesn’t affect them. And Nokia-Symbian application market been lukewarm at best. On the N-Gage front 400 thousands registered n-gage users doesn’t look impressive, considering only about 30% downloads convert to sales (which is rather good conversion rate).
So what can Nokia do to make situation better ?
The main problems for Nokia-Symbian application market are
1. Platform fragmentation.
2. Exhausting (and expensive sometimes) process of Symbain signing.
3. Even more difficult n-gage certification process:
“I think it’s easier to get an audience with the Pope than it is to get a game through certification at Nokia.” – and that is Electronic Arts Mobile Peter Parmenter speaking.
4. No single point of sale. N-Gage, Nokia Software Market, third party application shops, all with different certifications, policies, requirements.
5. Everything expensive. Professional tools are expensive, private API expensive, signing expensive, developer have to have several phones for testing which is not cheap. Remote Testing is a step in right direction, but amount of free testing is limited per user.

How the situation could be improved for third party Nokia-Symbian application market?
It obviously can not be made as smooth as iPhone, complexity of Symbian ecosystem is too high, but something could be done.
1. Fragmentation: better documentation and regular updates would definitely help.
2, 3, 5: Bundle everything – certification, tools, access to remote testing, private API into two packages, one entry level, and one professional. Entry level should cost about 100$ and professional no more than 300$. Ditch per-application signing fee.
4. Merge N-Gage and Nokia Software Market into single application store. Give all certified developers access to it.

Here are my musings about the situation. The only thing which keep me form iPhone is lack of official camera API.