Hi, I’m really tired today, yesterday was a very demanding day with very little sleep, a flight (with some delays for the way back), and again not enough sleep.
But it was a very productive day at the fair, though I still think that it’s not a fair for us to have an stand.
A friend from Unkasoft who is promoting their very promising mobile game advertising engine was so kind to invite us, they have an stand at the Spanish Pavilion. If you’re around there it’s worth to talk to them and see what they have to offer.
During the fair we had some meetings with interesting people, some arranged previously, some not. But probably the one of the most interesting and, probably, promising was meeting Robert Virkus, CEO of Enough Software, the company behind J2MEPolish. I think that there’s a way for collaboration between both parties that might lead to good opportunities and also to useful free applications. I’ll talk to Robert after the show so we can know more about each other’s platform.
I won’t talk about the thousands of new phones, android testing platforms, and so on. I think there’s enough people talking about that now
By the way, we’re a bit late with the development of the upcoming release which will fix the reported issues, and will include a new storage engine to override the current limits of many devices (S40, BlackBerry, Motorola, and more) and also a new version of the ui engine. As soon as we have a working beta version we will release it, but in the mean time we will probably publish a 3.0.2 release to just fix the current issues.
We’ve received many questions about different matters around OpenBaseMovil, and many times we ended up writing a brief history of the project to clarify concepts.
So we’ve thought that it’s better to write here the full history of the birth, growth and future of OpenBaseMovil. We’re sure that it will help many of you to understand the possibilities of the framework, what to expect now and in the near future, how to collaborate and many other matters (beware, this is a long story ):
The starting point and the problem
About three years ago we were trying to sell a SaaS application for the healthcare sector, and we found that we missed some important issues and the market was not ready for that. It will be, probably it’s now starting to be ready. But we needed another path to keep going, or say goodbay.
It happens that one of our partners is also co-owner of a large distribution firm, they produce ice cream and also distribute frozen food and other alimentary goods. They have been struggling with mobility solutions for years, and though they implied benefits that finally paid the TCO the ROI was poor in terms of the time it took and they faced many problems:
- The terminals are very expensive (around 2000€)
- They are very large and heavy
- They need a tech guy to install the software, and also to process the collected data each day and load it with fresh data for the next day.
- The tech guy also acts as the first line support for the sellers, but has no real knowledge of the inner workings of the system and needs to talk to the manufacturer, which in turn does not have access to the current customer data in order to diagnose. So support is kind of a nightmare.
- When the terminal brokes, and that happens more and more as they age, they are expensive to fix.The need to have spare terminals to have a fast replacement, usualy up to 50% of the salesperson base.
- The salesperson needs training, and when something happens is absolutely dependent on the tech guy at the company.
- They need to be in contact telephonic touch with the company to check for stock breaches or news.
We explored the market as it was by then, and new solutions were appearing to solve some of the above issues:
- Lowend PDAs with Windows CE (now Windows Mobile)
- The solve partially point 1, since they’re not as expensive
- High end ones might have a GPRS module, but are more expensive, and need to interchange data through FTP directly to the corporate office.
- Low end ones might have Bluetooth, and be capable of transmitting through a paired cellular phone.
But they introduce new issues:
- Connecting the PDA to a phone over bluetooth can be a nightmare to a non-tech guy such as salespersons.
- The require the company (for remote connection) to have a DSL or similar connection with a fixed IP (more costs and problems) or even dedicated modems on dedicated lines to handle the communications (even worse).
- Ruggedized PDAs also with Windows Mobile
- They are as expensive as the old terminals
- They share the same issues introduced with the low end PDAs.
- BlackBerry specific on-line solutions and WAP or Java based on-line solutions for Java ME
- Being on-line does not work for people that need their data everywhere and with fast access
- Even with 3.5G, the network latency is too big to give good response time for a good user experience and usability.
- They require the company to have a DSL or similar connection with fixed IP, a server for the communcations that it is at the same time connected to the backend server or database creating a dangerous and demanding link on the later.
- You don’t have data signal everywhere, even in highly populated areas.
- Sometimes because of the building you’re into
- Sometimes because the antennas give precedence to voice signal
- Sometimes because other are using data also, and the antennas have limited channels so they delay or drop packets when they cannot handle all the connections (we’ve tested this with as little as 7 phones connecting at the same time).
- Sometimes because the antennas have just voice channels or cover very big areas (in rural and coastal zones the antennas might be voice-only and have a range of up to 70km)
Then the inevitable question came to our heads: WHY are there no off-line (in fact some-times-connected) solutions for cellular phones?
They have many advantages out of the box:
- The user already has one
- He cares for it
- They are cheaper, harder and more easy to use than any of the above.
- They have increasing abilities that are “eating” other devices day by day (mp3, cameras, gps, etc.)
- They have more than enough storage space
- The user perceives the phone as a consumer good, a simple and easy to use tool, and not a computer (though they are now like small computers)
So we tried to do it and started to search for solutions to build our Sales Force Automation solution on Java powered cellular phones.
The beginning, BaseMovil 1.0
We created a prototype using the guidelines for RecordStore usage, with an object-oriented database approach, serializing and deserializing the data and storing and retrieving it with RecordStores.
Too soon we found that the RMS system is very, very primitive, well suited for storing game hi-scores but not for real databases with many tables and thousands of rows in each table. To begin with, just enumerating the records took a big deal of time, and then you needed to iterate over them all to perform a simple search. The search times were simply unusable.
And the object-oriented approach made it very difficult to synchronize the data with the backend server, since no reflection is present in Java ME and you need an specific approach on the server side to process each new object.
So we changed the approach to a table one (so serialization could be done with a single method for any object using a description of the table, the metadata), and we added a Btree index mechanism bound to each table so we could search efficiently for the data we needed (such as customer or product names and /or codes).
At the same time we started with the manufacturer-model-firmware nightmare, since every phone places buttons were it wants, every one has it’s own bugs, every one has it’s own limitations (number of open simultaneous RecordStores, size limits, memory leaks, and so on).
Finally, we had BaseMovil 1.0 has a Sales Force Automation solution with a custom database and synchronization engine that was connected to a backend server (ERP).
We did all of this always with the SaaS approach in mind, so the sync server was hosted and controlled by us, and the ERP was connected to it using an interchange protocol.
BaseMovil 2.0, a platform was born
Given the amount of work done, we realized that 80% of the code was not really part of the SFA solution, but could be easily reused for other applications.
So we refactored the code to split it in several libraries with well-defined functions, and modified the server to support multiple databases and applications.
Then we started to build the ui engine using XML that would in turn create the LCDUI forms and list to create menus, data forms, and data lists.
We also did some big changes to the sync engine to make it more network-proof (we have found errors that were not supposed to happen, errors that the TCP/IP stack was supposed to control), and to be able to handle more incoming requests.
BaseMovil 3.0, the platform gets mature
By now we had a real-world experience of two years, and that meant a lot. We had time to fix almost every bug in the database, to be able to repair it from many situations, and to improve it with new features.
But three main issues remained a problem: network failure tolerance, server load, and user interface usability.
We now had all the knowledge needed to address all of this problems, so we:
- created a really fault-tolerant database initialization and synchronization process, that it is able to recover from network outages and other issues such as the application being closed or battery being drained. And all of it imposing a very moderate pressure on the server, so it is now able to handle an almost unlimited number of requests. It will of course increase the response time of the server (that is when we just add new hardware), but the devices are ready to tolerate it and to recover to an improbable outage.
- we created new menus and data-list browsers using direct drawing (Canvas), so the user interface got much more pretty, usable and responsive. And it also addressed some of the differences between devices. This was a particular gain for SonyEricsson phones, whose native list’s are not very well suited to showing data.
Of course, no software is defect-free, and there’re still issues around for sure. But the truth is that now we receive little support calls, and most of them are in the first weeks of use and due to network problems (not software ones). And there’s something really important about support here: the user is very comfortable and confident with the phone, and is able to repair or reinstall the application without further assistance in the case that a problem arises (phone get’s broken, for example).
So we have finally addressed all the issues, and reached a 3.5 stage in our SaaS architecture:
- Phones are much affordable than most comparable PDAs or Ruggedized PDAs.
- They are small and light.
- The user can install the application just sending an SMS, with zero configuration. He can also send and receive just updates at anytime, and knows that the data is updated and introduced into the backend system. He does not need to go the the office anymore.
- There’s no need for a tech guy, or at least there’s no need for him to be devoted to the terminals. We can provide the user support, since we know the system and we have always the status of each phone.
- When the terminal brokes, it is far more easy and less expensive to replace. The end user can simply buy a new phone, send the SMS and get back to work in 15 minutes with the same info he had when he last synchronized.
- The salesperson needs no training, he knows how to do his job, he knows how to handle a phone and the application seems a natural extension of the phone’s capabilities.
- They have a frequently updated local database, that allows out-of-signal, fast operation, but at the same time has recent information. And he can always use the on-line capabilities to check out information.
OpenBaseMovil, BaseMovil goes Open Source
Almost since we released BaseMovil 2.0 we started to think about going open source. We believe in open source, we use it, and wanted to return something to the community and society.
But we were afraid that other players in our local markets could benefit of our work (we had some buying propositions by some of them), and Alas! we like what we do, but we and our families have the bad habit of eating each day.
Finally, we decided that we could open source a big deal of the platform so it could be used for free under a dual license (GPL and commercial), and at the same time keep the part of it that is the core of our business (the sync engine).
And thus, OpenBaseMovil was born.
The near future plans
- Reach other platforms
- Real multi-platform applications
- The consumer market
Reach other platforms
Why keep the platform just to Java ME devices? We were sure that it has to be the first, since it is by far the most spreaded one. But why not support other platforms?
Of course, there’s a first target for this plan: Google’s Android. It’s Java based, so we just need to change some things in the sync engine and ui engine and applications could be written once and deployed both on Java ME and Android.
But there’s Windows Mobile also, and we could even benefit by creating a Symbian specific port of the platform (but that is a low priority one since Symbian phones have all support of Java ME). But this last two are very tied to the next point:
Real multi-platform applications
One of the latest additions to the OpenBaseMovil platform was the scripting engine. We created it to build certain parts of the applications, but we’re sure that it can be used to build full applications. Applications that would be really platform independent and dynamically downloadable.
Since the scripting engine could be easily written with C# for .NET or C++ for Symbian, we could deploy script-based applications to almost any platform.
Of course, this would need the sync engine to be also ported to .NET, for example, but that is not a challenge since it would be pretty the same that the one we are planning for Android.
The consumer market
The scripting engine is the core of the OpenMidsets project, a host application for script-based applications. Very much like Nokia Widsets or Yahoo! Go 3, but with a database, connectivity to bluetooth devices, and absolutely open (it is Open Source and you can host and connect your application to wherever you want), you are not captive, not locked.
We will develop some applications for OpenMidsets once it’s available, and we hope that you take the challenge and write your own applications.
If you have read this far I’m really glad, I hope that the story is an interesting one and that it has helped you to better understand the project and the different alternatives.
Please, submit it to you preferred social site if you liked it, and feel free to comment. We’re always open to opinions and fruitful dialog.
All the feedback that you are giving us is great, and we’ve thought that a better place for discussion is a forum, because the post comments tend to create off-topic discussions that are difficult to track and less useful to everyone.
So we have created a forum. It’s now empty but for the few categories that we have created, but we hope that you find it useful and participate, suggest new categories, etc.
For support request, bug reports and feature requests the best place is the SourceForge tracker. It helps to keep open issues controlled and centralized. But for development discussions the best place is a forum, and we are looking forward for your collaboration and input.