While the use of Remote Display Protocols in the computing world has been widely adopted, there are clients for pretty much every modern mobile platform on the planet for most of the dominating remote terminal servers (e.g. RPC, VNC, RDP, HTML over Web servers). These protocols have long satisfied the enterprise requirements of granting their trusted employees access to business-sensitive data, without compromising their data’s security. Such a business case has indeed been the winning argument for companies to heavily, and often exclusively, use Terminal Services on the enterprise, for a Desktop Grade User Experience.
In this article I will give you an overview of the technical challenges involved in developing a remote display protocol, and how my company, Nubo, addresses them.
However, none of these ecosystems offer a cure for the following open wound: The Desktop-Grade User Experience does not scale to Mobile devices. It is as simple as that – the world has changed. Evolved. More sensors, more touch-based devices, more mobile orientation and an ever growing number of mobile applications that take into consideration the importance of having their design revolve around the mobile user experience. Business-class apps are no different. This led me to kick start the next inevitable evolution of mobile and enterprise relationship – a mobile terminal server infrastructure.
Building a new ecosystem introduces some very interesting challenges; The first challenge, of choosing the operating system for which the apps will run on was solved immediately. Android seemed to be the best choice since it is:
- Open Source
- Has an ever-increasing available apps
- Based on Linux
The latter was the clincher, given my rich experience in linux bring-ups. When I first heard Android runs a Linux kernel, it immediately occurred to me that if a platform runs Linux, I can make it run Android. The equation is simple: Nowadays, (almost) everything can run Linux, so (almost) everything can run Android. Bringing up Android on a server was the easier part. Dealing with Android’s display infrastructure was the hard part as it is different than any other flavor of Linux that I’ve ever come across. It is based on Frame Buffers, which makes it a candidate for VNC like solutions, but they are too slow and inappropriate for today’s highly dynamic mobile applications. In addition, it shares nothing with X11, Microsoft’s Windows display protocol, or anything like it, so my team had to invent a Remote Display Protocol for a widely unfamiliar system!
One can only imagine how challenging that could be… It goes without saying that there is absolutely no way around going into the publicly available platform source code and reverse engineering the platform constructs into design principles. This is the way to go when you want to know how to handle problems and trade-offs of any remote protocol solution such as:
- Providing best picture quality with lower bandwidth consumption
- Providing the most accurate and responsive feedback for sensors, buttons and touchscreen interaction
- Gracefully handling the performance degradation in high latency networks
- Resolving bandwidth vs. client memory consumption trade-offs
- Handling multi-users
Compared to other modern Operating Systems, the latter was extremely interesting to handle, since Android ripped off Linux’s multi-user legacy. Long before mobile device multi-user support was introduced in Android’s Jelly Bean versions, we had to devise our own solution that would both allow multi-users to work simultaneously on the same platform (via our remote display protocol), and conform to Android’s security guidelines. These challenges are just the beginning. We overcame them nicely at Nubo running our novel display protocol over distributed Android Servers, iOS, Android, and HTML5 clients.