Lurking in that low-cost Android tablet are the hardware and most of the software needed for building smart touchscreens into kiosks and other devices.
In light of the many cheap and capable Android tablets that have become so widely available, it is interesting to consider whether these could be adapted for use as smart touchscreens in many embedded systems currently serviced by more expensive — and less capable — platforms.
The reasons to consider Android tablets are many, including:
Of course, along with these advantages come a number of drawbacks, some obvious and some perhaps a bit more subtle:
For many applications, these drawbacks can be addressed satisfactorily, leaving you with a design option that is compelling.
Consider, for example, the use of an Android tablet as a smart touchscreen in a typical kiosk-style system.
Embedding an Android tablet in a kiosk
The usual solution would be a conventional color touchscreen with a 4- or 5-wire interface. This might be connected to the product’s control board and controlled directly by it, or perhaps controlled by a microcontroller dedicated to UI and having a UART or SPI interface to the control board and acting as a slave to it. While certainly a feasible, indeed conservative solution, there are reasons to regard this design direction with suspicion. By the time the display, touchscreen, interface board, and cabling are considered, the cost of this solution can easily substantially exceed that of most tablets.
Additionally, the software tools to manage GUI development on bare display/touchscreen hardware are not great. Some microcontroller manufacturers provide proprietary libraries to support GUI development. Microchip’s mTouch framework is typical. These libraries are quite specialized and it is unlikely anyone in your organization (or easily available to be hired) already has the expertise to use them. A few third-party libraries, such as Qt, are available but bringing them up on your hardware might constitute an additional integration step to be performed; and while somewhat more commonly found than the proprietary libraries, expertise in their use is not widespread.
In contrast, Android software development expertise is common; and once gained can be applied in many other contexts. If you don’t already know how to develop Android software, you can either hire someone relatively inexpensively or develop the ability yourself, confident that expertise will be marketable and applicable elsewhere in the longer term. Unlike with proprietary GUI APIs, you can be confident support for that environment will be provided as long as Android tablets and phones are popular. It is also certain the capabilities of Android will be upgraded to keep up with advances in mobile technology.
Some specific challenges must be met to achieve the goal of a superior user interface to your product. Below, we discuss technical issues relating to power, communications, security, lifecycle management, user interface, and more.
Making sure it turns on and keeps running
First, and perhaps most important, is the seemingly simple question of how to turn the unit on and off. The tablets have an on/off button that puts the unit in or out of sleep mode. Holding the button for several seconds turns the unit completely off. Such behavior is normal and expected for a tablet or phone, but constitutes a real puzzler for many embedded systems. The tablet must be mounted securely in a position usable as a user interface, but ideally one would like power to the entire system to be controlled by its power switch. The tablet on/off switch is usually not easily relocated or modified in any way due to the unit’s highly integrated design.
In fact, this seemingly trivial concern is fatal to the use of tablets in many embedded systems. One system I designed would spend much of its life in a closet, with no assurance it would be plugged in during that time. Once power was applied it had to come to life reliably. A tablet in sleep mode uses little power; some will drain the battery within a few days while better designed units might survive for several weeks in sleep mode. The tablet concept would otherwise have been a nice solution for the device I was designing, but the possibility of the unit not coming on when power is applied was not a tolerable risk, so we chose a more conventional SBC/touchscreen design for that product. In contrast, for a product that could be expected to be powered continuously aside from brief outages the Android tablet design is often a good one.
Dealing with power management
If you can satisfy yourself the power issue is not a fatal concern, the next step is to devise your power management plan. Fortunately, Android provides excellent capabilities, as one would expect from an operating system targeted to mobile devices.
In many kiosk applications you want the screen to be fully lit at all times when the unit is running and available, but to go into sleep mode if the device is unavailable. This is fairly easily and elegantly accomplished by combining some Android configuration options with the use of standard Android API facilities in your application.
Based on this, you can either add or clear the WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON flag to/from the current window to reflect whether you want the screen kept on. In this way, you ensure the screen will stay on as long as power is applied to the tablet. This technique must be applied for each Activity your application displays; I handle this by implementing the above logic in a class called BrightActivity, which inherits from Activity and from which all my own Activities inherit.
This handles the situation nicely, as when power is lost, the unit will go to sleep mode after the fifteen second timeout defined in the Android setup app, then wake up when power is restored. The method should work well in most situations in which the unit is normally powered, though as mentioned you have to convince yourself the situation would not be disastrous if power is off long enough to drain the unit in sleep mode, as doing so may require recharging and pushing the on/off button to restart.
With the EXTRA_PLUGGED mechanism in place, the application keeps running when power is disconnected, though in a reduced mode. Depending on your needs and preferences, you might also want to make use of the ACTION_POWER_CONNECTED and ACTION_POWER_DISCONNECTED intents. These are specified in your AndroidManifest.xml file and matched to a BroadcastReceiver for each. You might, for example, want to exit the app when power is removed and/or start a specific Activity when power is restored. You probably do want to make use of the BOOT_COMPLETED intent, which can be used to specify an Activity to be started on bootup. Normally you would specify the initial Activity you want loaded using this mechanism. This ensures the app will be running in case of a reboot for any reason — in embedded systems terms, you might think of it as a sort of watchdog mechanism.
Communicating with the tablet
Another problematic issue for the use of a tablet in an embedded system design regards how to power the unit while also communicating with it. Up until just a couple years ago, this was less an issue because many units had separate connectors for power and USB. You could simply power the unit via the dedicated power jack (typically 5 VDC) while controlling any peripherals or communicating with a separate control board via the USB port. This solution is no longer viable as virtually all the newer tablets and phones have only one option for charging, namely the single USB port. Potential inductive charging methods aside, for now we are stuck with USB charging.
This gets very sticky if you are also relying on the same port for communication. The USB protocol defines limits on the current that can be provided on a port that is used for communication and the conditions under which these limits can be exceeded. These limits are tricky on their own, and implemented in various ways on different devices. In my experience, these limits effectively prevent the use of a USB port for simultaneous power and communication if you need the screen lit constantly.
A number of interesting ad hoc solutions are available (see, for example, the active and successful efforts to hack the Nexus 7 tablet for automotive use), but these are fragile, complicated, and non-portable. However, all such solutions are specific to a particular tablet model, complicating efforts if you need to change models. They also inevitably involve the use of patches applied to third-party ROMs. They may work well for your use case, or they may have subtle bugs when used in an environment other than the one the developer anticipated. My advice is to avoid such solutions, in most cases.
This leaves the question of how to communicate with a device that has no wired port available for the purpose. With the USB port occupied solely with charging, a wireless approach must be used. Probably the best solution lies with Bluetooth. It is built into even the cheapest units, and support for it is as complete and mature in Android as one would expect in a mobile operating system. These factors make Bluetooth an obvious choice if only the tablet side is considered.
But what about the hardware to which it is talking? For new designs this can be managed by adding one of the many Bluetooth interfaces that are being marketed for embedded applications, though certainly the added cost must be taken into account when considering whether an Android tablet design is economic.
From an integration perspective Bluetooth is especially attractive as its RFCOMM protocol neatly emulates a serial port. A number of modules, for example the Microchip RN42, are available for design into your system; and RFCOMM is well supported in the Android API.
Another alternative is the IOIO, a small and inexpensive board that adds various types of I/O to an Android device. You can add a Bluetooth dongle to it, configure a couple of pins on the board as a UART, install the IOIO API software on the tablet, and talk to the device on the other end of the UART in much the same way as before you interposed the IOIO. The IOIO is especially attractive if you have other I/O that can be handled by its built-in functionality.
Bluetooth’s origins are in simple non-critical consumer products intended to replace wired devices, such as headphones and hands-free car kits. Consequently security was considered, but at the beginning was given low priority in trade-offs against convenience. Much progress has been made in Bluetooth security since the days of “Enter ‘0000’ for the PIN,” but a cautious, multilevel approach is warranted as in most areas of computer security. If you correctly implement the security features that are available, then design your overall system in a way the minimizes exposure, you may find any security risk to be manageable.
Bluetooth traffic is encrypted using a cipher that is considered mostly (though not entirely) safe, but as with most secure communication systems, successful efforts to break it have centered on examining the connection establishment protocol. One should actually say “protocols,” as the history of Bluetooth is littered with a series of mechanisms to establish connections, starting with the first generation pairing process with the often easily guessable (and always brute-forceable) PINs used to generate a session key, to the latest generation with its somewhat (though not entirely) credible key exchange protocol.
An ongoing problem with Bluetooth security is its insistence on continuing to support earlier, and clearly insufficient, security models as fallback mechanisms as the new versions are implemented. Devices with Bluetooth 2.1 Secure Simple Pairing (SSP) provide considerable protection against man-in-the-middle (MITM) and other attacks when properly used. The temptation is to allow the device to revert to the traditional pairing mechanism, which is insecure. This temptation should be resisted; the best policy is to use the latest Bluetooth hardware (this is an issue only for the control board end, as newer tablets generally have recent Bluetooth implementations) and perform pairing only with full SSP.
In the context of an embedded system, realistic threats consist largely of various forms of denial of service. Clearly, with any RF wireless link the possibility exists that the channel could be jammed. With SSP, there have been reports [PDF] of MITM attacks in which all frequencies are jammed, inducing both sides to conclude they have been disconnected and prompting a re-pairing sequence, at which point the attacker can insert itself in the middle. Re-pairing requires human interaction, so this attack contains a social engineering component. In an embedded context, with no operator present to complete the pairing, the MITM part of that attack is perhaps unlikely to be successful, but clearly it would leave the two sides disconnected. It appears that most other attacks, such as some possibilities for eavesdropping, also depend on a social engineering component, which is less likely to be successful (at least on systems with SSP) on an embedded system than on, say, a mass marketed cell phone.
Clearly one should not be complacent about the security of a protocol with a spotty history in that area. As with all designs, you should gain an understanding of the security model used in your implementation of Bluetooth, and analyze any potential weak points. One might also consider the nature of the traffic between the tablet and the I/O to which it is attached. In most realistic embedded systems it is unlikely to contain especially sensitive data, so you may be able to decide that even a successful attempt at eavesdropping is not a big concern. As with all risk analysis, a mature and realistic view of the potential threats to your system should drive the process. After all, the theoretical possibility that someone within Bluetooth distance of your system might be able to crash it with some elaborate procedure is probably not going to drive your design decision if the unit’s power plug is within reach of passers by.
Overall security of an Android based embedded system can be viewed as a trade-off between the considerable resources devoted to Android security in general, versus the large number of attackers targeting Android. Similar to the early days of Linux, in which security was often regarded with some complacency because most attacks were targeted toward Windows, many IoT developers have relied on the obscurity of their platforms in hoping not to be targeted.
Apart from the foolishness of the approach, that attitude is simply not available to anyone using a mass marketed Android device in their design. You know you will be targeted along with all other Android users — there is no alternative to monitoring security alerts and applying updates as needed. Naturally, not every update is applicable to a product that uses only a specific subset of Android capabilities, and the cost of applying updates is often high for embedded systems (unless you devise some automatic update scheme, which comes with its own set of vulnerabilities), but having a process in place to keep up with Android security status and updates is essential for a successful product.
The general approach of using a commodity Android tablet, charged over the USB port and communicating with the rest of the system via Bluetooth, provides a workable product design model for many embedded systems.
If you believe the tablet market will continue to make available tablets with the necessary features (i.e., of roughly the same dimensions, charged via USB, and provided with Bluetooth), and that future releases of Android will not break preexisting code, there is no reason not to design in a current model and expect to be able to replace it with new models as necessary. This provides a possible solution to the need for component purchasing continuity that embedded system designers traditionally have had.
Yes, the current tablet model will become obsolete long before your product is, but if you can expect to be able to choose from newer (and presumably better) models with compatible features, a minor redesign perhaps once per year may be a reasonable trade-off for the advantages of the Android tablet design.
Of course, you also should be aware of some concerns:
Taming Android’s look and feel
As a mobile operating system, Android supports and encourages a specific look and feel. This is understandable, as they want their users to have the same experience as they move from app to app on their devices. They also compete with other mobile operating systems, so they want their system to be unique and get good reviews. Finally, they want to provide certain status and navigation features that are common and always available on the device.
All of which makes perfect sense for a mobile OS, but has nothing to do with the way you want to design an embedded system user interface. In contrast to Google’s design goals for Android, your own goals are to achieve a distinctive look and feel for your application, either of your own new design or to provide continuity with a previous product generation. You also want the device locked into your environment — you certainly do not want to use Android’s standard navigation and status features.
Various solutions are available for bypassing Android’s navigation features and status displays. Recent versions of Android itself have an “immersive mode” that hides the status bar at the top of the screen and the navigation bar at the bottom. There have been a few attempts to hack the Android source code to fully remove them, and of course you can attempt to do so yourself. In my opinion, these attempts have been less than successful, at least from the point of view of someone wishing to design a maintainable and supportable system. Similarly, the immersive mode is a bit leaky in that you have to be careful about the conditions under which Android brings back the status and navigation bars; you might be surprised at times to find them there despite having requested to be in immersive mode.
The best solution is probably the most trivial: simply extend your device’s bezel down over the top status bar and up over the bottom navigation bar. Thus covered up, they will not be visible or clickable; but you will still have access to them during service (i.e., by removing the tablet from the device during configuration and service). It is also by far the easiest, cleanest, and most maintainable solution.
As for your application’s look and feel, there’s really no magic here. You probably will not want to use Android’s standard look for the simple reason that you don’t want your device to look as if it is controlled by an Android app. Similarly, you cannot use the standard means of navigation because it depends on the Android navigation bar. Like all GUI toolkits, Android’s allows you to redefine the look and feel of its widgets — you can define the graphics associated with buttons and other controls, redefine how buttons look when pressed, specify a background image for a screen, and so on. Like virtually all recent GUI toolkits, much of the user interface is defined in XML files, with a means to access that information from the main Java source, and with a half-hearted attempt at inheritance. This scheme allows you to define some qualities of your look and feel in general and derive specific controls from it, all with certain rather puzzling limitations. The trend toward XML-based user interface definition makes this old timer cranky, but Android’s scheme is neither better nor worse than others, and it is generally well documented and undeniably works.
Perhaps the worst case for UI design is if you want a unique (or at least uncommon) look and feel, but have not the desire or the resources to define the graphics from bottom to top. If you are accustomed to using third-party GUI toolkits, you might expect to choose from a number of predefined graphics options with good support. For the reasons described earlier, in Android this is difficult. Google wants you to use their look and feel, both for consistency across apps and to allow them to change it seamlessly from one Android release to the next. So you will not find many predefined look-and-feel options, as one would with third-party toolkits. Android is best for organizations with the resources to define their product look completely, or perhaps a few who can use the predefined look as is.
Perhaps the most attractive result of using a mass-produced Android tablet in an embedded system design is the features and qualities that essentially come for free. You purchase a low-cost computing platform with an attractive touchscreen and you get:
Who knows? One could imagine a situation in which an organization with a kiosk type device later discovers they have a vandalism problem. It would be easy enough to turn on the camera and record everything in a loop, or even use the built-in accelerometer to tell when someone is attempting to damage the unit, then turn on the camera and/or notify someone over the network interface. It is far nicer to have it there already than to try to add such capability to a preexisting product. And if you never need it? You didn’t pay anything for it, since you based your purchasing decision on only the processor and the touchscreen.
Repurposing an off-the-shelf Android tablet is clearly not a suitable solution for every embedded system designer’s user interface. But with a clear understanding of the trade-offs, you might find one to be a welcome addition to your toolbox for your next design.
Photo credit: The thumbnail image at the top of this post is licensed under the Creative Commons Attribution-Share Alike 3.0 Unported license. It depicts an Internet kiosk in Hemer, Germany that is not known to run Linux or Android software.
I’ve long thought that airlines in particular would be a good candidate for using tablet technologies in their seat-back units. Powered via USB (or directly to the battery terminals, since equipping a plane with hundreds of Lithium cells is a bad idea!), using USB-ethernet for data, the experience would be significantly better than the terrible touch screens that they currently use, as well as being much lighter, probably less demanding in terms of power consumption, etc, etc.
Sophisticated in-seat (gaming, etc) controllers could be simple USB HID devices, as could the regular brightness, volume up/down, on/off controls.
Given their volumes, they could almost certainly negotiate long-term availability guarantees, as well as selecting the hardware they want, such as eliminating the wifi and bluetooth controllers, adding an actual Ethernet port, supporting Power over Ethernet, etc, etc that would optimise the installation further.
Yeah. And likely much thinner than the units they have now, which we all could appreciate.
This is a well thought-out article, thanks. I had not considered buying a separate USB-powered board to drive other peripherals through the single USB port the tablet provides. At $30 though, the IOIO brings the system to a similar price point to an embedded Linux board + touchscreen.
I would mention these strengths for the Linux platform: 1) similar price point once you add in the separate peripheral board for an Android tablet. Bluetooth may not always be a good option due to possible lag, interference from other Bluetooth devices, and the greater risk of hacking. 2) far more software customization. I can setup Linux to run just my application, customize the look-and-feel of the entire software stack, and restrict it from tampering vs. trying to hide the bezel from the user or root the tablet and hope I don’t break something. 3) longer software lifespan if you choose a manufacturer with good drivers (such as Raspberry Pi or the imx6 platform).
Weaknesses of embedded Linux: 1) hardware selection: touchscreens with good driver support (not manufacturer-hacked custom kernels) are much more limited for the Linux platform. 2) APIs: I think Android wins because it was built for touch; Linux has QT which costs money for commercial use, GTK (no idea how well it works), and possibly Java.
I would be interested to see more articles regarding your decision-making process for Android vs. Linux in some of your projects.
Thanks for the feedback. Yeah, I’d agree the cost of an off-the-shelf IOIO narrows the cost advantage of the Android tablet approach considerably, especially if you also need to buy a Bluetooth dongle to go with it. On the other hand, if you already need to design custom hardware for the control part of the device, you can pretty cheaply add the functionality of the (open design) IOIO right into your board. Or you can skip the IOIO and just design a Bluetooth interface directly into your board. I think if you can do that the cost advantage turns back in the direction of the tablet. But yeah, in my opinion the talking-to-I/O part of the design is the most “interesting” one for the Android tablet approach. Up until a few years ago you could just power the unit off the separate 5V jack, then communicate over USB; some even had multiple USB ports. But now you really have to charge over USB and use some contactless method for communication. Bluetooth works well for some cases, but you’re certainly right it has some limitations. I may write some more about this issue.
I agree wholeheartedly with your summary of Linux vs. Android. The issues you raise are pretty much exactly the ones I consider most important. I’d say for most applications the balance is tilted toward a Linux SBC based design. The one persistently frustrating case to me is the class of kiosk products, where you want a largish touchscreen; and particularly for products in a regulated market. The number of larger touchscreens (say 7″ or larger) is quite limited in my view, and those that exist tend to be the components that require the most attention when doing EMC testing. That’s where a well integrated and already tested tablet is particularly nice.
Part 2 of this series is now available, covering the use of NFC or WiFi instead of Bluetooth, for contactless communication with the embedded tablet:
Communicating with an embedded tablet over NFC or WiFi