Virtual keyboard

Typing on an iPad's virtual keyboard

A virtual keyboard is a software component that allows the input of characters without the need for physical keys.[1] The interaction with the virtual keyboard happens mostly via a touchscreen interface, but can also take place in a different form in virtual or augmented reality.

Types

On a desktop computer, a virtual keyboard might provide an alternative input mechanism for users with disabilities who cannot use a conventional keyboard, or for bi- or multi-lingual users who switch frequently between different character sets or alphabets, which may be confusing over time. Although hardware keyboards are available with dual keyboard layouts (e.g. Cyrillic/Latin letters in various national layouts), the on-screen keyboard provides a handy substitute while working at different stations or on laptops, which seldom come with dual layouts.

Virtual keyboards can be categorized by the following aspects:

On the Internet, various JavaScript virtual keyboards have been created, allowing users to type their own languages on foreign keyboards, particularly in Internet cafes. Multitouch screens allow the possibility to create virtual chorded keyboards for tablet computers,[7] touchscreens, touchpads and wired gloves.[8][9]

Mobile devices

Virtual keyboards are commonly used as an on-screen input method in devices with no physical keyboard, where there is no room for one, such as a pocket computer, personal digital assistant (PDA), tablet computer or touchscreen-equipped mobile phone. Text is commonly inputted either by tapping a virtual keyboard or finger-tracing.[10] Virtual keyboards are also used as features of emulation software for systems that have fewer buttons than a computer keyboard would have.

Historical development


PDA

The four main approaches to enter text into a PDA were, virtual keyboards operated by a stylus, external USB keyboards, handwritten keyboards and stroke recognition. Many early PDA not primarily focused on virtual keyboards.[11] Microsoft’s mobile operating system approach[12] was to simulate a complete functional keyboard. Which resulted in a slightly overloaded keyboard layout.[13] The main problem that those old PDAs were not supporting the multi-touch technology and were causing as a result, usability problems for the user.

First iPhone

When Apple presented the first iPhone in 2007, the decision to not include a physical keyboard was seen as a detriment to the device.[14] But Apple brought the multi-touch technology into their new device, which enabled them to overcome the usability problems of PDAs. Apple's virtual keyboard design pattern has become a standard on mobile devices today.

Implementation and use

Both most common mobile operating systems, Android and iOS, give the developer community the possibility to individually develop custom virtual keyboards.

Android
Android Keyboard Settings

The Android SDK provides a so called InputMethodService.[15] This service provides a standard implementation of an input method, which final implementations can derive from and customize. What enables the Android development community to implement their own keyboard layouts. The InputMethodService ships with it on Keyboard View.[16] While the InputMethod Service can be used to customize key and gesture inputs, the Keyboard Class loads an XML description of a keyboard and stores the attributes of the keys. [17]

As a result, it is possible to install different keyboard versions on an Android device, and that the keyboard is only an application; a simple activation over the Android settings menu is possible.[18]

iOS
Apple iOS UIInputViewController

Apple also provides the possibility for the community to develop custom keyboards, but does not give any access to the dictionary or general keyboard settings. Further iOS is automatically switching between system and custom keyboards, if the user enters text into the text input field.[19][20]

The UIInputViewController is the primary view controller for a custom keyboard app extension. This controller provides different methods for the implementation of a custom keyboard, such as a user interface for a custom keyboard, obtaining a supplementary lexicon or changing the primary language of a custom keyboard.[21][22]

Text entry performance

Next to the classic virtual keyboard implementation Android, iOS and custom keyboards, such as SwiftKey for example, are providing different features to improve the usability and the efficiency of their keyboards.

Autocorrection and spelling checker

The Android platform offers a spelling checker framework that offers the possibility to implement and access spell checking in the application itself. The framework is one of the Text Service APIs offered by the Android platform. Based on provided text, the session object returns spelling suggestions generated by the spelling checker.[23][24]

iOS is using the class UITextChecker, an object used to check a string (usually the text of a document) for misspelled words, commonly known as Apple’s autocorrection. UITextChecker spell-checks are using a lexicon for a given language. It can be told to ignore specific words when spell-checking a particular document and it can learn new words, which adds those words to the lexicon.[25]

Word suggestions

Diverse scientific papers at the beginning of the 2000’s showed even before the invention of smart phones, that predicting words, based on what the user is typing, is very helpful to increase the typing speed. [26][27] At the beginning of development of this keyboard feature, prediction was mainly based on static dictionaries. Google implemented the predicting method in 2013 in Android 4.4. This development was mainly driven by third party keyboard providers, such as SwiftKey and Swype.[28] Both provide powerful word search engine with corresponding databases.[29] In 2014 Apple presented iOS 8 [30][31] which includes a new predictive typing feature called QuickType, which displays word predictions above the keyboard as the user types.

Gesture typing

iOS and Android allow developers to replace its keyboard with their own keyboard apps. This has led to experimentation and new features, like the gesture-typing feature that’s made its way into Android’s official keyboard after proving itself in third-party keyboards. Research by Google itself confirmed that gesture-typing is increasing the typing rate by 22% and is decreasing the error rate near to 0%.[32] Google further showed that the gesture-typing method is also useful on smart watches. Their scientific research is primarily based on research made by I. Scott MacKenzie and papers about modeling finger touch with fitts’ law.

Special keyboard types

Keyboards are needed in different digital areas. Not only smartphones need a virtual keyboards, also devices which create virtual worlds, for example virtual reality or augmented reality glasses, need to provide text input possibilities.

Optical virtual keyboard

An optical virtual keyboard was invented and patented by IBM engineers in 2008.[33] It optically detects and analyses human hand and finger motions and interprets them as operations on a physically non-existent input device like a surface having painted keys. In that way it allows to emulate unlimited types of manually operated input devices such as a mouse or keyboard. All mechanical input units can be replaced by such virtual devices, optimized for the current application and for the user's physiology maintaining speed, simplicity and unambiguity of manual data input.

Augmented reality keyboards

The basic idea of a virtual keyboard in an augmented reality environment is to give the user a text input possibility. A common approach is to render a flat keyboard into the augmented reality, e.g. using the Unity TouchScreenKeyboard. The Microsoft HoloLens enables the user to point at letters on the keyboard by moving his head.[34]

Another approach was researched by the Korean KJIST U-VR Lab in 2003. Their suggestion was to use wearables to track the finger motion to replace a physical keyboards with virtual ones. They also tried to give a audiovisual feedback to the user, when a key got hit. The basic idea was to give the user a more natural way to enter text, based on what he is used to. [35]

Virtual reality keyboards

The challenges, as in augmented reality, is to give the user the possibility to enter text in a completely virtual environment. One big issue is that most augmented reality systems on the market are not tracking the hands of the user. So many available system provide the possibility to point at letters. [36]

In September 2016 Google has released a virtual keyboard app for their Daydream[37] virtual reality headset. To enter text, the user can point at specific letters with the Daydream controller.[38]

In February 2017 Logitech presented experimental approach to bring their keyboards into the virtual environment. With the Vive Tracker and the Logitech G gaming keyboard it is possible to exactly track every finger movement, without wearing any type of glove. 50 of such packages were send to exclusive developers, enabling them, in combination of Logitche’s BRIDGE developers kit, to test and experiment with the new technology. [39][40]

Security considerations

Virtual keyboards may be used in some cases to reduce the risk of keystroke logging.[41] For example, Westpac’s online banking service uses a virtual keyboard for the password entry, as does TreasuryDirect (see picture). It is more difficult for malware to monitor the display and mouse to obtain the data entered via the virtual keyboard, than it is to monitor real keystrokes. However it is possible, for example by recording screenshots at regular intervals or upon each mouse click.[42][43]

TreasuryDirect login screen, showing virtual keyboard

The use of an on-screen keyboard on which the user "types" with mouse clicks can increase the risk of password disclosure by shoulder surfing, because:

  • An observer can typically watch the screen more easily (and less suspiciously) than the keyboard, and see which characters the mouse moves to.
  • Some implementations of the on-screen keyboard may give visual feedback of the "key" clicked, e.g. by changing its colour briefly. This makes it much easier for an observer to read the data from the screen. In the worst case, the implementation may leave the focus on the most recently clicked "key" until the next virtual key is clicked, thus allowing the observer time to read each character even after the mouse starts moving to the next character.
  • A user may not be able to "point and click" as fast as they could type on a keyboard, thus making it easier for the observer.

See also

Notes

  1. ↑ "virtual keyboard". PCMag.
  2. ↑ US application 4725694 Auer Carol M, Castagno Daniel L/AT&T: "Computer interface device" priority date 13.05.1986
  3. ↑ DE application 19734511 B. Kämmerer, C, Maggioni, H. Röttger/SIEMENS AG: "Kommunikationseinrichtung" filing date 08.08.1997
  4. ↑ WO 0003348 C. Maggioni, B. Kämmerer/SIEMENS AG: "Projection Device / Vorrichtung zur Projektion" priority date 10.07.1998
  5. ↑ EP 0554492 Hans E. Korth: "Method and device for optical input of commands or data" filing date 07.02.1992
  6. ↑ "Virtual keyboard".
  7. ↑ Chien-Hsu Chen. "Design and Evaluation of Chord Keyboard for Tablet Computer Manipulation". National Cheng Kung University.
  8. ↑ "ASETNIOP concept designed to make touch typing on a touchscreen a mite easier". Gizmag.com.
  9. ↑ "ASETNIOP invisible keyboard aims to get you touch-typing on the iPad". todaysiphone.com.
  10. ↑ "Swype finger-tracing text entry seems fast, hypnotic, and magical". Engadget. Retrieved 2016-03-24.
  11. ↑ "Personal digital assistant". Wikipedia. 2018-03-13.
  12. ↑ "Microsoft Windows Mobile". Wikipedia (in German). 2017-06-09.
  13. ↑ Georgiev, Tsvetozar; Georgieva, Evgenia (2008-06-12). Mobile application for determination of users' text entry speed.
  14. ↑ Titcomb, James (2017). "How the world reacted to the first iPhone 10 years ago". The Telegraph. ISSN 0307-1235. Retrieved 2018-03-16.
  15. ↑ "InputMethodService | Android Developers". developer.android.com. Retrieved 2018-03-19.
  16. ↑ "Create a Custom Keyboard on Android". Code Envato Tuts+. Retrieved 2018-03-19.
  17. ↑ "Keyboard | Android Developers". developer.android.com. Retrieved 2018-03-19.
  18. ↑ "keyboard - Google Play". Retrieved 2018-03-19.
  19. ↑ "Managing the Keyboard". developer.apple.com. Retrieved 2018-03-19.
  20. ↑ "UIInputViewController - UIKit | Apple Developer Documentation". developer.apple.com. Retrieved 2018-03-19.
  21. ↑ "App Extension Programming Guide: Custom Keyboard". developer.apple.com. Retrieved 2018-03-19.
  22. ↑ "Creating a Custom Keyboard Using iOS 8 App Extension". Retrieved 2018-03-19.
  23. ↑ "UITextChecker - UIKit | Apple Developer Documentation". developer.apple.com. Retrieved 2018-03-19.
  24. ↑ "Spelling Checker Framework | Android Developers". developer.android.com. Retrieved 2018-03-19.
  25. ↑ Liu, Croath (2016-04-25). "UITextChecker". NSHipster. Retrieved 2018-03-19.
  26. ↑ Tom Stocky, Alexander Faaborg, Henry Lieberman (2004). "A Commonsense Approach to Predictive Text Entry".
  27. ↑ Dunlop, Mark D. and Crossan, Andrew (2000). "Predictive text entry methods for mobile phones".
  28. ↑ Hattersley, Lucy. "How to use new QuickType and Swype keyboard options in iOS 8". Macworld UK. Retrieved 2018-03-19.
  29. ↑ "Apple - iOS 8 - QuickType". Apple. Retrieved 2018-03-19.
  30. ↑ "iOS 8". Wikipedia. 2018-03-17.
  31. ↑ "Using iOS 8's QuickType keyboard". CNET. 2014-09-19. Retrieved 2018-03-19.
  32. ↑ Gordon, Mitchell; Ouyang, Tom; Zhai, Shumin (2016-05-07). WatchWriter: Tap and Gesture Typing on a Smartwatch Miniature Keyboard with Statistical Decoding.
  33. ↑ EP 0554492 Hans E. Korth: "Method and device for optical input of commands or data" filing date 07.02.1992
  34. ↑ "Keyboard input in Unity". developer.microsoft.com. Retrieved 2018-03-16.
  35. ↑ Lee, Minkyung; Woo, Woontack (2003-01-01). ARKB: 3D vision-based Augmented Reality Keyboard.
  36. ↑ Poupyrev, Ivan. "Virtual Notepad: Handwriting in Immersive VR" (PDF).
  37. ↑ LLC, Google (2018-02-21), Daydream Keyboard, Google LLC, retrieved 2018-03-16
  38. ↑ "Google launches virtual keyboard for Daydream-ready phones". Haptical. 2016-10-19. Retrieved 2018-03-16.
  39. ↑ "Introducing the Logitech BRIDGE SDK - VIVE Blog". blog.vive.com. Retrieved 2018-03-16.
  40. ↑ "Logitech is experimenting with a keyboard built for virtual reality – TechCrunch". techcrunch.com. Retrieved 2018-03-16.
  41. ↑ "Virtual keyboard".
  42. ↑ Lake Software: Does not defeat keyloggers!
  43. ↑ Smith, David A. (2006-06-21), Outsmarting Keyloggers, PC Magazine, retrieved 2009-11-16
This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.