Ke Sun
Chun Yu
Skip Abstract Section
Abstract
The Qwerty soft keyboard is widely used on mobile devices. However, keyboards often consume a large portion of the touchscreen space, occluding the application view on the smartphone and requiring a separate input interface on the smartwatch. Such space consumption can affect the user experience of accessing information and the overall performance of text input. In order to free up the screen real estate, this article explores the concept of Sparse Keyboard and proposes two new ways of presenting the Qwerty soft keyboard. The idea is to use users’ spatial memory and the reference effect of spatial landmarks on the graphical interface. Our final design K3-SGK displays only three keys while L5-EYOCN displays only five line segments instead of the entire Qwerty layout. To achieve this, we employ a user-centered computational design method: first study the reference effect of a single landmark key (line segment) from empirical data, then make assumptions to generalize the effect to multiple landmarks, and finally optimize the best designs. To make the text entry function more complete, we also design and implement gestural interactions for editing operations and non-alphabetical characters’ input. User evaluation shows that participants can quickly learn how to type with K3-SGK and L5-EYOCN. After five 15-phrase typing sessions, participants achieve 88.1%--92.8% of the full Qwerty keyboard in terms of words per minute on the smartphone and 98.4%--99.1% on the smartwatch. The differences on character and word error rate between our keyboard designs and the full Qwerty keyboard are not significant. The results of out-of-vocabulary words input are also promising. In addition, participants can quickly recall the typing skills and maintain the input performance even after a few days. User feedbacks in real application contexts show that with the low occlusion keyboard, users can acquire more information and perform less scrolling on the smartphone and achieve a higher input efficiency on the smartwatch with a more fluent input experience.
- Ahmed Sabbir Arif, Benedikt Iltisberger, and Wolfgang Stuerzlinger. 2011. Extending mobile user ambient awareness for nomadic text entry. In Proceedings of the 23rd Australian Computer-Human Interaction Conference (OzCHI’11). ACM, New York, NY, 21--30. Google Scholar
Digital Library
- Ahmed Sabbir Arif and Ali Mazalek. 2016. A survey of text entry techniques for smartwatches. In Proceedings, Part II, of the 18th International Conference on Human-Computer Interaction. Interaction Platforms and Techniques - Volume 9732. Springer-Verlag, Berlin, 255--267. Google ScholarDigital Library
- Shiri Azenkot and Shumin Zhai. 2012. Touch behavior with different postures on soft smartphone keyboards. In Proceedings of the 14th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI’12). ACM, New York, NY, 251--260.Google ScholarDigital Library
- Nikola Banovic, Koji Yatani, and Khai N. Truong. 2013. Escape-keyboard: A sight-free one-handed text entry method for mobile touch-screen devices. International Journal of Human-Computer Studies 5, 3 (2013), 42--61. Google ScholarDigital Library
- Xiaojun Bi, Yang Li, and Shumin Zhai. 2013. FFitts law: Modeling finger touch with Fitts’ law. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’13). ACM, New York, NY, 1363--1372.Google ScholarDigital Library
- Thorsten Brants and Alex Franz. 2006. Web 1T 5-gram Version 1. Linguistic Data Consortium.Google Scholar
- Xiang ‘Anthony’ Chen, Tovi Grossman, and George Fitzmaurice. 2014. Swipeboard: A text entry technique for ultra-small interfaces that supports novice to expert transitions. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (UIST’14). ACM, New York, NY, 615--620.Google ScholarDigital Library
- Andy Cockburn, Carl Gutwin, Joey Scarr, and Sylvain Malacria. 2014. Supporting novice to expert transitions in user interfaces. ACM Computing Surveys 47, 2 (November 2014), Article 31, 36 pages. Google ScholarDigital Library
- Mary Czerwinski, Maarten Van Dantzich, George G. Robertson, and Hunter G. Hoffman. 1999. The contribution of thumbnail image, mouse-over text and spatial location memory to web page retrieval in 3D. In Proceedings of the INTERACT. 163--170.Google Scholar
- EnronData. 2008. Enron Email Datasets. Retrieved October 1, 2018 from https://enrondata.readthedocs.io/en/latest/data/enron-email-datasets/.Google Scholar
- Leah Findlater, Ben Lee, and Jacob Wobbrock. 2012. Beyond QWERTY: Augmenting touch screen keyboards with multi-touch gestures for non-alphanumeric input. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’12). ACM, New York, NY, 2679--2682. Google ScholarDigital Library
- Paul M. Fitts. 1954. The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology 47, 6 (1954), 381.Google ScholarCross Ref
- Brian Frey, Caleb Southern, and Mario Romero. 2011. BrailleTouch: Mobile texting for the visually impaired. In Universal Access in Human-Computer Interaction. Context Diversity. Springer, Berlin, 19--25. Google ScholarDigital Library
- Krzysztof Z. Gajos, Mary Czerwinski, Desney S. Tan, and Daniel S. Weld. 2006. Exploring the design space for adaptive graphical user interfaces. In Proceedings of the Working Conference on Advanced Visual Interfaces (AVI’06). ACM, New York, NY, 201--208.Google Scholar
- Mitchell Gordon, Tom Ouyang, and Shumin Zhai. 2016. WatchWriter: Tap and gesture typing on a smartwatch miniature keyboard with statistical decoding. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI’16). ACM, New York, NY, 3817--3821.Google ScholarDigital Library
- Aakar Gupta, Muhammed Anwar, and Ravin Balakrishnan. 2016. Porous interfaces for small screen multitasking using finger identification. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST’16). ACM, New York, NY, 145--156.Google ScholarDigital Library
- Carl Gutwin, Andy Cockburn, Joey Scarr, Sylvain Malacria, and Scott C. Olson. 2014. Faster command selection on tablets with FastTap. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’14). ACM, New York, NY, 2617--2626. Google ScholarDigital Library
- Niels Henze, Enrico Rukzio, and Susanne Boll. 2012. Observational and experimental investigation of typing behaviour using virtual keyboards for mobile devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’12). ACM, New York, NY, 2659--2668.Google ScholarDigital Library
- William E. Hick. 1952. On the rate of gain of information. Quarterly Journal of Experimental Psychology 4, 1 (1952), 11--26.Google ScholarCross Ref
- Ray Hyman. 1953. Stimulus information as a determinant of reaction time. Journal of Experimental Psychology 45, 3 (1953), 188.Google ScholarCross Ref
- Jussi P. P. Jokinen, Sayan Sarcar, Antti Oulasvirta, Chaklam Silpasuwanchai, Zhenxin Wang, and Xiangshi Ren. 2017. Modelling learning of new keyboard layouts. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI’17). ACM, New York, NY, 4203--4215.Google ScholarDigital Library
- David E. Kieras and Anthony J. Hornof. 2014. Towards accurate and practical predictive models of active-vision-based visual search. In Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems (CHI’14). ACM, New York, NY, 3875--3884.Google Scholar
- KlearKeys. 2013. KlearKeys: A Transparent Keyboard on Phone. Retrieved on November 25, 2018 from https://play.google.com/store/apps/details?id=com.touchscapesoftware.klearkeys.phone&hl===en_US.Google Scholar
- Gordon P. Kurtenbach, Abigail J. Sellen, and William A. S. Buxton. 1993. An empirical evaluation of some articulatory and cognitive aspects of marking menus. Human-Computer Interaction 8, 1 (1993), 1--23. Google ScholarDigital Library
- J. P. Lewis, Ruth Rosenholtz, Nickson Fong, and Ulrich Neumann. 2004. VisualIDs: Automatic distinctive icons for desktop interfaces. In Proceedings of the ACM SIGGRAPH 2004 Papers (SIGGRAPH’04). ACM, New York, NY, 416--423.Google ScholarDigital Library
- Frank Chun Yat Li, Richard T. Guy, Koji Yatani, and Khai N. Truong. 2011. The 1Line keyboard: A QWERTY layout in a single line. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology (UIST’11). ACM, New York, NY, 461--470. Google ScholarDigital Library
- Xianyun Liu, Matthew J. C. Crump, and Gordon D. Logan. 2010. Do you know where your fingers have been? Explicit knowledge of the spatial layout of the keyboard in skilled typists. Memory and Cognition 38, 4 (2010), 474--484.Google ScholarCross Ref
- Yiqin Lu, Chun Yu, Xin Yi, Yuanchun Shi, and Shengdong Zhao. 2017. BlindType: Eyes-free text entry on handheld touchpad by leveraging thumb’s muscle memory. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 2 (2017), 18:1--18:24.Google ScholarDigital Library
- I. Scott MacKenzie, Hedy Kober, Derek Smith, Terry Jones, and Eugene Skepner. 2001. LetterWise: Prefix-based disambiguation for mobile text input. In Proceedings of the 14th Annual ACM Symposium on User Interface Software and Technology (UIST’01). ACM, New York, NY, 111--120.Google ScholarDigital Library
- I. Scott MacKenzie and R. William Soukoreff. 2003. Phrase sets for evaluating text entry techniques. In Proceedings of the CHI’03 Extended Abstracts on Human Factors in Computing Systems (CHI EA’03). ACM, New York, NY, 754--755.Google Scholar
- I. Scott MacKenzie, R. William Soukoreff, and Joanna Helga. 2011. 1 thumb, 4 buttons, 20 words per minute: Design and evaluation of H4-writer. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology (UIST’11). ACM, New York, NY, 471--480. Google ScholarDigital Library
- I. Scott MacKenzie and Shawn X. Zhang. 2001. An empirical investigation of the novice experience with soft keyboards. Behaviour 8 Information Technology 20, 6 (2001), 411--418.Google Scholar
- Antti Oulasvirta. 2017. User interface design with combinatorial optimization. Computer 50, 1 (2017), 40--47. Google ScholarDigital Library
- Antti Oulasvirta, Anna Reichel, Wenbin Li, Yan Zhang, Myroslav Bachynskyi, Keith Vertanen, and Per Ola Kristensson. 2013. Improving two-thumb text entry on touchscreen devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’13). ACM, New York, NY, 2765--2774.Google ScholarDigital Library
- David Peebles and Corinna Jones. 2014. A model of object location memory. In Proceedings of the Annual Meeting of the Cognitive Science Society, Vol. 36.Google Scholar
- Ken Perlin. 1998. Quikwriting: Continuous stylus-based text entry. In Proceedings of the 11th Annual ACM Symposium on User Interface Software and Technology (UIST’98). ACM, New York, NY, 215--216.Google ScholarDigital Library
- George Robertson, Mary Czerwinski, Kevin Larson, Daniel C. Robbins, David Thiel, and Maarten van Dantzich. 1998. Data mountain: Using spatial memory for document management. In Proceedings of the 11th Annual ACM Symposium on User Interface Software and Technology (UIST’98). ACM, New York, NY, 153--162.Google ScholarDigital Library
- K. C. Salter and R. F. Fawcett. 1985. A robust and powerful rank test of treatment effects in balanced incomplete block designs. Communications in Statistics - Simulation and Computation 14, 4 (1985), 807--828.Google ScholarCross Ref
- Joey Scarr, Andy Cockburn, Carl Gutwin, and Andrea Bunt. 2012. Improving command selection with CommandMaps. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’12). ACM, New York, NY, 257--266. Google ScholarDigital Library
- Katherine Schramm, Carl Gutwin, and Andy Cockburn. 2016. Supporting transitions to expertise in hidden toolbars. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI’16). ACM, New York, NY, 4687--4698.Google ScholarDigital Library
- Wendy Ark, D. Christopher Dryer, Ted Selker, and Shumin Zhai. 1998. Landmarks to Aid Navigation in a Graphical User Interface. IBM Thomas J. Watson Research Division.Google Scholar
- Kristy M. Snyder, Yuki Ashitaka, Hiroyuki Shimada, Jana E. Ulrich, and Gordon D. Logan. 2014. What skilled typists don’t know about the QWERTY keyboard. Attention, Perception, 8 Psychophysics 76, 1 (2014), 162--171.Google Scholar
- Md. Sami Uddin and Carl Gutwin. 2016. Rapid command selection on multi-touch tablets with single-handed HandMark menus. In Proceedings of the 2016 ACM International Conference on Interactive Surfaces and Spaces (ISS’16). ACM, New York, NY, 205--214.Google Scholar
- Md. Sami Uddin, Carl Gutwin, and Andy Cockburn. 2017. The effects of artificial landmarks on learning and performance in spatial-memory interfaces. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI’17). ACM, New York, NY, 3843--3855.Google ScholarDigital Library
- Keith Vertanen, Haythem Memmi, Justin Emge, Shyam Reyal, and Per Ola Kristensson. 2015. VelociTap: Investigating fast mobile text entry using sentence-based decoding of touchscreen keyboard input. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI’15). ACM, New York, NY, 659--668.Google ScholarDigital Library
- Norman G. Vinson. 1999. Design guidelines for landmarks to support navigation in virtual environments. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’99). ACM, New York, NY, 278--285.Google ScholarDigital Library
- Daniel Vogel and Patrick Baudisch. 2007. Shift: A technique for operating pen-based interfaces using touch. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’07). ACM, New York, NY, 657--666.Google ScholarDigital Library
- Jacob O. Wobbrock, Leah Findlater, Darren Gergle, and James J. Higgins. 2011. The aligned rank transform for nonparametric factorial analyses using only anova procedures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 143--146. Google ScholarDigital Library
- Jacob O. Wobbrock, Brad A. Myers, and John A. Kembel. 2003. EdgeWrite: A stylus-based text entry method designed for high accuracy and stability of motion. In Proceedings of the 16th Annual ACM Symposium on User Interface Software and Technology (UIST’03). ACM, New York, NY, 61--70.Google Scholar
- Xin Yi, Chun Yu, Weinan Shi, and Yuanchun Shi. 2017. Is it too small?: Investigating the performances and preferences of users when typing on tiny QWERTY keyboards. International Journal of Human-Computer Studies 106 (2017), 44--62.Google ScholarCross Ref
- Xin Yi, Chun Yu, Weijie Xu, Xiaojun Bi, and Yuanchun Shi. 2017. COMPASS: Rotational keyboard on non-touch smartwatches. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI’17). ACM, New York, NY, 705--715.Google ScholarDigital Library
- Shumin Zhai, Michael Hunter, and Barton A. Smith. 2002. Performance optimization of virtual keyboards. Human--Computer Interaction 17, 2--3 (2002), 229--269.Google ScholarCross Ref
- Suwen Zhu, Tianyao Luo, Xiaojun Bi, and Shumin Zhai. 2018. Typing on an invisible keyboard. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI’18). ACM, New York, NY Article 439, 13 pages. Google ScholarDigital Library
Index Terms
- Exploring Low-Occlusion Qwerty Soft Keyboard Using Spatial Landmarks
Recommendations
Text Entry on Tiny QWERTY Soft Keyboards
CHI '15: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing SystemsThe advent of wearables (e.g., smartwatches, smartglasses, and digital jewelry) anticipates the need for text entry methods on very small devices. We conduct fundamental research on this topic using 3 qwerty-based soft keyboards for 3 different screen ...
The 1line keyboard: a QWERTY layout in a single line
UIST '11: Proceedings of the 24th annual ACM symposium on User interface software and technologyCurrent soft QWERTY keyboards often consume a large portion of the screen space on portable touchscreens. This space consumption can diminish the overall user experi-ence on these devices. In this paper, we present the 1Line keyboard, a soft QWERTY ...
Quasi-qwerty soft keyboard optimization
CHI '10: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsIt has been well understood that optimized soft keyboard layouts improve motor movement efficiency over the standard Qwerty layouts, but have the drawback of long initial visual search time for novice users. To ease the initial searching time on ...
Comments