Understanding finger postures when touching targets on the touchscreen of mobile devices1

Comprensión de las posturas de dedos al tocar objetivos en la pantalla táctil de dispositivos móviles

This paper presents the results of a preliminary study on the interaction of the fingertip and a mobile device’s touch screen. The objective of this study is to identify the finger postures that will be included in the main study on the contact area in the interaction between fingertip and a flat surface. Twenty participants (15 males and 5 females) took part in this study. They were asked to complete two tasks in a sitting posture. In the first task, they had to touch targets on the mobile device screens by tapping them sequentially, while in the second task they were asked to connect the targets with straight lines. The results showed that the participants used mainly their thumbs and index fingers to touch targets on the screen of the devices. Only a small number of participants used their middle finger, and only in a few touching activities.

Este trabajo presenta los resultados de un estudio preliminar sobre la interacción de la yema del dedo y la pantalla táctil de un dispositivo móvil. El objetivo del estudio es identificar posturas de los dedos que se incluirán en el estudio principal en el área de contacto en la interacción entre las puntas de los dedos y una superficie plana. Veinte participantes (15 hombres), participaron en este estudio. Se les pidió que completasen dos tareas en una postura sentada. En la primera tarea, tuvieron que tocar objetivos en las pantallas de dispositivos móviles de una forma secuencial, mientras que en la segunda tarea se les pidió que conectasen posiciones objetivo con líneas rectas. Los resultados mostraron que los participantes utilizan principalmente sus dedos pulgares e indicadores para tocar posiciones objetivo en la pantalla de los dispositivos. Sólo un pequeño número de participantes usó su dedo medio, y en pocas actividades.

DOI: https://doi.org/10.15446/dyna.v83n197.57587

Understanding finger postures when touching targets on the touchscreen of mobile devices

Mahrus K. Umami ^a, Pedro M. Arezes ^b & Álvaro M. Sampaio ^c

^a School of Engineering, University of Minho, Portugal, and School of Engineering, University of Trunojoyo Madura, Indonesia. mahrusku@gmail.com
^b Centre of ALGORITMI, School of Engineering, University of Minho, Portugal, parezes@dps.uminho.pt
^c Institute for Polymers and Composites IPC/I3N, School of Engineering, University of Minho, Portugal. amsampaio@dep.uminho.pt

Received: November 30^th, 2015. Received in revised form: March 10^th, 2016. Accepted: April 12^th, 2016.

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

---

Abstract
This paper presents the results of a preliminary study on the interaction of the fingertip and a mobile device's touch screen. The objective of this study is to identify the finger postures that will be included in the main study on the contact area in the interaction between fingertip and a flat surface. Twenty participants (15 males and 5 females) took part in this study. They were asked to complete two tasks in a sitting posture. In the first task, they had to touch targets on the mobile device screens by tapping them sequentially, while in the second task they were asked to connect the targets with straight lines. The results showed that the participants used mainly their thumbs and index fingers to touch targets on the screen of the devices. Only a small number of participants used their middle finger, and only in a few touching activities.

Keywords: finger posture; touch screen device; mobile phone; tablet.

Resumen
Este trabajo presenta los resultados de un estudio preliminar sobre la interacción de la yema del dedo y la pantalla táctil de un dispositivo móvil. El objetivo del estudio es identificar posturas de los dedos que se incluirán en el estudio principal en el área de contacto en la interacción entre las puntas de los dedos y una superficie plana. Veinte participantes (15 hombres), participaron en este estudio. Se les pidió que completasen dos tareas en una postura sentada. En la primera tarea, tuvieron que tocar objetivos en las pantallas de dispositivos móviles de una forma secuencial, mientras que en la segunda tarea se les pidió que conectasen posiciones objetivo con líneas rectas. Los resultados mostraron que los participantes utilizan principalmente sus dedos pulgares e indicadores para tocar posiciones objetivo en la pantalla de los dispositivos. Sólo un pequeño número de participantes usó su dedo medio, y en pocas actividades.

Palabras clave: postura; dedos; dispositivo con pantalla de toque; teléfono móvil; tableta.

---

1. Introduction

Nowadays, people use their mobile devices everywhere, adopting various postures with their fingers. They use their mobile phones when walking, standing, or sitting, as part of their daily activities. Karlson's study [1] identified that travellers operated their mobile devices using only one hand at an airport when walking, standing, or sitting. The observation involved users of mobile phones and PDAs. There were two types of mobile phones recorded in the study, the candy bar and the flip types. A larger survey by Hoober [2] showed that people use their mobile devices in standing, walking and sitting positions, and while travelling on public transportation, such as buses or trains. The survey observed how people held their mobile devices at common places, such as universities, parks, offices, public transportation, and shopping malls. Hoober recorded a Samsung Galaxy Note 2 as the largest device in his data set. From both surveys, it was possible to conclude that people hold their mobile devices in different ways: one hand, two hands, and cradle. Additionally, people can also use their mobile devices by putting them on a table, especially when they interacting with tablets.

People change their hand postures very often when interacting with their mobile devices, changing postures during usage time. It is possible to see that they often change the way they hold their mobile device easily while using it. Hoober repeatedly saw individuals that adopted various hand postures while using the same device, in a short amount of time. He recorded individuals that used one hand at the beginning, then used the other for a different activity, then adopted cradle posture, and then returned to the initial posture [2]. In accordance with Hoober's findings, Azenkot and Zhai [4] found that all participants used at least two methods to complete the tasks required in their study on the use of two thumbs, one thumb, and index finger in human interactions with mobile devices. Regarding the most commonly used fingers, Hoober [2] and Azenkot and Zhai [4] noticed that most people used their thumbs and index fingers.

It is also possible to see that the target size-defined by considering the screen size of the device and designed according to the size of a human finger pad-of a tablet is usually larger than that of smaller devices [5]. Microsoft's Windows Phone Silverlight development guidelines recommend that the touch target size can be greater than or equal to a 9 mm square [6]. For the minimum target size, Microsoft allows their developers to use a 7 mm square when the smaller hit target is warranted [6]. The minimum target size recommended by Microsoft is the same as the size recommended by Google. Google's Android UI Guidelines suggest a 7 mm square (48 pixels square) [5], although for targets that are rarely used by users, the developers can apply a size smaller than 7 mm square. A smaller minimum target size is recommended by Apple. The iOS Human Interface Guidelines allow a 6.5 mm square (44 pixels square) for their developers [7]. All of the operating system developers define the minimum targets size by considering the touching accuracy [5-7].

Few studies showed that the target size has a significant relation to the user errors, performance and satisfaction. Parhi's study, for example, found that user errors decline as the size of the target increases [8]. Park's study on touch key design, showed that user performance and subjective satisfaction with the larger touch key size was higher than the satisfaction with smaller sizes [9]. Both studies showed that the target size should fit to the fingertip contact area on the target.

It is commonly known that users try to touch the targets on the touch screen devices accurately by manipulating the finger posture. Wang and Ren [10] observed two different forms of fingertip contact: the vertical touch and the oblique touch. They found that the size of the fingertip contact area is significantly different with each of the two forms of touching. Holz and Baudisch [11] studied the users' mental model of touch in their efforts to minimize error. The study explored techniques used by the participants in targeting a crosshair accurately. They found that users tried to touch the target on their device by aligning the finger's feature and outline. Both studies observed the individuals touching the screen display in a flat posture on a table.

Since previous studies [2,4,10,11] have not completely described which fingers and respective postures are normally used by individuals in interacting with the touch screen devices, an in-depth study is required to acquire that information.

This paper presents the results of a pilot test on the interaction between the finger and the touch screen of mobile devices in various postures. The test is the preliminary observation of a proposed study on the contact area of fingertips and touch screen devices. The purpose of this study is to identify the user's typical finger posture in his interaction with the screen of a mobile device. The rationale for this observation is to determine the fingers and their pitches and rolls that should be included in the proposed main study on the fingertip contact area with the touch screen. A previous version of this paper with a smaller number of participants has been published elsewhere [3].

2. Materials and Methods

2.1. Participants

Twenty volunteer participants (15 males and 5 females) were involved in this study. Their age ranged from 19-58 years with a mean of 34.85 (SD. ± 9.76). All of them were right handed.

2.2. Equipment and tools

Two mobile devices, a Samsung Galaxy S Duos mobile phone (4 inch screen) and a Samsung Galaxy Note 10.1 tablet (10 inch screen), were used to display the targets. There were 8 targets (5 x 5 mm2 squares with blue outlines) set on the mobile phone screen, and 12 targets (7 x 7 mm2 squares with blue outlines) set on the screen of the tablet. Fig. 1 and 2 show the position of the targets on the screens of the mobile phone and tablet respectively.

A Sony DCR DVD-403E PAL video camera and an Olympus VR-340 compact camera were used to record the hand postures of the participants while performing the tasks. The Sony camera was placed next to the participants' hands and was used to record their finger pitch angles when performing tasks, while the Olympus camera was set behind the observation desk and was used to record the finger roll angles.

A table with a desk height (DH) of 74 cm was used to place the equipment on. A chair with a seat height (SH) of 55 cm was provided for the participants to be able to perform the tasks comfortably in a sitting posture.

An AutoCAD 2010 software application was used in the measurements of the finger pitch and roll angles. A Microsoft Excel for Windows was used to handle and to analyse the data.

2.3. Experiment design and procedures

This study was designed to consider the position of targets, the type of tasks, and the hand postures adopted by the participants to touch the targets as independent variables, while the chosen fingers, finger pitches, and finger rolls were considered as response variables.

In the planning phase, a number of postures that might be performed by the users when interacting with mobile devices were identified: four postures in the interaction with the tablet and four postures for the interaction with the mobile phone. From the eight postures, the two hands with two thumb-screen interaction for the tablet was ignored given that in our previous study [3], a few participants found it difficult to complete the tasks in this posture. The two-hand cradle posture with one thumb-screen interaction for mobile phone use was also ignored. It was assumed that the use of a single hand with one-thumb interaction had adequately represented the thumb postures for the observation. Only the use of an index finger or a middle finger in the cradle posture was observed. Table 1 listed the users' hand postures observed in this study.

In the experimental phase, participants were asked to perform two tasks in a sitting posture. In the first task, the participants were asked to touch targets appearing on the devices by tapping them in various hand postures sequentially. Then, in the second task, they were asked to touch each target by connecting them as they usually drag an icon or an image on the touch screen. The participants were allowed to choose which finger they wanted to use for the required tasks.

In sum, the experiment was designed to include 20 participants × 3 postures × 2 tasks × 12 targets = 1440 touches on the tablet and 20 participants × 3 postures × 2 tasks × 8 targets = 960 touches on the mobile phone.

The observation focused on finger pitches and rolls when touching the targets on the screen. The definition of the finger pitch and roll used by Holz and Baudisch [11] was adopted. The finger pitch is the angle between the finger and the device screen, while the finger roll is the rotation of the finger in a clockwise or anti-clockwise motion. Fig. 3 illustrates the definition of the finger pitch and the finger roll.

2.4. Data processing and analysis

The relevant images of the hand postures were captured from the videos recorded by two cameras. Then, the images were imported to the AutoCAD 2010 software, so that the finger pitch and the finger roll angle could be measured. To simplify data recording and computation, the finger pitch and roll angles measured from the images were classified by following a guideline shown in Table 2.

The data was summarised and analysed by using MS Excel for Windows on a computer. The frequency of the appearance of the finger pitch and the finger roll angles were counted manually and shown as the percentage of the sum of all the data in the related posture. All the data were analysed and presented in descriptive statistics.

3. Results and Discussion

It was found that most of the participants used their index finger when interacting with the tablet in all postures observed in this study, while only a small number of them used their middle fingers, and one participant was identified as not performing a cradle posture. It was also observed that two female participants used their middle fingers to touch the screen. They used their index and middle fingers, mainly because of their long fingernails.

In their interaction with the mobile phone, two participants did not use the single-handed posture. Besides, three participants did not use the cradle posture and four participants did not use the two thumbs posture. It was also observed that the participants performed the tasks by using their thumbs in the single- and two-handed interactions and their index fingers in the cradle interaction.

For the whole set of observations, we recorded 528 touching activities in 16 thumb postures and 288 touching activities in 9 index finger postures adopted by the participants when interacting with their mobile phones. Further, we recorded 1296 touching activities in 10 index finger postures and 120 touching activities in 3 middle finger postures adopted by the participants in their interaction with the tablet. Table 3 listed the number of the touching activities and the finger postures (pitch and roll angles combination) adopted by the participants.

3.1. Index finger postures in the interaction with tablet

In the interaction with the tablet in a tilt (45°) position, most of the participants adopted the 75° pitch and 0° roll (28.70%). For the flat position on the table, the participants equally performed 45° pitch and 0° roll, 60° pitch and 0° roll, and 75° pitch and 0° roll combinations. Fig. 4 and 5 respectively show the percentage of the finger pitch and roll combinations which occurred in the interaction with the tablet in tilt and flat positions on the table.

Fig. 6 shows the index finger pitch and roll combinations occurred in the interaction with the tablet screen in the cradle posture. We can observe the five combinations of finger pitch and roll angles that were most commonly used by the participants, namely: 60° pitch and 0° roll (21.81%), followed by 75° pitch and 0° roll (20.59%), 45° pitch and 0° roll (18.38%), 60° pitch and 30° roll (13.48%), and 75° pitch and 30° roll (9.80%).

Fig. 7 summarizes the occurrence of various index finger pitches and rolls angles performed by participants in their interaction with the tablet in all three postures observed in this study. We can see six combinations of finger pitch and roll angles that were most commonly used by participants, namely: 75° pitch and 0° roll (22.69%), followed by 60° pitch and 0° roll (20.68%), 45° pitch and 0° roll (17.59%), 60° pitch and 30° roll (12.96%), 90° pitch and 0° roll (9.72%) and 75° pitch and 30° roll (8.72%).

3.2. Index finger postures in the interaction with mobile phone

In the interactions with the mobile phone, the index finger can be seen to be used in the cradle posture. Fig. 8 shows the percentage of the index finger pitch and roll angles combinations used by the participants when touching targets on the mobile phone screen in the cradle posture.

We can see in Fig. 8 that the five most commonly used combinations of index finger pitch and roll angles for the interaction are: 60° pitch and 0° roll (29.17%), 45° pitch and 0° roll (16.67%), 75° pitch and 0° roll (15.28%), 60° pitch and 30° roll (9.72%) and 75° pitch and 30° roll (9.38%).

3.3. Index finger postures in the interaction with both devices

In Fig. 6 and 8, we can see that in the cradle interaction with the tablet or the mobile phone, the typical posture of the index finger adopted by the participants is the combination of 60° pitch and 0° roll angles. The other common postures are the combination of 75° and 45° pitches and 0° roll angles. Finally, it was also observed that there is no significant difference in the posture of the index finger adopted by the participants in the cradle posture when using both devices.

Fig. 9 shows the index finger postures most commonly used by the participants when touching the target on the screen of both devices. Fig. 9 allows us to conclude that for the whole observation, the participants performed the tasks by using their index finger in the 0° roll angles combined with 60°, 70°, and 45° pitches angles respectively.

3.4. Thumb postures in the interaction with mobile phone

Fig. 10 shows the thumb pitch and roll angle combinations used by the participants when interacting with the mobile phone screen.

Fig. 10 shows five combinations of pitch and roll angles of thumb most commonly used by participants in their interaction with the mobile phone, namely: 60° pitch and 0° roll (22.35%), 45° pitch and 0° roll (15.72%), 30° pitch and 0° roll (14.58%), 60° pitch and 30° roll (13.26%) and 45° pitch and 30° roll (7.77%).

It should be noted that two participants in this study used their middle fingers in the interaction with the tablet, performing combinations of 45° and 30° pitches and rolls respectively. They used these postures due to their long fingernails.

The current study found that the index finger postures commonly used by the participants are quite different from those observed in Holz's study [11] on the users' mental models of touch. Holz's study [11] found four finger pitch angles (65°, 45°, 25° and 15°) and five finger roll angles (-15°, 0°, 15°, 45° and 90°) from the exploration of the techniques used by the participants, which were included in his main study. This difference may occur given that in the current study, the participants were only asked to touch the targets freely regardless of the accuracy, while in Holz's study [11], the participants had to touch the targets accurately. Holz's findings suggest that users touch the targets precisely by aligning the finger's feature and outline [11].

The current study also completed our conclusions from the previous study [3] with a smaller number of participants. In both studies, the results showed that most of the participants performed the tasks by using their thumbs and index fingers in a combination of 60° pitch and 0° roll angles. The current study emphasized the conclusions of our previous study [3] in exploring the fingers and their postures, which will be included in our main study on fingertip contact area with touch screen devices.

4. Conclusion

This study is a preliminary observational study on the fingertip contact area with touch screen devices. The main purpose of this study was to understand finger postures when interacting with touch screen devices. It has been possible to conclude that the most common thumb and index finger posture used by participants in their interaction with the screen of a mobile device is a combination of 60° pitch and 0° roll angles. The study's results show that the female participants with long fingernails, touched targets on the mobile device screens by using their index or middle fingers, with a very low pitch. In sum, the findings of this study can help researchers determine the finger postures that should be included in the study of the fingertip contact area with touch screen devices.

Acknowledgments

This project has been funded with support from the European Commission under the scope of the AREAS Project of the Erasmus Mundus program. This publication reflects the views only of the authors, and the Commission cannot be held responsible for any use that may be made of the information contained herein.

This study also had the financial support of FCT-Fundação para a Ciência e Tecnologia of Portugal, under the project ID/CEC/00319/2013.

References

[1] Karlson, A.K., Bederson, B.B. and Vidal, J.L.C., Understanding single-handed mobile device interaction, Human-Computer Interaction Lab, University of Maryland, 2006.

[2] Hoober, S., How do users really hold mobile devices? [Online]. [Date of reference October 23^th of 2014], Available at: http://www.uxmatters.com/ mt/archives/2013/02/how-do-users-really-hold-mobile-devices.php

[3] Umami, M.K., Arezes, P.M. and Sampaio, A.M., Identifying finger postures when interacting with the touch screen of mobile devices. Proceeding of the International Symposium on Occupational Safety and Hygiene (SHO), 2015. pp. 415-417.

[4] Azenkot, S. and Zhai, S., Touch behavior with different postures on soft smartphone keyboards, Proceeding of the 14th international conference on Human-computer interaction with mobile devices and services (MobileHCI), 2012. pp. 251-260. DOI: 10.1145/2371574.2371612

[5] Google Developers. Page Speed Insights: Size tap targets appropriately [Online], [Date of reference March 26th of 2015], Available at: https://developers.google.com/speed/docs/insights/SizeTapTargetsAppropriately

[6] Windows Dev. Center. Motion and interaction for windows phone: Interaction and usability with windows phone [Online], [Date of reference March 26th of 2015], Available at: https://msdn.microsoft.com/en-us/library/windows/apps/hh202889%28v=vs.105%29.aspx

[7] iOS Human Interface Guidelines, UI Design Basics: Adaptivity and Layout [Online], [Date of reference March 26th of 2015], Available at: https://developer.apple.com/library/ios/documentation/UserExperience/Conceptual/MobileHIG/LayoutandAppearance.html

[8] Parhi, P., Karlson, A.K. and Bederson, B.B., Target size study for one-handed thumb use on small touchscreen devices, Proceeding of the 8th conference on Human-computer interaction with mobile devices and services (MobileHCI), 2006. pp. 203-210. DOI: 10.1145/1152215.1152260

[9] Park, Y.S., Han, S.H., Park, J. and Cho, Y., Touch key design for target selection on a mobile device. Proceeding of the 10th international conference on Human computer interaction with mobile devices and services (MobileHCI), 2008. pp. 423-426.

[10] Wang, F. and Ren, X., Empirical evaluation for finger input properties in multi-touch interaction. Proceeding of the SIG CHI conference on Human factors in computing system (CHI), 2009. pp 1063-1072. DOI: 10.1145/1518701.1518864

[11] Holz, C. and Baudisch, P., Understanding touch. Proceeding of the SIG CHI conference on Human factors in computing system (CHI), 2011. pp. 2501-2510. DOI: 10.1145/1978942.1979308

M.K. Umami, has a Sarjana Teknik (equal to BSc. Eng) degree in Mechanical Engineering and a MSc. degree in Industrial Engineering. He has been working at the Department of Industrial Engineering, School of Engineering, University of Trunojoyo Madura, Indonesia since 2004. He is currently a PhD student at the Department of Production and Systems, School of Engineering, University of Minho, Portugal. ORCID: orcid.org/0000-0002-1431-6060

P. Arezes, has a PhD. in Industrial and Systems Engineering from the University of Minho, Portugal, where he is currently a full professor in Ergonomics and Human Factors. He is also a visiting fellow at MIT's AgeLab in the USA. He leads the Human Engineering research group and coordinates the Engineering Design and Advanced Manufacturing (EDAM) area of MIT's Portugal Program at the University of Minho, where he is also chair of the Steering Board of the PhD program "Leaders for Technical Industries (LTI)". ORCID: orcid.org/0000-0001-9421-9123

Á.M. Sampaio, is an Industrial Designer, with a MSc in Design, Materials and Product Management, and a PhD in Polymer Engineering, he works on product design and development, more specifically, in the field of user-centered design, additive manufacturing and interaction design. He is a researcher in the HMIEXCEL project: Human Machine Interaction Excellence Project - Critical R&D in the framework of the development and production cycles of advanced multimedia solutions for automobiles, a partnership between the Bosch Car Multimedia Portugal, S.A. and the University of Minho. ORCID: orcid.org/0000-0002-8635-1682