Getting-up rehabilitation therapy supported by movement based interaction techniques


Palabras clave:

stroke rehabilitation, brain diseases, Kinect, movement interaction (en)


  • José Antonio Fernández-Valls Coordinadora Revistas Facultad de Minas
  • Victor M. R. Penichet Universidad de Castilla-La Mancha, Real, España
  • María Dolores Lozano Universidad de Castilla-La Mancha, Real, España
  • Juan E. Garrido Universidad de Castilla-La Mancha, Real, España
Every day the number of diseases related to brain problems increases dramatically. Medical facilities are full of people who need complex rehabilitation processes as a result of suffering from Parkinson's disease, Alzheimer’s disease, brain stroke, or multiple sclerosis, as well as other conditions. Patients often spend a long time travelling every day to get to the corresponding rehabilitation clinics. Physiotherapists carefully attend patients over a period of time, and the exercises are often extremely repetitive. The type of patient suffering from these type of illnesses is unable to perform many common every day actions such as getting-up from a chair, walking in a straight line, picking up objects, etc. The system we propose in this paper provides a way for patients to perform the common rehabilitation process of gettingup from a chair. It monitors and guides the patient when the specialist considers they are prepared to do so. The specialists can focus on the results and the way to improve each rehabilitation process and not just on iterations.


Getting-up rehabilitation therapy supported by movement based interaction techniques

Terapia de rehabilitación de levantarse guiada por medio de técnicas de interacción basada en movimiento


José Antonio Fernández-Valls a, Victor M. R. Penichet, María Dolores Lozano & Juan E. Garrido b


a Instituto de Investigación en Informática de Albacete (I3A), Universidad de Castilla-La Mancha, Albacete, España.
b Universidad de Castilla-La Mancha, Real, España. {Victor.Penichet, Maria.Lozano, Juanenrique.Garrido}


Received: February 18th, 2015. Received in revised form: March 24th, 2015. Accepted: September 30th, 2015.


This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Every day the number of diseases related to brain problems increases dramatically. Medical facilities are full of people who need complex rehabilitation processes as a result of suffering from Parkinson's disease, Alzheimer's disease, brain stroke, or multiple sclerosis, as well as other conditions. Patients often spend a long time travelling every day to get to the corresponding rehabilitation clinics. Physiotherapists carefully attend patients over a period of time, and the exercises are often extremely repetitive. The type of patient suffering from these type of illnesses is unable to perform many common every day actions such as getting-up from a chair, walking in a straight line, picking up objects, etc. The system we propose in this paper provides a way for patients to perform the common rehabilitation process of getting-up from a chair. It monitors and guides the patient when the specialist considers they are prepared to do so. The specialists can focus on the results and the way to improve each rehabilitation process and not just on iterations.

Keywords: stroke rehabilitation; brain diseases; Kinect; movement interaction.

Cada día aumenta el número de casos encontrados de enfermedades que afectan al cerebro y que necesitan tratamiento en un centro de rehabilitación, tales como el Parkinson, el Alzhéimer, los Ictus Cerebrales o la Esclerosis Múltiple. Algunas de ellas, suelen obligar al paciente a asistir a rehabilitación de forma ininterrumpida, lo cual exige desplazamientos diarios y la continua supervisión de un terapeuta. Muchos de estos pacientes necesitan reeducar alguno de los ejercicios básicos que se realizan en el día a día, como levantarse de una silla o caminar un tramo sin balancearse. Para ello, los terapeutas deben dedicar una cantidad de tiempo grande todos los días en comprobar que los pacientes realizan estos ejercicios de forma correcta. En este artículo, se presenta un sistema que monitoriza y guía en tiempo real al paciente en su proceso de rehabilitación para el ejercicio de levantarse de una silla.

Palabras clave: rehabilitación; interacción basada en movimiento; Kinect.


1. Introduction

Over time, the number of central nervous system diseases has been rapidly increasing. Most of them require a long period of rehabilitation to overcome or partially solve some of the problems they cause. The time devoted to this rehabilitation depends on each case, and it may even take the patient's whole lifetime: multiple sclerosis, and Parkinson's disease are clear examples of this. Furthermore, the kinds of exercises that patients have to perform require professional supervision. It takes a long time and a lot of manpower to properly attend every patient. The vast majority of specialized medical centers have long waiting times. The implementation of measures to reduce patient waiting times would improve patient satisfaction and patient rehabilitation. Specialists could increase the efficiency and accuracy of treatment.

Certain tasks took a long time to be performed in the past, but now they may be accomplished faster and automatically due to new technological advances. Technology is widely applied in the health field, although it is not particularly integrated into patient rehabilitation.

Due to the emergence of new interaction devices such as Microsoft Kinect, which allows user movement to be captured, systems have appeared which facilitate the rehabilitation process. These devices allow patients to interact with the application by simply using their body. They do not need to physically interact by manipulating any other hardware, which often proved to be too much trouble for them.

This paper presents a completely functional system for the rehabilitation of a patient performing one of the most common and frequent exercises: getting-up from a chair. The system makes use of movement-based interaction devices. It is focused on both the patient and the specialist. After a period of training together with the specialist, the patient may carry out part of the rehabilitation process at home. Unnecessary travel to healthcare facilities is avoided. The specialist may control the evolution of the patient via the application, and they do not need to be totally occupied on only one patient since the system monitors the patient for them. They are then free to accomplish many other tasks.

Thus, this proposal improves the quality of life for patients, and reduces waiting lists. These are both the main motivations for the implementation of the system we present in this paper.

The rest of the paper is organized as follows: Section 2 describes a number of related works and solutions that are found in this area. Section 3 provides a complete description of the system: The getting-up monitoring application that allows a patient to practice this frequent rehabilitation exercise. Section 4 presents the results obtained after the completion of two evaluations. Lastly, Section 5 provides some conclusions and future work.


2. Related work

The emergence of devices offering movement-based interaction in casual environments, such as video games, has created great interest with regard to their application in medical settings. Specifically, the rehabilitation processes, including physical exercise, is one of the main areas in which the research community has found a suitable environment in which to apply these techniques.

In the last few years we have being working on several functional movement-based interaction prototypes. Most of them were developed within the healthcare field: notably the automatic detection system for falls and fainting [2], and the balance disorder rehabilitation system [3]. The latter allows the patient to carry out the rehabilitation required that results from certain neurodegenerative disorders. Microsoft Kinect was used as the interaction device for these applications.

Regarding the systems that facilitate the rehabilitation process, a series of virtual games have recently appeared: the so-called serious games. Such applications attempt to motivate patients to carry out the rehabilitation process in a more effective, comfortable and friendly way [1,4,5].

In the following paragraphs, some of the current systems that use movement-based interaction devices for rehabilitation processes are described.

The first noteworthy related study is VirtualRehab [8], a new system for physical rehabilitation that uses the movement-based interaction provided by Kinect. The system allows the monitoring and tracking of patients from any location. The main objective of VirtualRehab is to offer patients an enjoyable way of completing complex rehabilitation processes at home. To this end, the equipment required comprises of a personal computer, Kinect for Windows and a screen. Additionally, VirtualRehab contains a management system that enables medical staff to plan, monitor and review each patient's progress. The system focuses on specific pathologies: acquired brain damage, neuromuscular diseases, and neurodegenerative diseases and reduced mobility in older adults. To treat these pathologies, VirtualRehab places patients in a virtual world in which they can work with nine games to perform specific movements in order to improve impairments (Fig. 1). These games make it possible to work with the affected parts of the body and physical symptoms, particularly regarding loss of motor ability, movement and posture disorder, and balance and coordination disorder.

SeeMe [7] is another major study on rehabilitation, but in this case, it corresponds to the assessment and treatment of unilateral spatial neglect. This project represents another system for rehabilitation that creates a virtual reality environment without the requirement for head-mounted displays or specialized equipment. SeeMe is a projected video capture of a virtual story in which the patient is embedded through their image. A representation of the patient is generated by capturing the patient performing the activities through a camera and by using specific algorithms for movement and position recognition and analysis. The participants should stand or sit in a specific area and view a monitor in which their virtual representation is shown during the exercises inside the virtual world. For example, patients can be embedded in a game in which they are touching virtual balls. Therefore, the participants can see themselves in a virtual story using trunk and limb movements. Additionally, the medical staff are able to change parameters of the virtual game (based on system indications and staff perceptions) during the rehabilitation procedure in order to adapt exercises and features according to patient progress and needs. In this way, SeeMe generates a set-up where patients and medical staff are together while the patient is working at the medical center.

Lastly, the Reflexion Rehabilitation Measure Tool (RMT) [9] is a Kinect-based system that allows the patient to perform rehabilitation exercises at home. The system gathers patient information, which is then sent to the specialists. It can then be analyzed and the therapy can be changed if necessary. Interactive feedback is provided to the patient. In this way, they receive entertainment while they properly perform the exercises. An assistant character guides the patient in their performance, and gives them some clues as to how to proceed. A colored shadow represents the patient instead of their real image.

The main difference between the system we propose and those mentioned above is the user interface and some key points regarding the interaction. The proposal we present shows the real image of the patient, even if it is sometimes altered. Patients may watch their real body on the screen, and this provides feedback about what they are doing incorrectly. Thus, they can be made aware of what they are not getting right, so they can change any incorrect postures as they can see themselves. They are able to see exactly what is wrong. They can see exactly where something is wrong. Providing the real image of patients in the interaction may reduce the digital gap for people who are not used to these kinds of applications. Otherwise, the avatars may cause people to make mistakes and to not feel comfortable. Our proposal is not a serious game, but an application for rehabilitation. It is not designed in terms of how games are designed, although some techniques and elements may be found entertaining. It is a real rehabilitation exercise, and patients should carry it out in the same way they would do if a specialist were with them.

VirtualRehab proposes non-functional exercises, i.e. exercises that are not based on real ones. They are not typical rehabilitation exercises, but specific movements that patients should accomplish. Obviously, these movements are not random but are thought to be useful in the rehabilitation process. Current rehabilitation processes tackle daily and common situations, such as getting-up from a chair. This is exactly what we intend to focus on.

The SeeMe System provides another way to carry out virtual exercises, but always under the supervision of a specialist. Our proposal allows patients to perform exercises at home without the need to be continuously supervised by a specialist. The system monitors a patient's movements, and gives them feedback about how to properly proceed. Important information is recorded in every session, and sent to the specialist to study the patient' evolution. It does require some training. The specialist decides if and when a patient is prepared to perform exercises at home, which is something that is already done in many cases with some basic rehabilitations.

RMT does not provide any kind of customization, or even personalization according to patient needs and evolution. Patients are always different, even if they suffer from the “same” problem. They evolve in very different ways. Some suffer from amputated limbs, reduced mobility, partial disability, etc. The system we propose considers these restrictions, and may adjust some parameters individually in accordance with every patient's circumstances.


3. Getting-up learning exercise

People with brain damage have disorders that hinder their daily life. They can be limited when walking, picking up objects, climbing stairs, etc. These common activities are natural and simple for people without brain handicaps, but those with a neurodegenerative disease or brain damage find them to be an important obstacle. Furthermore, as a consequence of certain diseases, patients must attend a rehabilitation center for long periods of time, in some cases their whole life. The rehabilitation process could be very burdensome for patients and their therapists, because patients have to perform a set of repetitive exercises and therapists may be required to conduct these exercises in order to correct them properly. This, together with the increase in diseases, which forces patients to attend rehabilitation processes, causes an increase in waiting times to obtain a place at one of these centers.

In this regard, the authors have developed a system, the main goal of which is to offer an aid to be used at rehabilitation centers or in the patients' homes in order to help patients in their specific disabilities.

Despite the large number of exercises that the cited patients can perform, the system developed focuses its efforts on one specific and concrete exercise. The exercise consists of guiding people with brain damage to improve or to learn how to get up from a chair. To that end, the system helps patients in the evolution of the exercise by continuously analyzing their postures and movements and guiding them. The patient receives indications to know how to preform and complete each of the steps. In this way, the system uses Kinect to analyze postures and movements, as is shown in Section 3.1.Via a screen, the user is shown the system interface, which contains appropriate indications, corrections and complementary information.

The system is an auxiliary tool that benefits all the main groups of people involved: the patients and their therapists. The patient can perform the exercises in their own home, thus avoiding a trip to the rehabilitation center. Regarding therapists, they find that the system offers them a tool that gives them more time and freedom to perform their work.

The next subsections describe the functionality and the foundations of the system, explaining how they work, how the patients interact with the system and how the system itself guides the patients via the interface.

3.1. Foundations and deployment of the system

Movement interaction is the main technological foundation of the proposal. Related devices help position and movement detection through continuous interaction with users who are in their vision area. They offer the possibility of analyzing each situation and reacting appropriately. However, a more important fact is that movement interaction offers the possibility to control the system by means of the users' natural movements, such as hand gestures and body postures.

Kinect was the device used to work with movement interaction in order to avoid interfering in the rehabilitation process. This device makes it possible to monitor and correct a patient automatically, while performing the rehabilitation exercises. Kinect has been chosen due for two key reasons. Firstly, Kinect was readily available to the authors and its cost is not too high in comparison with other devices, without sacrificing essential capabilities. And, secondly, this Microsoft device has already been used in several medical projects. Therefore, its integration in the field has been considerable; this allows us to know that it has a solid foundation.

The objective of the system is to allow the patient's posture and movements to be checked during the exercise without interruption. Posture and movement identification is performed with the use of the development kit that Microsoft provides. This process is a key element, so explaining how the system performs the identification is essential. The SKD allows for the recognition of all the body joints that Kinect is able to identify: 20 to be precise. These translate to points that are provided to the system through a class that represents the skeleton and offers the user access to them. Once the system obtains this class, the next step is to search for those points, by collecting their position in the obtained parameters that are necessary to identify correct postures. Each position is stored in a specific class that is made up of a set of three values that refer to the exact position of the point in space (x, y and z axes). Once the positions have obtained this representation, the detection of the patient's posture and movements starts while s/he is sitting on the chair.

The detection procedure is based on the consideration that there are pairs of body parts that may be aligned with a low error rate during some time periods. The following subsection describes the steps to be performed by the patient and which are the body points that have to be aligned in each step. For example, the points related to the shoulders have to be at the same height when the user starts the exercise and is sitting down; otherwise, the patient is badly positioned to start the exercise.

The deployment needed to achieve the objective of the system is composed of two main components. The first is that the server is responsible for holding information about the patients, the supervision of the exercises, the exercises themselves, and the relationships between these elements. This information is needed in order to adapt the interface and the conditions of each exercise in a personalized way. To that end, the Server offers this information through a set of Web Services. The second three components make up the deployment part, related to the place where the exercise is to be carried out: the Kinect device, a personal computer or laptop, and a screen. The deployment of the system is simple, and, if it is deployed in a patient's home, they can deploy it autonomously. The camera is connected to the personal computer or laptop on which the framework is executed. The Kinect device may be located in front of the chair that users will use during the exercise. At the beginning of the exercise, the system will display a line on the screen where the chair should be correctly positioned. And, finally, the screen is an essential element, showing the system interface. It is recommended to place the screen just above Kinect in order for the user to see their own image correctly. The interface guides the patients as an auxiliary tool by means of advice, corrections and information notes. The interface is described in depth in Section 3.4 due to its importance in the completion of the exercise.

3.2. Exercise description

This exercise is based on the interaction of a person moving from the sitting posture to the standing posture. To perform this exercise it is necessary to have a chair. The actual exercise in rehabilitation therapies is divided into several parts to be performed consecutively and in order:

  • Initial position: The patient should be seated without being tilted to one side. The back should be straight, forming an angle of ninety degrees with the thighs. The legs should be separated to the width of the shoulders. The knees should form an angle of ninety degrees.
  • Step 1: The patient should move their feet back, behind the knees. The space between the legs should be the same as in the initial position. The patient must keep the back straight, similar to the initial posture.
  • Step 2: The patient must move their arms forward, keeping the previous position. At the end, the hands must be ahead of the knees and approximately at chest height, regardless of the separation between them.
  • Step 3: The patient should tilt his back slightly forward (approximately forty five degrees), keeping to the restrictions of the previous two steps.
  • Step 4: At this moment, they should start to rise until achieving the standing posture. It is important that the patient does not rock while rising. The therapist should also check that the patient does not put their hands down during this step, as it will be essential to correct their balance when standing up.
  • Final position: The person should be standing with their back straight and their arms close to the body and feet to shoulder width.

After finishing a repetition, the patient should sit back and start the exercise again, until completing all the repetitions that their therapist has assigned them.

3.3. Overall functionality

Each step of the real exercise has been adapted to the program through states: a scan be seen in Table 1. Different states involve simple and essential actions that can be performed by people without any brain or degenerative damage. However, patients suffering from these types of disorder have difficulty in several situations, which makes their life more difficult. In this way, the system follows five simple steps to facilitate the instructions and to help patients to perform the activity.

Each state checks if the users can achieve the following state. In the case of every requirement being satisfied, the users progress to the next state. However, if they do not manage to satisfy the requirements, the system informs them about their mistakes and what they should do to correct them. The users can stay in the same state or they can return to a previous one, depending on the exercise restrictions. With regard to the movement and position analysis, which was described in section 3.1, the system knows the right indications which must be shown to the patient as well as telling them the right moment to do it.

There are two different ways to carry out the exercise: restrictively and permissively. The first is focused on patients who are learning the exercise, so when they fail in one of the previous states, the system returns to this state automatically. The restrictive way, however, will be used by patients who have previously done the exercise several times, and therefore know the way the system operates. In this case, although some mistakes will have been made by the users in one of the previous steps, the system will advise them and allow them to go on with the task without having to go back.

Each time the steps are finished successfully, the system shows, through an instructions area, the following actions that must be performed in order to perform the next step. At the same time, in the animation area, there is a guide picture in which the same instruction is performed.

The program includes several sounds and voice orders in order to help the user to perform the exercise correctly. After checking the state, if the step has been completed, a congratulation sound will be emitted and an animation picture will appear at the bottom of the screen in the instructions area. Apart from the text pointing out the next action to perform, the users can choose to select the option of a voice that reads what is written in the instructions area. In the case of the users making a mistake, a sound will be emitted to make the users aware of it. The sound and voice system will be described in greater detail in Section 3.5.

3.4. User Interface

The system guides patients in their training and learning to get up from the chair by following each step (previously described) through an intuitive interface. The analysis carried out by the system, using the Kinect device, defines the interface behavior. The system continuously checks the posture and movement of the patient, as described in Section 3.1. The results obtained allow the system to know if the patient has performed any incorrect movement in order to create adequate interface indications to help with the completion of the exercise.

The system's indications represent the main assistant element for patients and physiotherapists. The assistant helps during the execution of the exercise in a way that avoids the need for physiotherapists to physically be at the location where patients perform the therapy. In this way, the interface represents an essential component to be able to create a rehabilitation environment in which patients are corrected in the most natural possible way.

Three main areas make up the system interface: shown by Fig. 1. The broad area can be divided into the following components:

The first area indicates the action to be performed by the patient. To that end, the textual area shows the instructions to follow and uses a clear and large font. This component will be essential to guide users during the exercise, as it will show the next step if the current one has been accomplished, or a correction to be made to achieve successful completion.

The second area shows an animation in the form of a guide puppet that helps the instructions area to complete the feedback patients need to perform the exercise. The animation shown corresponds to the next action the patient has to perform, which is simultaneously indicated in the textual instructions area.

The main area covers the rest of the interface with the video signal offered by Kinect. The signal contains the patient working on the therapy with additional visual content. This information is essential because it indicates how the patients are performing the exercise, and then it allows them to check errors and to solve them. The main area can be divided into different parts, according to the information offered. Firstly, this area contains some animations, used to give indications for correction, and also to congratulate when the steps, or the whole exercise, are completed.

The correction animation is designed to indicate which body parts are badly positioned through a semi-transparent red area located in the related joints. As a result, the patients will be able to know where they are failing. Figs. 1-6 show an example in which a patient has positioned their feet incorrectly. In addition, the figures shows another correction using scales located in the upper area. This element represents the position of the patient's' back, in respect to the X axis. The scales lean to the left/right if the patient twists their back to the left/right, and change color indicating the need for the user to correct their posture. Otherwise, if the patient makes a short twist movement, the scales slightly change position (not color): enough to indicate the unbalanced movement of the patient. This element is essential due to the importance of positioning the back in a straight way during each stage of the exercise. Red indications appear when an error is found, following the standard procedure of associating red with errors. Continuing with the same area, a congratulations message appears when a step is performed correctly. The correct area is completed with two sets of circles.

The first one is located on the left side and represents the number of attempts made through ellipses that change color with an animation when the user makes progress. The other one is located in the bottom area of the interface and indicates the progress of the whole exercise. This area is responsible for notifying the user in which step of the exercise they are. Just like the attempts, with a set of animated ellipses, the system shows the patient's progress. In this case, each ellipse contains an internal image that represents the related step.

Finally, the system shows a box with errors committed by the patient when an attempt is completed and highlights those that appear more than once (Figs. 2-7).

3.5. Sound system

The system contains a set of functions to offer users feedback using voices and sounds. To that end, the user should enable the related option located that is located in the configuration window (Section 3.6).

The patients who make use of the system are people from a wide age group, from children through to older people. The percentage of people with visual problems increases with age: 30% for people between 15 and 44, increasing to 100% for people older than 45. Therefore, the system needs to take into account that many users will have visual problems. The use of sound and issuing voice instructions becomes, therefore, an essential element.

The system generates a light error sound each time an incorrect posture is detected, in order to notify patients of what has occurred. In addition, the patient can enable the option to be notified by voice messages by using the configuration windows. The source of each error is analyzed with the aim of avoiding constant error repetitions. If the source is the same as for the last one, the system will not emit the error sound until the patient makes an error with a different source.

At the same time, the patient hears sounds when they complete an attempt or the whole exercise. This information provides the user with positive feedback so that they know when a step is correctly performed, without the need to focus on the visual animations.

The voice system helps patients by dictating the phrase that appears in the instructions area. The system checks the phrase to know if it is different to the previous one, in order to avoid constant repetitions, as occurs with the error sounds. In this case, the system considers a margin of 5 seconds before repeating the instructions, if the current and the last one are identical. In this way, the system avoids constant repetition of the same instruction. If the user performs the current instruction during its dictation, the sound is cancelled and the system starts the dictation of the next one. The sound system, with the visual reinforcement of the instructions area and the animations area, guides the user during the completion of the exercise and overcomes the possible doubts that the patients may have, relating to either incorrect movements or to the next step to be completed.

3.6. Exercise settings

Thanks to the system, patients and physiotherapists have the possibility to modify some options to configure exercises in a personalized way. The system offers three different ways to configure an exercise.

The first option is the possibility to turn on the sound during exercise, as has been mentioned in the previous section. This option will be available to patients and physiotherapists.

The next option allows setting the correction mode that the exercise will follow: permissive or restrictive mode. A description of each of these methods is provided to facilitate the choice. This option will only be available to the therapist.

And, finally, the system allows changing the difficulty of the exercise and enabling or disabling joints in the body. The system has been created to be useful for any patient. For example, if a patient has an immobilized arm, the system will always detect errors because of the checks. In this way, the system is able to work with threshold values of difficulty at any check step by modifying the body parts involved.


4. Evaluation

In this section we present the tests performed by a set of users throughout the different stages of the system's development.

The first tests were performed with five users without previous experience of the project, and they were focused on finding problems related to usability and functionality.

The second tests were performed by two experts in rehabilitation, and were based on finding out the level of usefulness of the application and fixing the errors found in the design of the different exercises. Below, we show the results obtained from these tests and the changes to be undertaken. Finally, we summarize the conclusions reached after the performance of the tests, highlighting the most important points extracted from both evaluations.

4.1. First evaluation

The set of the tests were performed on five users with different ages, two teenagers with a high level of knowledge in new technologies, an adult person with a medium level of knowledge in new technologies and two older people with inadequate or no knowledge of technology. None of them had previous experience in using the system.

The number of users selected to perform the test (five) is enough to discover 85% of the problems and pitfalls, according to the usability expert, Jakob Nielsen [6].

The tests were performed in a 7x4 meter room with a 24” screen as visualization device. A Kinect device was placed just below the screen, at a height of one meter above the ground.

In order to check whether the interface elements and the program functionality were intuitive enough, the users were not informed previously about how they should perform the rehabilitation exercises. They were just informed about the purpose of the exercise, and we analyzed the time they required to finish the exercise as well as the problems they had in performing the exercise.

If a user got stuck while performing the exercise, they were given some instructions to help them proceed. When they finished the first repetition of the exercise, they were required to identify each one of the user interface elements and the functionality they thought these elements had.

After gathering this information, both the functioning of the system and the user interface elements were explained to them. Once they knew how to perform the exercise, they were required to repeat it four more times, and they were given help if they had any difficulty.

Some changes were made to the system based on the feedback gathered from this evaluation. The most important ones were the following:

  • Modification of the color of the scales, so that just the wrong half of was highlighted in red instead of highlighting the whole.
  • Most of the commands included in the instruction box were rewritten to make them easier to understand.
  • A voice and audio system was implemented to support the performance of the exercise, in order to help patients that can hardly read the commands displayed on the screen.
  • The dummy guiding the exercise was modified to make it bigger and more intuitive.
  • The color tone of the bones corrector was modified to make it semi-transparent so as not to interfere with the exercise.
  • The content of the steps panel was modified to give better feedback to the user regarding the current step they are performing.

4.2. Second evaluation

This set of tests was performed with two experts in rehabilitation therapies who were collaborating with the project: a physiotherapist and an occupational therapist. The purpose of these tests was to check the actual usefulness of the system, as both therapists know well the behavior patterns and limitations of rehabilitation patients. In this way, they could reproduce the performance of the exercises to check that the system really meets all the requirements.

These tests were focused on checking the correct functioning of the system and gave less importance to the user interface. Both therapists performed the exercises imitating the behavior of a real patient. They also checked that every step performed in the exercise matched the previously programmed one, and that the system worked properly in terms of identifying the different steps.

While performing the evaluation, the therapists realized that the exercise did not adapt to the possibility of a patient having a disability or impairment in any of their limbs. If they simulated an idle arm, the exercise failed in the first verifications and it was impossible to finish the exercise, leaving the user stuck in one of the steps.

To solve the problem of users with disabilities or impairments, we modified the design of the system in such a way that the therapists could choose which parts of the body the system should focus on in every step and the degree of freedom of each part should have.

Another aspect of the system that was changed was giving the patient the possibility of avoiding showing their body on the screen, and, instead, substituting it with an abstract human figure.

The experts also proposed showing more information on the screen related to the failures committed by the users during the execution of the exercise. The problem with this was mainly cluttering the screen with too much information that could distract the user from the exercise. Instead, we proposed the implementation of a software module to store the patient's statistics and failures during the exercise.

4.3. Evaluation conclusions

The first evaluation allowed us to improve the system in terms of simplicity and clarity of the user interface, making it more intuitive and easy to use. In addition, we were able to detect and fix the different kinds of problems found in every age range.

In general, the users of the first assessment concluded that the system was easy to use and comfortable. Thanks to the feedback comments received to improve the system we were able to fix most of the problems found during the assessment.

Regarding the heuristic evaluation and according to the therapists' opinions, they concluded that the system would have a high level of acceptance within the sector, as it would be really useful for the majority of patients following rehabilitation therapy. It was claimed that 90% of patients could use it without any problem.

The experts' general conclusions were that the system could be a very useful element in performing their work as it allows patients to perform the exercises in a correct way and with the information needed in case they are primarily not able to do them properly. More time could, therefore, be dedicated to other patients.


5. Conclusions and future work

In this paper, we have presented a system to support the rehabilitation therapies of patients with neurodegenerative problems and brain disorders when they need to perform common exercises, such as sitting down or getting up. We have developed a system to support one of the most commonly used exercises for the rehabilitation of disorders affecting the nervous system. This exercise has been designed exactly as it is performed in real life, including additional elements to allow the patient to perform the exercise individually without the therapist.

Apart from the exercises implemented in the system, we have also developed a system that allows the personalization of the exercises, so as they can be adapted to all types of patients.

With this system, and after a minimal training set by the experts, patients can perform the exercises by themselves. In this way, the system offers several benefits from both the point of view of the physiotherapists and the patients. Physiotherapists thus have a useful tool through which they can help patients without the need to be constantly present during the execution of the exercise.

Thus, physiotherapists may have more time to perform their tasks in a more efficient way. Additionally, patients can perform the exercises at the health center or even at home, which makes the rehabilitation process easier to follow by avoiding problems regarding time and distance; the patients can perform the exercises at any time and without the need for daily trips to the rehabilitation center.

The system acts as a personal assistant that guides patients during the rehabilitation process. In particular, the whole process is composed of a set of five states that are divided into three stages. The first stage is being seated on a chair and has two conditions: keeping the legs at a right angle and maintaining a straight back. The second stage consists in lifting the body, moving the back forwards and stretching the arms out. Finally, the patient should be able to get up autonomously.

They system offers enough indications to help patients complete the exercises correctly. The system continuously analyzes the postures and movements of the user. The system's indications and feedback depend on the result of this analysis. In this sense, the user interface becomes a key element in the system, as all its components have to be as clear and intuitive as possible in order to act as a guide for the exercise.

The interface includes textual indications, the current video signal, examples of how to perform each step in the exercise, and finally, images about corrections of postures and movements. Additionally, the interface offers animations to congratulate the correct performance of the exercise to encourage the patient.

Kinect is the device that allows movement-based interaction with the system. This device allows patients to interact with the system in a natural and intuitive way through postures and movements. In this paper we have shown how Kinect can be applied in health centers to solve a specific problem.

The system proposed presents a very promising future application. Firstly, we are working on the improvement of the user interface to make the interaction between the patient and the system as natural as possible.

Regarding the conclusions reached with the system evaluation (see section 4.3), we plan to create a new visualization mode that allows the user to choose an avatar to represent them in a virtual setting. In this way, we will solve the problem that some patients have when they see themselves on the screen, feeling uncomfortable with this situation.

Furthermore, we have the possibility of creating a system to store the statistics and failures committed by the patients while executing the exercise. In this way, the physiotherapist can control the patients' rehabilitation evolution more efficiently.

Finally, we are considering more exercises to include in the system. The system described in this paper is composed of a unique but complete exercise used as a treatment for specific brain disorders. The goal is to offer different solutions for different health problems by supporting the execution of a broad set of rehabilitation exercises.



This work has been partially funded by project TSI-100101-2013-147, belonging to the Spanish Ministry of Industry, Energy and Tourism. We also want to thank Carlos Gimena Bello for his work on this application.



[1] Fikar, P., Schoenauer, C., and Kaufmann, H., The Sorcerer's apprentice: A serious game aiding rehabilitation in the context of subacromial impingement syndrome. In Proceedings of the 7th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth '13). ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering), Brussels, Belgium, Belgium, pp. 327-330, 2013. DOI: 10.4108/icst.pervasivehealth.2013.252224

[2] Garrido, J.E., Marset, I., Penichet, V.M., Lozano, M.D. and Fernández, J.A., Automatic detection of falls and fainting. Journal of Universal Computer Science, 19 (8), pp. 1105-1112, 2013. DOI: 10.3217/jucs-019-07

[3] Garrido, J.E., Marset, I., Penichet, V.M. and Lozano, M.D., Balance disorder rehabilitation through movement interaction. In:Proceedings of the 7th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth '13). ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering), Brussels, Belgium, Belgium, pp. 319-322, 2013. DOI: 10.4108/icst.pervasivehealth.2013.252368

[4] Lange, B., Koenig, S., McConnell, E., Chien-Yen, Ch., Juang, R., Suma, E., Bolas, M. and Rizzo, A., Interactive game-based rehabilitation using the Microsoft Kinect. In Proceedings of the 2012 IEEE Virtual Reality (VR '12). IEEE Computer Society, Washington, DC, USA, pp. 171-172, 2012. DOI: 10.1109/VR.2012.6180935

[5] Lozano-Quilis, J.A., Gil-Gómez, H., Gil-Gómez, J.A., Albiol-Pérez, S., Palacios, G., Fardoum, H.M. and Mashat, A.S., Virtual reality system for multiple sclerosis rehabilitation using KINECT. In Proceedings of the 7th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth '13). ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering), Brussels, Belgium, Belgium, pp. 366-369, 2013. DOI: 10.4108/icst.pervasivehealth.2013.252208

[6] Nielsen, J., Why you only need to test with 5 users. [Online]. Available at:

[7] Sugarman, H., Weisel-Eichler, A., Burstin, A. and Brown, R., Use of novel virtual reality system for the assessment and treatment of unilateral spatial neglect: A feasibility study. Proceedings of the 2011 International Conference on Virtual Rehabilitation; Zurich, Switzerland. New York: IEEE; 2011. DOI: 10.1109/ICVR.2011.5971859

[8] Virtualware Group., 2013. [Online]. Virtual Rehab. Available at: [Accessed: 04-10th-2014]

[9] Westhealth Institute., Reflexion rehabilitation measure tool. [Online]. [Date of reference: April 06th of 2014]. Available at:


J.A. Fernández-Valls, received his MSc. in computer science from the Universitiy of Castilla-La Mancha, Spain. He has been a PhD. student at the University of Castilla-La Mancha in Albacete since 2013. He belongs to the Interactive Systems Everywhere (ISE) research group ( at the University of Castilla-La Mancha. His research interests include: Human-Computer Interaction (HCI), movement-based interaction, and interactive systems development. ORCID: 0000-0003-2466-3653

V.M.R. Penichet, has been an associate professor at the Department of Computer Systems at the University of Castilla-La Mancha, Spain since 2007. He belongs to the Interactive Systems Everywhere (ISE) research group ( at the University of Castilla-La Mancha. His current research is based on Human-Computer Interaction (HCI) and Distributed User Interfaces (DUI). He is author of nearly 100 publications in journals and international conferences. He is involved in various research projects. He has previously organized several workshops and conferences with substantive output (Springer books and JCR Journals): DUI 2011-2014, Interacción 2008, CADUI 2008, REHAB 2013, 2014. ORCID: 0000-0003-1125-9344

M.D. Lozano, has been an associate professor at the Department of Computing Systems at the University of Castilla-La Mancha in Albacete, Sapin, since 2003. She received her MSc. and a PhD. in computer science from the Technical University of Valencia, Spain. She co-directs the Interactive Systems Everywhere Research Group (ISE Research) of the Albacete Research Institute of Informatics. Her teaching and research areas cover software engineering and human computer interaction. She is author of more than 80 papers in journals, book chapters and international conferences. She is guest editor of some special issues in journals indexed in the Journal Citation Report. She has co-chaired some technical conference programs in the field of human-computer interaction, and she is a member of several Program Committees of different national and international conferences. Her research interests include interactive systems development, Model-Based User Interfaces, tangible interfaces, usability and collaborative environments. ORCID: 0000-0003-4069-2112

J.E. Garrido, was born in Albacete, Spain on April 24th, 1984. He received his PhD. in Computer Science at the University of Castilla-La Mancha, Albacete, Spain in 2014. His research interests include ubiquitous computing, context-awareness, collaborative environments and HCI. In particular, he is involved in the improvement of healthcare environments with current technology based on the above-cited topics. He worked as a research assistant at ReTICS (Real-Time and Concurrent Systems) from 2008 to 2009, when he started with the ISE Research group as a PhD student. ORCID: 0000-0003-4854-0914

Cómo citar


J. A. Fernández-Valls, V. M. R. Penichet, M. D. Lozano, y J. E. Garrido, «Getting-up rehabilitation therapy supported by movement based interaction techniques», DYNA, vol. 82, n.º 193, pp. 241–249, sep. 2015.


Fernández-Valls, J.A., Penichet, V.M.R., Lozano, M.D. y Garrido, J.E. 2015. Getting-up rehabilitation therapy supported by movement based interaction techniques. DYNA. 82, 193 (sep. 2015), 241–249. DOI:


Fernández-Valls, J. A.; Penichet, V. M. R.; Lozano, M. D.; Garrido, J. E. Getting-up rehabilitation therapy supported by movement based interaction techniques. DYNA 2015, 82, 241-249.


Fernández-Valls, J. A., Penichet, V. M. R., Lozano, M. D. y Garrido, J. E. (2015). Getting-up rehabilitation therapy supported by movement based interaction techniques. DYNA, 82(193), 241–249.


FERNÁNDEZ-VALLS, J. A.; PENICHET, V. M. R.; LOZANO, M. D.; GARRIDO, J. E. Getting-up rehabilitation therapy supported by movement based interaction techniques. DYNA, [S. l.], v. 82, n. 193, p. 241–249, 2015. DOI: 10.15446/dyna.v82n193.53500. Disponível em: Acesso em: 24 jul. 2024.


Fernández-Valls, José Antonio, Victor M. R. Penichet, María Dolores Lozano, y Juan E. Garrido. 2015. «Getting-up rehabilitation therapy supported by movement based interaction techniques». DYNA 82 (193):241-49.


Fernández-Valls, J. A., Penichet, V. M. R., Lozano, M. D. y Garrido, J. E. (2015) «Getting-up rehabilitation therapy supported by movement based interaction techniques», DYNA, 82(193), pp. 241–249. doi: 10.15446/dyna.v82n193.53500.


Fernández-Valls, J. A., V. M. R. Penichet, M. D. Lozano, y J. E. Garrido. «Getting-up rehabilitation therapy supported by movement based interaction techniques». DYNA, vol. 82, n.º 193, septiembre de 2015, pp. 241-9, doi:10.15446/dyna.v82n193.53500.


Fernández-Valls, José Antonio, Victor M. R. Penichet, María Dolores Lozano, y Juan E. Garrido. «Getting-up rehabilitation therapy supported by movement based interaction techniques». DYNA 82, no. 193 (septiembre 1, 2015): 241–249. Accedido julio 24, 2024.


Fernández-Valls JA, Penichet VMR, Lozano MD, Garrido JE. Getting-up rehabilitation therapy supported by movement based interaction techniques. DYNA [Internet]. 1 de septiembre de 2015 [citado 24 de julio de 2024];82(193):241-9. Disponible en:

Descargar cita

CrossRef Cited-by

CrossRef citations4

1. Alberto Mora Plata, Juan E. Garrido, Jose A. F. Valls, Victor M. R. Penichet, Maria Dolores Lozano. (2016). Looking for an Adequate Monitoring Mechanism for Rehabilitation Systems Based on Movement Interaction. Proceedings of the XVII International Conference on Human Computer Interaction. , p.1.

2. Juan E. Garrido, Victor M. R. Penichet, Maria Dolores Lozano, Alberto Mora Plata, Jose A. F. Valls. (2019). The use of joint coordinates to monitor patients in a movement-based interaction system. Universal Access in the Information Society, 18(1), p.3.

3. Víctor M. R. Penichet, María D. Lozano, Juan E. Garrido, Félix Albertos-Marco, Raymond Bond, Maurice D. Mulvenna. (2022). Designing postures for rehabilitation therapies in a multimodal system based on a 3D virtual environment and movement-based interaction. Multimedia Tools and Applications, 81(30), p.44445.

4. Juan E. Garrido, Víctor M. R. Penichet, María D. Lozano. (2020). A novel context-aware system to support healthcare environments. Universal Access in the Information Society, 19(1), p.17.



Visitas a la página del resumen del artículo



Los datos de descargas todavía no están disponibles.