Improving L2 Oral Accuracy and Grammatical Range Through Self-Assessment of Video Speech Drafts

Mejoramiento de la precisión y rango gramatical oral a través de la autoevaluación de versiones de videos

This action research study examines the effect of self-evaluation on grammatical range and grammar accuracy on the English speaking performance of 27 foreign language university and pre-university students enrolled in three different language centers, in three different cities in Colombia. Participants were asked to record themselves answering different IELTS-type tasks for four times during a 6-week period and score and reflect towards their performance using IELTS-type analytical scoring rubrics and journals. Researchers used journals to register impressions, thoughts, and judgments observed during the process. The findings led to conclude that learners highly benefit from applying self-assessment techniques using videos of their production and a language benchmark to compare with in the improvement of their oral language accuracy and grammatical range.

El presente estudio de investigación acción examina el efecto que tiene la autoevaluación sobre el rango y la precisión gramatical en la producción oral de 27 estudiantes universitarios y preuniversitarios matriculados en tres diferentes centros de idiomas de Colombia. A los participantes se les pidió que se grabaran, calificaran y evaluaran mientras respondían diferentes tareas similares a aquellas del examen IELTS cuatro veces, durante un período de seis semanas, implementando rúbricas de calificación tipo IELTS. Por su parte, los investigadores usaron un diario para registrar impresiones, pensamientos y juicios observados durante el proceso. Los hallazgos llevaron a concluir que los estudiantes se benefician enormemente de la aplicación de técnicas de autoevaluación, utilizando videos de su producción oral y un referente lingüístico comparativo en la mejora de la precisión del lenguaje oral y rango gramatical.

Martin Javier Caicedo Pereira*
British Council, Pereira, Colombia

Jhonny Alexander Lozano Bermúdez**
Universidad de Ibagué, Ibagué, Colombia

Luis Alfonso Vanegas Medina***
Universidad Surcolombiana, Neiva, Colombia

*martin.caicedo@oup.com
**jhonny.lozano@unibague.edu.co
***vanegasluisalfonso@gmail.com

How to cite this article (APA, 6th ed.):
Caicedo Pereira, M. J., Lozano Bermúdez, J. A., & Vanegas Medina, L. A. (2018). Improving L2 oral accuracy and grammatical range through self-assessment of video speech drafts. Profile: Issues in Teachers’ Professional Development, 20(2), 127-142. https://doi.org/10.15446/profile.v20n2.61724.

This article was received on December 23, 2016 and accepted on January 31, 2018.

This is an Open Access article distributed under the terms of the Creative Commons license Attribution-NonCommercial-NoDerivatives 4.0 International License. Consultation is possible at https://creativecommons.org/licenses/by-nc-nd/4.0/.

---

Key words: Grammatical range, oral language accuracy, self-assessment, videos.

---

El presente estudio de investigación acción examina el efecto que tiene la autoevaluación sobre el rango y la precisión gramatical en la producción oral de 27 estudiantes universitarios y preuniversitarios matriculados en tres diferentes centros de idiomas de Colombia. A los participantes se les pidió que se grabaran, calificaran y evaluaran mientras respondían diferentes tareas similares a aquellas del examen IELTS cuatro veces, durante un período de seis semanas, implementando rúbricas de calificación tipo IELTS. Por su parte, los investigadores usaron un diario para registrar impresiones, pensamientos y juicios observados durante el proceso. Los hallazgos llevaron a concluir que los estudiantes se benefician enormemente de la aplicación de técnicas de autoevaluación, utilizando videos de su producción oral y un referente lingüístico comparativo en la mejora de la precisión del lenguaje oral y rango gramatical.

Palabras clave: autoevaluación, precisión del lenguaje oral, rango gramatical.

---

Introduction

This research project aims to investigate how learning and being competent in English as a foreign language (EFL), specifically in the speaking domain, can be achieved by designing a teaching framework based on fostering in students the necessary abilities to carry out self-assessment of their speaking production, and hence, triggering the necessary devices to develop more autonomous, self-directed, and better language learners.

Aspects of self-assessment are primarily associated with the autonomy and self-directedness a language learner can display in learning settings. As self-directedness is a common trait of adult learners (Jarvis, 1985), this research focused on university English language learners enrolled in the language centers of Universidad Surcolombiana, Universidad de Ibague, and Universidad EAFIT. This allowed finding language learners whose level of motivation was not necessarily bound to extrinsic motivational factors such as a passing grade or graduation requirements, but to intrinsic factors such as personal and professional rewards that come with being fluent in English.

Theoretical Framework

This section comprises the pedagogical and theoretical pillars of the research, and it includes an objective analysis of previous work on the area of self-assessment of foreign and second languages. We first explain how the concept of self-assessment is interwoven with those of oral accuracy and oral grammatical range as well as its impact in the consolidation of the speaking domain in a second language, closing with a brief justification of the use of video in the implementation of pedagogical practices related to second language learning.

Self-Assessment

According to Chalkia (2012) self-assessment is the capacity that a person has to judge his/her performance and to make decisions about him/herself and his/her abilities. Other authors such as McMillan and Hearn (2008) have also asserted that self-assessment “stands alone in its promise of improved student motivation and engagement, and learning” (p. 1). “Self-assessment is more accurately defined as a process by which students: 1) monitor and evaluate the quality of their thinking and behavior while learning and 2) identify strategies that improve their understanding and skills” (p. 40). Therefore, for the purpose of this research, self-assessment is defined as the process by which students emit judgments of both the process and the outcomes of their linguistic products, comparing the results against defined criteria resulting in a conscious evaluation of where they are and the learning path to be followed in the journey of reaching the goals established by the criteria.

A term used commonly but erroneously when addressing the issue of self-assessment is self-monitoring. Shunk (as cited in McMillan & Hearn, 2008) states that self-monitoring is a constituent element embedded in the process of self-assessment as a necessary skill for effective self-assessment that involves focused attention to certain aspects of behavior or thinking. Another important element associated with the process of self-assessment is that of self-judgment. Self-judgment is the capacity of identifying and judging the progress towards the targeted performance (McMillan & Hearn, 2008); this judgment produces in students a significant idea of where they are, what they know, and what they still need to learn (Bruce as cited in McMillan & Hearn, 2008).

Implications of self-assessment in the classroom can also be recognized. In light of metacognition, which involves the capacity to monitor, evaluate, and know what to do to improve performance as McMillan and Hearn (2008) have claimed, self-assessment activates the necessary internal mechanisms that “includes conscious control of specific cognitive skills such as checking understanding, predicting outcomes, planning activities, managing time, and switching to different learning activities” (p. 42). These skills are positively related to increasing achievement and, most importantly, such skills can be taught to students (Shunk, 2004). Three particular elements can be associated with the benefits that self-assessment brings to the learning-teaching process within and beyond the classroom (McMillan & Hearn, 2008). When teachers and learners establish clear learning goals and articulate evaluative criteria that enable students to access to their own work:

1. English language learners actively engage and participate in the learning process and become more connected to the learning outcomes.
2. Self-assessment promotes scaffolding as teachers learn to pass the responsibility of evaluation to their students and this action includes pedagogical practices such as modeling, goal setting, evaluation, strategy adjustment, and reflection.
3. Also, self-assessment increases motivation and engagement when students believe that they can successfully complete a task.

Self-assessment of speaking skills is an area that is still in debt of providing practitioners with more empirical evidence that supports its effectiveness and sheds light on how to implement it in the classroom (Castañeda & Rodríguez-González, 2011). However, there are sources that orient the methodology that should be considered when implementing self-assessment in teaching practices, especially carried out by training students on how to do it. Castañeda and Rodríguez-González (2011) assert that in order for self-assessment to be successful and produce desirable outcomes, guidance and explicit awareness through training intervention are required. Several other researchers stress the importance of creating classroom practices that foster training students in the essentials of self-assessment for increasing its effectiveness. Researchers like AlFallay; Chen; Orsmond et al.; Patri, Stefani, and Taras (as cited in Castañeda & Rodríguez-González, 2011) demonstrated that in order to boost the efficacy of self-assessment and involve students in the evaluation process to activate long term learning, factors like intervention and feedback, practice, clear criteria, and training should be taken into consideration. They also investigated the methodology to be used when undertaking the endeavor of training students in how to conduct self-assessment. They claim that before actual assessment takes place, training, using a tutorial or workshop, should be developed in order to improve students’ understanding of the criteria, and thus enhance students’ assessment quality. Wiggins (1993) used examples of previously marked products for class discussion and analysis; and, despite coming from the research field of writing, scholars have argued that teacher intervention is determinant to successfully involve students in the self-assessment process (Min, 2005; Stanley, 1992).

The Speaking Skill, Oral Accuracy and Grammatical Range

According to Bygate (2006, 2009), De Saint-Léger (2009), and Shumin (1997), speaking entails a complex and unique cognitive processing task that is difficult for second language learners to accomplish. Speaking requires the learner to construct meaning by producing information while employing effective linguistic, non-linguistic, and contextual parameters such as language control, body language, and interlocutor-receptor relationships. Given the aforementioned, scholars agree on the importance of speaking as a pivotal skill in language learning and, hence, of its importance (Cohen, Weaver, & Li, 1995; Hughes, 2013; Martínez-Flor & Usó-Juan, 2006; Shumin, 1997).

The speaking skill must be analyzed from their three main dimensions of performance (Skehan, 1996): accuracy, complexity, and fluency. In his article, Skehan (1996) identifies accuracy as concerned with “a learner’s capacity to handle whatever level of interlanguage complexity she has currently attained” (p. 46). Based on this account, an English language learner who is trying to produce language that is more accurate will notice positive results in terms of his or her performance in the foreign language, as it becomes automatic due to the controlled nature of the linguistic features associated with speaking. Hence, accuracy is considered to be essential for the language to become automatic (Ahmadian & Tavakoli, 2011). As such, for the purposes of this research, the speaking skill is analyzed considering grammar accuracy and grammatical range.

The level of proficiency in speaking is, besides being determined by its cognitive demand, influenced by other factors amongst which we can count self-confidence and anxiety, as stated by Castañeda and Rodríguez-González (2011). De Saint-Léger (2009) states that over time, as anxiety decreases, self-confidence increases since learners perceive themselves as capable of performing a task, making the assertion that this inverse relationship prompts teachers to assist L2 learners in developing self-confidence and a sense of achievement (Graham, 2004). Self-assessment fosters self-confidence and, hence, enhances speaking production, which has been documented by De Saint-Léger (2009) when she claims that the self-perception developed by means of implementing self-assessment techniques evolves as positive outcomes in relation to L2 vocabulary, fluency, and self-confidence in speaking, establishing a direct connection between the benefits of self-assessment and the enhancement of the speaking skill.

It is also necessary to remark on the importance of oral production for learners when it comes to mastering a wide array of grammar structures. As Swain (as cited in Gómez, 2014) argues, “if learners are given the opportunity to practice language structures naturally in order to achieve grammatical competence, their production will likely be increased and it will also aid acquisition” (p. 54). The oral language outcomes that are envisioned to display improvement due to a controlled, though automatic use of grammatical structures, are framed in the task students are set to execute, and that task should elicit the expected linguistic features (accuracy and fluency) in a standardized environment. Although participants are prompted and given time to plan their answers in the speech drafts, they might eventually incorporate abilities acquired during the stages of these tasks in their daily-life oral production.

According to the English Language Testing System Handbook (IELTS, 2007), “grammatical range and accuracy refers to the range and the accurate and appropriate use of the candidate’s grammatical resource” (p. 12). One of the purposes of the current academic intervention is to expand the grammar structures employed by learners in their discourse; therefore, it is fundamental to define the most relevant elements of grammatical range and accuracy. They are then the pedagogical and theoretical pillars to sustain this research.

Video Speech Drafts in Learning Environments

Canning-Wilson (2000) defined video recordings as “the selection and sequence of messages in an audio-visual context” (p. 36), and added that video technique is, best-case scenario, characterized as the determination and arrangement of messages in a varying media setting. Austin and Haley (2004) argue that the educator who uses videos in the classroom permits second/foreign language learners the opportunity to view and effectively participate in lessons. Videos can be controlled; students can pause and rewind them in order to understand and/or listen again. In his studies, Hobbs (2006) suggests that videos increase motivation and boost language learning due to their capacity to catch students’ attention.

Research Question

For this study we attempted to answer this research question: “Does self-assessment of video speech drafts improve speaking grammatical accuracy and range in B1 learners?” This research concentrated on answering this question as we, teacher-researchers, are convinced that the speaking domain concerns one of the highest expectations second language learners have, which is to be able to fluently and clearly communicate their thoughts in an oral way (Young, 1990).

Method

We determined that the best course of action was using a research design which would respond to the context in which researchers were immersed in, that is, action research. Action research is the disciplined process of inquiry, conducted by those taking the action (Sagor, 2000); in this case, the researchers/teachers. Action research then presents itself as the most adequate approach to conduct the research as it allows the practitioners to be engaged in a research process with all the canonical tenets that need to be followed, but addressing a specific classroom situation, allowing for a simpler, yet sound framework to conduct the process. To produce results that were reliable and that effectively and objectively answer the research question, a combined approach to research was considered that would effectively analyze both quantitative and qualitative results. In this sense, research was designed to provide qualitative results in the form of students’ and researchers’ comments, judgments, and appreciations during the process and analyzed by searching recurring themes within the data previously collected. In the same vein, we foresaw the production of quantitative data related to the frequencies, means, and statistical deviation of the results yielded from the application of the experiment, and that were statistically analyzed to offer a correlation between both the quantitative and qualitative data and, in our doing so, being able to provide sound and reliable answers to the research question.

Researchers also generated conditions for learners to produce language orally by developing a task based on the one used in IELTS (International English Language Testing System) when evaluating speaking. IELTS speaking tests are normally an encounter between one candidate and one examiner and are designed to take between 11 and 14 minutes. During this test, three stages are developed: introduction, individual long turn, and two-way discussion. Researchers have selected the second moment of the exam, individual long turn, as the framework for performance, since it has been observed that it contains the required features for the intervention: grammatical range and accuracy. According to Seedhouse, Harris, Naeb, and Üstünel (2014), in long-turn tasks the candidate receives a card and is asked to orally respond to it. The candidate is given a minute to plan and two minutes to speak. In order to collect data in a reliable manner, researchers have opted for giving participants one minute to plan and one minute to produce.

Participants

27 students from the three institutions, 14 men and 13 women were chosen as they are part of EFL classes the researchers were teachers from. All of them have studied in the Language Institutes for at least one and a half years and take five, 100-minute sessions of English lessons per week. Their overall proficiency level is B1 according to the Common European Framework of Reference (CEFR). Their ages range from 17 to 27 years old: 34% are between 17 and 20, 48% are between 21 and 24, and the other 18% are between 25 and 27 years old. Most of them are digital natives and the others are digital immigrants (Prensky, 2001), meaning that all of them use technological devices and are acquainted with social media and web tools.

Materials

In order to collect the data in this study, information on the students’ self-assessment process was gathered and analyzed to determine to what extent self-assessment traits influenced their oral ability, specifically in grammar accuracy and grammatical range. The study was designed to be developed in four stages to cope with the four conditions suggested by Ross (2006) to carry out an appropriate self-assessment process:

• Define the criteria by which students assess their work.
• Teach students how to apply that criterion properly.
• Give feedback on their self-assessment.
• Help in using self-assessment data to improve their performance.

Data Collection Instruments

Based on the research question, researchers considered it pertinent to collect both kinds of data: qualitative and quantitative. Qualitative due to its suitability in knowing students’ perception in the use of self-assessment and to evaluate the effectiveness of the training model implemented; and quantitative to measure their progress in grammatical range and accuracy when speaking. Next, we give a description of each of the data collection instruments.

Pre-Test and Post-Test

One pre-test and one post-test related to oral production were developed in order to measure learners’ initial and final level of oral ability in terms of accuracy and grammatical range. The task performed by the students was based on the IELTS Speaking Test, Part 2, in which students have one minute to prepare a 2-minute dialogue about a topic prompted by a card. For this study, the task was shortened giving students one minute to prepare and 1-minute to speak. Students’ performance on each test (pre-test and post-test) was assessed by the teacher-researchers using an adapted version of the IELTS rubric (see Appendix). Researchers adjusted the rubric, so that only grammatical range and accuracy were assessed by it, designing a scale from 1 to 5, being 5 the highest score.

Semi-Structured Interviews

Two semi-structured interviews were applied along the research project, one during the first stage whose objective was to recognize students’ background knowledge of the concepts of self-assessment, oral accuracy, and grammatical range; a second interview was administered right after the last recording took place aimed at identifying what students’ understanding of the same concepts applied in Interview 1 was, and how they, the students, had changed during the research process. The main purpose was to gather information about students’ thoughts regarding the adoption of self-assessment strategies and their effect on their learning process. Each interview consisted of a total of 20 questions, ten open and ten closed questions and took approximately 10 minutes to answer. The teacher-researchers applied individual interviews using participants’ L1, with the purpose of getting more objective and accurate answers.

Teachers’ Journals

Along the study, we wrote our insights about the process in a journal. We completed an adapted version of the field notes template designed by Acero (2012) and implemented by Centeno, Montenegro, Montes, and Rodriguez (2013) in their research study. The template encloses questions related to students’ reactions towards the use of video recordings, attitude towards the self-assessment strategies implemented, learners’ participation in the tasks, and weaknesses and strengths experimented during the study. The aim of this instrument was to collect our insights along the process, and triangulate the information with the one provided by the students in their journals and in the interviews.

Students’ Journals

They served as the instrument students used to write their thoughts about the application of self-assessment techniques to the video speech drafts recorded during the training intervention. This activity was guided by a set of questions that helped students reflect upon their oral proficiency in accuracy and grammatical range, as well as feelings about the application of self-assessment as a strategy to improve their oral competence. The students were given a journal model consisting of five closed questions with the possibility of complementing their answers in a comments section; participants were asked to write their entries right after each evaluation of their video recordings.

Video Recordings

Subjects’ video recordings were the main source of data for analyzing oral production. Subjects recorded themselves at different stages (see below) to register and evaluate their performance and construct the path of improvement that would take them to an evidential enhancement of their oral performance.

Intervention Procedure

We designed an instructional device to provide learners with fundamentals of self-assessment, grammatical range, and accuracy as well as a detailed session to instruct learners on how to implement Mailvu, the online tool selected to video-record students’ spoken production, due to its practicality and facility to use.

The process of the intervention consisted of a sequence of nine stages as follows:

Stage 1: Self-Diagnosing and Sensitizing

In the first stage of the interventions students were required to reflect upon their spoken production abilities; to do this, learners were given a KWL form (What I know, what I want to know and what I learned) to elicit the features of grammar they effectively use when speaking. Prior to the completion of the form, learners and researchers undertook a brainstorm methodology to focus on the grammar forms students had been previously exposed to; in this manner, participants would have a better foundation of what items to include in the form. The KWL form gave us evidence of the extent to which learners were aware of their own needs regarding oral production in terms of grammar accuracy and grammatical range.

Stage 2: Grammar Features Analysis

After learners were exposed to the strategy to be used and reflected upon their own grammar needs for oral production, they were engaged in analyzing the grammar features of video samples provided by us. Teachers and learners watched two videos from YouTube of candidates performing the “long turn” task of IELTS, one posted by the AcademyEnglishHelp (2014) and the other by Fardin (2011). We paused the video at specific moments to identify the grammar features the candidates used when speaking plus we prompted learners to detect the possible candidates’ strengths and weaknesses when speaking. We and the learners observed samples of two rubrics to assess the grammatical range and grammar accuracy of candidates undertaking the “long turn” task of IELTS. The first rubric is the one used by IELTS and the second was designed only to correlate the results of the two rubrics. The purpose of observing the rubrics was to give learners clear models of grammar that could be identified when judging the candidates’ performance.

Stage 3: Awareness of Expected Performance

Unlike previous stages, this phase focused on modeling the type of performance participants were expected to achieve taking into consideration their level of proficiency. For this stage, learners were prepared to identify the features of the desired performance in the IELTS “long turn” task. During this phase participants and researchers analyzed two videos of candidates carrying out the “long turn” task of IELTS and focused on the grammar elements that a Band 3 and a Band 6¹ candidate applied during the test; these levels were selected as models of poor and expected performance for learners. While observing the videos, learners and researchers applied the adapted rubric to assess oral production in grammatical range and grammar accuracy. This analysis enabled learners to reflect on the needs they had in order to improve their oral production, having as a reference a model of speaking to follow; it also empowered learners to assess a task similar to the one they were to undertake, supported by a rubric.

Stage 4: Practice

We designed prompts similar to the ones that are used in the “long turn” task of IELTS and gave learners different opportunities to plan and produce language using them. Learners were asked to work individually and were required to record their performance using their mobile phones, tablets, or any technological device that permitted them to record their speaking; once learners had recorded their videos they were analyzed using the rubric. Afterward, learners were given new prompts and were asked to work in pairs; in this opportunity learners helped each other rank their performance using the rubrics.

Stage 5: Identifying the Gap

In this stage learners analyzed their own videos with the purpose of identifying the grammatical range and grammar accuracy, elements that they had in common with the samples that had been shown in previous stages. During this stage, participants identified how the mistakes they made in their videos affected the message they wanted to convey; researchers reviewed with students some basic and complex grammar structures in order to expand their options when speaking and to avoid mistakes that had been common among participants. Learners and researchers organized and consolidated the findings of this analysis in their journals.

Stage 6: Strategies to Improve

Learners socialized the content of their journals in class and brainstormed to seek possible solutions to the issues they encountered in their productions. Learners received training on how to record their own videos using the tool Mailvu. We selected Mailvu due to the facility this website offers to record and share video messages; moreover, this tool can be used for free.

Stage 7: Resources to Improve

We gave learners instructions about how to proceed to record a new video, using a prompt that contained a topic similar to the ones used in IELTS “long turn” tasks; these tasks usually ask candidates to plan a talk about a specific subject. Once videos were recorded, every learner was asked to observe and assess his or her performance by applying the same rubric designed for the task and that had been employed in previous stages. Upon completion of the recording, learners were asked to assign a mark to their performance in grammatical range and oral accuracy, then, learners created a new entry in their journals, in which they described the experience of applying the rubric to self-assess their performance.

Stage 8: Speaking Task

Learners were given instructions to work on a new video speech draft; on this occasion learners were assigned a new prompt to perform the “long turn” task. Then learners recorded the video and assessed their product; learners sent their videos and rubrics to researchers via email by sharing the link of the video retrieved from Mailvu and attaching their rubrics.

Stage 9: Comparing Assessments

We assessed the video drafts submitted by participants and applied the rubric designed to assess their performance focusing on grammatical range and accuracy. This phase allowed us to contrast and compare the self-assessment process undertaken by participants with the assessment done by researchers; this process was crucial to determine the weaknesses and strengths of learners to self-assess their performance. We also conducted an analysis to determine to what extent the oral production of learners had improved in terms of grammatical range and accuracy. Conclusions and observations of the self-assessment and videos of learners were socialized and discussed in plenary, so participants could have a clear grasp of what positive and negative aspects were encountered. Stages 8 and 9 of the intervention were repeated to give students the opportunity to be familiarized with the procedure.

Results and Data Analysis

Both quantitative and qualitative data were collected and analyzed in order to fulfill the objectives of this research study. Quantitative data were gathered through the numerical scoring of the oral production rubrics applied in Stage 9 of the pedagogical intervention; whereas qualitative data were examined following coding procedures for detecting salient themes from the collected information as proposed by Burns (2009), who suggests to identify the relevant topics that repetitively appear in the voice of participants collected in their journals and through interviews. Categories and sub-categories were organized to summarize and classify the collected data.

We revised the data collected from the 27 participants’ samples, then categories and sub-categories coding procedures, as suggested by Burns (2009), were applied to identify relevant data, repetitive ideas, and how patterns were connected among them and categories. Bearing in mind the objectives of this study and the previously described procedures, we found in the open coding phase the sub-categories shown in Table 1.

Table 1. Indicators From the Open Coding Phase

After identifying the categories, sub-categories were also recognized to detect relation among sub-categories which, after merging the indicators and the objectives of the study, resulted in three categories that grouped the identified indicators (see Table 2).

Table 2. Final Category Chart

Category 1: Self-Assessment Awareness

Mistakes Identification

Among the main aspects detected in this area and from the analysis of the qualitative data the participants registered in their journals as well as taking the videos as the source of their examination, the following are the aspects that were remarked upon during the analysis:

• Subjects recognized their strengths and weaknesses in their oral performance. Having learners assess their own products empowered them to tackle the aspects they needed to improve upon.
• Subjects exhibited elaborated judgments to refer to their mistakes; apart from spotting their slips, they could justify their observations in their journal entries, explaining the mistakes they made and providing empirical justifications for them.
• Subjects displayed a key factor of self-assessment: the capacity of making decisions that impact their learning process. They proposed paths to follow in order to solve those mistakes they had been able to recognize through the analysis of their video speech drafts.
• Subjects evidenced capability to focus their self-assessment on specific grammar features of their spoken language; this enabled them to seek improvement in the detected areas.

Sense of Improvement

Participants reported improvement in various aspects: confidence, clarity, error-free sentences, and verb forms. Video-recording their faces while undertaking the task was a situation that challenged the confidence of learners; researchers noted that participants seemed to be nervous and anxious.

Another aspect that was classified in this indicator was the improvement in the intelligibility of the message conveyed by participants. They could use the first recorded videos to compare their own performance and draw conclusions regarding how they have improved in this domain.

The correctness in the sentences that learners used in their discourse while doing the activity were favored, according to the data obtained from the journals, as they gave learners the sense that their production was getting better; additionally, a reduction of mistakes in their utterances was noticed.

Participants also reported improvement in specific aspects of grammar in which they may have presented difficulties; in this regard, subjects claimed that simple structures were used correctly, and this became another factor to determine the improvement they were experiencing. These findings were also observed and annotated by teacher-researchers in their journal: “Students show less mistakes when speaking, hence, it can be concluded that, in fact, their oral production was improved.”

Language Judgment

As previously defined in this paper, self-assessment is the capability of learners to make decisions about their performance based on the ideas they create upon observation of their own production. This definition is closely related to what we observed in the data obtained from participants; after the analysis process, it was noticed that learners registered in their journals judgments about their language; subjects demonstrated a wider view to describe the elements that made up part of their discourse; this might have contributed to a more critical perspective to evaluate their production.

An instance of this enhancement of critical thinking can be noticed in the first and the last entries that addressed the question: “Do you feel satisfied with your performance in this activity? Why?”

No, I felt nervous and I had a lot of mistakes . . . Now, I use more different tenses and words than I used before . . . When I hear myself I am aware if I use the right words to express what I want to. (Subject 4, Journal Entry 1)

Participants also revealed that the lack of grammar structures used in their video speech drafts was related to the deficiency in their knowledge of linguistic features of the language.

Category 2: Grammar Recognition

Indicators that suggested improvement in terms of accuracy and grammatical range of participants in their spoken production were consolidated and grouped in this category.

Correct Use of Grammar

Data analysis suggests that learners acknowledge the importance of grammar in their oral production. Some of the information gauged from participants’ journals reveals that participants were not only aware of how grammar can contribute to their oral production, but also valued the correct use of grammar when analyzing their video speech draft.

Now I am more aware of using a more accurate grammar. (Subject 22, Journal Entry 3)

I am more aware that I need to use correct grammar. (Subject 25, Journal Entry 3)

I answered what I needed with good grammar. (Subject 26, Journal Entry 3)

There was a close relationship between the self-assessment practices of learners and their improvement in grammar accuracy.

Range of Structures

Data collected suggest that learners attempted to incorporate complex structures into their oral production when recording their video speech drafts; by doing this, participants seemed to have expanded their grammatical range.

I used past continuous and present simple recording my video. (Subject 17, Journal Entry 4, Authors’ translation)

I have improved, because now I use more grammar structures. (Subject 13, Journal Entry 3, Authors’ translation)

Connecting participants’ ideas with Seedhouse et al.’s (2014) length and complexity are two traits of complex oral speech. Video after video, it could be noticed how participants increased the attempts to incorporate these two characteristics into their speech. It was also observed by researchers that verb forms that appeared to be difficult for learners were included in their oral production.

Category 3: Video Speech Drafts

The use of videos to record the learners’ performances in oral production and to apply self-assessment techniques enabled learners to have an artifact with which to review their own weaknesses and strengths. Data collected suggest that the usage of video speech drafts contributed in two aspects: students getting acquainted with the technique of recording themselves and the advantages of using video recordings to improve oral production. The following are the main results regarding this category:

• When using videos, subjects expressed having felt a certain level of anxiety and nervousness, even though they were given alone time for the recording; this feeling was reduced the more they got acquainted with and felt familiar with the tool.
• Another constraint participants had in the first video they recorded was the time they were given to plan versus the time they had to speak; participants complained about the limitation in terms of time to set their ideas and articulate them with their oral production.
• The use of video recordings was fundamental to achieve the objectives of this study; only through the usage of this tool could learners and researchers keep track on how self-assessment was impacting the oral production of participants in terms of grammar accuracy and grammatical range. Researchers and participants valued the use of video recordings as this strategy enabled them to refer, when necessary, to learners’ oral production to assess, identify improvement, and tackle weaknesses, among other decisions that were made during the intervention; for example, when researchers showed videos to provide learners with models of desired performance, learners were encouraged to compare their own production with the candidates’ production in the videos.

Quantitative Data Analysis

Complementing the qualitative data presented above, the quantitative data accounted for the numerical scoring of the oral production rubrics. The results shown in Table 3 were obtained after processing the scores participants gave themselves on their oral speech using the rubrics.

Table 3. Pre- and Post-Test Results Comparison for Grammar Accuracy and Grammatical Range for Subjects

In terms of grammar accuracy, it scored higher in the pre-test in comparison with grammatical range; nonetheless, when analyzing results of the post-test, grammatical range had a greater improvement in terms of relative results compared to grammar accuracy. The grammar accuracy post-test rose from 3.00 to 3.37 with a 0.37 increase, but grammatical range went from 2.56 to 3.04 resulting in an improvement of 0.48, 0.11 more than grammar accuracy. These results led us to ascertain that participants were more aware and applied more sound language strategies to expand the vocabulary and length of their oral utterances than focusing on how accurate their language was produced.

The above results were compared with the score researchers gave participants in both pre- and post-test. For validity purposes, both researchers and participants’ scores were analyzed at the end of the research process and participants’ marks were not revised up until researchers had revised students’ production (see Table 4).

Table 4. Pre- and Post-Test Results Comparison for Grammar Accuracy and Grammatical Range for Researchers

Notice that the tendency was similar in both cases (subjects’ scores and researchers’ scores). There was a higher mark for grammar accuracy in the pre-test; yet, the total increase was lower as the difference between post-test for both aspects showed how the growth was only 0.4, 0.7 lower than the participants’ own scores. Participants were also asked about their perception of self-assessment (see Table 5).

Table 5. Self-Assessment and Degree of Satisfaction

After processing the individual answers of the subjects, it can be evident, as seen in Table 5, that the number of subjects whose perception of having a positive attitude or feeling of satisfaction towards their performance increased from 19% in the pre-test, to 85% in the post-test.

In terms of their oral grammar accuracy, students’ answers are summarized in Table 6.

Table 6. Oral Ability: Grammar Accuracy

When processing and analyzing the frequency, subjects answered positively to the third question (see Table 7). It can be observed that, even though the number of students who expressed that they had, indeed, improved, the growth from the results in the pre-test compared to the ones in the post-test were not as considerable as the ones of grammar accuracy.

Table 7. Answers to Oral Ability: Grammatical Range

As seen in Table 7, 50% of the subjects (13), when answering the question about grammatical range in the pre-test, answered that their ability had somehow improved, and despite the increase in the number of subjects who answered positively (16), the increase only represented 12% (3 subjects). When paralleled with the results in the question regarding grammar accuracy, the difference is significantly lower, which can lead to validate the conclusion that subjects’ awareness of self-assessment impacted more significantly grammar accuracy than grammatical range.

Conclusions

Self-assessment, posterior application of improvement techniques based on mistakes identification, and subsequent implementation of learning paths to overcome those mistakes based on language benchmarks, have been mostly used in the domain of written production. Due to the transient nature of the spoken word (Hughes, 2013), self-assessment of oral production has not been explored which has left aside the application of self-assessment techniques as a means for oral speaking improvement. This is where technology comes to the aid of language teaching. By using video speech drafts and multimedia resources to exemplify speaking benchmarks, learners were able to undertake the daunting task of revisiting their own oral production in order to identify and analyze their mistakes or shortcomings, and based on this analysis define their own routes of improvement implementing their own improvement strategies. As an added bonus to this activity, the research yielded evidence that the use of video speech drafts to improve students’ oral skills in the domains of grammar accuracy and grammatical range, also had a positive effect in the learners’ sense of improvement, that is, their motivation was positively impacted as they could see their progress as they saw the different videos they recorded.

Regarding the grammatical areas taken into consideration for the current study, participants were more likely to detect their flaws in terms of accuracy, and raise awareness of the correct use of grammar which led to an improvement of other elements inherent to oral production, such as fluency and use of vocabulary. On the other hand, grammatical range appears as a more challenging aspect, since learners displayed difficulties to widen the structures they used in their video speech drafts.

About the usage of video recordings, it was established that they can become a useful source of reflection for students. Participants of this study constantly highlighted this technique as a good opportunity to practice and assess their own oral performance and, at the same time, to lead them to improvement in various areas of spoken production. The findings of this project can be taken as a reference to incorporate self-assessment practices in English language teaching contexts, and may also be included in curriculum designs attempting to promote self-direction among learners.

---

¹Band 3 and Band 6 refer to the performance based on the rubric applied by IELTS on its exams. IELTS rubric places candidates in a 1-9 band system.

---

References

AcademyEnglishHelp. (2014, January 24). IELTS speaking with native English speaker subtitles: Score 9, Example 2 [video file]. Retrieved from https://www.youtube.com/watch?v=eGoA-Exkupg.

Acero, C. (2012, September 26). Facilitating collaborative language learning [Web log post]. Retrieved from http://generalsuccessfullearningstrategies.blogspot.com/.

Ahmadian, M. J., & Tavakoli, M. (2011). The effects of simultaneous use of careful online planning and task repetition on accuracy, complexity, and fluency in EFL learners’ oral production. Language Teaching Research, 15(1), 35-59. https://doi.org/10.1177/1362168810383329.

Austin, T. Y., & Haley, M. H. (2004). Content-based second language teaching and learning: An interactive approach. Boston, US: Allyn & Bacon.

Burns, A. (2009). Doing action research in English language teaching: A guide for practitioners. New York, US: Routledge.

Bygate, M. (2006). Areas of research that influence L2 speaking instruction. In E. Usó-Juan & A. Martínez-Flor (Eds.), Current trends in the development and teaching of the four language skills (pp. 159-186). Berlin, DE: Mouton de Gruyter. https://doi.org/10.1515/9783110197778.3.159.

Bygate, M. (2009). TESOL and Linguistics. In J. Culpeper, F. Katamba, P. Kerswill, R. Wodak, & T. McEnery (Eds.), English language: Description, variation and context (pp. 642-655). Basingstoke, UK: Palgrave Macmillan.

Canning-Wilson, C. (2000). Practical aspects of using video in the foreign language classroom. The Internet TESL Journal, 6(11).

Castañeda, M., & Rodríguez-González, E. (2011). L2 speaking self-ability perceptions through multiple video speech drafts. Hispania, 94(3), 483-501. Retrieved from http://www.jstor.org/stable/23032122.

Centeno, E., Montenegro, G. M., Montes, S. M., & Rodriguez, C. (2013). Collaborative learning teams: A strategy to foster self-directed language learning in A1 Colombian students (Master’s thesis). Universidad de La Sabana, Chía, Colombia.

Chalkia, E. (2012). Self‐assessment: An alternative method of assessing speaking skills. Research Papers in Language Teaching and Learning, 3(1), 225-239.

Cohen, A. D., Weaver, S. J., & Li, T-Y. (1995). The impact of strategies-based instruction on speaking a foreign language (Research Report). Minneapolis, US: Center for Advanced Research on Language Acquisition, University of Minnesota. Retrieved from ERIC database (ED394322).

De Saint-Léger, D. (2009). Self-assessment of speaking skills and participation in a foreign language class. Foreign Language Annals, 42(1), 158-178. https://doi.org/10.1111/j.1944-9720.2009.01013.x.

Fardin, F. (Adapter). (2011, August 30). Real IELTS speaking band 7 [Video file]. Retrieved from https://www.youtube.com/watch?v=_2oawkQIgrw.

Gómez, D. (2014). The impact of peer and self-assessment on teenage B2 students’ use of present prefect simple, present perfect continuous and past perfect simple in their spontaneous spoken production (Master’s thesis). Universidad de La Sabana, Chía, Colombia.

Graham, S. J. (2004). Giving up on modern foreign languages? Students’ perceptions of learning French. Modern Language Journal, 88(2), 171-191. https://doi.org/10.1111/j.0026-7902.2004.00224.x.

Hobbs, R. (2006). Non-optimal uses of video in the classroom. Learning, Media & Technology, 31(1), 35-50. https://doi.org/10.1080/17439880500515457.

Hughes, R. (2013). Teaching and researching: Speaking. Chicago, US: Routledge.

IELTS. (2007). International English Language Testing System Handbook. Cambridge, UK: Author.

Jarvis, P. (1985). The sociology of adult and continuing education. Beckenham, UK: Croom Helm.

Martínez-Flor, A. & Usó-Juan, E. (2006). A comprehensive pedagogical framework to develop pragmatics in the foreign language classroom: The 6Rs approach. Applied Language Learning, 16(2), 39-63.

McMillan, J. H., & Hearn, J. (2008). Student self-assessment: The key to stronger student motivation and higher achievement. Educational Horizons, 87(1), 40-49.

Min, H.-T. (2005). Training students to become successful peer reviewers. System, 33(2), 293-308. https://doi.org/10.1016/j.system.2004.11.003.

Prensky, M. (2001). Digital natives, digital immigrants (Part 1). On the horizon, 9(5), 1-6. https://doi.org/10.1108/10748120110424816.

Ross, J. A. (2006). The reliability, validity, and utility of self-assessment. Practical Assessment, Research, and Evaluation, 11(10).

Sagor, R. (2000). Guiding school improvement with action research. Alexandria, US: ASCD.

Seedhouse, P., Harris, A., Naeb, R., & Üstünel, E. (2014). The relationship between speaking features and band descriptors: A mixed methods study. IELTS Research Reports Online Series, (2). Retrieved from https://www.ielts.org/-/media/research-reports/ielts_online_rr_2014-2.ashx.

Shumin, K. (1997). Factors to consider: Developing adult EFL students’ speaking abilities. English Teaching Forum, 35(3), 8-13.

Shunk, D. H. (2004). Learning theories: An educational perspective. Upper Saddle River, US: Merrill Prentice/Hall.

Skehan, P. (1996). A framework for the implementation of task-based instruction. Applied Linguistics, 17(1), 38-62. https://doi.org/10.1093/applin/17.1.38.

Stanley, J. (1992). Coaching student writers to be effective peer evaluators. Journal of Second Language Writing, 1(3), 217-233. https://doi.org/10.1016/1060-3743(92)90004-9.

Wiggins, G. P. (1993). Assessing student performance: Exploring the purpose and limits of testing. San Francisco, US: Jossey Bass.

Young, D. J. (1990). An investigation of students’ perspectives on anxiety and speaking. Foreign Language Annals, 23(6), 539-553. https://doi.org/10.1111/j.1944-9720.1990.tb00424.x.

---

About the Authors

Martin Javier Caicedo Pereira holds an M.A. in English language teaching for self-directed learning from Universidad de La Sabana (Colombia), and a Graduate Diploma in TESOL from Anaheim University (USA). He currently works as adjunct professor at the Master of English didactics at Universidad de Caldas, as well as Regional Assessment Specialist for Oxford University Press in Latin America. His research interests focus on assessment, ICTs in language learning, and self-directed learning.

Jhonny Alexander Lozano Bermúdez is an English language teacher with more than 10 years’ experience working with varied ages and levels. His research interests mainly focus on autonomous learning and self-assessment. He holds a degree in social communication, an ELT diploma from Anaheim University and a Master degree from Universidad de La Sabana.

Luis Alfonso Vanegas Medina is an EFL teacher-researcher invited by the Foreign Language Department at Universidad Surcolombiana. He holds a Graduate Diploma in TESOL from Anaheim University and a Masters in English Didactics from Universidad de La Sabana. His current research interests include CLIL methodology and the use of TICs in EFL lessons.

---

Appendix: Rubric to Self-Assess Oral Ability (Accuracy and Grammatical Range)

In order to keep a record of your performance in each recording, we invite you to fill in the following chart.

---