Statistical literacy refers to an individual’s ability to understand and critically evaluate statistical information and results encountered in various contexts (Gal, 2004; Gönülal, 2019). Statistical literacy involves various skills, including critical thinking; the selection of statistical methods; and data management, analysis, interpretation, and accurate reporting (Gönülal, 2018, 2019; Loewen et al., 2014, 2020; Plonsky, 2021). To inform the design of this study’s statistics course, several components of statistical literacy were reviewed, which are described in the following subsections.

Previous research syntheses have identified the prevalence of quantitative research in applied linguistics journals, with classical statistical methods such as t-tests, correlations, chi-square tests, ANOVA, and MANOVA being commonly employed (Khany & Tazik, 2019). While parametric tests dominated, non-parametric equivalents were less frequent. More recent reviews also confirmed the prevalence of quantitative research (Farsani et al., 2021) and basic statistical techniques (Khany & Tazik, 2019) in research articles published in prominent applied linguistics journals.

Survey-based studies revealed a gap in statistical literacy among applied linguists. Loewen et al. (2014) surveyed 331 applied linguistics doctoral students and professors from North America, Australia, Europe, and Asia and found positive attitudes toward quantitative research. However, only 14% of students and 30% of professors felt adequately trained in statistics. Gönülal et al. (2017) investigated the development of statistical literacy among 50 graduate students, noting substantial improvements in basic descriptive and common inferential statistics but weaker gains in advanced statistics. Later, Gönülal (2018) explored predictors of statistical literacy among 120 SLA doctoral students in North America, emphasising the significance of taking more statistics courses, having a quantitative research orientation, and engaging in self-training.

Gönülal’s (2019) subsequent investigation with 120 SLA doctoral students in North America identified challenges in interpreting common and advanced statistics, suggesting improvements in course content and format. Loewen et al. (2020) assessed the statistical knowledge of 198 applied linguistics and SLA researchers from North America and Europe, revealing a gap in advanced statistics and emphasising the positive influence on performance of quantitative research orientation and the number of statistics courses taken. More recently, Zhang and Han’s (2023) survey of 211 applied linguistics master’s students in China highlighted a deficiency in overall statistical literacy despite positive attitudes toward statistics. The number of statistics courses taken, a quantitative research orientation, and self-training were identified as crucial predictors of statistical literacy, attitudes, and self-efficacy.

The findings from research syntheses and survey-based studies emphasise the significance of statistical literacy in applied linguistics research. While quantitative research is prevalent, there is a notable gap in statistical knowledge, emphasising the need for targeted training and improvements in course content. Direct evidence of participants’ ability to perform statistical analyses is needed to understand the effectiveness of statistical training. The present study addresses this gap by incorporating performance and perception data, providing insights into the effectiveness and challenges of statistics training.

This study was conducted within an applied linguistics doctoral programme on English language teaching education at Universidad del Valle, Colombia. The pilot course stemmed from the need to develop the PhD candidates’ statistical abilities and identify the appropriate content and methodology for these participants and the context. The participants were five third-year doctoral candidates—middle-aged language teachers with extensive teaching experience—who were enrolled in the statistics course I taught. I hold a PhD in applied linguistics and have experience with quantitative research. By obtaining online consent, the study secured confidentiality and anonymity. As shown in Table 1, the participants were unfamiliar with statistics and shared common academic and professional backgrounds.

Table 1

	Antonio	Hector	Claire	Atahualpa	James
Gender/Age	Male/40s	Male/30s	Female/40s	Female/30s	Male/30s
Qualifications	BA and MA in Languages	BA and MA in Languages Diploma in Marketing and Administration	BA and MA in Languages	BA and MA in Languages	BA and MA in Languages
Years of professional experience	15	15	20	15	14
Topic of qualitative study	Integration of ICC in the curriculum	Material development with cultural aspects	Interculturalism in the curriculum	Multilingualism and interculturality	Meaning-making and language ideologies
Familiarity with quantitative research	Familiar with item reliability	Familiar with descriptive statistics	Unfamiliar	Unfamiliar	Familiar with descriptive statistics

Note. All the participants’ professional experience was as school and university language teachers. ICC = Intercultural communicative competence.

Action research was implemented to track the development of the participants’ statistical literacies. Action research is suitable for positive change and improvement in the participants’ social situation from a “socio-constructivist approach in which teachers are seen as agentive actors and investigators within their social contexts” (Burns, 2019, p. 166). As the course instructor and researcher, I collected classroom data and conducted three cyclical phases of action research: diagnostic, intervention (with cycles), and evaluation (see Figure 1).

Figure 1

The diagnostic phase inquired into participants’ initial statistical performance through a pre-test and self-perception data, establishing a baseline for the 16-week intervention phase. The intervention phase consisted of implementing the course, utilising a theoretical-practical approach that featured collaborative learning. The course syllabus (see Appendix) was designed based on reputable textbooks (e.g., Lowie & Seton, 2013; Phakiti, 2014) and prioritised classical statistical methods due to the participants’ needs. Software for quantitative data analyses included Excel, SPSS, Langtest, and JASP. Ten real-life statistics tasks included performing, interpreting and analysing data, reporting results, and evaluating the quality of quantitative research reports. The course was delivered online through Zoom, as most participants reside in different regions of Colombia. During the sessions, the participants’ oral reflections provided valuable insights into their learning process and experiences with the course; however, due to space constraints, these interactions are not reported here. Their reflections related mainly to the pacing of the course content. The evaluation phase employed a post-test, graded assignments, and an oral final reflection to examine participants’ performance, learning experiences, and perceptions of the course.

To address Research Question 1, I employed video recording, pre-test and post-test results, and three graded assignments. In the video, participants reported their qualifications, professional experience, research designs, familiarity with statistics, and opinions on Loewen et al.’s (2014, 2020) articles about statistical literacy among applied linguistics and SLA researchers. This provided qualitative insights into the participants’ characteristics and their familiarity with quantitative research prior to the course.

The online pre-test and post-test (administered through Google Forms), along with three graded assignments, provided participants’ performance data on the development of statistical literacy. The pre-test and post-test consisted of three sections (see Table 2), each containing 34 multiple-choice questions and one open-ended question. The option “I do not know” ensured thoughtful responses, avoiding random choices. Both tests’ questions encompassed critical thinking, statistical methods selection, data analysis, interpretation, and reporting, and were informed by the research of Gönülal (2018, 2019), Gönülal et al. (2017), and Loewen et al. (2014, 2020). Prior to implementation, the pre-test was reviewed and piloted by a colleague with experience in quantitative research.

Table 2

Sections	Purpose	35 items/points
Descriptive statistics	Understanding of descriptive statistics (e.g., types of variables and scales, mean, mode, standard deviation)	15
	Understanding visual data (e.g., outliers, normal distribution, box plots)	4
Inferential statistics	Understanding inferential statistics by choosing the appropriate statistical methods (parametric and non-parametric tests)	8
	Interpreting data results (significance and effect sizes)	7
Writing up the report	Writing/reporting results	1

The three graded assignments (see Table 3) gauged the participants’ statistical learning. Table 4 shows a sample of Assignment 1. Continuous feedback was provided during the course through practical individual and group tasks, mirroring the structure of the graded assignments.

Table 3

Assignment	Description	Marking criteria (50 points each)
Critical analysis of a quantitative research report chosen by the instructor (30%)	Assessment checklist of 24 items divided into 10 sections, based on Norris et al.’s (2015) guidelines: population and participants, sampling and assignment, measurement, design and procedures, analysis, descriptive statistics and graphical analyses, effect sizes and confidence intervals, assumptions, statistical tests and final evaluation Instruction: Participants select whether the information was reported, partially reported, or unreported, and then justify their answers	60-64% (13 correct answers): 3.5 65-69% (14-15 correct answers): 3.8 70-74% (16 correct answers): 4.0 75-79% (17 correct answers): 4.3 80-84% (18 correct answers): 4.5 85-89% (19-20 correct answers): 4.8 90-100% (21-24 correct answers): 5.0
Quantitative data analysis and reporting (30%)	Participants analyse a fictional study given the purpose and research design Instruction: Choose one out of four studies and perform the appropriate analysis. Report all the information required (e.g., assumptions, descriptive and inferential statistics, confirmation of results from the non-parametric test) and the results	Assumption checks and report: 10 points Descriptive and inferential statistics reports: 15 points Confirmation of results using the non-parametric test counterpart: 10 points Written report: 15 points
Interpreting and reporting quantitative research results (40%)	Participants interpret and write the results of a published quantitative study when provided with the context, graphs, and tables Instruction: Participants receive the description, research questions, and results of three published quantitative studies. They must interpret the results of the tables and answer the research question	Provide detailed interpretations and reports of descriptive and inferential statistics. Answer the research questions directly and clearly for each study Study 1: 15 points Study 2: 15 points Study 3: 20 points

Table 4

Objective	Analyse critically the report and robustness of a quantitative study
Instructions	Read the article by Al-Sumait et al. (2022) “Multicultural Educational Experiences and ICC in Arab Context” and Norris et al. (2015) “Guidelines to Report Quantitative Methods Results.” Then, read the list of reporting recommendations for methods sections and results sections in this grid. Then, read the article again and tick the boxes whenever the information required in the grid is reported in the article. Answer the questions and provide your final assessment of the robustness and quality of the report of this quantitative study.
Reporting Recommendations for Method Sections
Population and Participants	Yes	Partially	No	N/A	Comments
Define the human or other population(s) of interest for the study, including key demographic characteristics or other parameters that limit generalizability and help operationalize who/what is under investigation, such as age, gender, language (first, second, other), proficiency (using standardized or otherwise generalizable measures), language learning/use setting.	*				Al-Sumait et al. (2022) provided a complete description of population and participants. Participants in the study were undergraduates and graduates (N = 797). There are percentages showing how participants are distributed (genre and languages).
Report precise frequencies of participants reflecting each key demographic characteristic or case feature, including how they were determined (e.g., language proficiency measures utilized and their range, minimum and maximum scores, central tendency, such as mean, and dispersion, such as standard deviation, estimates for participant scores).	*				Most of the respondents were female (66%), reflecting a social trend in Kuwaiti universities. Further, 76% of the students were between the ages of 20-24, with a mean age of the overall sample of 22.75 with a standard deviation of 6.87, and values ranging from 17 to 77 years of age. Additionally, demographic factors is the section in which the authors provided detailed information.

To answer Research Question 2, data were gathered through a final oral reflection in the last class session. Individual responses were recorded and then transcribed using the tool Otter.AI. This oral reflection included the following questions:

How familiar were you with statistical methods before the course, and how familiar are you now?

Due to the study’s nature and small sample size, inferential statistics were deemed inappropriate (Gönülal et al., 2017). Descriptive statistics (mean and standard deviation) were employed for reporting the results of both tests and the graded assignments. Qualitative data, primarily derived from the participants’ reflections, were analysed through content analysis, identifying the participants’ learning experiences, strengths, weaknesses, and motivation toward statistics. The participants’ perceptions of the course were analysed in relation to its components: objectives, content, methodology, resources, and assessment.

This study drew on applied linguistics theory and research for conceptualisation and instrument design. The instruments were reviewed and piloted by an expert in the field. To enhance triangulation and the validity of findings, data were collected from multiple sources (pre- and post-tests, assignments, and student reflections), which were compared to identify patterns and discrepancies, ensuring that findings were not solely reliant on any single measure. Member-checking was also incorporated to allow participants to provide feedback to their peers and comment on preliminary interpretations of their performance and views, thereby reducing potential misrepresentations of their perspectives.

To ensure reliability and provide an impartial perspective on participants’ engagement, interrater agreement was employed by having two researchers independently code reflection data from two randomly chosen participants. Discrepancies were resolved through discussion until consensus was reached. For the remainder of the data, I conducted iterative data analysis.

I was granted permission to conduct this study, and participation was voluntary. Given that I was also the course instructor, I mitigated potential biases that could influence data collection and interpretation by engaging in constant reflection through my lesson plans and interactions with participants about my practices and expectations throughout the pedagogical intervention. To minimise desirability bias, where participants might provide overly positive evaluations due to the researcher’s dual role, they were explicitly encouraged to critique the course and its materials. Feedback sessions emphasised the value of constructive criticism.

At the outset, all participants acknowledged their lack of statistical knowledge and reported perceiving a deficiency in the quantitative component of Colombian undergraduate and graduate applied linguistics programmes. They expressed concerns about the low statistical literacy among applied linguistics researchers globally, referencing the findings of Loewen et al. (2014, 2020). They highlighted a broader concern about the scarcity of statistical literacy research in Latin America, emphasising the need for more studies in the region, as existing research mainly focuses on universities in North America, Europe, and Asia.

For instance, Antonio stated that he had experience with item reliability but was unfamiliar with other statistical concepts. He considers that “our students are unfamiliar with statistics [and that] our academic community’s attitudes are negative toward numbers.” Claire reported that she had never taken statistics courses. She usually “skip[ped] the results of quantitative research articles and jump[ed] to the conclusions.” Atahualpa said she was “a qualitatively oriented researcher but was open to learning statistics,” while Hector claimed that “there is a need for more studies in Latin America and more training on statistics in applied linguistics programmes.”

This shared concern shaped the participants’ initial engagement with the course, with all of them acknowledging the relevance of statistics to applied linguistics in our research context.

Before the course, the participants encountered some struggles and cognitive dissonance, particularly in understanding and reporting quantitative research outputs. Their performance in the pre-test allowed them to realise the need to develop statistical literacies. However, as the course progressed—and through guided practice and peer collaboration—all participants improved their performance and reported key breakthroughs.

As shown in Figure 2, the group’s pre-test results (M = 4.6, SD = 5.3) were lower than the post-test outcomes (M = 32.2, SD = 2.9), indicating improvement in the teacher-researchers’ statistical literacy at the end of the course.

Figure 2

At the individual level, Table 5 also shows substantial gains (above 60 %) in statistical knowledge and performance, indicating advancements after completing the course. Claire and Atahualpa exhibited the greatest improvements (above 90 %) as seen in their pre-test and post-test results. Overall, these results suggest that the course is effective in fostering participants’ statistical literacy at both the group and individual levels.

Table 5

	Pre-test (35 points) Sum (%)	Post-test (35 points) Sum (%)	Individual gain (%)
Antonio	5 (14%)	27.5 (79%)	22.5 (64%)
Claire	0	33.5 (96%)	33.5 (96%)
Atahualpa	0	33.5 (96%)	33.5 (96%)
Hector	13 (37%)	35 (100%)	22 (63%)
James	5 (14%)	31.5 (90%)	26.5 (76%)
Group average M (%)	4.6 (13%)	32.2 (92%)	27.6 (79%)

Table 6 provides a detailed report of the statistical literacy components assessed in the three sections of the pre-test and post-test.

Table 6

Section 1: Descriptive statistics (numerical and graphical data)
Statistical components	# of items	Pre-test scores M (SD)	Post-test scores M (SD)
Scales	9	2.6 (3.7)	8.6 (0.5)
Variables	2	0.4 (0.9)	2.0 (0.0)
Mean	1	0.2 (0.4)	1.0 (0.0)
Median	1	0.2 (0.4)	1.0 (0.0)
Mode	1	0.2 (0.4)	1.0 (0.0)
Standard deviation	1	0.2 (0.4)	1.0 (0.0)
Histograms	1	0.4 (0.5)	1.0 (0.0)
Box and whisker plots	1	0	1.0 (0.0)
QQ & PP plots	1	0.2 (0.4)	0.8 (0.4)
Outliers	1	0	1.0 (0.0)
Subtotal	19	4.4 (5.3)	18.4 (0.5)
Section 2: Common inferential statistics (t-tests, ANOVAs, correlations, non-parametric tests) and significance level
Statistical components	# of items	Pre-test scores M (SD)	Post-test scores M (SD)
Parametric vs non-parametric tests	5	0	4.6 (0.5)
Selection of appropriate statistical tests based on a study design.	3	0.2 (0.4)	3.0 (0.0)
P-value and effect sizes interpretation	7	0	5.2 (2.4)
Subtotal	15	0.2 (0.4)	12.8 (2.5)
Section 3: Reporting of results
Statistical components	# of items	Pre-test scores M (SD)	Post-test scores M (SD)
Writing up, including descriptive and inferential results	1	0	0.6 (0.2)
Total average	35	4.6 (5.3)	32.2 (2.9)

As shown in Table 6, the participants demonstrated improvement in their knowledge of statistics across several components. Regarding Section 1, the teachers demonstrated enhanced comprehension of descriptive statistics in numerical and graphical data, which is essential for recognising patterns and assumptions in the data. In Section 2, the participants progressed in their understanding of common inferential statistics and in their ability to differentiate between parametric and non-parametric tests, choose appropriate statistical tests, and interpret alpha values and effect sizes. Section 3 also shows progress in participants’ ability to report statistical analysis results, indicating an improvement in conveying quantitative findings.

Furthermore, the three course assignments showed notable improvements in the participants’ development of statistical literacy, as evidenced by their high marks (see Table 7). Assignment 1 (critical analysis of a published quantitative study) posed fewer challenges to the participants compared to Assignments 2 and 3, which were related to performing, interpreting, and reporting quantitative data analyses and results. This suggests that it was easier for the participants to review and analyse a quantitative research report than to conduct more specific procedures. Nevertheless, the results from these assignments suggest that the participants increased their knowledge and use of descriptive and inferential statistics, assumptions, quantitative analysis, and use of statistical tools, as well as their ability to generate, interpret, and report data and critically evaluate quantitative studies.

Table 7

Assignments	Grading mark from 1 to 5 M (SD)
Assignment 1: Critical analysis of a quantitative research report (30%)	5.0 (0.0)
Assignment 2: Quantitative data analysis and reporting (30%)	4.1 (0.0)
Assignment 3: Interpretation and reporting of quantitative research results (40%)	4.3 (0.0)
Total	4.5 (0.0)

Note. The marking scale goes from 1 (poor) to 5 (excellent).

My feedback to the participants on their assignments identified their strengths and weaknesses. Regarding Assignment 1:

Regarding Assignment 2, my feedback highlighted the participants’ ability to perform data analysis and generate and interpret numerical and graphical outcomes. In contrast, participants needed to improve the written report, interpretation, and discussion of results, considering the research questions and theoretical and pedagogical implications:

My feedback for Assignment 3 highlighted the participants’ ability to interpret data, although they had to be more accurate in the written report:

After the course, most participants claimed their view of statistical literacy had changed, moving from concerns and struggles to confidence. Their reflections indicated a sense of empowerment and a shift in mindset, as they realised they could engage with quantitative data.

All the participants perceived a substantial increase in their statistical knowledge and skills, and were able to identify their strengths and weaknesses. They reported competence in identifying variables and scales; performing and interpreting descriptive and inferential statistics; using Excel, SPSS, and Langtest; selecting appropriate statistical tests; interpreting visual and numerical outputs; writing the results; and critically analysing quantitative research reports. However, they expressed a need for additional practice to strengthen concepts and procedures and enhance report-writing skills:

A shift in perception emerged as they moved from limited knowledge to conceptual understanding, recognising patterns in data and developing confidence in their interpretations.

In their reflections, the participants showed how their evolving statistical literacy reshaped their academic identity as applied linguistics researchers. They emphasised both their improved skills and their motivation and confidence in engaging with quantitative research in the future:

The participants also reported that guidance and the course resources were positive methodological strategies that influenced their learning and motivation. For instance, Claire said she “liked the practical activities that guided [her] step by step.” Similarly, Hector reported that he “loved designing tables using APA 7th style and performing analysis in SPSS and Langtest.” James also “enjoyed performing the analyses in the software programmes because [he] didn’t know about these programmes before.” This newborn confidence suggests that, beyond skill acquisition, the intervention fostered a sense of agency in participants as researchers.

The participants’ positive experience with the course allowed them to develop a critical view of the relevance of introducing statistics in applied linguistics programmes. In their reflections, they acknowledged the appropriateness of the course components in their formation. Regarding the course content, participants considered it appropriate, although it was advanced. For Antonio, “this was not an introductory course but an advanced one.” Hector said that “the feedback provided, planning, and course methodology allowed [him] to learn the course contents.” The course learning objectives were also appropriate and achieved by the participants, as Hector claimed. Concerning the course methodology, the participants valued the course planning and practical activities, which were based on real-life statistical tasks. The ten practical tasks focused on understanding, performing, and reporting data analyses and results from correlations, t-tests, and ANOVAs. Participants also designed research studies based on real issues, their contexts, and realities and analysed published quantitative articles in their research areas.

The online modality was also appropriate, although participants preferred face-to-face interactions. James, Claire, and Atahualpa agreed that “on-site classes promote interaction, but online courses are necessary for students from other regions.” Antonio mentioned that “the online modality did not negatively impact [their] learning. It’s just a matter of preference.” The course resources (e.g., statistics books in applied linguistics and social sciences, research articles, videos on statistics, and software) were relevant and authentic, as highlighted by the participants. Special guests also complemented their knowledge. Participants preferred Langtest over Excel and SPSS because it is user-friendly and practical. However, SPSS offers good visual features, including histograms, box plots, and outlier detection. The limitation of Langtest is the lack of APA format tables. JASP was introduced at the end of the course and was not sufficiently explored.

Regarding course assessment, the participants valued the detailed feedback I provided on each practical activity, which helped them scaffold their learning process. The participants expressed that the pre-test, post-test, and three assignments were appropriate for measuring their performance. As they noted, practical activities should have been graded as part of the process. On the other hand, collaborative work and group assessment allowed them to identify their strengths and weaknesses in statistics.

Based on their positive experiences and analytical views, the participants suggested that at least two statistics courses should be offered in the doctoral programme: one compulsory introductory statistics course in the first year to familiarise candidates with quantitative research designs, and an elective course in the middle of the curricular stage for advancing this knowledge.