Language assessment literacy (henceforth LAL) is an area of ongoing debate in the field of language testing. The research on this topic has focused on the components of LAL (Davies, 2008; Inbar-Lourie, 2013b), models for describing LAL (Giraldo, 2018b; Malone, 2017), definitions (Fulcher, 2012), and the shape of this construct across different stakeholders (Pill & Harding, 2013; Taylor, 2013). In essence, LAL represents the different levels of knowledge, skills, and principles required to engage in language assessment, either from a development perspective (i.e., designing and evaluating language assessments) or from a knowledge perspective, that is, understanding and using scores from assessments to make decisions about people’s language ability.

Much research, especially when it comes to language teachers, has not used the term LAL explicitly but has clearly studied areas that deal with language assessment in practice. For example, various research studies have examined what teachers do in the classroom for language assessment (Hill & McNamara, 2011; Rea-Dickins, 2001), what they think about language assessment, that is, their beliefs (Díaz, Alarcón, & Ortiz, 2012; López & Bernal, 2009), and the instruments they use for collecting information about students’ language ability (Cheng, Rogers, & Hu, 2004; Frodden, Restrepo, & Maturana, 2004; Tsagari & Vogt, 2017). Thus, it can be argued now that LAL has solidified as a general research and conceptual framework to scrutinize the meaning, scope, and depth of this construct in language testing in its three overarching components (Davies, 2008): knowledge, skills, and principles for language assessment.

A clear trend in the research has been the prominent use of psychometric measures to research LAL. Specifically, scholars have used questionnaires to study LAL as it reflects content from language testing courses (J. D. Brown & Bailey, 2008; Jin, 2010; Lam, 2015) and, in the case of language teachers, their training in LAL, current level of LAL, and needs to further their understanding of language assessment (see specifically Fulcher, 2012; Vogt & Tsagari, 2014). Given statistical interpretations, data from questionnaires can be used to sensibly derive generalizations about populations (i.e., language teachers), as the data can describe extensive aspects of LAL, including terminology and technicalities of test design. On the other hand, few studies have used a mixed-methods approach (Tsagari & Vogt, 2017) to further understand, now in depth, what LAL means for language teachers. Although quantitative and mixed-methods studies have indeed yielded useful results to conceptualize LAL, further research is needed to capture other areas of language teachers’ LAL and, therefore, provide a more valid account of what this construct means for this population.

My purpose with this article, then, is to reflect on the need to have a broader perspective towards researching the LAL of language teachers. To do so, I first explain why the focus on language teachers’ LAL is necessary and then review current research challenges surrounding LAL. Lastly, I put forward two major proposals within a post-positivist research paradigm: the need to expand LAL constructs and a related need to expand research methodologies.

Taylor (2013) explained four differential profiles of stakeholders in language assessment: test writers, classroom teachers, university administrators, and professional language testers. The author argues that these people should have different levels of knowledge, skills, and principles for doing language assessment. Such levels refer to aspects including knowledge of theory, technical skills, principles and concepts, language pedagogy, sociocultural aspects, and others. In general, the call that Taylor makes is to conduct research to examine her proposal so that the field can increase its awareness of LAL among these stakeholders.

While research for these different profiles is welcomed, language teachers have remained a central stakeholder group, arguably because they are the ones more directly engaged in doing language assessment (Giraldo, 2018b; Tsagari & Vogt, 2017). The research and discussions about language teachers’ LAL has given two central trends. On the one hand, teachers are expected to possess quite a wide array of knowledge, skills, and principles, as several authors have emphasized (Fulcher, 2012; Giraldo, 2018b; Inbar-Lourie, 2013a; Stabler-Havener, 2018). On the other hand, research has consistently shown that pre- and in-service language teachers need and want training across the board (Fulcher, 2012; Giraldo & Murcia, 2018; Vogt & Tsagari, 2014). Additionally, current studies have started to shed light on the complexity of LAL in its sociocultural milieu, that is, language teachers’ institutional contexts of assessment (Hill, 2017; Scarino, 2013; Sultana, 2019). For example, in Sultana (2019) one of the language teachers stated the following when asked about a public examination:

The excerpt above attests to the fact that the sociocultural context of language teachers shapes and even constrains their LAL (Inbar-Lourie, 2012, 2017a). Thus, the research arena in LAL, and this includes language teachers of course, is going through a process of exploration and refinement (Inbar-Lourie, 2017b). In regard to language teachers, it can be argued that their LAL involves three moments for scrutiny: the before, the now, and the after. The before in LAL refers to teachers’ prior training (or lack thereof) in language assessment. The now refers to language teachers’ current practice in language assessment and what this process implies. Finally, the after includes the level of LAL growth once teachers have finished professional development experiences in LAL; this focus includes their perceived improvement in LAL and how they put new learning into practice. In synthesis, the LAL of language teachers should be carefully studied for the following four reasons:

They are the ones most directly engaged in planning, implementing, and interpreting language assessments, with the corresponding responsibility to gauge students’ level of language ability.

Although numerous articles exist defining what LAL is, the field of language testing has not ultimately reached a consensus as to what the construct means at the granularity level. Thus, a first challenge in researching LAL is trying to operationalize what it means (Inbar-Lourie, 2013a): There is no solidified, agreed upon knowledge base. However, this is not necessarily a negative aspect of LAL-in reality, it invites further research. The complexity lies in how to operationalize the construct for research purposes.

Another related challenge is to identify who the authorities are for establishing the aforementioned knowledge base. Stabler-Havener (2018) argues that a group of scholars should come forward and formulate ways to define what LAL means and implies, specifically, for language teachers. Efforts to provide broad guidelines in language testing exist; for instance, the guidelines for practice and code of ethics by the International Language Testing Association (2000, 2007). This association was formed by scholars in language testing, and their documentation is generally taken as sound. In the case of defining LAL, however, there still is not an established body of thinkers willing to define it.

As briefly commented on earlier, language teachers’ LAL is complex, so another challenge in researching this construct is to manageably state where the concept stands for this stakeholder group. To illustrate, the works by Davison and Leung (2009) and Hill and McNamara (2011) have provided thick descriptions of what teachers do and why they do so in classroom language assessment. In Davison and Leung (2009), the authors describe what they call “key steps in teacher-based assessment” (p. 396), some examples of which are shown in Table 1.

Table 1

The actions in Table 1 can be conceptualized against the overarching components of LAL, that is, knowledge, skills, and principles. Selecting appropriate methods, for example, requires knowledge of assessment instruments so they are fit-for-purpose; the use of varied methods for assessment may require skills in design, administration, and evaluation. Finally, checking whether an assessment can be trusted reflects back on principles for doing sound assessment; specifically, an instrument whose information cannot be trusted may lead to unfair practices.

In conclusion, the LAL of language teachers embodies what is a potentially large set of knowledge, skills, and principles. For example, in Giraldo (2018b), readers can find 66 descriptors that seek to explain part of the LAL for language teachers in eight dimensions: awareness of applied linguistics, awareness of theory and concepts, awareness of one’s own language assessment context; instructional skills; design skills for language assessments; skills in educational measurement; technological skills; and awareness of and actions towards critical issues in language assessment. Additionally, as it has come to be accepted, language teachers’ contexts of assessment are one more ingredient in the LAL puzzle. Scarino (2013, 2017) has been emphatic in explaining that efforts to cultivate LAL among teachers should include acknowledgement of their life-worlds, or interpretive frameworks, where their beliefs, values, experiences, and contextual knowledge play a role in LAL.

A last challenge in this review refers to the use of questionnaires for researching LAL. As stated elsewhere, questionnaires can compile large amounts of data on varied topics of LAL, which can then be used to interpret trends in the construct. However, these instruments come with their own limitations when researching teachers’ LAL, some of which are internal to the field and others which relate to the use of questionnaires in general, as I discuss next.

Fulcher (2012) explained that the use of a survey in his study led to two problematic issues, namely, low variation in responses and the idea that teachers need to improve LAL across the board. The answers in this survey suggested that they thought “all topics within language testing are important” (Fulcher, 2012, p. 127) and needed for training. As Fulcher states, the fact that the respondents were self-selected may account for this result. This sentiment is also observed in the studies by Vogt and Tsagari (2014) and Yan, Fan, and Zhang (2017) with in-service teachers in Europe and China respectively; and Giraldo and Murcia (2018) with pre-service teachers in Colombia. The results then beg the question of whether these teachers do indeed think all of the items they see in questionnaires are truly important for their LAL. Further, Giraldo and Murcia warn that the use of pre-determined questionnaires needs to be examined carefully. In their study, the researchers used the survey developed by Fulcher and then realized it lacked a more fine-grained definition of classroom-based assessment, where issues such as portfolio use were not included.

One last internal limitation that I see in the use of questionnaires to tap into language teachers’ LAL is the use of technical jargon. To illustrate, validity in language testing and in other fields (e.g., psychology) traditionally has represented the degree to which an assessment instrument measures what it is supposed to measure and nothing else (H. D. Brown & Abeywickrama, 2010). However, it can be ascertained with confidence that this is no longer an accurate definition in language testing, which has embraced Messick’s (1989) view on the matter. So, in a survey where language teachers select validity as a concept (in fact quite a far-reaching and ongoing debate) to learn about in language assessment, some may not be aware of what the term actually implies. In other words, how can I know that something is important (or that I need training in it) if I am not sure what that something really is? Or perhaps my definition may be inappropriate or incomplete. Table 2 includes sample items from two different questionnaires for researching the LAL of language teachers. The items refer to long-standing debates in language testing and require, to my belief, a good deal of knowledge to understand them. I have highlighted them in bold and made minor modifications to the format of the original questionnaires.

Table 2

Lastly, Dörnyei and Taguchi (2010) warn practitioners of the possible disadvantages of using questionnaires, among which they explain the superficiality of answers, unreliable answers, and low levels of literacy, which the authors define as reading and writing; as the present paper implies, low levels of LAL may affect respondents’ answers and their validity. Additionally, the authors comment on social desirability (wanting to choose answers to please the researcher), self-deception (respondents deviating from what is true about them), and acquiescence bias, or what they call “yeasayers” (p. 9) who would agree with items that look right at face value. Finally, Dörnyei and Taguchi warn of fatigue effects, which can have a negative impact on the last items in a questionnaire.

To reiterate, questionnaires have been useful in researching LAL as they have allowed the field to operationalize this construct across different stakeholders. However, given the complexity of language teachers’ LAL, complementary approaches to research should be welcomed so that the field delves into the intricacies of the matter. To such end, I now move on to suggesting possible expansions of LAL research.

The reflection I am proposing is grounded on a general research philosophy. Positivist approaches to research see nature as measurable, easily observable, and quantifiable (Cohen, Manion, & Morrison, 2007). Conversely, a post-positivist approach sees reality as amenable to varied interpretations where probabilities, rather than absolute truths, are sought and understood (Guest, Namey, & Mitchell, 2013). Particularly in LAL, a post-positivist view to research embraces teachers’ LAL as situated practice. Hill (2017), for example, argues that a precondition for teachers’ LAL is a close examination of the contexts where they do assessment, what Scarino (2013) calls their interpretive frameworks or life-worlds. The implication for post-positivist research is that LAL research must look at language teachers’ reality in naturalistic contexts (Cohen et al., 2007).

Such research philosophy can have the advantage of listening to teachers’ voices regarding their LAL. As Inbar-Lourie (2017a) argues, their voices need “to be heard loud and clear” (p. 268), as this attitude can help to unveil the complexities of LAL for these stakeholders. A positivist view would not be fit for such purpose-it is not its intention, really-but a post-positivist view can be.

In test-based teaching countries such as those reported in Sultana (2019) and Baker and Riches (2017), a post-positivist and interpretive lens helped these researchers to realize that LAL is shaped by teachers’ cultures. Specifically, teachers can at times accept large-scale tests unquestioningly (Sultana, 2019), an issue Vogt and Tsagari (2014) see as problematic. Thanks to a post-positivist philosophy to LAL research, these problematic areas arise.

The ongoing research on language teachers’ LAL has provided valuable insights into what they lack, need, do, and believe. Because this research has done so, the field of language testing is expanding its boundaries to invite new research paradigms to raise awareness of the construct, which may lead to what Inbar-Lourie (2017b) calls an era of language assessment literacies. My purpose in this paper was to propose ways in which the field’s invitation can be answered.

Language teachers are constantly making decisions about student learning based on data generated by assessments. Thus, they are a crucial stakeholder group for conducting comprehensive research on LAL, especially because research studies have suggested burning needs in teachers’ LAL. To have a more fine-grained picture of LAL for this group, I propose the use of a post-positivist and interpretive research philosophy to operationalize research constructs through qualitative methodologies. Specifically, the field can benefit from research studies on language teachers’ use of local policies for assessment, design of assessment instruments vis-à-vis these policies; uses and misuses of assessments; teacher perceptions towards professional development opportunities in LAL and their impact on teacher learning; the shape and impact of language testing courses on pre- and in-service teachers; and, overall, the impact of these programs once teachers are implementing new ideas and approaches to language assessment.

To tap into the aforementioned constructs, I suggest qualitative methods for data collection, including interviews, document analysis, and observations. Such research will not only listen to teachers’ situated LAL voices and their messages-loud and clear-but also use such data to further conceptualize LAL. The methods will also allow for a more complete, informative picture of the expanded research construct on LAL and, in turn, unveil the intricacies of the matter. Collectively, this information will be useful for practitioners (e.g., professional language testers, language teacher educators) to engineer approaches to support language teachers to improve their LAL, which will hopefully have a positive impact on student learning. That should be the ultimate goal of researching language teachers’ LAL.