Monash University Publishing | Contacts Page
Monash University Publishing: Advancing knowledge

Learning Discourses and the Discourses of Learning

CHAPTER 17

WASHBACK OF HIGH-STAKES ASSESSMENT

YEAR 12 JAPANESE

This chapter will examine the Victorian Certificate of Education (VCE) Japanese Second Language assessment in terms of the notion of ‘washback’ and its effects on teaching. The study investigates how the VCE curriculum is implemented in classrooms, and examines the relationship between curriculum, teaching and assessment. In order to explore the nature and scope of the washback, interviews were conducted with VCE teachers of Japanese.

Factors that may be affected by washback (such as implemented curriculum, teaching materials, content of teaching, and teaching approach and method) and factors that may affect washback (such as teacher variability, the school, and the perceived stakes of the assessment) are discussed. The current study identifies a gap between externally prescribed curriculum and courses implemented by individual teachers, and argues that this discrepancy derives to a large extent from the washback of the assessment on teaching.

INTRODUCTION

It is widely believed in the fields of education and applied linguistics that testing influences teaching and learning. This concept is referred to as ‘washback’ (Wall and Alderson 1993), ‘backwash’ (Hughes 1993, cited in Bailey 1996: 262–264), or ‘test impact’ (Bachman and Palmer 1996). According to Eckstein and Noah (1993), washback exists in any type of assessment in which test results affect test-takers’ futures, and thus are regarded as high-stakes tests. Examples of such tests are the Scholastic Assessment Test (SAT) in America, the Higher School Certificate (HSC) in New South Wales, Australia, and the Victorian Certificate of Education (VCE) examinations in Victoria, Australia, which are used for university selection.

Although the importance of washback studies has been recognised, research to date has not fully established the nature and scope of washback on teaching and learning. Moreover, whereas the majority of washback research has been undertaken within the field of English language teaching, washback research on languages other than English has not received as much attention. In addition, washback research has often ignored the relationship between externally prescribed curriculum, assessment and implemented curriculum by individual teachers, although prescribed curriculum and assessment would be the powerful determinants of implemented curriculum (Adamson 2004; Little 2003). The focus of this study is on the relationships between the prescribed curriculum, assessment and implemented curriculum in the context of year 12 Japanese programs in Victoria. The two questions that are addressed in this study are:

  1. What is the washback of the assessment?
  2. What is the relationship between the prescribed curriculum, the requirements of the assessment and actual programs implemented by teachers?

OVERVIEW OF THE VCE JAPANESE SECOND LANGUAGE PROGRAM

The Victorian Certificate of Education refers to the final two years of schooling in Victoria, Australia. Curriculum and assessment are set by a central authority, and students’ results in their final year (year 12) contribute directly to the score utilised by universities for selection into courses. Thus, the VCE assessment can be said to be high-stakes. Japanese is one of a number of Languages Other Than English (LOTE) which are offered, and which share a common curriculum and assessment design. The VCE curriculum emphasises ‘the overall education of students’ in the areas of communication, cross-cultural understanding, cognitive development, and literacy and general knowledge’, which would provide students with ‘access to the culture of communities which use the language, and promote understanding of different beliefs and values within and beyond the Australian community’ (Victorian Curriculum and Assessment Authority [VCAA] 2004: 7). To achieve these aims, the VCE curriculum recommends ‘[a]ny communicative teaching method or combination of methods’ (p. 41). This might be based on the belief that communicative language teaching (CLT) brings about increased use of TL for purposeful language use in the classroom, which matches with the communication aims of the curriculum.

The VCE LOTE examinations are comprised of two parts, the School-Assessed Coursework (SAC) that makes up 50 per cent of the final mark, and the external examinations, that include the oral examination (12.5 per cent) and the written examination (37.5 per cent) (VCAA 2004). The nature and number of SACs, as well as the marking criteria for assessment, are prescribed externally, but the tasks themselves are set and marked internally by the classroom teacher. The VCE Japanese examinations are held annually, and are taken by approximately 1,200 candidates each year, usually in the final year (year 12) of their secondary schooling (VCAA 2005).

WASHBACK RESEARCH

Although the existence of washback is widely acknowledged, not much attention was paid to identifying the nature of washback until Alderson and Wall (1993: 120–121) presented 15 hypotheses on washback, which are as follows:

  1. A test will influence teaching.
  2. A test will influence learning.
  3. A test will influence what teachers teach; and
  4. A test will influence how teachers teach.
  5. A test will influence what learners learn; and
  6. A test will influence how learners learn.
  7. A test will influence the rate and sequence of teaching; and
  8. A test will influence the rate and sequence of learning.
  9. A test will influence the degree and depth of teaching;
  10. A test will influence the degree and depth of learning.
  11. A test will influence attitudes to the content, methods, etc. of teaching and learning.
  12. Tests that have important consequences will have washback; and conversely
  13. Tests that do not have important consequences will have no washback.
  14. Tests will have washback on all learners and teachers.
  15. Tests will have washback effect for some learners and some teachers, but not for others.

Subsequent to Wall and Alderson’s study, Alderson and Hamp-Lyons (1996: 295–296) in their research on TOEFL preparation classes, presented an expansion of the fifteenth Washback Hypothesis:

15’. Tests will have different amounts and types of washback on some teachers and learners than on other teachers and learners.

This expansion of the Washback Hypothesis has taken into account the variability of teachers, learners and educational contexts to better understand the complexity of washback.

A number of studies (e.g. Alderson and Hamp-Lyons 1996; Alderson and Wall 1993; Andrews et al. 2002; Cheng 1999; Shohamy et al. 1996; Wall and Alderson 1993; Watanabe 1996a; Watanabe 1996b) have examined the impact of assessment on teaching and learning, and suggested that it can lead to positive or negative washback, depending on whether or not the processes it encourages contribute to learners’ actual interlanguage development (Alderson and Wall 1993:Bailey 1996).

Spratt (2005) was concerned with factors affected by washback such as curriculum and materials, and factors affecting washback such as teachers, the schools and the exam itself. With respect to curriculum, Wall and Alderson (1993) reported that both positive and negative washback were observed in the content of teaching. The positive washback they identified was that teachers taught the content in the textbook to bring about higher marks in the examination. The negative washback was that teachers did not teach the textbook thoroughly, since they realised that some skills would not be assessed in the examination. On the contrary, no evidence of washback on teaching methodology was identified in Cheng’s research on the revised high school examination in Hong Kong. Cheng (1999: 268–269) argued that the reason why teachers did not change teaching methodology was that the examination did not influence teachers’ beliefs about teaching and learning. On the other hand, Alderson and Hamp-Lyons (1996) found that the examination influenced both the content and teaching method, but the effect varied from teacher to teacher.

Spratt (2005) contended that instructional materials may be influenced by washback, as had earlier been identified in Alderson and Hamp-Lyons (1996: 285). The authors claimed that in order for learners to be well prepared for the examination, teachers utilised commercial materials that had the same format as the TOEFL examination. Watanabe (1996a) reported that some teachers employed self-made materials in the exam preparation class to help students achieve higher results in tests. In this respect, Smith (1991: 10) claimed that teacher-designed materials might result in ‘narrowing of the curriculum’.

In relation to teacher variability, Watanabe (1996a) investigated two teachers teaching the same courses, and reported that they employed quite different teaching methodology. He argued that teachers’ educational background affects the way they teach. Another study by Watanabe (1996b: 232) suggested that not only teacher variability, but also the difference between schools was significant in washback. Shohamy et al. (1996) emphasised the role of the school, which can either promote or inhibit positive washback, since the type and degree of the support that teachers could receive differed in schools.

Popham (1987) argued that a high-stakes test would drive the teacher to prepare students for the test. Shohamy et al. (1996) investigated the washback on the modification of two tests, Arabic as a Second Language (ASL) and English as a Foreign Language (EFL) in Israel. Whereas the modification in ASL did not produce any observable change, the slight modification in EFL yielded a significant change in terms of classroom activities, materials, increased time allotment, enhanced anxiety, fear and pressure on teachers, students and parents. Shohamy et al. claimed that this is because the ASL test was considered a low-stakes test as opposed to the EFL test, which was viewed a high-stakes test.

The general picture that emerges from the literature above is that the phenomenon of washback is much more complex than it first seems. It would seem that there are a diverse range of factors influencing and influenced by washback, and they are all interrelated and interacting with each other (Cheng 1999: 2). Furthermore, Cheng contends that an examination carried out in different educational contexts taken by different participants at different times would reveal quite different pictures, since washback is a complex phenomenon in which various factors interact with each other (pp. 7–8).

DETAILS OF THE STUDY

Few of the above studies closely examined the linkage between externally prescribed curriculum and assessment, and their effects on teaching, although it can be expected that curriculum and assessment would be two of the most powerful determinants of teaching (Adamson 2004;Little 2003). Consequently, the present study aims to investigate how the externally prescribed curriculum is implemented and how the assessment tasks and examinations, which have been designed for and with that curriculum, affect its implementation. I will draw on Alderson and Wall’s (1993) Washback Hypotheses and Spratt’s (2005) categorisation of factors affected by washback, such as implemented curriculum, and teaching approach and method; and factors affecting washback such as teacher variability, the school and the perceived stakes of the assessment.

The study utilises curriculum evaluation, analysis of instructional materials, and semi-structured interviews to examine how the externally prescribed curriculum is implemented in classrooms, and the relationships between curriculum, assessment and teaching. Interviews with three female teachers (two Anglo-Australian teachers and one native Japanese teacher) were undertaken to examine not only materials, but also teachers’ perspectives on their teaching, students’ learning, and the difference in their teaching contexts. Table 1 presents information about the background of informants. For convenience, the informants are named A1, A2 and J1 (A = Australian; J = Japanese).

Table 1 Background of the informants

FACTORS AFFECTED BY WASHBACK

IMPLEMENTED CURRICULUM

CHOICE AND USE OF INSTRUCTIONAL MATERIALS

The interviews revealed an interesting aspect of washback on material production and selection. The teachers employ teaching materials, which they believe would help their students be well prepared for the VCE assessment.

In most secondary schools in Victoria, including all the schools where my informants teach, Kookoo Seikatsu (life in senior secondary school) Book 1 (Aitchison 2001a) and 2 (Aitchison 2001b) are used as the main textbooks for VCE students. However, the use and dependency on the textbook varies across the teachers. A1 uses the textbook to introduce grammatical patterns, and her students complete the drill exercises in the textbook, but skip most of the reading texts. A2 also skips some of the reading texts if she thinks the texts do not suit her students, in terms of the difficulty, length and content, or if the texts are not compatible with the SACs that A2 writes. On the contrary, J1 attempts to cover not only grammatical patterns, but also the reading texts in the textbook. This is because J1 believes that the more learners read Japanese texts, the better they could be prepared for tests. Consequently, it seems that dependency on the textbook varies across teachers and it is influenced by washback. The fact that the washback is different for different teachers relates to their different perceptions of the demands of the assessment tasks, and of the best ways of preparing for them.

In order to supplement the weaknesses of Kookoo Seikatsu; that is, the points on which teachers judged the textbook to be incompatible with or insufficient as preparation for the VCE assessment, the teachers introduced other instructional materials in the classroom. For example, A1 produces her own materials with the help of her Japanese husband and the Japanese husband of her colleague and a native Japanese assistant, in order to explicitly introduce certain text types, which appear in the SACs that she writes. A1 said that she needs to produce materials, especially listening and reading texts, since she writes listening and reading SACs. A1 first writes the SACs, and then creates some texts that contain the same text types and the same or similar vocabulary, grammar elements, and kanji so that her students can be well prepared.

J1 reported that she always conducts two or three practice SACs, which have the same format and contain similar elements to the SACs. As a consequence, it seems that the production of instructional materials is significantly affected by washback. In this respect, Smith (1991: 10) claimed that teacher-designed materials that focused on certain tests, could result in ‘narrowing of the curriculum’, which could be the case with my informants in their VCE class. However, it could also be said that the materials that the informants in the current study produced are widening the curriculum, compared to the textbook.

The interview showed that not only material production, but also material selection was affected by washback. For example, A2 reported that since the main textbook does not introduce a sufficient number and types of text types and kinds of writing, she utilises other commercial textbooks to provide her students with a range of texts. A1 stated that although she employs a variety of authentic materials for her year 7, 8, 9 and 10 students such as websites, advertisements and various realia to motivate her students, she does not use any authentic materials in her VCE class. This is because A1 believes that authentic materials are not utilised in any VCE examinations. Furthermore, although the use of ICT is strongly advocated in the study design, A1 does not use it in the VCE class. A1 noted that she utilises computers in her junior classes such as typing Japanese script, searching information for cultural projects or using PowerPoint for presentation, yet, she does not do so in her VCE class, because computer skills such as Japanese typing skills are not tested in any VCE assessment.

In terms of listening materials, Kookoo Seikatsu does not include any listening exercises. Thus, all the three teachers utilise a textbook, Excel (Reekie 2000) which is published for the Higher School Certificate (HSC) in NSW. The listening tapes are available for Excel, yet the format of the listening questions follows the HSC, in which all the questions are to be answered in English, which is incompatible with the VCE. Moreover, some teachers in the VCE complain that the speech speed of the listening texts in Excel is too slow in comparison to the listening section in the VCE written examination. All the teachers in this study reported the utilisation of the listening materials, ‘Listening tasks for senior school students of Japanese’ published by the MCJLE with the help of a group of Japanese teachers in Melbourne (MCJLE 2001). This listening pack comes with three CDs, questions and the script of the texts. This listening material was produced to meet the needs of VCE teachers and learners. Before this listening pack was released, many VCE teachers complained that there were no adequate listening materials, which could prepare their students for the examination, since the listening tasks in the examination were far more difficult than any available materials. Further, the content of the listening pack reflects the VCE written examination, in terms of a range of text types, complexity of texts, speech speed and contact situations. Moreover, this listening package has the same format as the listening section in the written examination.

CONTENT OF TEACHING
VOCABULARY AND KANJI

All the informants believed that enhanced vocabulary and kanji are crucial skills for students to achieve a higher score in the assessment. The teachers acknowledged that the best way for their students to learn vocabulary and kanji is by learning them in contexts such as in listening, reading or speaking. However, the tight curriculum schedule of year 12 students do not allow the teachers to implement what they think is the best for their students. A2 commented that ‘Conversation using (newly introduced) vocabulary would help, but no time. It’s important, but so many other things to do, so run out of time’. A1 stated, ‘Repetitive reading is the one, and they can make cards, but I’m not really into that. We haven’t got enough time for games. The more you read the text is the best way’. A1 also said that she uses quizzes to encourage her students to build vocabulary. She said, ‘Start from the early years in quizzing. Year 10 still quizzing vocab. I don’t quiz as much as I should in year 12, still do, once a fortnight’.

The use of quizzes is also reported by the informants in teaching of kanji. The interviews revealed that the most common way of teaching kanji is quizzing. A2 reported that for junior year levels, she employs games to promote kanji learning, but for year 10, 11 and 12, A2 introduces three kanji in each lesson, and conducts a kanji quiz every five lessons. J1 reported that her students have to buy the Kookoo Seikatsu kanji book (Aitchison 2000). Her students learn 10 kanji every week by themselves, and the 10 kanji are tested in a weekly kanji quiz in J1’s class. J1 said that in this way, her students learn all the 200 prescribed kanji by the end of term 1, year 11, which leaves more time for her students to be familiarised with and to practice the prescribed kanji in contexts such as in reading or writing texts. In this respect, the washback of the VCE assessment in relation to teaching of vocabulary and kanji extends beyond year 12 into previous years.

GENRE

The VCE curriculum prescribes 43 text types, involving 22 text types that students might be required to produce in the written examination, and 21 text types for receptive skills. A2 commented that although she introduces as many text types as possible in year 10 and 11, it is still quite challenging for her year 12 students to master text types, since rules on the use of genkooyooshi (squared paper traditionally used for formal compositions) vary from one text type to another.

All informants reported that students are required to produce only certain kinds of text types in the assessment such as personal letter, formal letter, speech script, story, diary entry and personal account. Consequently, they only focus on these text types in the classroom. It is interesting to note that although teachers acknowledge that text types such as story, diary entry and personal account frequently appear in the writing section in the written examination where students need to select one out of five questions, they tell their students to avoid choosing questions that involve such text types. This is because these text types require more linguistic and cognitive skills than other text types such as personal letter, formal letter or speech script. Thus, in the classroom, A2 consciously focuses on letter and speech script, but not other text types. Such a focus on certain text types demonstrates ‘narrowing’ of the content of teaching, which is influenced by washback.

The study design provides a generic guide for all languages other than English (LOTEs) on the main characteristics of common text types and kinds of writing (VCAA 2004: 66–67). The VCAA also published a supplementary booklet for the VCE Japanese, which provides advice on the use of genkooyooshi (lined paper for writing Japanese) for some of the prescribed text types (VCAA 2000). Yet, the booklet does not cover some of the prescribed text types such as email. Moreover, although the features of kinds of writing vary across LOTEs, the curriculum does not provide any language specific advice on kinds of writing. This might create confusion for teachers and students of all LOTEs, especially, LOTEs such as Japanese which is non-roman alphabetical and character language, coming from a very different cultural background to English. A great number of VCE teachers raised concerns about the use of genkooyooshi. The interviews revealed teachers’ frustration in teaching text types, relating especially to the rules for use of genkooyooshi. A1 commented that:

There really isn’t, with using genkooyooshi, I suppose come from experience and enough books tell you how to use it. Hardest one of the text types is email, and how to set in genkooyooshi. It is really silly. The person really should know the rules. Letters too. In reality, there is no set format. I probably go with whatever PDs I’ve been to, how to set out genkooyooshi. The text types, excel (name of a text book) has a variety of text types, article, diary. That’s really the way I go. There is no ‘This is it’. I don’t think it should be either. We have to be flexible. For example, article comes with variety. Examiners need to be flexible in using genkooyooshi. I think they are.

A1 expressed the view that it is not always appropriate to use genkooyooshi, because most of the prescribed text types ‘would not naturally be written on genkooyooshi, so writing the acceptable format and conventions is very difficult’ (MCJLE 2006a). Despite this, A1 teaches the rules of genkooyooshi, because the assessment employs genkooyooshi.

As seen in A1’s comment above, she does not know what rules or standards examiners apply for the acceptable use of genkooyooshi, since A1 has never been appointed as an assessor for the written examination. In response to concerns about this situation and a lack of response from the authorities, the MCJLE published a booklet on the use of genkooyooshi in 2006 (MCJLE 2006b), although this booklet is not endorsed by the VCAA. This genkooyooshi booklet could be said to be a by-product of the washback of the VCE assessment in that it actually established rules for genkooyooshi that will never be used outside of the VCE assessment context. The publication of instructional materials by the MCJLE is consistent with Hughes’ (1993, cited in Bailey 1996: 262–264) trichotomy of the backwash model which suggests that the ultimate product of washback is learners’ interlanguage development, but also it could include the production of new materials by publishers.

The study of genres can be employed to facilitate learning of linguistic elements in relation to culture, since genres embody culturally influenced social practices involving language. However, in teaching genres such as text types, the informants focused only on linguistic items and the use of genkooyooshi. This might be because, as reported in this section, the informants were teaching to the assessment.

TOPIC

The teachers claimed that decisions about the scope and sequence of the content are deeply affected by the VCE assessment. Although the externally prescribed curriculum provides teachers with teaching content such as themes and topics, grammatical elements, text types and kinds of writing, it does not prescribe the scope and sequence of the content. In other words, the rate, sequence, degree and depth of teaching are left up to teachers.

A1 reported that in her year 11 class, she teaches sub-topics that might not appear in the written examination, and introduces frequently appearing sub-topics in greater depth in her year 12 class. A1 added that she spends more time in class when she teaches some topics such as technology and school life, which, she claims, appear frequently in the written examination. This is because she believes that this will help her students to perform better in the written examination. Further, J1 said that she covers one of the prescribed themes, the personal world, in her year 11 class, since the first half of the oral examination requires students to talk about their personal world. If her students complete this theme in year 11, they are able to start conversation practice for the oral examination well in advance.

J1 maintained that in assessing her year 10 and 11 students, she utilises the year 12’s marking criteria. J1 claimed that the employment of the year 12 assessment scheme in year 10 and 11 tests would increase the awareness of the VCE assessment in earlier years, which would lead her year 10 and 11 students to achieve higher results in the VCE examinations. This is another example of the washback of the VCE assessment extending beyond year 12 into previous years.

The above findings revealed that the washback of the VCE assessment is significant in teaching topics. As reported above, washback was found in the rate, sequence, degree and depth of teaching, thus, of learning. This is consistent with Alderson and Wall’s (1993: 120–121) Washback Hypotheses, which are: ‘A test will influence the rate and sequence of teaching’, hence, ‘A test will influence the rate and sequence of learning’; and ‘A test will influence the degree and depth of teaching’, consequently, ‘A test will influence the degree and depth of learning’.

SYLLABUS DESIGN

The curriculum emphasises that all areas of study such as themes and topics, text types, kinds of writing, vocabulary and grammar, should be introduced in an integrated way in implemented courses, to help VCE students achieve the curriculum outcomes (VCAA 2004: 12). In other words, the curriculum documents suggest the employment of topical, situational, functional, genre-based, structural and task-based focuses in the design of the syllabus. It is individual teachers’ responsibility to select an appropriate syllabus for their VCE program, which should derive from the situational analysis of their teaching contexts.

However, all the informants reported the employment of a primarily structural syllabus in their classrooms. This is because the teachers believe that the prescribed assessment criteria emphasise mastery of grammar, kanji and vocabulary. In order to help their students to achieve a higher mark in the assessment, the teachers employed a structural syllabus as the basis of their curriculum design. The informants were also conscious that they focus on the prescribed topics, text types and kinds of writing. Thus, it appears that the teachers utilised topical and genre-based sub-syllabuses, although a structural syllabus was the dominant syllabus focus in the classrooms.

These findings are consistent with four of Alderson and Wall’s (1993: 120–121) 15 hypotheses: ‘A test will influence teaching’, consequently, ‘A test will influence learning’; and ‘A test will influence what teachers teach’, which leads to ‘A test will influence what learners learn’.

TEACHING APPROACH AND METHOD

To help students to achieve a higher mark in the assessment, the teachers focused on teaching of test taking strategies. A2 reported that ‘It is difficult to focus on the written exam, since I don’t know what’s in there. But I focus on techniques, the same in reading. Skim read, looking for answers’. A1 noted that she always encourages her students to employ strategies when completing listening exercises. When A1’s students do listening exercises in class, they are always reminded to read the questions first, and not to attempt to understand the whole text, but to pick up key words responding to the questions. A1 maintained that her year 12 students are well prepared for the written examination, since they are exposed not only to the language skills that they might be required in the examination, but also to the strategies of how to respond to the tasks appropriately and effectively.

Faerch and Kasper (1983) claimed that second/foreign language learners utilise communication strategies to compensate for their lack of linguistic knowledge. They argued that communication strategies employed by L2 learners might include ‘topic avoidance’, which occurs when a chosen topic requires a speaker to employ linguistic forms that are problematic to the speaker. Instead, the speaker changes the topic to achieve communicative goals. Informal conversations with experienced VCE teachers of Japanese revealed that test taking strategies such as topic avoidance, are emphasised in the classroom. Students are trained to lead assessors to ask questions on certain topics that they are well prepared for, by strategically changing the subject.

Teaching of test taking strategies in the VCE classroom might be encouraged by the teachers’ belief that the assessment does not test underlying communicative skills so much as carefully prepared conscious knowledge. This might lead the teachers to believing that CLT is not the best way to prepare learners for the assessment in the limited class time, even though CLT is recommended in the study design (VCAA 2004: 41). A2 commented that she utilises CLT in her junior year classes, but not in the VCE level, since she believes that the VCE assessment requires students to accurately demonstrate linguistic skills. Another reason the teachers did not employ CLT is that they believe that accuracy is emphasised in the assessment. This conflicts with CLT which stresses fluency, while accuracy is less strongly emphasised.

The failure to employ CLT in the VCE classroom is clearly apparent in the preparation process for the oral examination. It is probable that test developers included the oral component to promote the development of communication skills, which reflects the curriculum aims. However, in preparing for the oral examination, A2’s students write drafts, and learn the script by heart, which allows them to conduct a conversation in Japanese. This strategy seems to be a common practice among Japanese teachers and students, according to informal conversations with a number of VCE teachers. Drafting and memorising conversation scripts encourages rote learning, but not the development of communication skills, which is intended by curriculum developers. This rote learning might help students speak Japanese, yet may encourage the use of formulae, but not flexible, fluent and meaningful use of Japanese. The rote learning might be due to the nature of the oral examination, because teachers and students know the topics covered in the oral examination in advance.

The VCE curriculum implies that teachers need to employ a teaching approach/method that allows students to achieve nonlanguage outcomes listed in the prescribed curriculum such as problem solving, teamwork, communication, use of information and communications technology, planning and organising, self-management and initiative (VCAA 2004: 43). In particular, it seems important for students to learn ICT skills such as electronic forms of communication and wordprocessing, in that such skills are essential in communicating with people in the modern world. Nonetheless, no nonlanguage outcomes are tested in the assessment, and thus teachers do not pay a great deal of attention to teaching methods that would bring about the achievement of the nonlanguage outcomes. Instead, teachers utilise an ‘academic’ style teaching approach designed to equip learners with the knowledge and skills which they need for the assessment. This is another example of how teaching methodology is greatly affected by washback.

The findings of the current study regarding the teaching approach contrast with Wall and Alderson’s (1995) and Cheng’s (1999) research, which identified that while content of teaching was affected by washback, teaching methodology was not influenced by washback. However, the findings of the current study are consistent with the fourth and sixth of Alderson and Wall’s (1993: 120) Washback Hypotheses: ‘A test will influence how teachers teach’; hence, ‘A test will influence how learners learn’, as well as the eleventh hypothesis; ‘A test will influence attitudes to the content, methods, etc. of teaching and learning’.

FACTORS AFFECTING WASHBACK

TEACHER VARIABILITY

The influence of the assessment on implemented curriculum has been discussed above, and it is clear that not all teachers were influenced by the assessment tasks in exactly the same way. Teachers’ variability was also identified with reference to what and how teachers teach. A1 reported that she always tells students that everything they have learned in class will be on the SACs, thus harnessing the washback to motivate students. It is in this way that A1 attempts to engage her students with SACs.

The interviews provided the researcher with the impression that among the informants, A1 is the one who is most affected by the impact of the tests. This could be seen by the utilisation of past examination papers. A1 commented that she frequently uses tasks from past examination papers as a class exercise, so as to familiarise her students with the written examination format. On the other hand, A2 and J1 usually only employ past examination papers for practice examinations during the school practice examination period. Thus, it is clear that the washback effect differs from teacher to teacher. This finding is consistent with Alderson and Hamp-Lyons’ (1996: 295–296) expansion of the fifteenth Washback Hypothesis; ‘Tests will have different amounts and types of washback on some teachers than on other teachers’.

THE SCHOOL

The school plays an important role in washback in terms of support for or pressure on teachers. Pressure from the school was reported by J1 who work at a prestigious private school. At J1’s school, after the study score is released, the school principal checks the study score of each student and each subject against the general achievement test (GAT) score to see whether the student achieved as well as they could have. J1 reported that if any of J1’s students obtain a lower score than expected, as compared with GAT, J1 has a meeting with the principal to discuss the matter. According to J1, this quality assurance procedure pushes J1 to teach to the tests. Thus, it appears that the school can be a factor affecting washback. On the contrary, A2 reported that her school does not evaluate her based on her students’ study score. She commented that the school does not put any pressure on her to produce a higher score in the VCE assessment, and she highly values this freedom and confidence shown in her by the school. A2 added that she believes that teachers can help students to learn, but students should take responsibility for their own learning. To sum up, it appears that there is a significant difference between schools in terms of support for and pressure on teachers even within this small and biased sample.

THE PERCEIVED STAKES OF THE VCE ASSESSMENT

Washback experienced by schools and teachers was discussed in the preceding sections. It is probable that this washback effect found in schools and teachers is due to the high-stakes nature of the VCE assessment. For example, as mentioned previously, if the schools consider the assessment as high-stakes, they treat the study score seriously, which was identified in J1’s school reported earlier. Furthermore, my informants also perceived the assessment as high stakes, which led them to teaching for the SACs and external examinations to produce the highest possible marks. The interviews also identified that the washback of the high-stakes assessment is exerted not only on schools and teachers, but also on students, which was reported by A2. She commented that:

The biggest effect that it has is that it (the VCE assessment) makes them (students) very depressed. Because they (the VCE examinations) are very difficult... they (students) want to be able to understand everything, and they want to be able to read something easily. Well, it’s not gonna be like that. For them (students) it’s a very threatening thing. Because when we do the practice ones, every exam, I tell them that it’s silly if they didn’t expect there to be something in the exam they didn’t understand, but it’s about conquering the fear and focusing on the answers on questions and things like that... One of our students was very distressed because she said that ‘There’s nothing. I don’t have anything that I can study for the end of year exam’.

The washback of the VCE assessment on A2’s students was identified in the form of enhanced anxiety, fear, pressure and a desire to focus study efforts on items that are relevant to examination performance, effects which were also found in Shohamy et al.’s (1996) study. To sum up, it would appear that the washback of the high-stakes VCE assessment, is consistent with the twelfth of Alderson and Wall’s (1993: 120) Washback Hypotheses, ‘Tests that have important consequences will have washback’.

CONCLUSION

The current study explored the washback of the VCE assessment, and the relationship between the externally prescribed curriculum, assessment and implemented curriculum by individual teachers. Utilising Wall and Alderson’s (1993) Washback Hypotheses and Spratt’s (2005) washback categorisation, I examined factors affected by washback such as implemented curriculum, and teaching approach and method, and factors affecting washback such as teacher variability, the school and the perceived stakes of the assessment. With reference to implemented curriculum, each VCE teacher is required to write a VCE course based on the externally prescribed curriculum, by reflecting their own teaching context. To deliver the required content and help students achieve higher marks, the teachers put significant effort into designing and implementing their own curriculum, through choice and use of instructional material, content of teaching, and teaching approach and method, and this study revealed that implemented curriculum was significantly influenced by washback. Aspects of the external curriculum which were not assessed (such as ability to use technology) were ignored or even avoided by teachers, whereas those aspects considered important to test performance (such as grammatical accuracy) were a key focus in planning for the implemented curriculum.

The current study also found teacher variability in washback. Not all teachers were influenced by the assessment tasks in the same way, which might have emerged from the difference in the perceived stakes of the assessment, as well as other differences in teacher beliefs. The variability was also identified in schools. Some schools put more pressure on teachers for their students to obtain higher scores, which led teachers to teach to the tests. This might be derived from the high-stakes nature of the assessment. Not only schools and teachers, but also students were affected by the high-stakes nature of the tests. This was identified in the form of increased anxiety, fear, pressure and a desire to focus study efforts on items that are relevant to examination performance.

The findings in the current study suggest important implications for curriculum planners and LOTE teachers. Notably, as identified in the current study, washback is very strong in the VCE context. Thus, curriculum developers ought to take washback into careful consideration in writing assessment specifications in order to promote positive washback and inhibit negative washback. Moreover, this study revealed gaps between the prescribed curriculum and implemented curriculum. The former is developed by VCE curriculum planners and the latter by VCE teachers. Hence, it would seem to be beneficial for both curriculum planners and teachers to have an opportunity to directly exchange their views on the curriculum and assessment. This would help identify problematic and ambiguous areas related to the VCE system such as the intention and interpretation of the curriculum, and feasibility of prescribed curriculum in practice. This would facilitate bridging the currently existing gaps between the externally prescribed curriculum and implemented programs.

The washback literature (e.g. Cheng 2000;Watanabe 2004) indicates that teachers’ perceptions obtained by interviews may not be reliable since what teachers think they do might be quite different from what they actually do. Longitudinal classroom observation ought to be utilised to investigate washback in the VCE context in the future. Such research, in conjunction with follow-up interviews with teachers and students, will yield the nature and scope of washback in more detail, and offer valuable insights into the relationship between the externally prescribed curriculum, assessment and programs implemented by teachers.

REFERENCES

Adamson, B. 2004. ‘Fashions in language teaching methodology’. In The Handbook of Applied Linguistics, edited by Davies, A; Elder, C. Malden, Massachusetts: Blackwell.

Aitchison, K. 2001a. Kookoo Seikatsu Book 1: Japanese for Secondary Students. South Melbourne: Macmillan.

Aitchison, K. 2001b. Kookoo Seikatsu Book 2: Japanese for Secondary Students. South Melbourne: Macmillan.

Aitchison, K. 2000. Kookoo Seikatsu Kanji Workbook: Japanese for Secondary Students. South Melbourne: Macmillan.

Alderson, J. C; Clapham, C; Wall, D. 1995. Language Test Construction and Evaluation. Cambridge: Cambridge University.

Alderson, J. C; Hamp-Lyons, L. 1996. ‘TOEFL preparation courses: A study of washback’. Language Testing 13 (3): 280–297.

Alderson, J. C; Wall, D. 1993. ‘Does washback exist?’ Applied Linguistics 14 (2): 115–129.

Andrews, S; Fullilov, J; Wong, Y. 2002. ‘Targeting washback – A case study’. System 30: 207–223.

Bachman, L. F; Palmer, A. S. 1996. Language Testing in Practice: Designing and Developing Useful Language Tests. Oxford: Oxford University.

Bailey, K. M. 1996. ‘Working for washback: A review of the washback concept in language testing’. Language Testing 13 (3): 257–279.

Cheng, L. 2000. ‘Washback or Backwash: A Review of the Impact of Testing on Teaching and Learning’. ERIC ED442280.

Cheng, L. 1999. ‘Changing assessment: Washback on teacher perceptions and actions’. Teaching and Teacher Education 15: 253–271.

Eckstein, M. A; Noah, H. J. 1993. ‘The politics of examinations: Issues and conflicts’. In Secondary School Examinations: International Perspectives on Policies and Practice, edited by Eckstein, M. A; Noah, H. J. New Haven: Yale University Press.

Faerch, C; Kasper, G. 1983. Strategies in Interlanguage Communication. London: Longman.

Little, D. 2003. ‘Learner autonomy and public examinations’. In Learner Autonomy in the Foreign Language Classroom: Teacher, Learner, Curriculum and Assessment, edited by Little, D; Ridley, J; Ushioda, E. Dublin: Authentik.

Melbourne Centre for Japanese Language Education. 2006a. ‘New resource!!!!! Writing rules!! A text type guide for senior secondary students’. MCJLE News 33 (February): 2.

Melbourne Centre for Japanese Language Education. 2006b. Writing Rules!! A Text Type Guide for Senior Secondary Students. Clayton, Victoria: Melbourne Centre for Japanese Language Education.

Melbourne Centre for Japanese Language Education. 2001. Listening Tasks for Senior School Students of Japanese. Melbourne: Melbourne Centre for Japanese Language Education.

Popham, W. J. 1987. ‘The merits of measurement-driven instruction’. Phi Delta Kappa 68 (9): 679–682.

Reekie, F. 2000. Excel Senior High School Japanese: Study Guide and Exam Preparation. Glebe, New South Wales: Pascal Press.

Shohamy, E; Donitsa-Schmidt, S; Ferman, I. 1996. ‘Test impact revisited: Washback effect over time’. Language Testing 13 (3): 298–317.

Smith, M. L. 1991. ‘Put to the test: The effects of external testing on teachers’. Educational Researchers 20 (5): 8–11.

Spratt, M. 2005. ‘Washback and the classroom: The implications for teaching and learning of studies of washback from exams’. Language Teaching Research 9 (1): 5–29.

Victorian Curriculum and Assessment Authority. 2005. Statistical Information: VCE Assessment Program. East Melbourne: VCAA.

Victorian Curriculum and Assessment Authority. 2004. Japanese Second Language: Victorian Certificate of Education Study Design. East Melbourne: VCAA.

Victorian Curriculum and Assessment Authority. 2000. VCE Bulletin Supplement 1 – October 2000. East Melbourne: VCAA.

Wall, D; Alderson, J. C. 1993. ‘Examining washback: The Sri Lankan impact study’. Language Testing 10 (1): 41–70.

Watanabe, Y. 2004. ‘Methodology in washback studies’. In Washback in Language Testing: Research Contents and Methods, edited by Curtis, A; Cheng, L; Watanabe, Y. Mahwah, New Jersey: Lawrence Erlbaum.

Watanabe, Y. 1996a. ‘Does grammar translation come from the entrance examination? Preliminary findings from classroom-based research’. Language Testing 13 (3): 318–333.

Watanabe, Y. 1996b. ‘Investigating washback in Japanese EFL classrooms: problems of methodology’. Applied Linguistics Association of Australia, Series S. 13: 208–239.

Cite this chapter as: Ryumon, A. 2007. ‘Washback of high-stakes assessment: Year 12 Japanese’. In Learning Discourses and the Discourses of Learning, edited by Marriott, H; Moore, T; Spence-Brown, R. Melbourne: Monash University ePress. pp. 17.1 to 17.15.

© Copyright 2007 Akiko Ryumon
All rights reserved. Apart from any uses permitted by Australia’s Copyright Act 1968, no part of this book may be reproduced by any process without prior written permission of the copyright owners. The fact that this book is published online does not mean that any part of it can be reproduced without first obtaining written permission: copyright laws do still apply. Inquiries should be directed to the publisher, Monash University ePress: http://www.epress.monash.edu/contacts.html.

Learning Discourses and the Discourses of Learning

   by Helen Marriott, Tim Moore and Robyn Spence-Brown