Abstract
The importance of textbook evaluation in our educational system has been reinforced through many studies done in this regard. However, none of them has focused on Vision 1, the new tenth grade high school English course book. In addition, the studies done had different orientations in their evaluation, but evaluation based on authors’ mentioned objectives is missing. To fill the gap, the present study attempted to investigate whether Vision1 has fulfilled the authors’ objectives or not. To this end, 1000 high school students and 100 school English teachers filled in two questionnaires. The results of the questionnaire data were triangulated by student interviews. The results showed that the course book received almost a mid-point evaluation both by the teachers and students. The student interviews also supported the results.
Key Words: textbook evaluation, objectives, Vision 1
Introduction
In a foreign language class, there are many important factors including the teacher, the student, the course book, and the teaching aids. Among these, course books play a crucial role. Sheldon (1988) considers course books as “the visible heart of any [English Language Teaching (ELT)] program and as a universal element of ELT teaching” (p. 237).
Since teachers spend a lot of time using the school course books, materials developers should pay attention to teachers’ expectations of the course book and be aware that their attitude toward it will affect the success or failure of a course book in practice. Wette (2010) provides empirical support for the view when he asserts “making instructional curriculum is a dynamic process in which teachers draw on their professional knowledge to construct the curriculum by synthesizing and acting on feedback from their learners’ needs within what is feasible in a particular teaching context” (p. 577).
If we agree on the importance of ELT course books and the importance of language teachers’ attitude toward them, we understand how important it is for the language teacher to be aware of how to evaluate the course book he/she is using. Huchinson and Waters (1987) consider evaluation as the assessment of appropriateness and suitability of materials for a special purpose. Byrd (2001) explains the appropriateness as “the fit between the materials and (1) the curriculum (2) the student (3) the teacher” (p. 416). Following the same line of thought, Tomlinson (2011) believes that materials evaluation refers to the systematic evaluation of the materials in relation to the course objectives to see how much success has been gained in achieving these objectives.
Review of the Related Literature
The previous studies carried out to evaluate course books have different dimensions. The study done by Alimorad (2016) concluded that the cultural values hidden in the imported textbooks has a negative cultural effect on students and, as a result, emphasized the importance of modifying or producing a local course book for school English education. In two other studies by Jahangard (2008) and Rashidi and Kehtarfard (2014), the previous high school books were analyzed and studied. They mentioned that those school textbooks did not cover students’ needs. They further added the shortcomings of those textbooks and the necessity of a change in them. In a different approach, Talebinezhad and Mahmoodzadeh (2011) evaluated internationally and locally developed ELT materials and concluded the superiority of international materials in terms of authenticity and meaningful contexts.
Although many studies have been conducted to investigate school EFL textbooks, none of them has paid enough attention to the objectives of textbook developers and whether they have been successful in achieving their objectives or not. Sheldon (1988) observes that success or failure of a selected course book is determined during or after its classroom use. He argues that as language learners are supposed to follow some educational goals, course books should be evaluated in terms of their integration with and contribution to these long-term goals.
The purpose of this study was to investigate whether Vision1 fulfils the objectives mentioned by the textbook developers or not. Curriculum objectives provide guidelines for the development of texts, activities, and exercises. Therefore, it is crucial both for teachers and students to be aware of course objectives to know what each of them should do to meet the course requirements, get involved in the course, and appreciate the teaching/learning experience they are engaged in. According to Nunan (1988), students who know the objectives will learn faster than those who do not know them.
Tyler (cited in Nunan, 1988) who was the predecessor of objective oriented evaluation defines objectives as “what the learner should be able to do as a result of instruction” (p. 63). Through a phone call to the head of the group of Vision1 authors, the writers of the present paper realized that there were no approved objectives set by the Ministry of Education for this book. Therefore, the objectives or the approaches mentioned by the materials developers in the preface of Vision1 were considered as the set objectives. They are as follows:
1. Attending simultaneously to four language skills
2. Using a variety of activities in the language learning process
3. Emphasizing experiential learning
4. Using rich, meaningful, and understandable texts in the book
5. Increasing cooperative learning situations
6. Providing suitable corrective feedback to learner errors
7. Paying attention to the importance of students’ emotions in the class (From the above list, numbers six and seven depend on teachers’ performance and cannot be considered as the objectives of the book. Instead the researchers added two other objectives which are crucial for any course book).
8. Providing a complete and an accessible pack
9. Ensuring that the textbooks have a suitable layout and acceptable physical appearance
The following research question was formulated to guide the present study:
Does Vision1 fulfill the objectives specified by the authors of the course book?
Method
To investigate this research question, the researchers first studied different syllabuses and came to the conclusion that the syllabus used in Vision1 is situational and topical, a syllabus in which different lessons are ordered based on specific topics and situations in which they happen. Later, a number of ten book evaluation checklists were studied and among them Skierso checklist edited by Celce Murcia (1991) was considered suitable for this purpose. A translated version of the checklist was used for the evaluation of Vision 1.
Participants
The participants of this study were composed of two groups, students and teachers. The students were sixteen-year-old high school boys and girls. The participants were studying English at such disciplines as Mathematics, Natural Sciences, and Humanities. The students were studying Vision1 for three hours a week, for nine months in a scholastic year. The selected eight public high schools were located in the north, south, east, and west of Tehran, four boy schools and four girl schools. The total number of students was 1000. They were asked to fill out the questionnaires. The teachers were one hundred male and female high school English teachers, teaching the tenth graders all over the country.
Instrument
The instruments used in this research were two questionnaires, and the interview. Vision 1, the textbook selected for investigation, is the first book from the senior high school English book series. Vision1 consists of two books, the student book and the work book. The student book contains four lessons ordered based on situations and topics. The work book includes exercises related to different parts of the book.
Two questionnaires were used in this study. The student questionnaire, developed based on the authors’ objectives, consisted of 26 questions. It was first piloted with 30 students so that the researchers could remove the probable problematic items. The teacher questionnaire, a translation of the Skierso checklist (Celce Murcia, 1991, pp. 445-8) included 76 questions. The items were carefully analyzed and modified based on the selection of words and item classifications. Then, it was filled out by some colleagues to check face validity of the questionnaire. Both questionnaires were in Likert scale format ranging from strongly disagree to strongly agree.
An interview schedule was used as the second instrument for data collection in the present study. A total number of 30 students were interviewed. The interview included topics related to the seven mentioned book objectives. In fact, the interview was used to triangulate the results of the research.
Design
The study had a descriptive quantitative design. It is quantitative because the researchers used two different questionnaires which were answered by 1000 students and 100 teachers to understand attitudes toward the book. It is descriptive because the researchers used the quantitative data gathered and the interview results to describe whether the course book has achieved the objectives of the authors or not. The sampling used in this study was convenient sampling. Almost every accessible tenth graders or teachers of Vision1 that accepted to take part in this research were welcomed to participate in this study.
Data Analysis
The data gathered in this research through student and teacher questionnaires were studied separately. In the student questionnaire, the items related to each objective were identified, and the reliability of the questionnaire was measured using Cronbach’s Alpha. The reliability ranged from .74 to .84 which is acceptable. The teacher questionnaire, on the other hand, was studied many times, and its different items were classified into the seven mentioned objectives. The reliability of items for each objective was again measured using Cronbach’s Alpha which ranged from .65 to .92. As the lowest alpha was achieved only by eight items of the seventy-six questionnaire items, this is also acceptable.
Results and Discussions
The results in percentages and mean scores achieved by 100 teachers and 1000 students are presented and discussed below:
The results concerning the first objective -attending to four language skills simultaneously -revealed that both teachers (%58) (Mean 2.97) and students (%58) (Mean 3.09) believed that language skills were not well-attended separately or in integration in the course book. In fact, their ratings were around the mid-point. These findings might be justified with reference to the results of the interviews. Some students expressed that language skills do not support each other, and, consequently, they found it difficult to deal with each skill separately. It seems that listening, speaking, and reading activities, as well as grammar exercises, were not integrated appropriately.
In relation to the second objective -using a variety of activities -both teachers (%58) (Mean 2.98) and students (%67) (Mean 2.85) believed that the book had failed to enhance the learning process through the incorporation of varied and interesting activities and tasks. These findings were justified on the account that most students preferred shorter lessons with various texts, dialogues, tasks, and activities, rather than few long lessons. Teachers expected more various exercises, activities, puzzles, and games to consolidate learning and add to the learners’ enjoyment.
Concerning the third objective -promoting experiential learning -teachers’ ratings were at the midpoint. That is to say, teachers (%41) (Mean 3.01) believed the book helped the students eaperience the language use, to some extent. While students (%69) (Mean 2.80) rated this objective negativity suggesting that they believed opportunities for experiential learning were not abundant as the authors claimed. The lack of experiential learning opportunities might originate from the fact that only some students considered the listening, reading, speaking, and writing activities genuine, while others believed that these activities did not provide them with real experience.
Findings in relation to the fourth objective -using understandable, rich, and meaningful materials- showed that, in contrast to teachers (%49) (Mean 3.19) who had a somehow positive view, students (%62) (Mean 2.96) believed that the course book did not include highly understandable, rich, and meaningful materials, that is, material is coming in a familiar context based on learners’ prior knowledge, experience, feelings, and interest. As reflected in the interviews, students believed the pronunciation part, as well as the grammatical points, listening/speaking activities, and quizzes did not make good sense, and that the texts were highly difficult for them to understand.
Analysis of data on the fifth objective -facilitating cooperative learning - showed that both teachers (%40) (Mean 3) and students (%46) (Mean 3.21) believed that activities and exercises included in the course book encouraged cooperative learning to some extent. These findings may result from the fact that Vision1 did include some cooperative learning and pair work activities which seemed to be engaging for the students.
Concerning objective 6 - providing a complete and accessible pack - the results indicated that neither teachers (%70) (Mean 2.55) nor students (%68) (Mean 2.68) believed that the course book was available as a complete pack and contained the necessary components. This is not surprising. The audio CDs were not distributed attached to the book itself. Furthermore, there were no accompanying test models, no self-study CD-ROM, and no companion updated website.
In relation to the seventh objective -providing a suitable layout and physical appearance - both teachers (%44) (Mean 3.03) and students (%36) (Mean 2.96) believed that the physical layout and appearance of the textbook was suitable although students’ ratings were a little lower. It is evident for every user that the book is not appealing considering the color and picture quality of the cover and some pages inside the book. For example, the picture on the cover page is somewhat irrelevant, misleading, and unattractive.
Conclusion
The results are summarized as follows: The mean scores based on Likert scale range from 2.55 to 3. 21. These figures showed that although the results of the research did not show a great success for achieving the mentioned objectives, they did not show a total failure either.
The first high mean score was allotted to the fifth objective by the students (3.21) (46% positive answers) which was learning in a cooperative manner. Doing exercises or answering the questions on their own is a stressful activity for teenagers in the tenth grade, especially if they were going to receive grades for their answers. In cooperative learning activities, students could share ideas to compensate for their own or other members’ lack of knowledge and achieve success. Teachers (3.00) (40% positive answers) also scored this objective highly since it helps students find the related answers more quickly, and the class can move at a faster pace in the learning activity.
The second high mean score was given to the fourth objective by the teachers (3.19) (49% positive answers). They believed that the materials were rich, meaningful, and understandable. Of course, the recently-published book with updated topics on daily themes like nature, tourism... should be interesting for teachers. However, they criticized some unnecessary vocabulary, such as the name of many animals, which usually children learn in pre-school English classes or the reading-like conversations with artificial sentence structures that do not encourage students to use language. On the other hand, students (2.96 ) ( 61% negative answers) who seemed to be poor in grammar, nagged about learning a lot of ambiguous grammatical points in one lesson. They were not satisfied with the difficulty level of the reading materials either. They believed that the difficulty level of the reading materials did not increase smoothly.
The third high score was given to the first objective by the students (3.09) (42% positive answers). Although this objective which is attending to four language skills simultaneously was not believed to be fully achieved by the students, it was a good step forward in this area. Students try to listen, imitate, copy, and use English, of course to a very limited extent. Teachers (2.97) (42% positive answers), on the other hand, spend a lot of time and energy on listening and speaking, but they do not receive the necessary feedback from the students because the teacher is usually the only model in the class and there are no teaching aids like films, clips, and slide shows. Writing activities are also very limited in the course book. Students unscramble letters to make words. They also put given words in correct order to make sentences. However, teachers believe more writing is needed.
The fourth high mean score was given to the book layout and physical make-up. This objective received a rather acceptable mean score by teachers. (3.03) (45% positive answers).They believed that the type size is appropriate for the learners, the texts are attractive and appealing to the intended students, and the material is clearly organized. They also believed that the pictures, colors, and graphic devices were helpful to the learners. However, students (2.96) (63% negative answers) gave lower score to the book layout. For example, considering grammar, students believed one grammatical point is enough for each lesson. They did not like the inductive way of teaching grammar either. They preferred to learn the rules and then go through the examples deductively. Also, they liked to have one book instead of two (student book and work book). They believed that the work book must be colorful, instead of black and red, and include puzzles and games to motivate learners to work on. The book cover is very much like their other course books, and it happens that students make a mistake while preparing their school bag.
The fifth high mean score was given to the third objective that is experiential learning activities by the teachers (3.01) (41% positive answers). The teacher guide asks teachers to provide some questions before playing the audio files of the conversations or reading texts to engage students in manipulating and using the materials. The students are also asked to talk about their personal experiences using the book topics or grammatical points. However, the students (2.80) (68% negative answers) gave a lower score to this objective because they thought the book texts and activities did not familiarize them with the culture of the native speakers of English and introduce artificial conversations and dialogues to them.
The sixth objective which is rated below three both by teachers (2.98) (55% negative answers) and students (2.85) (63% negative answers) was the variety of activities in the language learning process. For example, teachers believed that in grammatical exercises the authors should have considered Bloom’s taxonomy in ordering various exercises as “knowledge, comprehension, analysis, synthesis, and evaluation” (Richards & Schmidt, 2010, p. 59). But the work book exercises are repetition of the exercises in the student book. This objective is rated by the students even lower than the teachers. Students preferred to have ten shorter lessons with various topics, vocabularies, and grammatical points instead of four long lessons in the book. They also preferred learning diacritical marks for pronunciation rather than sentence intonation.
The seventh or the least mean score was received by Vision1 pack. The teachers (2.55) (68% negative answers) gave even lower score than the students (2.68) (67% negative answers) to this pack. No CDs or DVDs accompanied the book in the beginning of the scholastic year. In the second term, a CD with a poor quality was sent to schools. The schools did not provide any facilities for playing the files. The teachers’ book was not available in time. There were no slides or video clips to be used as teaching aids. There were no communicative test models given to the teachers.
Final Remarks
Materials development follows specific steps; if not observed, it causes problems. The first step is performing a needs analysis. It is not possible to predict the new generation’s needs and interests without careful study. When the book is going to be used all over the country, paying attention to the needs and interests of both girls and boys should not be considered trivial.
Another problem was hasty preparation and use of Vision 1. The book was not piloted before use so that the problems could be identified. When the course book came to the market and a little later to schools, it was not really a pack. The only part of the pack which was available in time was the student book and the work book. The other parts of the pack like teachers’ guide and audio files were probably available in some schools, and some parts of the pack, like test models, related films or slides were not accessible.
REFERENCES
Alavi Moghadam, S. B., Kheirabadi, R., Rahimi, M., & Davari, H. (2016). Vision1. English for schools. Tehran: Iran’s School Book Publication.
Alimorad, Z. (2016). Cultural values represented in first certificate master class taught in Iran. Retrieved from http://www.creativecommons.org/licences/by/3.0/
Byrd, P. (2001). Textbooks: Evaluation for selection and analysis for implementation. In M. Celce Murcia (Ed.) Teaching English as a second or foreign language ( 3rd edi, pp. 415-427). Boston: Heinleand Heinle.
Celce Murcia, M. (Ed. 1991). Teaching English as a second or foreign language. Massachusette: Heinle and Heinle Publishers.
Huchinson, T. & Waters, A. (1987). English for specific purposes. Cambridge: CUP.
Jahangard, A. (2008). Evaluation of EFL materials taught at Iranian public highschools. The Iranian EFL Journal, 1, 31-43
Nunan, D. (1988). Syllabus design. Oxford: OUP.
Rashidi, N., & Kehtarfard, R. (2016). A needs analysis approach to the evaluation of Iranian third-grade high school English textbook. Retrieved from http://sgo.sagepub.com
Richards, J. C., & Schmidt, R. (2010). Longman dictionary of language teaching and applied linguistics(4th ed.). Edinburgh: Pearson Educational Limited.
Sheldon, L. E. (1988). Evaluating ELT textbook and materials. ELT Journal, 42(4), 237–246.
Talebinezhad, M.R., & Mahmoodzadeh, M. (2011). Re-evaluation of internationally and locally developed ELT materials in Iran: A bimodal approach to material development. Journal of Language Teaching and Research, 2(6), 1204-1216.
Tomlinson, B. (2011). Introduction: Principals and procedures of material development. In B. Tomlinson (Ed), Material development in language teaching. (2nd ed., pp. 1- 31). Cambridge: CUP.
Wette, R. (2010). Professional knowledge in action: How experienced ESOL teachers respond to feedback from learners within syllabus and contextual constraints. System, 38, 569- 579.