M4
_may be defined as a procedure or a systematic appraisal measuring the potential value(s) of materials on learners in relation to their objectives (Tomlinson 1998, 2003).
Materials evaluation
_ means a principled process of providing useful information about the targeted matrials in order to select and/or develop them in a reliable and valid approach.
materials evaluation
Purposes of materials evaluation Mukundan (2004, 2009)
1. to select the _
2. to determine the _ of the coursebooks while they are used.
3. for _ of material. Tomlinson and Masuhara (2004)
coursebook
effectiveness
re- development
Types of evaluation (Stages)
_: It involves making predictions about the potential value of materials on people who use them (Tomlinson 1998, 2003). He indicates that this type of evaluation is often impressionistic, subjective and unreliable since a teacher scans a book quickly to gain an impression about its value.
_: It measures the value of the materials whilst observing or using them. Mukundan (2009), for example, indicates that retrospective evaluation is the re-evaluation of materials while they are in-use to decide if the materials work
_: It is the most important and valuable type of evaluation because it can measure the actual effects of the materials on the users and provide reliable information (Tomlinson 1998, 2003). Tomlinson maintains that it can measure short term effects such as motivation.
Pre-Use Evaluation
(In)-Use Evaluation
Post-Use Evaluation
Principles in materials evaluation
There are two principles in materials evaluation: _ and _ (Ellis 1998 in Tomlinson 1998).
_: Is a course book effective in meeting the needs of the learners
. Compare what the learners knew and were able to do before they used the course book with what they know and are able to do after they have used the book;
_: Does a course book meet the needs of the learners more effectively than some alternative course books?
Compare the learning gains evidenced by using one course book with the gains evidenced by another course book.
effectiveness, efficiency
Effectiveness principle
Efficiency principle
Approaches to materials evaluation
A) Ad hoc impressionistic evaluation vs. systematic evaluation
An _ is based on intuitions, impressions, and experience of using materials.
In an _, valid methodological considerations may give way to an eye-catching appearance.
_ can be biased by misconceptions about what ‘desirable’ materials should look like. Both classroom teachers and specialized researchers can have these misconceptions. Think about why.
Ad hoc impressionistic evaluation
Approaches to materials evaluation
A) Ad hoc impressionistic evaluation vs. systematic evaluation
A _ is based on a specification of objectives, principles and procedures adopted or embedded in materials. A _ is best done in a longitudinal fashion, which has three stages: pre-use evaluation, whilst-use evaluation and post-use evaluation.
_ needs more theoretical background in language learning and more expertise in materials development.
systematic evaluation
Approaches to materials evaluation
B) Internal evaluation vs. external evaluation
An _ focuses on the internal accountability of the materials, e.g. the theoretical assumptions behind the materials; the intended objectives and the extent to which the objectives have been realized; the justification of language selection and grading; the design of activities and tasks.
An _ examines the extent to which a set of materials meets the needs of a particular group of learners, the syllabus, and the examination, i.e. whether a set of materials is suitable for a group of students.
_&_distinguishes between “external evaluation” and “internal evaluation”, both of which focus on the evaluation of materials “as they are”, rather than relate what’s in the materials to the needs of the students. So both types belong to internal evaluation outlined above.
internal evaluation
external evaluation
McDonough & Shaw (1993)
Why do we evaluate materials?
To identify the popular and successful features of _, which will help us to produce better materials.
To choose _ materials for language programmes.
To adapt the materials to make them better suit the needs of the _ and the needs of _.
To identify how _ the materials used in the classroom are.
To examine whether the materials _ the prescribed course
To examine the extent to which materials permit students to _ learning objectives.
To identify whether the designs of the materials are _ for a task-based syllabus.
existing materials
suitable
students&teaching
successful
fulfill
achieve
suitable
Frameworks and checklists for materials evaluation
In order to make materials evaluation more operable and easier to conduct, many frameworks or checklists have been developed.
A) _ framework:
PHASE ONE: Initial questions
1. What do the materials aim to do and what do they contain?
2. What do the materials make your learners do while they are learning?
3. How do the materials expect you to teach the learners in the classroom?
4. Are the materials the only resource in classroom language learning?
PHASE TWO: Your learners and the materials
1. Are the materials appropriate to your learners’ needs and interests?
2. Are the materials appropriate to your learners’ own approaches to language learning?
3. Are the materials appropriate to the classroom teaching/learning process?
4. Seven design features of materials for classroom work
5. Discovering learners’ criteria for good materials
B) _
FRAMEWORK
THE UNITS
SUBJECT-MATTER
FORM
COURSE COMPONENTS
C) _
Publication:
1. Place of learners’ materials in the set
2. publication form: worksheets vs. bound book; durable vs. consumable
3. Subdivision of materials into sections;
4. Subdivision of sections into sub-sections
5. Continuity
6. Route (whether teaching order is predetermined; i.e. can a section be skipped)
7. Access (how access into the materials are supported, for example, whether there are contents lists, wordlists, and indexes)
Design
1. Aims
2. Principles of selection
3. Principles of sequencing
4. Subject matter and focus of subject matter
5. Types of learning/teaching activities
6. Participation: who does what with whom
7. Learner roles
8. Teacher roles
9. Role of materials as a whole
D) _
E) _
Breen and Candlin’s Framework (1987)
Dougill’s framework
Littlejohn’s framework for analyzing material
Cunningsworth’s (1984) checklist
Grant’s (1987) evaluation criteria