by Bob Wilson, Professor Emeritus, Queen’s University
Issue 1, 2007
A good assessment item, whether on a classroom test or on an EQAO assessment, is one that fulfills its purpose. A major purpose for teachers is to assess students so that both the teacher and the students can use the information to shape future teaching and learning. When this is your purpose, the following ideas might help.
Many test items ask students to recall discrete bits of factual information, recognize or define a particular term or show that they can perform a specific operation, such as multiplication, in mathematics. While the information about student achievement these items provide is sometimes useful and important, it is very limited.
Items that are more useful to inform future teaching and learning ask students not only to produce bits of information but also to link them to other related subject knowledge or to make personal connections between them and other knowledge and experience. They ask students to use newly learned information to solve problems, make judgments and go beyond what they have learned in school by showing how it is relevant to real-life situations.
Such items are learning as well as assessment activities for students. They are engaging and interesting tasks that encourage knowledge and skill retention by linking newly acquired knowledge to what the student already knows.
Many learning expectations have multiple parts or are part of a process, and the most valid test items allow for the complexity or process to be assessed. For example, many times teachers ask questions that require a paragraph or an essay response. These items can be made more useful by broadening the test to include the stages of preparation prior to writing the paragraph or essay. A part of the assessment might ask for a plan or draft. Reading comprehension can also be part of the assessment. Did the student understand the material that contained the basic knowledge needed for the response? Such elaborated items help students and their teachers discover where blocks to achievement are and may even suggest means of overcoming them. Multi-stage problems in mathematics, social studies and science provide excellent opportunities to develop such items.
Other methods teachers currently use for assessment include projects (group and individual), demonstrations (history museum, science fair), dramatic performances, formal or informal debates and portfolios. Tools for evaluating these kinds of assessments include student and teacher observation checklists; teacher, peer and self evaluation; and reflective student work journals for extended assessment activities.
Good assessment items such as those described above are useful for diagnosis. These items provide useful information at the lower as well as the higher levels of student achievement. If a student is not able to respond at the simplest level to the vocabulary or operations required by the item, this tells both the teacher and the student where they have to start. If a student shows an understanding of the basics but is not yet able to see how various elements are connected, this indicates to both the teacher and the student what the next steps in the learning process need to be.
The reason students do not do well on assessments typically has more to do with their conceptual abilities than with the particular content they are being asked to learn. For example, students may grasp onto the first idea they encounter in a passage or a question and be unable to add or link more information to that concept for a more sophisticated response.
See the two examples at the end of this article of how student responses can be analyzed to understand student misconceptions and learning needs.
The more students can see that assessment is something that happens for a purpose (to support their progress) and not just something that happens to them, the more they will involve themselves in it and benefit from it. There are many ways to engage students in the assessment process. They can be asked to write or report orally on the quality of the learning they have demonstrated on an assessment. This reflection can include a section on what they need to work on to improve. Learners can also prepare assessment items themselves to give to fellow students. They can be asked to include a statement about why the knowledge required by the item is important and to report on how well the item has assessed the knowledge and skills of their classmates.
Metacognition (reflection on thinking and learning processes) is often referred to as the key to improved student learning. The revised
Ontario Curriculum, Grades 1–8: Language (2006) includes overall and specific expectations related to metacognition in all four language strands in every grade.
Some of the most useful information for a teacher and a student is about progress over time. Looking at the differences in achievement by the class and by individuals within the class on the same items at different times will identify areas where the teaching and learning have been successful and areas where more work has to be done. This understanding is enhanced when both student and teacher can point to work that is arranged according to “what I could do then” and “what I can do now.”
An alternative to repeating items requires parallel tests with similar items but different contexts. Published items such as those provided by EQAO on its Web site are a good source of parallel items.
Reading a student’s test straight through, noting the strengths and weaknesses as you go, is an alternative to scoring and grading it. After reading it, you can make some notes on what learning was demonstrated (strengths and weaknesses) and what this means for the student and perhaps for your teaching (next steps).
All assessments must produce accurate (reliable) information about learning (i.e., the information must be valid). The goal of large-scale assessments is to provide snapshots of achievement mainly at a group level for comparison to a standard over time. So large-scale assessments must sample the curriculum effectively and require students to work under common conditions in order to yield meaningful data about the diverse range of students being assessed. Classroom assessments must also be reliable and valid, but, since their goals are frequently different, they need not always share all the characteristics of large-scale assessments. For example, if a teacher is not interested in comparing one student to another, there is no need to be rigid about standardization.
Example 1: Reading Comprehension (Question from the EQAO sample assessments Spring 2007 Primary Division, Language)
Question: According to the text, what animal are scientists certain is a reptile?
The selection of the correct option shows understanding of information stated explicitly in the passage. The selection of an incorrect option says something about the reader’s approach. Because
Tyrannosaurus rex, for example, is given lots of space in the article, a developing reader may choose it for that reason alone even though it is not the correct response. A read-aloud session would help to validate this judgment.
Example 2: Mathematics (Question from the EQAO sample assessments Spring 2007 Junior Division, Mathematics)
To answer the question correctly, the student must understand and use the concept of ratio. The selection of an incorrect option says something about the student’s approach. For example, students who merely count the sections labelled “Red” may be swayed by answer F. Students looking at the top half of the circle and seeing that three of the four sections here are “Red” may use this information to select answer H. In both cases, the students are telling you something about how they have approached the question.