Standard-Setting Process

A historical document outlining the process used to define the standards for the OSSLT.

The standard was established by parallel panels of educators and citizens working in English and French. The process was completely comparable in the two languages.

Participants

• In the English- and French-language panels, there was equal representation of people currently employed in the elementary and secondary school systems and members of the public.
• Educators were nominated by Directors of Education. Members of the public were nominated by senior education officials and community leaders on the basis of their experience with schools and students, or their familiarity with areas in which reading and writing skills are required.
• Participants were not designated representatives of any special-interest groups or organizations.
• The diversity of the province was represented, insofar as was possible in eighteen-member panels.

The Process

• The English-language panel (nine educators and nine members of the public) came together on May 30, 2000, and worked for three days, including evening sessions. The parallel French-language panel met June 13 to 15, 2000.
• Educators familiar with assessment facilitated small groups of panel members until the final stage of decision-making, during which the groups functioned independently. Staff and consultants from EQAO implemented the process and recorded the results.
• For the English-language sessions, assessment experts from three universities (University of British Columbia, Queen’s University and the University of Ottawa) provided advice to ensure the formal validity and reliability of the results. An expert with international experience in standard setting for the Organisation for Economic Co-operation and Development tests for 15-year-olds was also involved.
• Similar arrangements were in place for the French-language sessions.
• In each working session, the 18 panel members were presented with background information, materials relevant to the Grade 10 assessment and demographic information about the diversity of Ontario students.
• The panels reviewed samples of student work from the March 2000 Grade 10 field test.
• Based on the samples, the panels developed descriptions of students’ performance for both reading and writing at the incomplete, complete and advanced levels of proficiency.

The panels then applied these descriptors in reviewing student test papers to determine which should pass (“complete”) and which should not (“incomplete”).

The descriptions of student work below were created by the committees; they have not been changed except to correct small mechanical errors. Many of the words and phrases were subject to lengthy discussion and occasionally to votes.

For purposes of sorting student work, the committees also developed a description of work that was “advanced” or well beyond the standard required for a pass. Since the test is reported only as pass/fail, the third category is not required in the reporting process.