Scoring Activities

• EQAO selects potential scorers to score the open-response questions completed by students. Potential scorers must
  • create a profile and keep it up to date, and
  • accept a scoring-specific invitation (if you receive one) by the specified deadline to express your interest in participating in that activity.

Note that EQAO is unable to accept inquiries related to scoring activities by e-mail or by telephone. EQAO does not directly offer an employment position; individuals are hired by a third-party employment vendor of record (Quantum) and then assigned back to EQAO.

Training and scoring are self-directed through the e-assessment scoring system, which is available during specific hours, seven days a week (except on statutory holidays), during the scoring window. All the training and scoring resources are available in the e-assessment scoring system.

Once scorers have completed their training and have passed the qualifying test on an item, they will be able to select batches, which they will have a set time frame to complete. A batch takes approximately one hour to score.

If a scorer completes the selected batches before the time frame ends, the scorer can select one or more additional batches, for which a new time frame for completion will be set. Toward the completion of scoring, EQAO may shorten the time frame to complete batches to two hours or less, and the number of batches that can be claimed at a time to one (to enable the distribution and availability of batches for everyone). EQAO can also revoke claimed batches at any time to facilitate the completion of scoring.

Online scoring is self-directed. Although scorers do not need to score daily, they must score a minimum of 14 batches per week, if available, during the entire scoring window and meet the minimum validity threshold.

If a scorer falls below the minimum rolling validity threshold, the scorer may have an opportunity to retrain and pass another qualifying test, depending on their cumulative validity metrics. If successful, the scorer will be able to continue scoring. If unsuccessful, the scorer may be withdrawn from their current scoring assignment. Scorers can score as many batches as are available but must not score for more than 44 hours in a work week (from Sunday to Saturday).

Scoring generally takes place two to three weeks before the end of the administration windows of the Ontario Secondary School Literacy Test and the Assessments of Reading, Writing and Mathematics, Primary and Junior Divisions. Scoring invitations are generally sent to qualified candidates six weeks before the start of the administration window.

• You will be required to register with Quantum, the employment vendor of record, and sign an employment agreement.
• You will receive a confirmation letter and information that includes the relevant details, next steps and key dates leading up to the beginning of training and scoring.

The name in your EQAO profile must match the name in your profile on the OCT website. If you change your name, you must ensure that you update your name with the OCT. Once the name change is reflected on the OCT website, the name will automatically be updated in your EQAO profile. As EQAO’s database is linked with the OCT’s database, a different name will create communication errors between the two sites. If you have a middle name, you can try adding your middle name to the first-name field or the last-name field one at a time, which may rectify the error. Otherwise, please ensure that your name is the same in both profiles.

If you cannot remember your password, click Forgot Password, and an e-mail will be sent to you with instructions on how to reset your password. If you no longer have access to the e-mail account on file with EQAO, please contact us at [email protected] or 1-888-327-7377.

Some scorers, upon making changes to their EQAO profile, may find that a scoring opportunity is no longer available to them. This is a normal occurrence, as the automated system verifies that the details, qualifications and current employment status in an applicant’s EQAO profile match the requirements for the scoring activity. It is therefore important that each scorer’s profile be maintained with accurate information regarding address, qualifications, current role in education, experience, etc.

Openings for scoring activities are limited, and fill up quickly on a first-come, first-served basis, as candidates accept their invitations to score. Openings become available if scorers withdraw. Candidates are encouraged to check their EQAO profiles regularly, as openings can become available at any time during the administration windows.

Qualified candidates will receive e-mails with offers to score. As e-mails may be stopped by firewalls or sent automatically to junk or spam folders, it is recommended that users check their EQAO profiles regularly for scoring offers. Users should also add the following e-mail addresses to their “safe senders” list to ensure they receive EQAO e-mails:

• [email protected]
• EQAO/[email protected]
• EQAO/[email protected]

Please note that EQAO is unable to accept inquiries related to scoring activities by e-mail or by telephone.

This message means that a scoring opportunity was made available but has already been filled, as multiple candidates received an invitation for the same opening. Candidates are encouraged to check their EQAO profiles regularly, as additional opportunities can become available at any time.

When you log into your EQAO profile, all the scoring opportunities that you have accepted are listed under “Status” on your profile dashboard page. For more information about each scoring opportunity, click Details beside each opportunity.

You are seeing more than one opportunity because you have qualified for more than one opening. You may accept as many scoring opportunities as you like, provided the scoring dates are not conflicting. If that is the case, please select and accept only one offer.

To accept an opening, select Accept/Decline beside the opening and continue to follow the instructions until you click Confirm.

Unfortunately, the task you have been offered cannot be changed. However, you may be offered a different task to score based on your validity measure during the scoring activity.

You must immediately withdraw from your accepted invitation to score. To do so, please follow the instructions below. Note that if you do not withdraw, you will be considered a “no show” and may not be invited to participate in future EQAO events.

Log into your EQAO profile and locate the offer you wish to decline or withdraw from on your profile dashboard page. Next, select Details and then click Withdraw. If you are not able to withdraw through your EQAO profile, please contact EQAO at [email protected] or 1-888-327-7377.

Do not accept an invitation to score until you can commit to the scoring activity.

You must update your proof of enrolment in each year of your program. Once you receive your Ontario College of Teachers (OCT) certificate and your information is on the OCT website, you will need to update your current educator role and enter the OCT number in your EQAO profile. Ensure that the first name and last name in your EQAO profile match those on the OCT website.

Your EQAO profile may not be up to date or your qualifications and experience may not have met the requirements to perform scoring activities. Additionally, there may not be any scoring opportunities available at the moment.

Please refer to the Payment Guidelines for details on remuneration.

No. EQAO is not directly offering an employment position by offering involvement in EQAO activities. Individuals involved in EQAO activities will be hired by a third-party employment vendor of record and then assigned to perform EQAO activities including scoring activities, committee-related activities or any other activities designated by EQAO.

EQAO will not cover any additional expenses, including supply teacher cost, Internet charges, equipment rentals, or other related expenses. Employees who are currently employed by another employer (e.g., school, school board, etc.) shall not perform duties assigned by EQAO during their regular work hours. EQAO will not reimburse the expenses incurred by an employer for “supply workers” to cover for absent or missing employees.

The following terms are most commonly used:

• Scoring Window: a specific period (with predefined start and end dates) during which the e-assessment scoring system is available for training and scoring
• Item: a question or task that appears on an assessment or test
• Response: a student’s response to an item on an assessment or test
• Training Module:
• Training Module: material for the training session, which includes an introductory video, a scoring guide, scoring rubric, anchors, a practice test and a qualifying test
  • Scoring Guide: the core instructional tool used during scorer training that contains the specific scoring rubric alongside item-specific anchor responses. It outlines the exact criteria and exemplar student work required to assign performance codes (Codes 10, 20, 30 and 40) consistently across all student responses for a given question.
  • Scoring Rubric: an assessment tool provided as either a generic or item-specific scale used by scorers to evaluate student open-responses against provincial expectations.
  • Anchors: a sample student response selected by EQAO’s range-finding committee to serve as the recommended baseline example for each specific code level on the scoring rubric.
  • Practice Test: a test that scorers complete as part of the training module, used to prepare scorers for the qualifying test.
  • Qualifying Test: a test used to assess a scorer’s ability to accurately score student responses.

• Batch: a predefined number of student responses to a test question (each batch is expected to take an average of one hour to score.)
  • Partial Batch: a batch that is partially scored, or a batch with a partial number of responses to be scored, which was automatically assigned by the system
• Expert Scorer: an experienced scorer with high validity whose scores are reviewed and confirmed by committee members and/or EQAO education staff.
• Validity: the measure of a scorer’s accuracy assessed by examining the agreement between the scores assigned by the scorer and those assigned by expert scorers. The following indices are computed: percentage of exact agreement, percentage of exact-plus-adjacent agreement, percentage of adjacent-low agreement and percentage of adjacent-high agreement.
• Adjacent: a difference of one score code between the score assigned to a certain response by the scorer and that assigned by the expert scorers
  • Adjacent-Low: the score assigned to a certain response by a scorer is one score code below the score assigned by the expert scorers.
  • Adjacent-High: the score assigned to a certain response by a scorer is one score code above the score given by the expert scorers.
  • Non-Adjacent: the difference between the score assigned to a certain response by a scorer and that assigned by the expert scorers is greater than one score code
• Exact Agreement: the score assigned to a certain response by the scorer and the expert scorers is the same
  • Cumulative Exact Validity Percentage: the total percentage of validity responses scored that match the score preassigned
  • Cumulative Exact + Adjacent Validity Percentage: the percentage of validity responses scored that are either a match with the score given by the expert scorers or are one score code different (higher or lower)
• Rolling Validity: the average validity achieved on a rolling batch. Average rolling validity is calculated and made available only when the pre-determined number of validity items are scored. (This average is used for comparison against the minimum validity threshold, which is required for scorers to be able to score.)
• Validity Response: a response that matches the anchors in the corresponding scoring guide
• Minimum Validity Threshold: the minimum average rolling validity required to continue to score without retraining
• Cumulative Validity Incentive Threshold: the minimum cumulative exact and cumulative exact + adjacent validity percentage required to receive incentive pay