What are the commonly used terms for scoring that I should be familiar with?
The following are the commonly used terms:
- Batch: a predefined number of student responses to a test question (Each batch is expected to take an average of one hour to score.)
- Rolling Batch: a set number of scored batches used to calculate the scorer’s average validity of those batches (This average is used to compare against the minimum validity threshold.)
- Item: a question or task that appears on an assessment or test
- Training Module: the training session, which includes an introductory video, the scoring guide, training responses, a practice test and the qualifying test (see below)
- Qualifying Test: the test used to assess a scorer’s ability to accurately score student responses
- Scoring Window: a specific time period (with a start and end date) during which the e-assessment scoring system is available for training and scoring
- Validity: the measure by which a scorer’s accuracy is determined through the use of student responses that have already been scored by an expert
- Cumulative Validity Incentive Threshold: the final average validity attained during the scoring activity that is needed for scorers to receive incentive payment
- Minimum Validity Threshold: the minimum average rolling validity required to continue to score without having to be retrained
- Average Rolling Validity: the average validity achieved on the rolling batches (This average is used for comparison against the minimum validity threshold, which is required for scorers to be able to score).
What is the scoring process?
Training and scoring will be self-directed through the e-assessment scoring system, which will be available 24 hours a day, seven days a week, with the exception of statutory holidays, during a specific time period referred to as a scoring window. This means that you can participate at your own time and from any location, if you meet the necessary technical requirements and qualifications. All the resources required for training and scoring will be available in the e-assessment scoring system.
Once you have completed your training and passed your qualification test on the e-assessment scoring system, you will be able to select from one to four batches, which you will have 24 hours to complete. A batch is estimated to take an average of one hour to score.
Please do not select any batches unless you are able to score them in the 24-hour period. If you complete your selected batches before the 24-hour period expires, you can select an additional batch (or batches), for which you will have another 24 hours to complete.
Online scoring is self-directed. You do not need to score daily as long as you score a minimum of seven batches per week during the entire scoring window and meet the minimum validity threshold. This is outlined in the Confidentiality, Non-Disclosure and Employment Agreement for an EQAO Scorer (Online) or Committee Member (Online) document.
If on one occasion you fall below the minimum validity threshold, you will have the opportunity to retrain and pass the qualification test again. If you are successful, you will be able to continue scoring. Otherwise, you will not be able to score and will be withdrawn from the scoring activity.
You can score as many batches as are available, but you will not be able to exceed 48 hours in a workweek (Sunday to Saturday).
What happens after I confirm a position?
You will receive an e-mail from firstname.lastname@example.org two days prior to the start of your scoring window. The e-mail will contain a link that will allow you to register for an account on the e-assessment scoring system and will include important dates by which you must register for your account, complete your training and start scoring. If you do not complete these steps by the dates specified, you will be withdrawn from the activity.
If you do not see the e-mail, check your spam or junk folders (you can add this e-mail address — email@example.com — to your “safe senders” list). If you have not received the e-mail one day prior to the scoring activity, please contact 1-888-444-4017.
When I log in to the e-assessment scoring system, I don’t see any batches to score. What does this mean?
The system updates in real time. This means that while schools are still administering the test, student responses are being captured and uploaded into the system. Batches can become available at any time during the administration window as students complete their test. You may check daily for additional available batches.
What is my pay?
Please refer to the Payment Guidelines for details on remuneration.
I used to have an EQAO account that I accessed with a marker PIN. What is my User ID?
For user accounts created prior to September 2014: If you used to access your EQAO profile using your marker PIN, your user ID is the e-mail address that was associated with your marker account. Your password remains the same. If the e-mail address associated with the old account is no longer being used, you can change it once you are logged into your profile. Ensure that the new address has not been entered in the “alternate e-mail address” field on your contact information page. Then change your User ID. If you are not able to access the account, contact us at firstname.lastname@example.org or 1-888-327-7377. Do not attempt to create a new account.
For user accounts created in or after September 2014: Your User ID is the e-mail address you used when you created your profile.
What is my EQAO profile password?
If you cannot remember your password, click Forgot Password, and an e-mail will be sent to you with instructions on how to reset your password. If you no longer have access to the e-mail account on file with EQAO, please contact us at email@example.com or 1-888-327-7377.
What if the name used in my EQAO profile doesn’t match the name in my Ontario College of Teachers (OCT) profile?
Your name in your EQAO profile must match the name used in your profile on the OCT Web site. As EQAO’s database is linked with OCT’s, a non-matching name will create communication errors between the two sites. You can try adding your middle name to either the first-name field or the last-name field one at a time to see if this helps to rectify the error. Otherwise, please take action to ensure that your name is the same in both profiles.
How can I tell if scoring opportunities are no longer available at the moment?
Some scorers may have received an invitation (or multiple invitations) to score, only to discover that scoring opportunities are no longer available at the moment. This is because the scoring positions that were available have been filled quickly, and there are no more scoring opportunities at the moment. Applicants should keep checking their EQAO profiles, as scoring positions can become available at any time.
How does EQAO determine how scoring positions are filled?
There are a limited number of scoring positions. As applicants accept offers to score, the scoring positions are filled quickly on a first-come, first-served basis. Sometimes additional scoring positions are added, or scoring positions can become available if scorers withdraw from their previously accepted scoring positions. Applicants should keep checking their EQAO profiles, as scoring positions can become available at any time.
How do I know if I have been invited to score?
Qualified candidates will receive e-mails with offers to score. As e-mails may be stopped by firewalls or sent automatically to junk or spam folders, it is recommended that applicants check their EQAO profiles regularly to see if they have any scoring offers. Applicants can add the e-mail addresses below to their “safe senders” list to avoid having the e-mail scoring invitations end up in a spam or junk folder. It’s always a good idea to check your spam or junk folders periodically.
List of EQAO e-mail addresses to add to your “safe senders” list:
Please note that EQAO is unable to accept job applications or employment inquiries by e-mail or by telephone.
Why didn’t I get an offer to score?
Your EQAO profile may not be up to date, or your qualifications and experience may not have met the requirements of the scoring position—or, simply, there may not be any more scoring opportunities available at the moment.
What happens to my chances of receiving scoring offers if I make changes to my EQAO profile?
Some scorers, upon making changes to their EQAO profile, may find that a scoring opportunity is no longer available to them. This is a normal occurrence, as the automated system verifies that the details, qualifications and current employment status in an applicant’s EQAO profile match the requirements for the scoring position. As a result, it is important that a scorer’s profile be maintained with accurate information regarding address, qualifications, current role in education, experience, etc.
How many offers to score can I accept?
You can accept as many scoring offers as you want, as long as the dates do not conflict with one another. Once you accept a scoring opportunity, you will not be able to accept another if the dates conflict with the dates of the offer that you have already accepted.
Please note: In some cases, you will be permitted to accept only one session in the initial round of invitations. If scoring opportunities are still available after that, you may receive notice to accept a second session.
Why am I seeing more than one scoring opportunity on my EQAO profile dashboard page?
This means you have qualified for more than one scoring position. You may accept as many of the scoring opportunities as you want, as long as the dates do not conflict with one another. If the dates conflict, please select and accept only the scoring offer that you prefer. To accept an offer to score, click Accept/Decline Position beside the scoring position on your profile dashboard page, then click Accept and continue on in the process until you hit Confirm.
How do I decline an invitation to score after I have already accepted it?
Log in to your EQAO profile, and then locate the offer you wish to decline on your profile dashboard page. Next, click Details, and then click Withdraw. If you are not able to withdraw through your EQAO profile, please contact us at firstname.lastname@example.org or 1-888-327-7377.
What if I want to change a particular task that I have been offered?
Unfortunately, EQAO cannot offer changes to the task you have been offered.
How can I view the scoring offers that I have accepted?
Log in to your EQAO profile. All of the scoring opportunities that you have accepted are listed under “Status” on your profile dashboard page. For more information about each scoring opportunity, click Details beside each opportunity.
What happens if I am no longer able to score after I have already accepted an invitation to score?
What if I am not sure if I can participate in scoring this year?
Do not accept any invitation to score until you can commit to the scoring opportunity.
I am a faculty of education student. How can I make sure that I am invited to scoring and assessment committees?
You need to update your proof of enrolment in each year of your program. Once you receive your Ontario College of Teachers (OCT) certificate and your information is on the OCT website, make sure that you update your current educator role and enter the OCT number in your EQAO profile. Ensure that the first name and last name in your EQAO profile match exactly what is on the OCT website.