SET Evaluator Training
SET Evaluator Training and Evaluation Process
Note: If you already completed the SET evaluator webinar training and reliability test, go to #4.
Training – As soon as possible, you (the named SET evaluator) should complete the SET evaluator training webinar. This is about a 2-hour webinar that can be viewed at your convenience; it can be stopped and re-started where you left off. Follow this link to register for the webinar:
You should complete the webinar training before a SET evaluation is scheduled at your school.
Reliability Test – Next, you will be required to observe a SET evaluation that will be conducted by an experienced SET evaluator to be reliability tested. All SET evaluators must establish inter-rater reliability of 85% or higher before they complete a SET on their own. Learn more about Reliability Testing here:https://www.youtube.com/watch?v=k823pj0ZZ4o&list=PLzbBB85oLWG
*For SET evaluators whose schools are in Year 1 of PBIS training (cohort 9): An experienced SET evaluator will contact your school's administrator to schedule the baseline SET at your school at a mutually agreeable date and time, during September and October 2013. You should complete your reliability test during the SET at your school, if possible. The reliability test can also be conducted at another school if you are not available to do the reliability test on the date that your school's SET is scheduled. Contact Wilder Research at email@example.com or 651-280-2960 to schedule this if needed.
*For all other SET evaluators (cohorts 1-8): An experienced SET evaluator will contact you to arrange to conduct your reliability test at another school at a time that is mutually convenient, sometime during September and October 2013.
Schedule SET evaluation – Call and email the school administrator (or PBIS coach if you are having trouble reaching the administrator) to schedule a date/time for conducting the SET at the school where you are assigned (see below). When scheduling the SET evaluation with the administrator, confirm all of the school's contact information.
BE PROACTIVE. It is your responsibility to ensure the SET is scheduled to be completed at this school during the scheduled timeframe (after you have completed the training webinar and reliability test). Please contact Wilder Research if you are having difficulty getting in touch with the school where you have been assigned.
Note: No-cost exchange match assignments were made by Wilder Research. Factors in match assignments include previously known school or district partnerships; geographic proximity between schools; and availability in the pool of eligible schools. Adjustments to match assignments can be made if needed with prior consent from Wilder Research.
Confirm the date of SET evaluation. Once a date has been scheduled to conduct the SET at your assigned match school, please email the school administrator and Wilder Research at firstname.lastname@example.org with: Name of school where you will conduct the SET; your name; your school; your cell/phone number; date/time of evaluation.
Enter SET score. Enter SET scores into the data entry page at http://pbisevalmn.org/set-scoring.php.
Write and send the SET narrative report. Within 7 days of completing the SET, please email the completed SET report to email@example.com AND the administrator at the school where the SET evaluation was conducted.
What is a reliability test?
The SET is a rigorous measurement tool that requires accuracy and precision to administer. Therefore, before someone can be considered a "SET Evaluator" they must shadow an experienced SET Evaluator on at least one complete SET assessment. The SET Evaluator-in-training watches the experienced SET Evaluator conduct the assessment, including leading the observations and interviews.
The SET Evaluator-in-training scores the SET and so does the experienced SET Evaluator. After the assessment is complete, the experienced SET Evaluator reviews the SET Evaluator-in-training's scoring against their own to determine if there is adequate consistency (i.e., "reliability") in scoring.
It is important to remember that although the reliability testing may seem like an intimidating process, we want to ensure that all SET Evaluators are scoring schools consistently, so we can understand how PBIS is being implemented across the state. The reliability test can also be looked at as another opportunity to really understand how to identify the critical elements of PBIS in a school.
Learn more about Reliability Testing in this short video:
If you need more information about the SET training session, please contact Wilder Research at firstname.lastname@example.org or 651-280-2960.