What is the SET?

The School-wide Evaluation Tool, or SET, is a research tool designed to assess and evaluate the critical features of school-wide PBIS. The SET is completed during a two-hour site visit to your school by an independent (not staff of your school) SET Evaluator. The SET assessment includes observations in classrooms and common areas, and conducting informal interviews with staff and students.
Your school will receive a report after your school's SET is completed. This report will show how well your school is implementing the universal tier of PBIS. The SET measures specific features of PBIS implementation so your PBIS team can identify areas of strength and target areas for improvement.
*If your school is not familiar with the SET, we strongly recommend using the TFI or BoQ instead as there is less training and time involved for school teams, administrators, and new evaluators. Please contact Wilder Research at pbisevalmn@wilder.org if you have any questions.

When does my school need to have a SET?

If your school chooses to use the SET, the Minnesota Department of Education recommends that all schools that are entering PBIS training have a SET completed in the fall of their first year of training, and annually in the spring thereafter, until the school reaches full implementation.  Full implementation is defined as having a score of 80 or better overall and 80 or better in the domain of Behavioral Expectations Taught, or "80/80".

Click here for a printable version of the Minnesota Department of Education's recommended assessment schedule for PBIS schools that are in Years 1 and 2 of training.

How to become a SET evaluator

Note: If you already completed the SET evaluator webinar training and reliability test, go to #4.

1. Training – As soon as possible, you (the named SET evaluator) should complete the SET evaluator training webinar. This is about a 2-hour webinar that can be viewed at your convenience; it can be stopped and re-started where you left off. Follow this link to register for the webinar:
   https://www.youtube.com/watch?v=O_oYqXLqNhc&list=PLzbBB85oLWG
   F74sWzRQ16aTlHKZgLkKul&index=3
   You should complete the webinar training before a SET evaluation is scheduled at your school.
2. For this webinar – You will need to download and print (if possible):
   SET Evaluation Tool
   Case Study #2
3. Reliability Test – Next, you will be required to observe a SET evaluation that will be conducted by an experienced SET evaluator to be reliability tested. All SET evaluators must establish inter-rater reliability of 85% or higher before they complete a SET on their own. Learn more about Reliability Testing here:https://www.youtube.com/watch?v=k823pj0ZZ4o&list=PLzbBB85oLWG
   F74sWzRQ16aTlHKZgLkKul&index=2
4. Schedule SET evaluation – Call and email the school administrator (or PBIS coach if you are having trouble reaching the administrator) to schedule a date/time for conducting the SET at the school where you are assigned (see below). When scheduling the SET evaluation with the administrator, confirm all of the school's contact information.

BE PROACTIVE. It is your responsibility to ensure the SET is scheduled to be completed at this school during the scheduled timeframe (after you have completed the training webinar and reliability test). Please contact Wilder Research if you are having difficulty getting in touch with the school where you have been assigned.

5. Confirm the date of SET evaluation. Once a date has been scheduled to conduct the SET at your assigned match school, please email the school administrator and Wilder Research at pbisevalmn@wilder.org with: Name of school where you will conduct the SET; your name; your school; your cell/phone number; date/time of evaluation.
6. Enter SET score. Enter SET scores into the data entry page at [QUICKBASE LINK].
7. Write and send the SET narrative report. Within 7 days of completing the SET, please email the completed SET report to pbisevalmn@wilder.org AND the administrator at the school where the SET evaluation was conducted.

Here are the Templates for the reports:
SET Narrative Report – Template
SET Narrative Report – Example

What is a reliability test?

The SET is a rigorous measurement tool that requires accuracy and precision to administer. Therefore, before someone can be considered a "SET Evaluator" they must shadow an experienced SET Evaluator on at least one complete SET assessment. The SET Evaluator-in-training watches the experienced SET Evaluator conduct the assessment, including leading the observations and interviews.

The SET Evaluator-in-training scores the SET and so does the experienced SET Evaluator. After the assessment is complete, the experienced SET Evaluator reviews the SET Evaluator-in-training's scoring against their own to determine if there is adequate consistency (i.e., "reliability") in scoring.

It is important to remember that although the reliability testing may seem like an intimidating process, we want to ensure that all SET Evaluators are scoring schools consistently, so we can understand how PBIS is being implemented across the state. The reliability test can also be looked at as another opportunity to really understand how to identify the critical elements of PBIS in a school.

Learn more about Reliability Testing in this short video: https://www.youtube.com/watch?v=k823pj0ZZ4o&list=PLzbBB
85oLWGF74sWzRQ16aTlHKZgLkKul&index=2

SET Narrative Report – Template

SET Narrative Report – Example

If you need more information about the SET training session, please contact Wilder Research at pbisevalmn@wilder.org or 651-280-2960.