Standard Data Review Protocol

Why should you establish a standardized process for Data Review?

Standardizing a Data Review process helps set a clear expectation that the end product of Data Review is not knowledge, but action. Any Data Review Protocol should ensure that raw data is converted into a clear and digestible format before the reflection process so that reviewers can focus their energies on reflecting on the data, rather than synthesizing the data.

By separating the work of creating and refining a Data Review process from the work of implementing the process in practice, a standardized Data Review Protocol helps you focus your designated Data Review time on what you’re reviewing, not how to review it.

How should you conduct a Data Review?

When: As soon as possible after collecting relevant data. The more dated the data, the less relevant it will be for making timely decisions.
Why: The goal is to learn and improve — not to assign blame for shortfalls. Set norms accordingly.
Who: The facilitator guides the conversation, but they do not have all the answers. Every voice matters.
What: Don’t just review aggregate data. Disaggregate by demographic to reveal impact across lines of difference.
How: Prioritize quality over speed, but adjust the time allotment based on how comprehensive the dataset is:
- A single tutor reviewing daily assessment data for all of their students should only need about 15 minutes to complete the protocol.
- An entire team reviewing the past year’s worth of data could take half a day to complete the protocol.

Standard Data Review Protocol
This is a standard protocol you can use for a wide variety of reflections. It is broadly applicable whenever someone has data (qualitative, quantitative, or both) to review. Your organization might apply it to end-of-year outcome data; a head of program might apply it to training data at the end of tutor preservice training; a leadership team might apply it to quarterly parent feedback. There are also versions of this protocol specifically tailored for tutors reviewing student data.
Step	Purpose	Possible Questions
Step 1: WHAT did we want to happen?	Ensure all participants are on the same page about what the goal or intended outcome was.	What was our goal? (Refer to any relevant performance expectations from the Performance Measurement Plan.) What was our plan for reaching this goal?
Step 2: WHAT actually happened?	Ensure all participants are on the same page about what the actual outcome or result was. Explore the divergences between expectations and realities.	Did we meet our goal? What did we achieve? Did we follow our plan? If not, where did we diverge from it? Where were the differences between our intent and our impact?
Step 3: SO WHAT did we learn?	Reflect on successes and failures during the course of the project, activity, event or task. The question ‘Why?’ generates understanding of the root causes of these successes and failures.	What worked? What didn’t work? What could have gone better? Was our plan a success? Why or why not?
Step 4: SO WHAT can we do better in the future?	Generate clear, actionable recommendations and next steps for future projects.	What would we do differently next time? What advice would you give yourself if you were to go back to where you were at the start of the project? What two or three key lessons would you share with others? What should be different one year from now (or after the next similar project) given this conversation? What comes next for us on this project? Are there any lessons for you, personally, to internalize?
Step 5: NOW WHAT changes do we need to make to our project and individual plans?	Incorporate key lessons into your future actions. Document all key lessons for those who may inherit this project in the future.	Add reflections and next steps to individual plans. Add reflections and next steps to project plans.