3.3 Evaluation Questions
You probably have some idea of why you are conducting an evaluation—possibly one of the reasons listed in What is an evaluation? For instance, evaluations can directly affect future actions or decisions or change someone’s attitude or way of thinking.1 Evaluation questions transform a vague idea into specific elements that are tied to one or more outcomes and can be measured through assessments.
All evaluations answer the same three fundamental questions, according to the Principal Investigator’s Guide:2
- What? (What happened or happens and what are the results?)
- So what? (Why did the results occur the way they did, and what are the implications?)
- Now what? (What actions, decisions, or recommendations can be made based on the results?)
| DON'T | DO |
|---|---|
| Ask more questions than you have time or resources to answer | Prioritize questions based on the value of their answers and your ability to answer them2 |
| Come up with all the questions yourself | Talk to stakeholders for their input |
| Try to answer all questions from every stakeholder | Use stakeholder input to inform the evaluation design, not determine it |
| Ask questions that are too broad or aspirational | Ask questions that can be answered realistically |
EVALUATION QUESTION EXAMPLES
How financially sustainable is the robotics lab?
Does the robotics lab help teens develop collaboration skills?
| EVALUATION TYPE | DESCRIPTION | EXAMPLES |
|---|---|---|
| Front-end evaluation | Used in informal science education; similar to audience or user research and used to inform the initial design of a program.2 | What aspects of this topic are our youth interested in? What is our audience’s current skill level? |
| Formative evaluation | Periodic evaluations conducted during the development and implementation of a program. Used for course-correcting if necessary. Formative evaluation can look at how a project is progressing towards its goals whether the implementation of the project going according to plan.3 | Are participants learning what we expected? Are there any issues with implementation of the program? |
| Summative evaluation | Focuses on conditions at the end of the program and is an evaluation of the entire process. It may report results from earlier formative evaluations as well. | Did participants learn what and as much as we wanted? What was the value of the partnership? |
1 Evaluation Use, by Marvin C. Alkin, in the Encyclopedia of Evaluation.
2: Principal Investigator’s Guide: Managing Evaluation in Informal STEM Education Projects, by R. Bonney, K. Ellenbogen, L. Goodyear, & R. Hellenga, p. 13, 16, 61. Center for Advancement of Informal Science Education, 2001.
3: The 2010 User-Friendly Handbook for Project Evaluation, pp. 8-9, by J. F. Westat. National Science Foundation, 2010.
