3.3 Evaluation Questions

You probably have some idea of why you are conducting an evaluation—possibly one of the reasons listed in What is an evaluation? For instance, evaluations can directly affect future actions or decisions or change someone’s attitude or way of thinking.1 Evaluation questions transform a vague idea into specific elements that are tied to one or more outcomes and can be measured through assessments.

All evaluations answer the same three fundamental questions, according to the Principal Investigator’s Guide:2

  1. What? (What happened or happens and what are the results?)
  2. So what? (Why did the results occur the way they did, and what are the implications?)
  3. Now what? (What actions, decisions, or recommendations can be made based on the results?)
DON'TDO
Ask more questions than you have time or resources to answerPrioritize questions based on the value of their answers and your ability to answer them2
Come up with all the questions yourselfTalk to stakeholders for their input
Try to answer all questions from every stakeholderUse stakeholder input to inform the evaluation design, not determine it
Ask questions that are too broad or aspirationalAsk questions that can be answered realistically

EVALUATION QUESTION EXAMPLES

How financially sustainable is the robotics lab? 
Does the robotics lab help teens develop collaboration skills?


EVALUATION TYPEDESCRIPTIONEXAMPLES
Front-end evaluationUsed in informal science education; similar to audience or user research and used to inform the initial design of a program.2What aspects of this topic are our youth interested in?

What is our audience’s current skill level?
Formative evaluationPeriodic evaluations conducted during the development and implementation of a program. Used for course-correcting if necessary. Formative evaluation can look at how a project is progressing towards its goals whether the implementation of the project going according to plan.3 Are participants learning what we expected?

Are there any issues with implementation of the program?
Summative evaluationFocuses on conditions at the end of the program and is an evaluation of the entire process. It may report results from earlier formative evaluations as well. Did participants learn what and as much as we wanted?

What was the value of the partnership?

1 Evaluation Use, by Marvin C. Alkin, in the Encyclopedia of Evaluation.

2: Principal Investigator’s Guide: Managing Evaluation in Informal STEM Education Projects, by R. Bonney, K. Ellenbogen, L. Goodyear, & R. Hellenga, p. 13, 16, 61. Center for Advancement of Informal Science Education, 2001.

3: The 2010 User-Friendly Handbook for Project Evaluation, pp. 8-9, by J. F. Westat. National Science Foundation, 2010.