WEEK 3: Evaluating Programs & Human Performance Technology

1.

I was intrigued by the models of Instructional Design and Evaluation. The two models that I would like to focus on are Brinkerhoff’s Success Case Method and Patton’s Utilization-Focused Evaluation.

Let’s begin by looking at Brinkerhoff’s Method. This model is focused on finding out what worked in a particular training or other organizational intervention. The founder of this method designed it with it being “a quick and simple process that combines analysis of extreme groups with case study and storytelling….to find out how well some organizational initiative is working.” Brinkerhoff’s model has five steps to ensure this process is carried out effectively.

• Identify and Define the program to be evaluated, explicate its purpose.
• Explicate the major program goals and listing all impacts/outcomes that are hoped for or expected to result from the program.
• Conduct a survey to identify the best cases and worst cases.
• Schedule and conduct in-depth interviews with both success cases and those that were not successful.
• Write-up and communicate the findings discovered by the evaluation methods.

This method is appealing to me because the steps are properly sequenced and there is a clear motive and purpose behind each. Following them will allow us to arrive to accurate data concerning the program being evaluated. I personally like that it looks at both sides of the spectrum. It does not only focus on the positives, but looks at the negatives to see where the program “went wrong” so to say.

The second method, Patton’s Utilization-Focused Evaluation or U-FE is “evaluation done for and with specific intended users for specific, intended uses.” In this method, the evaluator focuses on use from the beginning until the end of the evaluation, and during that time, he or she continually facilitates use and organizational learning or any other process that helps ensure that the evaluation results will continue to be used once the evaluator leaves the organization. (Reiser and Dempsey, 2012.) This method follows the nine steps highlighted below:

• Conduct a readiness assessment.
• Identify the “primary intended users” and develop a working relationship with them.
• Conduct a situational analysis.
• Identify the “primary intended uses.”
• Focus the evaluation. (i.e., high priority issues and questions)
• Design the evaluation.  
• Collect, analyze, and interpret the evaluation data.  
• Continually facilitate evaluation use.
• Conduct a metaevaluation (i.e., evaluation of the evaluation)

As one can see from the specific and detailed steps, this method is committed to being a full approach evaluation. I really like how the book recommends one take courses in specific areas and fields to be an evaluator in this approach. This highlights the seriousness of the system and the ownership one must have to take full advantage of this method.

I would use Brinkerhoff’s Success Case Method to measure the effectiveness of one of our programs, Odyssey Ware. This program is an online curriculum where students work at their own progress. It would be ideal to evaluate because there are students who experience great success but also those who do not. Brinkerhoff’s method would allow us to get to the root of the success cases as well as those who do not make it.

2.

I believe importance must be given to the importance of funds or Monetary Values. This is an important question that allows us to ensure funds are being spent on programs that are yielding positive results. These evaluation methods are an exceptional means to indicate that we are spending on effective and efficient methods. I also like how the book mentions “The Value Shift.” This challenges us to view through the lens of results versus activity. Return on investment and management of resources should be considered. We must be good stewards of what is available to us. We are to be held accountable and if our programs are not performing, it is our responsibility to view other programs that will be beneficial.

3.

An example that comes to mind in regards to a non-instructional approach and human performance in solving a problem is an Incentive Lunch reward that our school implemented a few years back to address a high amount of tardies. Students who did not receive a tardy Monday-Thursday were rewarded with an extra twenty minutes of lunch on Friday. This cut our tardies down from approximately 500 a week to approximately 40. A simple reward of twenty extra minutes has had a drastic impact on minimizing students distracting instruction by walking into class late.  

Reiser, R. A., & Dempsey, J. V. (2017). Trends and issues in instructional design and technology. Boston: Pearson Education.