In this webinar, we will give practical advice about evaluation reporting in the ATE context, including report content and structure, integrating evaluation report content into annual reports to NSF, and using results. We will provide step-by-step guidance for developing an ATE evaluation report that balances the competing demands that reports be both comprehensive and concise. We’ll discuss the how-where-and-what of including evaluation results in NSF annual reports and project outcome reports. Finally we’ll address how to use evaluation results to inform project-level improvements and build the case for further funding. Participants will leave the webinar with a clear strategy for creating effective ATE evaluation reports that meet NSF accountability requirements and support project-level improvement.

*This webinar was in inspired, in part, by the work of Jane Davidson, author of Evaluation Methodology Basics: The Nuts and Bolts of Sound Evaluation (Sage, 2005).

For evaluation reports to be meaningful and useful, clients and other stakeholders need to be able to find the forest of the trees. What’s the bottom line? Is the project on track? What’s the project’s quality? What do these numbers mean? To convey clear answers to these types of questions, evaluators need to interpret, synthesize, and report results in meaningful and efficient ways. In this webinar we’ll share strategies for transforming data into genuinely evaluative conclusions and presenting results in visually compelling ways to aid in understanding and using results.

The use of surveys for data collection is ubiquitous in evaluation, so writing good survey questions is an essential skill for any evaluator. In this webinar, we’ll cover the essential DOs and DON’Ts of writing survey items. Crafting good questions is an art as well as a science and requires careful attention to context, including respondents, evaluation purposes, and intended use of results. We’ll dissect examples of good and bad question phrasing and response options, explore the implications of various question-and-answer formats for data analysis, and offer strategies to ensure that surveys will yield meaningful and useful data for your evaluation.

In this webinar, participants will learn about evaluating ATE initiatives in practical, yet meaningful ways. The Kirkpatrick “Levels” model for evaluation is a systematic approach for assessing a project’s quality and effectiveness in terms of participants’ satisfaction, learning of the material, application of new skills or content, and resulting impacts. Participants will learn about what questions should drive data collection at each level, steps to take to obtain data of sufficient quantity and quality, and how to interpret and use the evaluation results. Proven strategies for tracking students' employment outcomes will be shared by Kevin Cooper, PI for the Regional Center for Nuclear Energy Education and Training at Indian River State College.

A grant proposal that includes a strong evaluation plan linked to a sound project logic model will be reviewed more favorably than one that does not. Will it make or break your chances for funding? No, but attending to evaluation matters as you develop your proposal, especially in terms of how you will assess your project’s intellectual merit and broader impacts, is likely to strengthen your overall proposal and give you a competitive edge. Considering your proposal through an evaluative lens as you are crafting it can help you avoid common proposal pitfalls, such as writing goals that are either too lofty or too simplistic or failing to demonstrate a logical relationship between your activities and your intended outcomes. In this webinar, we’ll share two tools specifically developed for ATE proposers: a checklist for developing evaluation plans for ATE proposals and a template for creating simple, yet powerful project logic models. There will be ample time with our expert panel, which includes an ATE evaluator, PI, and program officer.

By the end of the webinar, participants will
1. Understand how to prepare an ATE proposal that meets NSF’s requirements for evaluation
2. Know how to establish a working relationship with an external evaluator and what to expect from him or her (both before and after award)
3. Be able to create a logic model to convey your proposed project’s activities and intended outcomes
4. Understand the role of evaluation in ATE projects and how to align an evaluation plan with project goals

Presenters:
Lori Wingate, PI, EvaluATE
Michael Lesiecki, PI, MATEC
Penny Billman, REGS Consulting, LLC
Connie Della-Piana, NSF