Print Page | Sign In | Join AESP
Certificate of Excellence – Foundations of Impact Evaluation



In cooperation with the United States Department of Energy, AESP Institute is proud to launch an in-depth training program on DSM program impact evaluation.  Upon completion of all necessary coursework and meeting other requirements, participants will receive AESP’s “Certificate of Excellence – Foundations of Impact Evaluation.”

The learning objectives of the certificate program are:

·         Understand the demand side management industry, including its basic purpose, components, and function.

·         Identify the purpose of evaluation, measurement and verification.

·         Understand and apply foundational program impact evaluation knowledge that grounds and guides professional practice.

·         Identify and define program impact evaluation methodology terms.

·         Demonstrate the ability to frame impact evaluation questions.

·         Determine appropriate impact evaluation methods, including quantitative, qualitative and mixed methods. 

This course covers the basics of the energy industry and demand-side management industry, before diving into the principles of program impact evaluation.

In this cohort-based program, participants enroll at the same time as their peers and experience the learning journey together (communicating via an online discussion board and eventually meeting in person), culminating in a final project which they complete as a team. 

Individually, participants will be required to complete two courses and a Project. The first course will be online and self-paced. The second course will be a two day in-person course. The Project will be introduced during the in-person course.


2020 Cohorts:

Waltham, MA - April 8-9, 2020



Course Outline

1. EE Programs
- EE programs, their basic purpose, components, and functioning
- History of EE programs and why EE programs were originally created
- Basics of the utility business including load profiles, obligation to serve, load duration curves, load factors, rudimentary rate making (which includes differentiation between fixed and variable costs)
- Different types of EE and DER programs and purposes of each, savings taxonomy (e.g. portfolios, programs, projects, measures)
- Compare demand-side and supply-side resources, IRPs, and common cost-effectiveness assessments (e.g. California Standard Practice Manual and National Standard Practice Manual)
- Types of customers and participants (i.e., various utility sectors and their customer characteristics)
- Players within the EE program sphere and the political realities of each
- Types of regulatory bodies and how they interact with EE programs

2. Program Evaluation in General
 - Foundations of program evaluation e.g. standards, guidelines, principles, competencies, approaches, and theories.    
- Key program evaluation concepts e.g. evaluability, impacts, processes, worth, etc.
- How evaluation and EE programs interact; how evaluation timing may affect a program
-  The different goals of evaluation
- Types of evaluation associated with EE/Demand Response/Distributed Energy Resource programs e.g. impact, process
- Evaluation standards/guidelines e.g. the JCSEE standards and the AEA guidelines
- Roles and relationships of the evaluator in the program design/implementation/evaluation cycle, in regard to regulators and program administrators
- What is included in evaluation frameworks

3. EE Impact Evaluation    
- Methodology that grounds inquiry in program evaluation practice    
- Key EE impact evaluation terms such as effective useful life, net and gross energy, and non-energy impacts
- Various impact designs, when to apply them, and tools of each e.g. quasi-experimental, experimental designs
- Counterfactual, and how baselines are related to the counterfactual
-  EE evaluation resources e.g. SEE Action publications, Technical Reference Manuals (TRMs), etc.
- Relationships and differences between impact evaluations and cost effectiveness evaluations, process evaluations, or market evaluations

4. Key Impact Evaluation Methods and Tasks

- Appropriate methods, including quantitative, qualitative, and mixed methods (to the extent practical, absent significant professional experience)
- How to plan an impact evaluation and the tasks that must occur prior to choosing or implementing any impact method e.g. research goals, questions, etc.
- How a sampling frame can affect an impact evaluation
- Different types of impact evaluation approaches/methods along with their uses, strengths, limitations, risks, and relative costs
- Important tasks within four key impact methods
- Complications that may arise during each task of estimating energy savings
- Savings persistence and savings interactions
- Relationship between evaluation costs, accuracy, and timeliness
- Relative precision, absolute precision, and accuracy, as well as the differences in each

5. Framing the Evaluation 
- Appropriate evaluation purposes
- Evaluation questions
- How evaluation goals and research questions support choice of impact method(s)
- What a report should include and how to structure a report depending on the audience
- How to write about sampling and non-sampling errors within a report


Target Audience

The program is designed with new evaluation staff in mind, those with less than 3 years’ experience in the energy industry. Its alternatives are a formal in-house training program, or more commonly “learning-on-the-job” where a supervisor or colleague guides new staff over several months or years.

Ideal candidates for this program include new staff entering a research and evaluation function in DSM consultancies, consultancies that specialize in research and evaluation, government, state and regional energy organizations and large utilities with in-house evaluation needs (National Grid, etc.).



The course will be delivered in two parts:

  • Course 1: Delivered online. Self-paced.
  • Course 2 and capstone project: Course participants will meet in-person for 2 days in a city and date that will have been pre-determined before the start of the course. Here they will attend live lectures to complete Course 2, then work collaboratively with a team to complete the project. 



CEUs credits will be awarded upon completion of the course requirements.

Course 1 (Online) - 0.4
CEU Course 2 (In-person) - 0.7
CEU Total - 1.1 CEU



US$699, AESP member

US$899, non-member

or 4 group member points

For more information, contact


Latest Tweets

Contact Us

15215 S 48th St, STE 170
Phoenix, AZ 85044
Tel: 480-704-5900

Privacy Policy