At least two evaluation functions or purposes should be considered:

Formative evaluations provide information to improve a product or process. Formative evaluation of new instructional materials would ideally be conducted with experts and selected target audience members prior to full-scale implementation. Expert review of content by instructional designers or subject-matter experts may provide useful information for modifying or revising selected strategies. Learner review is the process of determining if students can use the new materials, if they lack prerequisites, if they are motivated, and if they learn. In both types of formative review, problems may be discovered such as as spelling errors, confusing organizational structures, confusing media interfaces and navigation, or a need for more illustrations and examples. These voiced concerns would lead to revised and improved instruction.

Summative evaluations provide short-term effectiveness or long-term impact information to decide whether or not to adopt a product or process. Summative evaluation can occur just after new course materials were implemented in full (i.e., effectiveness evaluation), or several months to years after the materials were implemented in full (i.e, impact evaluation). It is important to specify what decisions will be made as a result of the evaluation, then develop a list of questions to be answered by the evaluation. You may want to know if learners met your objectives, if the innovation was cost-effective, if the innovation was efficient in terms of time to completion, or if the innovation had any unexpected outcomes. Data could include quantitative results of the innovation including criterion-referenced assessments to determine how well students met specified objectives. Data could also include qualitative interviews, observations, and artifact analyses.