Evaluating educational software

Useful evaluation measures for educational software require consideration of a variety of constraints. In addition to developing the correct questions to obtain the information you or your institution needs, the form or measure needs to be easy for the evaluator to use. Consider layout and design of the measure, including font size and length. Think about how the measure is distributed to the evaluator(s), how it will be returned to the individual(s) who will be making decisions on the purchase, and if/what information needed after the purchase decision is made.

On the evaluation tool, be sure to include an area identifying the instructional material.

General concepts to keep in mind when considering instructional materials:

Who made it? Is it valid and reliable?
Who is the audience?
Is it easy for someone to use?
Is it interesting?
Is this product using the tools available to it?
How much does it cost?
Is it useful?

Sample questions (in no particular order):

Likert style
Short answer

Likert style	Not applicable	Poor/Low	Excellent/High
Able to adapt hardware/software to make program operational	Not applicable	Poor/Low	Excellent/High
Able to distinguish choices made	Not applicable	Poor/Low	Excellent/High
Able to enter and exit program as needed	Not applicable	Poor/Low	Excellent/High
Able to identify where you are in program	Not applicable	Poor/Low	Excellent/High
Able to make minor modifications in content to match curriculum	Not applicable	Poor/Low	Excellent/High
Accurate and clear content	Not applicable	Poor/Low	Excellent/High
Adapts to user input	Not applicable	Poor/Low	Excellent/High
Appropriate use of computer	Not applicable	Poor/Low	Excellent/High
Authenticity of on-screen characters	Not applicable	Poor/Low	Excellent/High
Benefits/frequency of use worth cost now and in future	Not applicable	Poor/Low	Excellent/High
Can be used independently with minimal instruction	Not applicable	Poor/Low	Excellent/High
Can review previous screens	Not applicable	Poor/Low	Excellent/High
Capable of producing hardcopy as needed	Not applicable	Poor/Low	Excellent/High
Captures and sustains user interest	Not applicable	Poor/Low	Excellent/High
Clear instructions	Not applicable	Poor/Low	Excellent/High
Color/sound/animation/video/graphics used appropriately	Not applicable	Poor/Low	Excellent/High
Consistent with curriculum	Not applicable	Poor/Low	Excellent/High
Content accurate and current	Not applicable	Poor/Low	Excellent/High
Content free of stereotypes and cultural bias	Not applicable	Poor/Low	Excellent/High
Content organized and clearly presented	Not applicable	Poor/Low	Excellent/High
Content present effectively	Not applicable	Poor/Low	Excellent/High
Content relevant	Not applicable	Poor/Low	Excellent/High
Control amount of time on each screen	Not applicable	Poor/Low	Excellent/High
Correct spelling, grammar, & punctuation	Not applicable	Poor/Low	Excellent/High
Creative design	Not applicable	Poor/Low	Excellent/High
Documentation is clear and complete	Not applicable	Poor/Low	Excellent/High
Duplicates existing materials available	Not applicable	Poor/Low	Excellent/High
Ease in begining program	Not applicable	Poor/Low	Excellent/High
Ease of exit from program	Not applicable	Poor/Low	Excellent/High
Ease of exit to various areas of the program	Not applicable	Poor/Low	Excellent/High
Ease of running entire program	Not applicable	Poor/Low	Excellent/High
Effective branching	Not applicable	Poor/Low	Excellent/High
Effective cures for knowledge retention	Not applicable	Poor/Low	Excellent/High
Effective use of color	Not applicable	Poor/Low	Excellent/High
Effective use of graphics	Not applicable	Poor/Low	Excellent/High
Effective use of interactivity	Not applicable	Poor/Low	Excellent/High
Effective use of sound	Not applicable	Poor/Low	Excellent/High
Effective use of video vs. computer to present content	Not applicable	Poor/Low	Excellent/High
Effectively captures user's attention	Not applicable	Poor/Low	Excellent/High
Efficient use of user time	Not applicable	Poor/Low	Excellent/High
Evaluation provided formative	Not applicable	Poor/Low	Excellent/High
Evaluation provided summative	Not applicable	Poor/Low	Excellent/High
Feedback helpful	Not applicable	Poor/Low	Excellent/High
Feedback objective	Not applicable	Poor/Low	Excellent/High
Feedback supports learning objectives	Not applicable	Poor/Low	Excellent/High
Format enhances learning	Not applicable	Poor/Low	Excellent/High
Free from stereotypes	Not applicable	Poor/Low	Excellent/High
Free pf programming errors	Not applicable	Poor/Low	Excellent/High
Has mechanism for keeping content updates as needed	Not applicable	Poor/Low	Excellent/High
Holds attention	Not applicable	Poor/Low	Excellent/High
Immediate and satisfying feedback	Not applicable	Poor/Low	Excellent/High
Individual by use of varied feedback	Not applicable	Poor/Low	Excellent/High
Instructional design achieves purpose	Not applicable	Poor/Low	Excellent/High
Instructional design evident	Not applicable	Poor/Low	Excellent/High
Instructions easy to follow	Not applicable	Poor/Low	Excellent/High
Interaction is frequent and varied	Not applicable	Poor/Low	Excellent/High
Key concepts defined	Not applicable	Poor/Low	Excellent/High
Learning objectives clearly stated	Not applicable	Poor/Low	Excellent/High
Length of video/audio segments	Not applicable	Poor/Low	Excellent/High
Motivating	Not applicable	Poor/Low	Excellent/High
Multiple paths or levels provided	Not applicable	Poor/Low	Excellent/High
Objectives of programs met	Not applicable	Poor/Low	Excellent/High
Personalized individualized by use of branching	Not applicable	Poor/Low	Excellent/High
Program runs smoothly without glitches and blind loops	Not applicable	Poor/Low	Excellent/High
Protects privacy of user responses	Not applicable	Poor/Low	Excellent/High
Purchase option: discount on multiple copies	Not applicable	Poor/Low	Excellent/High
Purchase option: lease	Not applicable	Poor/Low	Excellent/High
Purchase option: sale	Not applicable	Poor/Low	Excellent/High
Purchase option: site license/network	Not applicable	Poor/Low	Excellent/High
Quality of narration	Not applicable	Poor/Low	Excellent/High
Quality of video/audio	Not applicable	Poor/Low	Excellent/High
Reading level appropriate	Not applicable	Poor/Low	Excellent/High
Record keeping capacity adequate to meet future needs	Not applicable	Poor/Low	Excellent/High
Requires additional instruction materials (ex: workbooks)	Not applicable	Poor/Low	Excellent/High
Scope and depth of content appropriate for intended use	Not applicable	Poor/Low	Excellent/High
Screen display aesthetic	Not applicable	Poor/Low	Excellent/High
Screen display is clear with consistent format	Not applicable	Poor/Low	Excellent/High
Screen transitions smooth	Not applicable	Poor/Low	Excellent/High
Size and style of text adjustable	Not applicable	Poor/Low	Excellent/High
Size and style of text easy to read	Not applicable	Poor/Low	Excellent/High
Speed of presentation under control of user	Not applicable	Poor/Low	Excellent/High
Student participation promoted	Not applicable	Poor/Low	Excellent/High
Touch panels clearly identified	Not applicable	Poor/Low	Excellent/High
User can control of order of presentation	Not applicable	Poor/Low	Excellent/High
User controls pace and sequence	Not applicable	Poor/Low	Excellent/High
Uses graphics and/or animation	Not applicable	Poor/Low	Excellent/High
Uses rapid calculations	Not applicable	Poor/Low	Excellent/High
Varies with user input	Not applicable	Poor/Low	Excellent/High
Vender support available	Not applicable	Poor/Low	Excellent/High
I would give this program a rating of:	Not applicable	Poor/Low	Excellent/High

Example of short answer questions

This program could be used for:
Title of program:
Source (publisher/producer/distributor)
Date produced:
Update frequency:
Cost:
Program version:
Type of Computer (Mac, PC, OS)
Computer Memory required/recommended:
Number of disk drives required:
Hard disk required (); preferred ()
Copyrighted
Back-up copy permitted
Networked: yes (); no ()
Web distribution: yes (); no ()
Web browser dependent: yes (); no (); if yes: IE (); Netscape (); Other ()
Other equipment required:
Printer required: yes (); no ()
Printer memory required/recommended:
Other software plugins needed: yes (); no (); if yes, what:
Additional instructional materials required:
Suggested running time:
Intended audience: student; staff; nurse; physician; patient (reading level, age)
Web access security: