Performance Assessment

Rationale

For the past several years the dissatisfaction with objectively scored tests has intensified significantly. Teachers are rebelling; the press has joined in the criticism. Test companies have tried to respond but not altogether successfully. The basis of the complaint lies in the fact that objectively scored tests are not valid measures of what is important to learn in school. Objectively scored tests--multiple choice, completion, short answer--emphasize the acquisition of and the memorization of information. They cannot be appropriately used to measure many higher level thinking abilities nor can they be used to measure some other important goals of schooling. To deal with this reality, educators have begun to develop and use "more authentic" measures of student learning--measures which do, in fact, assess higher level thinking and other skills and abilities which students will use throughout their lives.

The term "authentic" as it is currently used to describe this new genre of tests offends many test writers. It seems to suggest that objectively scored tests are phony or artificial. Not so, but they are limited in the kinds of learning that they can measure. Hopefully, this lesson will not cause you to throw the baby out with the bath water but rather will cause you to think carefully about your purpose for testing students and then select the kind of test (items) which will best serve your purpose. For some of your purposes objectively scored items will be appropriate, for others you will need more authentic measures.

What is authentic assessment?

To begin this lesson, carefully review the three samples which follow.

Look for things that all three have in common.

Notice ways that they are different from more traditional "objective" paper and pencil tests. Pay particular attention to the skills and abilities that they are testing.

Water Quality (performance test)

Designing a Classification System (performance test)

Not In My Backyard (performance task)

*******************************************************************************************

P	9th Grade Science	Standard: 04
Objective: .01 Relate the properties of water to Earth system
ILO: 2d. Collect and record data using procedures designed to minimize error 2e. Analyze data and draw warranted inferences 6d. Construct tables, graphes, charts, diagrams, and models to describe and summarize data

Water Quality

You have been given 3 samples of water. Using the materials available, test the samples for pH, hardness, and suspended solids. Make a complete record of your results. Based on the tests, Describe where each sample of water may have come from.

Materials Needed

Litmus paper, diluted detergent, three clear vials with lids, graduated cylinder, filter paper, transparent grid, beaker. (Materials will vary depending on what test you want the students to perform; these are just suggestions.)
(tap water, dirty river water, distilled water, mineral water, etc.)

Scoring Guide (out of 100%)

data table ....................................................... 20%
correct pH readings .............................................. 20%
correct comparison of hardness ................................... 20%
correct reading for solids ....................................... 20%
reasonable sources ............................................... 20%

*******************************************************************************************

P 7th Grade Science Standard: 05

Objective: .03 Compare and contrast a variety of classification schemes

ILO:
1b. Develop and use categories to classify observations
2e. Analyze data and draw warranted inferences
4d. Recognize the personal relevance of science in daily life
6d. Construct tables, graphes, charts, diagrams, and models to describe and summarize data
7a. Understand that the goal of science is to produce a systematized body of knowledge consisting of concepts, theories, and generalizations that have explanatory and predictive power.

Designing a Classification System

The Problem

Classification systems are based on levels of shared characteristics. Organisms may be classified according to similarties in structure, or methods of carrying out life processes. Maybe you organize your clothing according to style, color, season, or other similar characteristics. The way we classify is based on the characteristics we examine. In this activity, you will have the opportunity to design a classification system for a particular set of objectives.

Materials Needed

A set of 15-20 photographs of animals. (You could use leaves, pieces of fabric, buttons, etc.)

Investigation

(Could be used as a group assessment.)

A. Planning your investigation

Make a list of characteristicsthat you could use to classify your animals. Try to list as many characteristics as possible. Make your list based on the actual animal, not on the photograph alone. Use your textbook or other classroom sources if necessary. Think about whether some characteristics would result in dividing the animals into two large groups. Then consider other characteristics that could be used to divide each group into several smaller groups. Decide on the system you wish to use and determine how many classification groups you have in all.

B. Conducting your investigation

Develop a system or set of directions to classify your animals according to the system you established. You may change your system if you feel it is not working well. Make a chart showing the classification groups you used and how many animals are in each group.

C. Communicating you results

Write reasons why you selected certain characteristics rather than others. List and describe any difficulties you had using your system. Tell how your classification system would be useful. Finally compare your system with other in class and list advantages and disadvantages of each. Is there only one "correct" classification system?

Going Further

Design a classification system that you could use for a store you are familiar with that might make it better. It can be a store of your choice, i.e., sporting goods, clothing, department, etc. Tell why the categories you have chosen would better serve the customer and improve sales.

Grading. In this test, the process is more important that the outcome. Use a part 'A', Planning the Investigation, to evaluate the plan. After the investigation is complete, use the other three sections of the outline to evaluate student work.

Evaluation. As part of each groups' grade, they will use the classification system designed by another group, to follow the things they classified, They will also use the attached scoring guide. The scoring guide would be available before the test so all students will know the criteria they will be graded on.

Teacher Notes

1. For consistency, give each group pictures of the same animals. 2. Also make sure the student compare physical structures instead of habitats or behaviors. 3. You may want to make an example key to shopw students how to get started. 4. This activity may take more than one class period.

*******************************************************************************************

Not In My Backyard -- Teacher Notes

Student Objectives

Students will investigate a problem, plan a solution to it, and report these to the class.

Grade Level: Seventh grade or higher

Abilities to be Measured

Students should be able to independently:

Gather data pertinent to a problem

Classify data which they have collected

Analyze data pertinent to a problem

Cooperate with each other assigned students on a task relating waste disposal

Make estimates and approximations

Develop alternative solutions to a problem

Orally communicate their ideas in a clear organized manner

Identify environmental problems associated with waste disposal

Demonstrate in some way, an inderstanding of (1) the importance of each individual's contribution to the quality of life (2) the implications of limited natural resources

Write clear, complete sentences

Be serious about this assignment

Groups of Students should be able to:

Work together to complete an assigned task.

Organize time to complete a project.

Use group time appropriately.

Materials Needed for a Class of Thirty:

30 large garbage bags
three or four scales to weigh garbage

*******************************************************************************************

Background Information and Requisite Abilities

How to operate weight scales

Basic information on how to prepare tables and graphs

The amount of trash produced by the student's town in one year

Vocabulary:

Crisis Landfill Hazard

Typical day Assumption Environmental

Condominiums

Scoring Student Performance

Score student's work and record scores on the "Not In My Backyard Scoring Form" on the next page. Use the key to judge each student's performance. In the students' written work you will find evidence of how well they have done on many of the "abilities." Some evidence will be available through your observations of student's as they work on the tasks assigned. For example, evidence of a student's ability to "develop alternate solutions to a problem" will be found in his or hers response to the problem in Part III. The ability to "write clear, complete sentences" may be found in the student's written answers to Part i questions and the Part iii problem.

As you assign students the tasks of this performance exercise, be sure to help them understand both the abilities to be measured and how you will make judgments about each student's abilities.

Remember to mark the rating form as students work on those abilities that require direct observation of them at work. Do not leave these judgments unilt later.

*******************************************************************************************

"Not in My Backyard!"
Scoring Form: 0 = needs improvement, 1 = good, 4 = excellent
Student Name: ________________________________________________________

Gather Data
Classify Data
Analyze Data
Cooperation
Estimates and Approximations
Develop Alternative Solutions
Communicate Ideas
Environmental Problems
Write Clear, Complete Sentences
Be Serious About Task
Work Together
Organize Time
Use Time Wisely

*******************************************************************************************

NOT IN MY BACKYARD -- Part I

In America, every adult and child produces about 3.5 pounds (1500 grams) of trash per day. The Environmental Protection Agency (EPA) estimates that this amount will increase to 4 pounds by the year 2000.
Getting rid of this waste is a problem and will soon be a crises because there is less room available to properly dispose of the trash. Many landfills, the place where garbage is taken, are closing because they are dangerous or hazardous to our health and our environment.

In this activity you will investigate a problem, come up with solutions to the problem and make an advertisement to increase peoples' awareness of the waste disposal problem.

Complete the Following on Your Own

Collect and carry all of your trash or garbage (don't count something trash" if you usually recycle it) in a 30-gallon trash bag. Do this for ONE day. Begin today! This bag goes everywhere you go in twenty-hours. Please take this assignment seriously.

Example: If tonight you help fix or clean up dinner, the potato peelings, empty milk carton, left over beans will go in your bag. If you clean the house and throw away the newspaper or an old shirt; or if you weed the garden and usually throw the weeds in the garbage, put all of these in your bag.

Safety Consideration

Do not collect broken or sharp materials or materials that may be harmful like paint thinner, ammonia and other chemicals. Personal hygiene products and other items that you feel may cause safety problems should be carefully considered.Any food containers should be rinced out before you place them in your bags.

Tomorrow bring the trash you have collected to class with you.

After you have brought your trash to class you may answer the following questions on a piece of paper. Be sure to keep the paper because you will need to turn it in with your final report. Take about 15 minutes to complete the questions. (Keep your answers to be turned in later.)

Make a list of everything you collected.

Is the amount of trash you collected a typical, or a regular, day of trash for you?
How is your trash different from what it might be on other days?

Do you think the amount of trash is similar to other peoples' trash such as:

the members of your class?

the members of your family?

the members of your community?

the people in other countries?

*******************************************************************************************

NOT IN MY BACKYARD -- Part II COMPLETE THE FOLLOWING IN YOUR GROUP

First:

Organize the trash that was collected by your group in different types or kinds
You need to decide as a group how you will do this.

Next:

Weigh each type of trash and classify this information.

Now:

Throw away the trash in an appropriate manner (Be sure to ask if you are not sure what is appropriate).

Then:

Write answers to the following questions on a separate paper.
Use the data you collected:

How much of each type of trash would your group produce in one year?
In your lifetime?

How much trash do you think your city or town produces in one year?

Explain how you came up with these answers and what influenced you decisions.

What environmental problems are created by the types of waste your group collected?

Think of and list some solutions to the problems you listed in question 4.

Now:

In a one page report, describe what you would do in your daily activities to carry out these solutions. Explain how practical, how hard, or how easy they would be to do. (Keep answers to the above questions and your report to be turned in later.)

Next:

Do one of the following:

Develop and pass out a questionnaire or a poll to find out what members of your school or community know about the waste disposal problem. Find out how they feel about this problem.

Develop and conduct a study to find out how much trash is produced in, for example, your school lunch room, or in a fast food restaurant, or how much paper is thrown away at a business office or bank.

Finally:

Prepare a poster that uses the data and information you collected from your studies to increase the awareness of the waste disposal problem in your community. You will present your poster, ideas, and solutions to the problem to the class. Plan in an oral report, what you each will say and how you are going to give your report.

NOTE:

You may believe that I have not given you enough information or complete direction to do this assignment. This is done on purpose. Work with your group members. Do your best work.

*******************************************************************************************

NOT IN MY BACKYARD -- Part III

FINISHING BY YOURSELF

Your community is considering building condominiums near an abandoned landfill. You are being asked as an expert on waste disposal to make a recommendation to the local planning and zoning board of your city, if they should or should not go ahead with plans to build on the site. Write what you would recommend to be done. Include data and not just your opinion. Include in your report what environmental problems could result from the landfill and what could be done by the community to solve these problems.

Now organize all of your material and turn it in. You should submit:

Questions on the trash you collected (Part I)

Data and information on the trash (Part II)

One page report on environmental problems and solutions (Part II(

Poster

Presentation to the class

Final report / Recommendation (Part III)

*******************************************************************************************

Comparing the Three Examples

You should have noted many of the following similarities in the four examples:

They test both individual and cooperative abilities
They require students to integrate knowledge in several subjects
In addition to recall they require higher level thinking.
They cannot be scored objectively, but instead they require scorers to observe and use a scoring guide. You should have noticed that the two performance tests are less complex. They test only one major skill; they can usually be done in one class period; they are also simpler to score.
etc.

Developing a Performance Test

There are five dimensions which you should consider as you prepare a more authentic measure:

Length
only be assessed over time, such as the ability to plan and carry out a project, or solve a complex problem, or show persistence.

The multiple choice question is the ultimate short task. They typically take about a minute to answer and therefore can only assess very focused abilities such recall or the kind of focused judgment involved in discriminating among alternatives. But a performance test takes up to one class period. A performance task requires several days. Long tasks can get at more than short tasks -- for example, time management, detailed planning, use of a variety of out-of-class resources, etc.
Amount of Structure Provided
You cannot talk about a structured versus an unstructured performance task because some tasks are highly structured in some ways and unstructured in other ways. In order to discuss structure in performance tests or tasks, it is necessary to look at four separate components of the task, (Problem definition, scaffolding, alternate strategies, alternate solutions). For example, the waste disposal task --

Problem definition -- Highly structure

Scaffolding -- Moderate to highly structured

Alternate strategies -- Moderate structure

Alternate solutions -- Low to moderate structure

Another good example of how the same task can have elements of high and low structure is the following. Assume that students have been asked simply to build a scale model of the solar system. AS far as problem definition goes, this test has fairly high structure -- you have a specific goal to meet. However, there is very little scaffolding -- students are not told what materials to use, or what proportions to use, or where to get information on the planetary distances and orbits. There are a lot of alternate pathways to the solution -- consider the fact that no two models will look exactly alike, and will vary in terms of materials used, scale employed, special features included (such as moons, orbital speeds, etc.). And in a way there's one best solution, a perfectly scaled model of the solar system.

Structure is a critical dimension deserving of a great deal of research and study on our part. We do know that the ability to respond effectively to situations of low structure is a major dimension of individual student differences, and we do know that multiple choice questions do not address this dimension.

A challenge ahead of users of performance tasks is getting kids used to less structure, given that typical classroom assignments tend to be highly structured. This may take some gradual introduction to problems with less and less structure.
Task Participation (Individual, Group or Mixed)
An important issue raised by more authentic assessment deals with students working in groups, both to take advantage of the principles of cooperative learning and to assess group interaction skills. For this reason, some performance tests and all performance tasks involve group activities. The tasks usually involve students working in both group and individual work.
Evaluation Focus (Process or Product)
The advantage of products, from an assessment point of view, is that they're concrete. You can see them, feel them, reexamine them at your leisure and reflect on how you're going to evaluate them, and use them as a point of reference when you explain your evaluation to the student.

BUT many educators value process far more than product, the thinking and decision-making process more than the outcome. It will take creativity to assess process reliably, but for some traits this is an important goal.

Example of above: group interaction skills are pure process, they don't produce a concrete product.
Performance Modes
One feature that makes performance testing so rich in its potential is the variety of communicative modes that students might use (as opposed to the "blackened bubble" mode of the standardized test). Responses can be essays or other forms of narrative; charts or graphs; scale models; oral reports; dramatizations; portfolios or logs; the demonstration of effective group skills such as leadership or consideration; etc.

Guidelines for Reviewing A Performance Assessment

Essential & Valid. The measure is connected to core standards and objectives in the curriculum
1. Tests one or more objectives
2. Tests one or more science process skills
3. Tests one or more intended learning outcomes
Authentic. The problem and the process simulate, mimic, or parallel activities in the larger world, e.g., problems are authentic.
Integrative. The problem requires the student to put together a variety of identifiable skills, knowledge, and concepts and demonstrate attitudes and habits of mind.
Rich. As student works through the problem, questions will continue to emerge e.g., the measure is open-ended.
Engaging. The problem is engaging for the student, e.g., it promotes persistence.
Freeing. The measure encourages students to be divergent thinkers and thoughtful risk takers.
Feasible. Activities are safe and feasible.
Equitable. The measure allows for a variety of learning styles.
Group Work. The use of group work stimulates individual thinking
Individual Accountability. Although group work is used, the individual's performance is easily observable.
Structure. There is an appropriate amount of definition of the problem. There is an appropriate amount of definition of the procedure. Directions are clear.
Feedback & Intervention Loop. The experience for the student allows for feedback and revision cycles.
Choice. The student has some choice over the format for the final product and/or presentation.
Criteria for Quality. The criteria which will be used to judge the quality of the process and product are made clear to the student from the beginning of the activity. Scoring guide is easy to use.

The Performance Test Development Process.

Following the steps listed here should help you develop an assessment of your own.

Select the objectives of the measure.
What will this test tell you about students? What knowledge/attitudes/attributes will the students have to display in order to successfully handle this task.

Write a complete statement, including purpose, and suggestions on strategy and focus. Keep your original objectives in mind throughout! Focus the test in the direction of these objectives.
Continue with an idea
From a textbook or another book
From another test
From a newspaper or magazine article
From a life experience
From a conversation with colleagues or others
Random thought
Divine inspiration
Test the idea
Is it important? Does it center on an important concept or issue in the subject taught?

Is it contextualized? Does it tie the concept or issue to real life? Does it lead students to deal with the concept or issue instead of just memorizing it? Does it make students use it, understand it, explain it to others, or otherwise take some ownership of it?
Consider embellishments...
Can the task be structured to elicit attitudes and attributes which can be measured (e.g., group cooperation, persistence, resourcefulness, etc.)?

Can the task be structured to include a group activity?
Consider what a teacher will need to know to administer the test
Where does this test fit into the curriculum? What needs to be taught before this test is administered?

What materials and equipment are needed?

What problems or difficulties are likely to occur?
What kinds of assistance or intervention should the teacher be prepared to provide?
What kinds of assistance should the teacher not provide? How should such interventions be treated in scoring?

Develop Notes to the Teacher to include all the above.
Design a scoring approach to the problem. (Review "the development of scoring guides" in the lesson.)
Consider your original objectives -- how will they show themselves in students' responses?

Decide wither you are assessing processes or products.

Identify either dimensions of performance or aspects of the product which a) reflect the objectives you had for the test; and b) can be observed and rated with reasonable objectivity.

Weigh the dimensions in proportion to their importance, using your own judgment and that of colleagues.

Develop levels of performance which you feel are likely to be present in student performance or products.

Build a section within the "directions" to communicate to students how their performance will be evaluated.
Try the test out!
Have one or more colleagues review it "cold" and critique it.

Administer it in the classroom in a relatively "low-stakes" setting:

Get feedback from you students -- what was good or bad about the task? What would improve it?

Try out the scoring system. What was good about it? What was bad about it? How can it be enriched by the examples of performance you have now collected?
Revise based on your experience.