CA2284529A1 - System and method for evaluating raters of scoring constructed responses to test questions - Google Patents

System and method for evaluating raters of scoring constructed responses to test questions Download PDF

Info

Publication number
CA2284529A1
CA2284529A1 CA002284529A CA2284529A CA2284529A1 CA 2284529 A1 CA2284529 A1 CA 2284529A1 CA 002284529 A CA002284529 A CA 002284529A CA 2284529 A CA2284529 A CA 2284529A CA 2284529 A1 CA2284529 A1 CA 2284529A1
Authority
CA
Canada
Prior art keywords
constructed responses
rater
raters
stations
training
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CA002284529A
Other languages
French (fr)
Other versions
CA2284529C (en
Inventor
Frank J. Romano
Mark Grant
Marisa D. Farnum
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Educational Testing Service
Original Assignee
Educational Testing Service
Frank J. Romano
Mark Grant
Marisa D. Farnum
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Educational Testing Service, Frank J. Romano, Mark Grant, Marisa D. Farnum filed Critical Educational Testing Service
Publication of CA2284529A1 publication Critical patent/CA2284529A1/en
Application granted granted Critical
Publication of CA2284529C publication Critical patent/CA2284529C/en
Anticipated expiration legal-status Critical
Expired - Fee Related legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B7/00—Electrically-operated teaching apparatus or devices working with questions and answers
    • G09B7/02—Electrically-operated teaching apparatus or devices working with questions and answers of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student

Abstract

System and method for presentation to raters of constructed responses to test questions in electronic workfolders. The system and method provide selection and designation of constructed responses into categories for use in quality control functions and training applications. A storage device is used to relationally store data regarding raters, the constructed responses, and the scores awarded for use in selecting constructed responses for use in quality control and training. The system also comprises rater stations (40a, 40b, ... 40n) and processing system (14) for selectively transmitting constructed responses to the rater stations (40a, 40b, ... 40n) and receiving scores awarded by the raters assigned to the rater stations (40a, 40b, ... 40n) during both training and actual production scoring. The system further comprises rater stations (40a, 40b, ... 40n) and processing system (14) for displaying rater performance data, topic performance data, constructed responses volume data, and candidate sample volume data.
CA002284529A 1997-03-21 1998-03-20 System and method for evaluating raters of scoring constructed responses to test questions Expired - Fee Related CA2284529C (en)

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
US4431997P 1997-03-21 1997-03-21
US60/044,319 1997-03-21
US6178197P 1997-10-10 1997-10-10
US60/061,781 1997-10-10
PCT/US1998/005803 WO1998043222A1 (en) 1997-03-21 1998-03-20 System and method for evaluating raters of scoring constructed responses to test questions

Publications (2)

Publication Number Publication Date
CA2284529A1 true CA2284529A1 (en) 1998-10-01
CA2284529C CA2284529C (en) 2005-11-15

Family

ID=26721398

Family Applications (1)

Application Number Title Priority Date Filing Date
CA002284529A Expired - Fee Related CA2284529C (en) 1997-03-21 1998-03-20 System and method for evaluating raters of scoring constructed responses to test questions

Country Status (3)

Country Link
CA (1) CA2284529C (en)
GB (1) GB2338334B (en)
WO (1) WO1998043222A1 (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
AU783979B2 (en) * 1999-10-27 2006-01-12 Eclipse Computing (Australia) Pty Ltd A data collection method
AUPQ368799A0 (en) * 1999-10-27 1999-11-18 Arc Research & Development Pty Limited Assessment and reporting system
JP2008251006A (en) * 2007-03-28 2008-10-16 Nv Organon Method for accurately assessing disease severity in clinical trials concerning psychopathology

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4895518A (en) * 1987-11-02 1990-01-23 The University Of Michigan Computerized diagnostic reasoning evaluation system
US4978305A (en) * 1989-06-06 1990-12-18 Educational Testing Service Free response test grading method
US5059127A (en) * 1989-10-26 1991-10-22 Educational Testing Service Computerized mastery testing system, a computer administered variable length sequential testing system for making pass/fail decisions
US5204813A (en) * 1990-06-08 1993-04-20 Assessment Systems, Inc. Computer-controlled testing process and device for administering an examination
US5524193A (en) * 1991-10-15 1996-06-04 And Communications Interactive multimedia annotation method and apparatus
US5565316A (en) * 1992-10-09 1996-10-15 Educational Testing Service System and method for computer based testing
US5509127A (en) * 1992-12-04 1996-04-16 Unisys Corporation Transmission logic apparatus for dual bus network
US5433615A (en) * 1993-02-05 1995-07-18 National Computer Systems, Inc. Categorized test item reporting system
US5437554A (en) * 1993-02-05 1995-08-01 National Computer Systems, Inc. System for providing performance feedback to test resolvers
WO1995010095A2 (en) * 1993-09-30 1995-04-13 Educational Testing Service A centralized system and method for administering computer based tests

Also Published As

Publication number Publication date
GB2338334A (en) 1999-12-15
WO1998043222A1 (en) 1998-10-01
CA2284529C (en) 2005-11-15
GB9922049D0 (en) 1999-11-17
GB2338334B (en) 2001-03-07

Similar Documents

Publication Publication Date Title
CA2398095A1 (en) Scoring method and scoring system
Shohamy The validity of direct versus semi-direct oral tests
CA2107413A1 (en) Categorized test item reporting system and method
Shepard Setting Performance Standards for Student Achievement. A Report of the National Academy of Education Panel on the Evaluation of the NAEP Trial State Assessment: An Evaluation of the 1992 Achievement Levels.
CA2310616A1 (en) Method and apparatus for preserving human saliva for testing
EP1306663A3 (en) A method of operating an automatic sample testing machine
CA2158616A1 (en) Method and Apparatus for Network Analysis
WO2004038542A3 (en) Method for automated mass screening for visual dysfunction in children
CA2294911A1 (en) Method for assessing disposable absorbent articles
CA2347742A1 (en) Generating a nonlinear model and generating drive signals for simulation testing using the same
MXPA99005828A (en) Positive fertility testing and reproductive health system.
CN107194836A (en) A kind of Multifunctional English teaching management device and its teaching method
CN109035890B (en) Remote demonstration teaching method of mobile network
CN107832742A (en) Measure of supervision and robot applied to robot
CA2402842A1 (en) Wireless electronic libretto display apparatus and method
CA2284529A1 (en) System and method for evaluating raters of scoring constructed responses to test questions
EP0831330A3 (en) Method of automatic analysis of samples
Gardner Communication games: Do we know what we're talking about?
US20030203346A1 (en) Method for interactive testing
US20020138269A1 (en) Voice recognition maintenance inspection program
WO2004093303A3 (en) Human capital management assessment tool system and method
WO1998059285A3 (en) System and method for simulation, modeling and scheduling of biopharmaceutical batch process operations
CN108399804A (en) A kind of intelligence music learning training device
AU6415394A (en) Test kits and methods for rapidly testing for contamination by microorganisms
Glenn Teacher Empowerment.

Legal Events

Date Code Title Description
EEER Examination request
MKLA Lapsed

Effective date: 20170320