GB2338334A - System and method for evaluating raters of scoring constructed responses to test questions - Google Patents

System and method for evaluating raters of scoring constructed responses to test questions

Info

Publication number
GB2338334A
GB2338334A GB9922049A GB9922049A GB2338334A GB 2338334 A GB2338334 A GB 2338334A GB 9922049 A GB9922049 A GB 9922049A GB 9922049 A GB9922049 A GB 9922049A GB 2338334 A GB2338334 A GB 2338334A
Authority
GB
United Kingdom
Prior art keywords
constructed responses
rater
raters
stations
training
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
GB9922049A
Other versions
GB2338334B (en
GB9922049D0 (en
Inventor
Frank J Romano
Mark Grant
Marisa D Farnum
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Educational Testing Service
Original Assignee
Educational Testing Service
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Educational Testing Service filed Critical Educational Testing Service
Publication of GB9922049D0 publication Critical patent/GB9922049D0/en
Publication of GB2338334A publication Critical patent/GB2338334A/en
Application granted granted Critical
Publication of GB2338334B publication Critical patent/GB2338334B/en
Anticipated expiration legal-status Critical
Expired - Fee Related legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B7/00—Electrically-operated teaching apparatus or devices working with questions and answers
    • G09B7/02—Electrically-operated teaching apparatus or devices working with questions and answers of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student

Abstract

System and method for presentation to raters of constructed responses to test questions in electronic workfolders. The system and method provide selection and designation of constructed responses into categories for use in quality control functions and training applications. A storage device is used to relationally store data regarding raters, the constructed responses, and the scores awarded for use in selecting constructed responses for use in quality control and training. The system also comprises rater stations (40a, 40b, ... 40n) and processing system (14) for selectively transmitting constructed responses to the rater stations (40a, 40b, ... 40n) and receiving scores awarded by the raters assigned to the rater stations (40a, 40b, ... 40n) during both training and actual production scoring. The system further comprises rater stations (40a, 40b, ... 40n) and processing system (14) for displaying rater performance data, topic performance data, constructed responses volume data, and candidate sample volume data.
GB9922049A 1997-03-21 1998-03-20 System and method for evaluating raters of scoring constructed responses to test questions Expired - Fee Related GB2338334B (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US4431997P 1997-03-21 1997-03-21
US6178197P 1997-10-10 1997-10-10
PCT/US1998/005803 WO1998043222A1 (en) 1997-03-21 1998-03-20 System and method for evaluating raters of scoring constructed responses to test questions

Publications (3)

Publication Number Publication Date
GB9922049D0 GB9922049D0 (en) 1999-11-17
GB2338334A true GB2338334A (en) 1999-12-15
GB2338334B GB2338334B (en) 2001-03-07

Family

ID=26721398

Family Applications (1)

Application Number Title Priority Date Filing Date
GB9922049A Expired - Fee Related GB2338334B (en) 1997-03-21 1998-03-20 System and method for evaluating raters of scoring constructed responses to test questions

Country Status (3)

Country Link
CA (1) CA2284529C (en)
GB (1) GB2338334B (en)
WO (1) WO1998043222A1 (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
AU783979B2 (en) * 1999-10-27 2006-01-12 Eclipse Computing (Australia) Pty Ltd A data collection method
AUPQ368799A0 (en) * 1999-10-27 1999-11-18 Arc Research & Development Pty Limited Assessment and reporting system
JP2008251006A (en) * 2007-03-28 2008-10-16 Nv Organon Method for accurately assessing disease severity in clinical trials concerning psychopathology

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4895518A (en) * 1987-11-02 1990-01-23 The University Of Michigan Computerized diagnostic reasoning evaluation system
US4978305A (en) * 1989-06-06 1990-12-18 Educational Testing Service Free response test grading method
US5204813A (en) * 1990-06-08 1993-04-20 Assessment Systems, Inc. Computer-controlled testing process and device for administering an examination
US5433615A (en) * 1993-02-05 1995-07-18 National Computer Systems, Inc. Categorized test item reporting system
US5437554A (en) * 1993-02-05 1995-08-01 National Computer Systems, Inc. System for providing performance feedback to test resolvers
US5509127A (en) * 1992-12-04 1996-04-16 Unisys Corporation Transmission logic apparatus for dual bus network
US5513994A (en) * 1993-09-30 1996-05-07 Educational Testing Service Centralized system and method for administering computer based tests
US5524193A (en) * 1991-10-15 1996-06-04 And Communications Interactive multimedia annotation method and apparatus
US5565316A (en) * 1992-10-09 1996-10-15 Educational Testing Service System and method for computer based testing

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5059127A (en) * 1989-10-26 1991-10-22 Educational Testing Service Computerized mastery testing system, a computer administered variable length sequential testing system for making pass/fail decisions

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4895518A (en) * 1987-11-02 1990-01-23 The University Of Michigan Computerized diagnostic reasoning evaluation system
US4978305A (en) * 1989-06-06 1990-12-18 Educational Testing Service Free response test grading method
US5204813A (en) * 1990-06-08 1993-04-20 Assessment Systems, Inc. Computer-controlled testing process and device for administering an examination
US5524193A (en) * 1991-10-15 1996-06-04 And Communications Interactive multimedia annotation method and apparatus
US5565316A (en) * 1992-10-09 1996-10-15 Educational Testing Service System and method for computer based testing
US5509127A (en) * 1992-12-04 1996-04-16 Unisys Corporation Transmission logic apparatus for dual bus network
US5433615A (en) * 1993-02-05 1995-07-18 National Computer Systems, Inc. Categorized test item reporting system
US5437554A (en) * 1993-02-05 1995-08-01 National Computer Systems, Inc. System for providing performance feedback to test resolvers
US5513994A (en) * 1993-09-30 1996-05-07 Educational Testing Service Centralized system and method for administering computer based tests

Also Published As

Publication number Publication date
GB2338334B (en) 2001-03-07
CA2284529A1 (en) 1998-10-01
CA2284529C (en) 2005-11-15
GB9922049D0 (en) 1999-11-17
WO1998043222A1 (en) 1998-10-01

Similar Documents

Publication Publication Date Title
Duker Listening and reading
EP0176854A3 (en) Method and apparatus for improved monitoring and detection of improper device operation
CA2107413A1 (en) Categorized test item reporting system and method
CA2398095A1 (en) Scoring method and scoring system
Hartmann Four perspectives on dictionary use: a critical review of research methods
WO2002052373A3 (en) Collecting user responses over a network
WO2001024058A3 (en) Automated internet research tool
CA2220388A1 (en) Method and apparatus for assessing slide and specimen preparation quality
Olivares A conceptual and analytic critique of student ratings of teachers in the USA with implications for teacher effectiveness and student learning
CN107832742A (en) Measure of supervision and robot applied to robot
GB2338334A (en) System and method for evaluating raters of scoring constructed responses to test questions
WO2003049063A1 (en) Test result analysis apparatus, method, and program, and learning support system and method
Lundberg Integrating on‐line technology into counseling curricula: Emerging humanistic factors
Gardner Communication games: Do we know what we're talking about?
Lee Spiegel et al. Teacher responses to miscues during oral reading by second-grade students
Luera et al. Linking math, science, and inquiry-based learning: An example from a mini-unit on volume
US20030203346A1 (en) Method for interactive testing
JPS6447948A (en) Automatic cell sorter
JP2003066823A (en) Communication correction system, server, terminal device, and recording medium
WO1998059285A3 (en) System and method for simulation, modeling and scheduling of biopharmaceutical batch process operations
CN109035891B (en) Remote demonstration teaching system of mobile network
Devine Listening and Reading.
ITTO920578A1 (en) AUTOMATIC TRAINING AND LEARNING CONTROL SYSTEM THROUGH REAL TIME ANALYSIS OF REPLIES TO QUESTIONNAIRES.
EP0896277A3 (en) Image tester for game machine
JP2002169457A (en) Learning system, terminal equipment and learning method

Legal Events

Date Code Title Description
PCNP Patent ceased through non-payment of renewal fee

Effective date: 20160320