CN102509483B - A kind of distributed Auto-Evaluation System of SET and method thereof - Google Patents
A kind of distributed Auto-Evaluation System of SET and method thereof Download PDFInfo
- Publication number
- CN102509483B CN102509483B CN201110336179.4A CN201110336179A CN102509483B CN 102509483 B CN102509483 B CN 102509483B CN 201110336179 A CN201110336179 A CN 201110336179A CN 102509483 B CN102509483 B CN 102509483B
- Authority
- CN
- China
- Prior art keywords
- scoring
- task
- module
- examinee
- score calculation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Electrically Operated Instructional Devices (AREA)
Abstract
A kind of distributed Auto-Evaluation System of SET, comprise a scoring management devices and manage some examination client devices, scoring management devices communicates with at least one scoring task scheduling apparatus, a scoring task scheduling apparatus manages some score calculation devices, module wherein for automatic scoring comprises: recording module, for detecting and recording examinee's voice; Speech processing module, for processing examinee's voice signal, extracts acoustic feature, identifies and obtains the time boundary of acoustic elements by the text that voice are corresponding; Pronunciation evaluation module, for the result exported according to topic information and speech processing module, carries out assessment and analysis to the different aspect of user pronunciation; Flow characteristic extracting module, for extracting the flow comprehensive characteristics of reflection examinee spoken language proficiency, comprises the feature of acoustics, syntax and semantics; Scoring submodule: utilize flow comprehensive characteristics, comprehensive grading is carried out to user speech; Wherein except recording module, other grading module can be carried out dynamic dispatching according to the computational load of the score calculation device on examination client device and backstage and offered load and dispose.
Description
Technical field
This patent relates to computer speech signal processing and analyzing, language assessment, pattern-recognition and Computer Assisted Instruction (CAI) field, particularly computer assisted distributed extensive SET automatic scoring method and system field.
Background technology
Tradition examination can be divided into two large classes, and a class is written exam; One class is SET (the conversational language ability of high spot review student).Along with deepening continuously of Quality Education, oracy is more and more taken seriously, and correspondingly, oral English teaching and examination also constantly universal, such as, all start to test oracy in national examination and provincial examination.In addition, in various professional grade examination, oracy also becomes test emphasis, such as PSC, College English grade examination, IELTS, the TOEFL etc.
Different with written exam, SET has two kinds of modes usually: one is that examinee and examiner carry out face-to-face exchange; Another kind is that the spoken voice recording examinee in special context is analyzed to examiner.The scoring of SET also has two kinds of modes, and one is that examiner provides scoring according to interchange experience scene; Another kind is by listening cls analysis to mark to examinee's voice.Visible, no matter which kind of mode, SET all needs the man power and material of at substantial, especially in Large-scale Examinations.On the other hand, in Large-scale Examinations, usually require in finite time, complete all scorings, need to improve scoring efficiency.In addition, SET belongs to subjective item, and its scoring workload is huge, and subjectivity is comparatively large, and consistance is poor.Therefore, in extensive SET, need a kind ofly high consistency not only can be provided to mark but also there is examination system and the method for high-efficiency and low-cost.
Computer technology has been applied to examination of education field, can reduce examination cost, improves examination efficiency.Relatively more typical application comprises: general Examination management system [CN101964152A, CN1801244A, CN1732490B, CN101827124A], for the Examination system [CN101419795B, the design and implimentation (Chinese University of Science and Technology's Master's thesis) of Oral English Exam system] of spoken language, network grading points-scoring system [CN1700200A, CN1598845A, CN1987889A] etc.But these work all do not relate to organization and administration and the automatic scoring problem of extensive SET.
For the subjectivity problem of current SET expert analysis mode, the present invention utilizes computing machine and speech analysis techniques, examinee's voice is carried out to the automatic scoring of high consistency.For the huge problem of extensive SET scoring task workload, the present invention utilizes computer network and calculation server to carry out distributed parallel scoring to extensive examinee's voice.
Summary of the invention
The present invention discloses distributed Auto-Evaluation System and the methods of marking thereof of a set of SET, solve in current SET the problem of the low and consistance difference of efficiency of marking, especially for extensive SET provides the scoring solution of a set of reliable, efficient and low cost.
The invention provides a kind of distributed Auto-Evaluation System of SET, comprise a scoring management devices and manage some examination client devices, scoring management devices communicates with at least one scoring task scheduling apparatus, a scoring task scheduling apparatus manages some score calculation devices, and the module wherein for automatic scoring comprises:
Recording module, for detecting and recording examinee's voice;
Speech processing module, for processing examinee's voice signal, extracts acoustic feature, identifies and obtains the time boundary of acoustic elements by the text that voice are corresponding;
Pronunciation evaluation module, for the result exported according to topic information and speech processing module, carries out assessment and analysis to the different aspect of user pronunciation;
Flow characteristic extracting module, for extracting the flow comprehensive characteristics of reflection examinee spoken language proficiency, comprises the feature of acoustics, syntax and semantics;
Comprehensive grading module: utilize flow comprehensive characteristics, carries out comprehensive grading to user speech; Wherein
Except recording module, other grading module can be carried out dynamic dispatching according to the computational load of the score calculation device on examination client device and backstage and offered load and dispose.
Additionally provide a kind of distributed automatic scoring method of SET, the concrete steps of wherein marking comprise:
Step 1, sound pick-up outfit detect examinee's voice and record;
Step 2, extract phonetic feature, utilize acoustic model and language model, voice are identified, obtain corresponding text and phoneme unit time boundary thereof and likelihood score score;
Step 3, utilize phoneme unit time boundary and likelihood score score and exercise question relevant knowledge, analysis and evaluation is carried out to voice, obtains the pronunciation accuracy of different phonetic unit, rhythm accuracy, pronunciation integrity degree and pronunciation fluency etc.;
Step 4, integrated voice result, assessment result and topic information, extract the acoustics of reflection examinee spoken language proficiency, the flow comprehensive characteristics of syntax and semantics aspect;
Step 5, based on the model of training in advance and design rule, flow comprehensive characteristics is used to carry out comprehensive grading to examinee's spoken language proficiency.
Further provide a kind of distributed automatic scoring method of SET, specifically comprise the steps:
Step 1, examinee obtain exam question and answer on request, examination client device records examinee's voice, according to topic information and requirement, first stage score calculation is carried out to examinee's voice, according to data exchange standard, examinee's voice and appraisal result are sent to scoring management devices;
Step 2, scoring management devices receive examinee's answer and the first stage appraisal result of the transmission of examination client equipment, resolve appraisal result, if appraisal result is not final scoring, then according to data exchange standard, examinee's voice and scoring intermediate result and topic information are organized into scoring task, are sent to scoring task scheduling apparatus after encryption and mark further;
Step 3, scoring task scheduling apparatus receive the scoring task that scoring management devices sends, resolve scoring task and obtain score data and task priority, required grading module and algorithm is selected according to arranging of grading module territory, generate score calculation program, the score calculation device being submitted to backstage according to task priority calculates;
Step 4, score calculation device receive the scoring procedures that task scheduling apparatus sends, and perform subordinate phase scoring, final appraisal result is sent to scoring task scheduling apparatus, and task scheduling apparatus of then marking is forwarded to scoring management devices appraisal result again.
Accompanying drawing illustrates:
The institutional framework schematic diagram of the distributed Auto-Evaluation System of Fig. 1 SET.
The distributed Auto-Evaluation System of Fig. 2 SET and the scoring schematic flow sheet of method.
The workflow schematic diagram of the distributed Auto-Evaluation System of Fig. 3 SET.
The deployment schematic diagram of the distributed automatic scoring module of a kind of typical SET of Fig. 4.
Fig. 5 accesses the institutional framework schematic diagram of the distributed Auto-Evaluation System of the SET of voice cloud device.
The structural representation of the distributed Auto-Evaluation System environment of the SET that Fig. 6 simplifies.
Embodiment
Below in conjunction with the drawings and specific embodiments, the present invention is described in further detail:
Fig. 1, gives the institutional framework schematic diagram of system disclosed in the present invention.A kind of distributed Auto-Evaluation System of SET is made up of four bigger devices: examination client device 101, scoring management devices 102, scoring task scheduling apparatus 103 and score calculation device 104.
Examination client equipment 101 is terminal devices that examinee provides examination service, is generally computing machine or embedded mobile device etc.At least there is following hardware configuration: audio playing unit, audio recording unit, computing unit, network communication unit.
Alternatively, graphic user interface and touch control operation is supported.
It shows examination paper content by graphical interfaces to examinee, examination paper audio frequency is play by audio playing unit (as: headphone or loudspeaker), examinee's voice are recorded by audio recording unit (as: microphone), by local computing unit check on one's answers (i.e. examinee's voice) carry out pre-service and certain score calculation (referred to as first stage scoring), send examinee's answer and appraisal result (normally appraisal result) to scoring management devices by network or other modes.And examinee's voice can be kept in, carry out off-line asynchronous transmission.
A special examination client program or one can also be configured there is the browser of examination plug-in unit to complete above-mentioned recording and scoring work on examination client equipment.
Scoring management equipment 102 obtains examinee's answer and first stage appraisal result, tissue submit the computing machine of scoring task to.
It obtains from examinee's answer of examination client equipment and first stage appraisal result, utilizes this appraisal result and topic information structure scoring task, is sent to scoring task scheduling apparatus, and receives the final appraisal result from scoring task scheduling apparatus.
There is reliable mass-memory unit, general configurable high data transmission broadband.Preserve examinee's answer and final appraisal result, and the temporary next middle appraisal result to first stage scoring.
Scoring task scheduling equipment 103 is for receiving scoring task, parsing construct score calculation task, submit the computing machine of calculation task and management score calculation device to.
It receives the scoring task that scoring management equipment sends, parsing task, extract score data, automatically score calculation program is organized according to information such as topic information and the required modules of scoring, generate score calculation task, submit to calculation task to score calculation device by dispatching algorithm, finally obtain final appraisal result from computing equipment.
Especially, described scoring task scheduling apparatus has reliable mass-memory unit, and generally can have high data transmission broadband.
Especially, described scoring task scheduling apparatus retains one section of memory headroom, safeguards the calculation task queue of a prioritization.
Especially, the task scheduling algorithm of described scoring task scheduling apparatus is: if task queue is discontented with, then task is inserted queue relevant position according to its priority; Task priority is higher than some task priority in queue else if, then task minimum for priority is moved out to buffer area or external memory storage, and this task is inserted queue relevant position; Otherwise this task is stored into buffer area or external memory storage; The queue of timing query task, if having living space, then adds queue buffer area task by its priority; The each score calculation equipment of automatic regular polling, if there is CPU idle, then task of getting from task queue carries out computing to this CPU.
The computing cluster that score calculation equipment 104 is made up of some high-performance CPU, all CPU can concurrent working.It performs the score calculation program that scoring task scheduling apparatus is submitted to, through calculating appraisal result and exporting to scoring task scheduling apparatus.
Its feature of distributed Auto-Evaluation System of SET is that described scoring task scheduling apparatus can form one multiple scoring taskings together can the calculation task of batch processing, submits to the score calculation equipment on backstage.
Its feature of distributed Auto-Evaluation System of SET is that described examination client device can dispose multiple grading module.In the impregnable situation of guarantee examination Recording Process, utilize local cpu to complete the scoring work of first stage.The scoring work of described first stage can be carried out with examination simultaneously, for avoiding having any impact to examination, and also can free time timing operation outside examination.
Its feature of distributed Auto-Evaluation System of SET is that described scoring task scheduling apparatus can be connected by wide area network with scoring management devices, also can be connected by LAN (Local Area Network).
Its feature of distributed Auto-Evaluation System of SET is that the score calculation device on described backstage can be common computer, also can high performance computing service device.These score calculation devices are connected by LAN, form a computing cluster.
The structure of the distributed Auto-Evaluation System of described SET is: a scoring management devices manages some examination client devices, scoring management devices communicates with at least one described scoring task scheduling apparatus, and a scoring task scheduling apparatus manages the score calculation device on some backstages.
The network structure of the distributed Auto-Evaluation System of described SET is: by LAN (Local Area Network) or dedicated Internet access between scoring management devices and examination client device; Connected by LAN (Local Area Network) between scoring task scheduling apparatus and the score calculation device on backstage; The data transfer mode that data communication between scoring management devices and scoring task scheduling apparatus had both also been maintained secrecy by other by any refined net is carried out.
Specifically, mark between management devices and examination client device and between scoring management devices and scoring task scheduling apparatus and also can carry out data transmission by movable storage device.
Fig. 2 gives the distributed Auto-Evaluation System of SET and the grading module of method and flow process.The distributed Auto-Evaluation System of SET and the nucleus module of method comprise: recording module 201, speech processing module 202, pronunciation evaluation module 203, flow characteristic extracting module 204 and comprehensive grading module 205.
Recording module 201: detect and record examinee's voice.
Described recording module utilizes sound end detecting method automatically to detect examinee's voice, or starts recording by receiving examinee's manual operations order.
Speech processing module 202: process examinee's voice signal, extracts acoustic feature, identifies and obtains the time boundary of acoustic elements by the text that voice are corresponding.
Its feature of described speech processing module is to support real-time online process and off-line batch processing two kinds of modes.Speech recognition process in described speech processing module both can adopt local service, and also can have been come by network access voice cloud service.
Pronunciation evaluation module 203: the result exported according to topic information and speech processing module 202, carries out assessment and analysis to the different aspect of user pronunciation.
Described its feature of pronunciation evaluation module is that pronunciation evaluation at least comprises four aspects: the pronunciation accuracy of different units, rhythm accuracy (as stress, tone, intonation etc.), pronunciation integrity degree and pronunciation fluency.
Flow characteristic extracting module 204: the flow comprehensive characteristics extracting reflection examinee spoken language proficiency, comprises the feature of acoustics, syntax and semantics.
Described flow characteristic extracting module is characterized in that extracted a feature part comes from pronunciation evaluation module 203, and a part is from topic information, and some is from speech processing module 202.
Described exam question information at least comprises topic types, topic requirements, item content, investigation emphasis and model answer text etc.
Scoring submodule 205: utilize flow comprehensive characteristics, comprehensive grading is carried out to user speech.
Described scoring submodule is characterized in that flow comprehensive characteristics comes from flow characteristic extracting module 204, and comprehensive grading algorithm statistics Sum fanction combines.
The methods of marking of the distributed automatic scoring method of SET comprises the steps:
Step one, detects examinee's voice by recording module and records;
Step 2, extracts phonetic feature, utilizes acoustic model and language model, identify voice, obtains corresponding text and acoustic elements time boundary thereof and likelihood score score;
Step 3, utilizes acoustic elements time boundary and likelihood score score and exercise question relevant knowledge, carries out analysis and evaluation to voice, obtains the scoring of the pronunciation accuracy of different phonetic unit, rhythm accuracy, pronunciation integrity degree and pronunciation fluency etc.;
Step 4, extracts the acoustics of reflection examinee spoken language proficiency, the flow comprehensive characteristics of syntax and semantics aspect from the Output rusults and exam question information of speech processing module and pronunciation evaluation module;
Step 5, based on the model of training in advance and the rule of design, uses flow comprehensive characteristics to carry out comprehensive grading to examinee's spoken language proficiency.
The distributed automatic scoring method of described SET is characterized in that it being that front and back rely between each grading module, and the input information of each grading module comes from the calculating output of last module, and wherein the computational complexity of speech processing module 202 is the highest.
Fig. 3 gives the workflow of the distributed Auto-Evaluation System of SET:
The first step, examinee obtains exam question and answers on request, examination client equipment records examinee's voice, carries out first stage score calculation, according to data exchange standard, examinee's voice and appraisal result are sent to scoring management devices according to topic information and requirement to examinee's voice;
Second step, scoring management devices receives examinee's answer and the appraisal result of the transmission of examination client device, resolve appraisal result, if appraisal result is not final scoring, then according to data exchange standard, examinee's voice and scoring intermediate result are organized into scoring task, are sent to scoring task scheduling apparatus after encryption and mark further;
3rd step, scoring task scheduling apparatus receives the scoring task that scoring management devices sends, resolve scoring task and obtain score data and task priority, required grading module and algorithm is selected according to arranging of grading module territory, generate score calculation program, the score calculation device being submitted to backstage according to task priority calculates;
4th step, score calculation device receives the scoring procedures that scoring task scheduling apparatus sends, execution subordinate phase is marked, and final appraisal result is sent to scoring task scheduling apparatus, and task scheduling apparatus of then marking is forwarded to scoring management devices appraisal result again.
Its key features of distributed Auto-Evaluation System of disclosed SET is that automatic scoring process can be divided into two stages: the subordinate phase scoring being positioned at the first stage scoring on examination client device and being positioned on the score calculation device on backstage.
The distributed Auto-Evaluation System of disclosed SET it is mainly characterized in that described automatic scoring module can belong to the first scoring stage and the second scoring stage respectively.
The distributed Auto-Evaluation System of disclosed SET it is mainly characterized in that described first stage scoring unit is deployed on examination client device, and subordinate phase scoring unit is deployed on score calculation device.
The distributed Auto-Evaluation System of disclosed SET it is mainly characterized in that the subordinate phase scoring be deployed on score calculation device supports that the batch of extensive scoring task calculates, and multiple score calculation task independent parallel performs.
Its key feature of distributed Auto-Evaluation System of disclosed SET is also that subordinate phase score calculation task can comprise the scoring of the multiple tracks exercise question of multiple examinee simultaneously.In same score calculation task, different examinee can share identical scoring resource with the identical grading module of examination paper, reduces the Time and place waste that resource repeats to load.
The distributed Auto-Evaluation System of disclosed SET is further characterized in that both supports the scoring of large-scale off-line batch, also supports online scoring on a small scale.
The distributed Auto-Evaluation System of described SET is further characterized in that supports online scoring on a small scale, and task of namely at every turn marking only processes one examination paper answer of an examinee, direct feedback score after examination completes.
The distributed Auto-Evaluation System of disclosed SET it is mainly characterized in that can carry out dynamic dispatching according to the computational load of the score calculation device on examination client device and backstage and offered load except other grading module of recording module other places disposes.
The distributed Auto-Evaluation System of disclosed SET it is mainly characterized in that in the grading module be deployed on examination client device, recording module has limit priority, ensures that the execution of other modules can not affect sound-recording function.
According to the position that grading module is disposed, the distributed Auto-Evaluation System of SET has following four kinds of grading module deployment schemes:
Four kinds of grading module deployment schemes of the distributed Auto-Evaluation System of table 1 SET
Specifically, in scheme 4, scoring task no longer needs the score calculation device sending into backstage to calculate.
The grading module that Fig. 4 gives scheme 1 disposes schematic diagram.The distributed Auto-Evaluation System of whole SET can be divided into two large modules: front-end module and rear module.
Described front-end module comprises examination client device and scoring management devices.Wherein examination client device deploy has recording module and speech processing module.
Described rear module comprises the score calculation device on scoring task scheduling apparatus and backstage.Wherein the score calculation device deploy on backstage has pronunciation evaluation module, flow characteristic extracting module and comprehensive grading module.
Typically, scheme 1 is disposed recording module and speech processing module on front root module, and other module is deployed in rear module.
The benefit of deployment scheme 1 be make full use of examination client device computing power to complete language process function, the pressure of the score calculation device on backstage can be alleviated, be simultaneously unlikely to again cause the load of examination client device too heavy and affect recording work.
The distributed Auto-Evaluation System of described SET is further characterized in that examination client device is connected with speech cloud end system by network, can make full use of cloud computing resource.
The deployment scheme 1 of the distributed Auto-Evaluation System of described SET is further characterized in that, the speech recognition calculating of speech processing module can be submitted to voice high in the clouds and complete, thus alleviates the computational load of examination client.
Fig. 5 is the institutional framework schematic diagram of the distributed Auto-Evaluation System of the SET of access speech cloud system.Examination client device 101 is by network insertion speech cloud end system 105.
Speech cloud end system 105 deploy has powerful computational resource and speech model resource accurately.Speech recognition process in speech processing module is the most consumption calculations resource of whole scoring process, by utilizing computing power and the model resource of voice cloud device, not only can alleviate the computational load of examination client, but also recognition result more accurately can be obtained.
When grading module deployment scheme 1, first examination client device 101 records examinee's voice; Then speech processes is carried out, the acoustic feature of examinee's voice or examinee's voice is sent on speech cloud end system and carries out speech recognition, and obtain voice identification result, after completing other speech processes work, the result of topic information, examinee's voice and speech processes is sent to scoring management devices 102 together, carries out follow-up scoring by scoring management devices 102 and deal with the work.
The distributed Auto-Evaluation System of described SET is characterized in that if the Marking apparatus on backstage and scoring management devices are in same LAN (Local Area Network), or data are conveyed through movable storage device, then the scoring management devices in the distributed Auto-Evaluation System of described SET and scoring task scheduling apparatus can merge.This system is the distributed Auto-Evaluation System of a SET of simplifying.
Fig. 6 gives a kind of structural representation of distributed Auto-Evaluation System environment of SET of simplifying.
The distributed Auto-Evaluation System of the described SET of simplifying comprises examination client device 301, task management equipment 302 and score calculation device 303.
Examination client device 301 is identical with configuration with the function of 101 modules in Fig. 1.
Score calculation device 303 is identical with configuration with the function of 104 modules in Fig. 1.
Task management equipment 302: receive examinee's answer and appraisal result, resolves appraisal result, and in the middle of utilizing, score data and topic information structure score calculation program, submit score calculation task to, receive final appraisal result.
Specifically, can utilize movable storage device that the examinee's answer on each examination client device and appraisal result are moved to scoring management devices.Scoring supervisory routine on scoring management devices resolves appraisal result, score data, examinee's voice and topic information in the middle of extracting, be connected with the scoring procedures of required grading module, gather multiple scoring task and generate batch score calculation task, the score calculation equipment then submitting to backstage calculates.
Its feature of distributed Auto-Evaluation System of described SET is also to have a set of calculating fault tolerant mechanism, to ensure the stability of score calculation.Concrete measure comprises:
1) spare computing devices is configured: for the CPU of the score calculation Equipments Setting redundancy on backstage is to tackle burst hardware fault.If there is score calculation equipment to break down, scoring task scheduling equipment enables score calculation equipment for subsequent use automatically, and the scoring task on this computing machine is resubmited, and sends a warning.Score calculation equipment for subsequent use is in waiting status always, and scoring task scheduling equipment can enable this equipment at any time.Usually arranging score calculation equipment can with 1/10th of CPU as spare computing devices.
2) daily record of calculation process is recorded: the scoring task for every problem all keeps a journal file, until obtain last scoring from recording, record the implementation status of each step in each scoring submodule, at least comprise the instruction of execution, data directory, execution time, the information such as executing state.
3) detect recording abnormal: before examination formally starts, recording module requires examinee's rehearsal sound.By analyzing the sound signal obtained from examinee's sound pick-up outfit, detect the exception of sound pick-up outfit, such as: do not have voice, the abnormal conditions such as the too large or volume of volume is too little all will point out examinee.
4) score calculation task abnormity is detected: the scoring task scheduling apparatus of rear module monitors the computation process of each scoring task, if find that there is task computation failure, if computing hardware problem causes, then enable computational resource for subsequent use, and resubmit calculation task; Otherwise misregistration information, give a warning information.
5) detect data transmission exception: examination client device and scoring management devices monitoring data transmission process, if there is data transmission fails, the interval some time transmits again, and misregistration information, give a warning information.
6) data backup: interim result of preserving examinee's voice and first stage scoring on examination client device, until this data Successful transmissions.Scoring management devices in front end is preserved examinee's voice, scoring task and final appraisal result.Interim preservation scoring task and final appraisal result on the scoring task scheduling apparatus of rear end.
Certainly; the present invention also can have other various embodiments; when not deviating from the present invention's spirit and essence thereof; those of ordinary skill in the art are when making various corresponding change and distortion according to the present invention, but these change accordingly and are out of shape the protection domain that all should belong to the claim appended by the present invention.
Claims (12)
1. the distributed Auto-Evaluation System of a SET, comprise a scoring management devices and manage some examination client devices, scoring management devices communicates with at least one scoring task scheduling apparatus, a scoring task scheduling apparatus manages some score calculation devices, and the module wherein for automatic scoring comprises:
Recording module, for detecting and recording examinee's voice;
Speech processing module, for processing examinee's voice signal, extracts acoustic feature, identifies and obtains the time boundary of acoustic elements by the text that voice are corresponding;
Pronunciation evaluation module, for the result exported according to topic information and speech processing module, carries out assessment and analysis to the different aspect of user pronunciation;
Flow characteristic extracting module, for extracting the flow comprehensive characteristics of reflection examinee spoken language proficiency, comprises the feature of acoustics, syntax and semantics;
Comprehensive grading module: utilize flow comprehensive characteristics, carries out comprehensive grading to user speech; Wherein
Except recording module, other grading module are carried out dynamic dispatching according to the computational load of the score calculation device on examination client device and backstage and offered load and are disposed;
Wherein, examination client device, for recording examinee's voice by described recording module, by local computing unit, pre-service is carried out to examinee's answer, calculate certain middle appraisal result, by network communication unit or other modes, examinee's answer and middle appraisal result thereof are sent to scoring management devices;
Scoring management devices, for obtaining examinee's answer from examination client device and middle appraisal result thereof, in the middle of utilizing, appraisal result and topic information structure scoring task, be sent to scoring task scheduling apparatus, and receive the final appraisal result from scoring task scheduling apparatus;
Scoring task scheduling apparatus, for receiving the scoring task that described scoring management devices sends, resolve and extract score data, automatically score calculation program is organized according to topic information and the required module information of scoring, generate score calculation task, submit to calculation task to score calculation device by dispatching algorithm, finally obtain final appraisal result from score calculation device;
Score calculation device, for receive and perform described scoring task scheduling apparatus send scoring procedures, final appraisal result is sent to described scoring task scheduling apparatus.
2. the distributed Auto-Evaluation System of SET as claimed in claim 1, wherein examination client device disposes multiple grading module, in the impregnable situation of guarantee Recording Process, utilize local cpu to complete certain scoring work; Described scoring work is carried out with examination simultaneously, or for avoiding having any impact to examination, described scoring is operated in the free time timing operation outside examination.
3. the distributed Auto-Evaluation System of SET as claimed in claim 1, management devices of wherein marking configures reliable mass-memory unit with scoring task scheduling apparatus, has high data transfer bandwidth; Described scoring task scheduling apparatus safeguards the calculation task queue of a prioritization in internal memory.
4. the distributed Auto-Evaluation System of SET as claimed in claim 1, wherein examination client device, other data transfer mode of network data transmission mode of marking between management devices and scoring task scheduling apparatus substitute, and all the other devices and workflow still remain unchanged; Consider for data confidentiality, transmit data and be encrypted.
5. the distributed Auto-Evaluation System of SET as claimed in claim 3, the task scheduling algorithm of wherein said scoring task scheduling apparatus is: if task queue is discontented with, then task is inserted queue relevant position according to its priority; Task priority is higher than some task priority in queue else if, then task minimum for priority is moved out to buffer area or external memory storage, and this task is inserted queue relevant position; Otherwise this task is stored into buffer area or external memory storage; The queue of timing query task, if having living space, then adds queue buffer area task by its priority; The each score calculation device of automatic regular polling, if there is CPU idle, then task of getting from task queue carries out computing to this CPU.
6. the distributed Auto-Evaluation System of SET as claimed in claim 3, wherein said scoring task scheduling apparatus forms one multiple scoring taskings together can the calculation task of batch processing, submits to score calculation device.
7. the distributed Auto-Evaluation System of SET as claimed in claim 1, wherein flow characteristic extracting module extracts flow comprehensive characteristics according to the Output rusults of speech processing module and pronunciation evaluation module in conjunction with exam question information; Described exam question information at least comprises topic types, topic requirements, item content, investigation emphasis and model answer text.
8. the distributed Auto-Evaluation System of SET as claimed in claim 1, its automatic scoring process is divided into two stages: first stage scoring unit is deployed on examination client device; Subordinate phase scoring unit is deployed on score calculation device; Wherein: first stage scoring comprises recording module.
9. the distributed Auto-Evaluation System of SET as claimed in claim 8, its automatic scoring module successively relies on; Except recording module, other modules are flexible allocation in first stage scoring and subordinate phase scoring; Concrete scheme is: first stage scoring also comprises speech processing module; Subordinate phase scoring is made up of pronunciation evaluation module, flow characteristic extracting module and comprehensive grading module.
10. the distributed Auto-Evaluation System of SET as claimed in claim 8, the subordinate phase scoring be deployed on score calculation device supports that the batch of extensive scoring task calculates, and multiple score calculation task independent parallel performs; Each score calculation task agent of subordinate phase scoring contains the scoring of the multiple tracks exercise question of multiple examinee or the answer of a whole set of examination question.
The distributed Auto-Evaluation System of 11. SETs as claimed in claim 1, described Auto-Evaluation System comprises a set of calculating fault tolerant mechanism, specifically comprises:
1) computational resource for subsequent use is configured: for the CPU of backstage score calculation device configuring redundancy is to tackle burst hardware fault; If there is score calculation device to break down, described task scheduling apparatus enables score calculation device CPU for subsequent use automatically, and resubmits the scoring task on this computing machine, and sends a warning; Score calculation device for subsequent use is in waiting status always, and described task scheduling apparatus enables the CPU of this device at any time;
2) daily record of calculation process is recorded: the scoring task for every problem all keeps a journal file, until obtain last scoring from recording, record the implementation status of each step in each grading module, at least comprise the order of execution, data directory, execution time and execution state information;
3) detect recording abnormal: before examination formally starts, described recording module requires examinee's rehearsal sound, by analyzing the examinee's sound signal obtained, detecting sound-recording function abnormal, and pointing out examinee to adjust;
4) detect score calculation task abnormity: described scoring task scheduling apparatus monitors the computation process of each scoring task, if find that there is task computation failure, if computing hardware problem causes, then enable computational resource for subsequent use, and resubmit task; Otherwise misregistration information, give a warning information;
5) detect data transmission exception: described examination client device and scoring management devices monitoring data transmission process, if there is data transmission fails, the interval some time transmits again, and misregistration information, give a warning information;
6) data backup: described examination client device preserves the result of user speech and first stage scoring temporarily, until this data Successful transmissions; Described scoring management devices preserves examinee's voice, scoring task and final appraisal result; Described scoring task scheduling apparatus preserves scoring task and final appraisal result temporarily.
The distributed automatic scoring method of 12. 1 kinds of SETs, the concrete steps of wherein marking comprise:
Step 1, detect examinee's voice recording by recording module;
Step 2, extract phonetic feature, utilize acoustic model and language model, voice are identified, obtain corresponding text and phoneme unit time boundary thereof and likelihood score score;
Step 3, utilize phoneme unit time boundary and likelihood score score and exercise question relevant knowledge, analysis and evaluation is carried out to voice, obtains the pronunciation accuracy of different phonetic unit, rhythm accuracy, pronunciation integrity degree and pronunciation fluency;
Step 4, integrated voice result, assessment result and topic information, extract the acoustics of reflection examinee spoken language proficiency, the flow comprehensive characteristics of syntax and semantics aspect;
Step 5, based on the model of training in advance and the rule of design, flow comprehensive characteristics is used to carry out comprehensive grading to examinee's spoken language proficiency;
Specifically comprise the steps:
Step 1, examinee obtain exam question and answer on request, examination client device records examinee's voice, according to topic information and requirement, first stage score calculation is carried out to examinee's voice, according to self-defining data exchange standard, the first stage appraisal result of examinee's voice and acquisition is sent to scoring management devices;
Step 2, scoring management devices receive examinee's answer and the first stage appraisal result of the transmission of examination client device, resolve appraisal result, if appraisal result is not final scoring, then according to self-defining data exchange standard, examinee's voice and scoring intermediate result and topic information are organized into scoring task, are sent to scoring task scheduling apparatus and carry out subordinate phase scoring;
Step 3, scoring task scheduling apparatus receive the scoring task that scoring management devices sends, resolve scoring task and obtain score data and task priority, required grading module and algorithm is selected according to arranging of grading module territory, generate score calculation program, the score calculation device being submitted to backstage according to task priority calculates;
Step 4, score calculation device receive the scoring procedures that task scheduling apparatus sends, and perform subordinate phase scoring, final appraisal result is sent to scoring task scheduling apparatus, and task scheduling apparatus of then marking is forwarded to scoring management devices appraisal result again.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110336179.4A CN102509483B (en) | 2011-10-31 | 2011-10-31 | A kind of distributed Auto-Evaluation System of SET and method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110336179.4A CN102509483B (en) | 2011-10-31 | 2011-10-31 | A kind of distributed Auto-Evaluation System of SET and method thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102509483A CN102509483A (en) | 2012-06-20 |
| CN102509483B true CN102509483B (en) | 2015-10-21 |
Family
ID=46221559
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201110336179.4A Active CN102509483B (en) | 2011-10-31 | 2011-10-31 | A kind of distributed Auto-Evaluation System of SET and method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102509483B (en) |
Families Citing this family (28)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102779435A (en) * | 2012-07-30 | 2012-11-14 | 无锡南理工科技发展有限公司 | Wireless intelligent answering system supporting multi-network and multi-terminal and data transmission method thereof |
| CN103151042B (en) * | 2013-01-23 | 2016-02-24 | 中国科学院深圳先进技术研究院 | Full-automatic oral evaluation management and points-scoring system and methods of marking thereof |
| CN103258544B (en) * | 2013-04-15 | 2016-05-25 | 深圳市海云天科技股份有限公司 | A kind of examination system |
| TWI494891B (en) * | 2013-06-19 | 2015-08-01 | Learning Digital Inc | Course market learning system in internet cloud |
| CN105556564A (en) * | 2013-07-16 | 2016-05-04 | 株式会社倍乐生 | Portable information processing device, test assistance system, and test assistance method |
| CN104347071B (en) * | 2013-08-02 | 2020-02-07 | 科大讯飞股份有限公司 | Method and system for generating reference answers of spoken language test |
| CN103605492B (en) * | 2013-11-28 | 2016-08-17 | 中国科学院深圳先进技术研究院 | A kind of self adaptation speech training method and platform |
| CN103685313A (en) * | 2013-12-26 | 2014-03-26 | 北京中电普华信息技术有限公司 | Communication method and system applied to online examination |
| CN104575490B (en) * | 2014-12-30 | 2017-11-07 | 苏州驰声信息科技有限公司 | Spoken language pronunciation evaluating method based on deep neural network posterior probability algorithm |
| CN104715640A (en) * | 2015-03-14 | 2015-06-17 | 陕西学前师范学院 | Language teaching system |
| CN106205635A (en) * | 2016-07-13 | 2016-12-07 | 中南大学 | Method of speech processing and system |
| CN106409283B (en) * | 2016-08-31 | 2020-01-10 | 上海交通大学 | Man-machine mixed interaction system and method based on audio |
| CN107316255A (en) * | 2017-04-07 | 2017-11-03 | 苏州清睿教育科技股份有限公司 | A kind of efficient competition method competed online that shuttles |
| CN107578778A (en) * | 2017-08-16 | 2018-01-12 | 南京高讯信息科技有限公司 | A kind of method of spoken scoring |
| CN107767862B (en) * | 2017-11-06 | 2024-05-21 | 深圳市领芯者科技有限公司 | Voice data processing method, system and storage medium |
| CN109785698B (en) * | 2017-11-13 | 2021-11-23 | 上海流利说信息技术有限公司 | Method, device, electronic equipment and medium for oral language level evaluation |
| CN108877308A (en) * | 2018-07-14 | 2018-11-23 | 汤凯 | A kind of VR safety education evaluating method |
| CN109087667B (en) * | 2018-09-19 | 2023-09-26 | 平安科技（深圳）有限公司 | Voice fluency recognition method and device, computer equipment and readable storage medium |
| CN109147765B (en) * | 2018-11-16 | 2021-09-03 | 安徽听见科技有限公司 | Audio quality comprehensive evaluation method and system |
| CN109889938B (en) * | 2019-01-17 | 2021-03-26 | 东南大学 | Oral interpretation examination recording device |
| CN110164447B (en) * | 2019-04-03 | 2021-07-27 | 苏州驰声信息科技有限公司 | Spoken language scoring method and device |
| CN110164422A (en) * | 2019-04-03 | 2019-08-23 | 苏州驰声信息科技有限公司 | A kind of the various dimensions appraisal procedure and device of speaking test |
| CN110827835A (en) * | 2019-11-21 | 2020-02-21 | 上海好学网络科技有限公司 | Spoken language examination system and method |
| CN111161577A (en) * | 2019-12-25 | 2020-05-15 | 上海好学网络科技有限公司 | Spoken language examination system and method |
| CN111833853B (en) * | 2020-07-01 | 2023-10-27 | 腾讯科技（深圳）有限公司 | Voice processing method and device, electronic equipment and computer readable storage medium |
| CN113572995A (en) * | 2021-04-09 | 2021-10-29 | 深圳市即构科技有限公司 | Interview video recording device, interview system and interview cabin |
| CN113674766A (en) * | 2021-08-18 | 2021-11-19 | 上海复深蓝软件股份有限公司 | Voice evaluation method and device, computer equipment and storage medium |
| CN114245194A (en) * | 2021-12-23 | 2022-03-25 | 深圳市优必选科技股份有限公司 | Video teaching interaction method and device and electronic equipment |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101094104A (en) * | 2007-07-30 | 2007-12-26 | 中兴通讯股份有限公司 | Method and device for carrying out device management through proxy of security network management |
| CN101197084A (en) * | 2007-11-06 | 2008-06-11 | 安徽科大讯飞信息科技股份有限公司 | Automatic spoken English evaluating and learning system |
| CN101739868B (en) * | 2008-11-19 | 2012-03-28 | 中国科学院自动化研究所 | Automatic evaluation and diagnosis method of text reading level for oral test |
| CN201570038U (en) * | 2009-12-09 | 2010-09-01 | 丁肇芬 | Blinding control system for spoken language tests |
-
2011
- 2011-10-31 CN CN201110336179.4A patent/CN102509483B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN102509483A (en) | 2012-06-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102509483B (en) | A kind of distributed Auto-Evaluation System of SET and method thereof | |
| CN108399923B (en) | More human hairs call the turn spokesman's recognition methods and device | |
| CN107680019A (en) | A kind of implementation method of Examination Scheme, device, equipment and storage medium | |
| WO2019095586A1 (en) | Meeting minutes generation method, application server, and computer readable storage medium | |
| CN102629936B (en) | A kind of method of mobile terminal process text, relevant device and system | |
| CN106898349A (en) | A kind of Voice command computer method and intelligent sound assistant system | |
| CN107610702A (en) | Terminal device standby wakeup method, apparatus and computer equipment | |
| CN104346480B (en) | information mining method and device | |
| US20220301546A1 (en) | Method for testing vehicle-mounted voice device, electronic device and storage medium | |
| CN106781763A (en) | A kind of university's applied mathematics Teaching System | |
| CN114328852A (en) | Text processing method, related device and equipment | |
| CN112287082A (en) | Data processing method, device, equipment and storage medium combining RPA and AI | |
| CN106067092A (en) | The student realized based on the Internet asks for leave management method and system | |
| CN205814303U (en) | A kind of Chinese standard aphasia evaluation system | |
| CN110390614B (en) | Method and system for immersive teaching based on emotion monitoring and real-time interaction | |
| CN211264953U (en) | Intelligent interrogation system | |
| CN205193801U (en) | Otology detects data management unit | |
| CN106448296A (en) | Intelligent English teaching system for English teaching | |
| CN107403629A (en) | Far field pickup method of evaluating performance and system, electronic equipment | |
| CN208094741U (en) | A kind of intelligent microphone based on speech recognition technology | |
| CN110163777A (en) | A kind of public good assistance to improve schooling in backward areas management method and device | |
| JP2019074865A (en) | Conversation collection device, conversation collection system, and conversation collection method | |
| CN108831449A (en) | A kind of data interaction system method and system based on intelligent sound box | |
| CN113593553B (en) | Voice recognition method, voice recognition apparatus, voice management server, and storage medium | |
| JP2003066820A (en) | Communication correction system, terminal device, and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| DD01 | Delivery of document by public notice |
Addressee: Suzhou gallop Mdt InfoTech Ltd patent controller (collect) Document name: Notification that Application Deemed not to be Proposed |
|
| DD01 | Delivery of document by public notice |
Addressee: Suzhou gallop Mdt InfoTech Ltd patent controller (collect) Document name: Notification that Application Deemed not to be Proposed |
|
| ASS | Succession or assignment of patent right |
Owner name: SUZHOU CHISHENG INFORMATION TECHNOLOGY CO., LTD. Free format text: FORMER OWNER: SUZHOU AISPEECH INFORMATION TECHNOLOGY CO., LTD. Effective date: 20140926 |
|
| C41 | Transfer of patent application or patent right or utility model | ||
| COR | Change of bibliographic data |
Free format text: CORRECT: ADDRESS; FROM: 215021 SUZHOU, JIANGSU PROVINCE TO: 215000 SUZHOU, JIANGSU PROVINCE |
|
| TA01 | Transfer of patent application right |
Effective date of registration: 20140926 Address after: 102, room 210, Xinghai street, Suzhou Industrial Park, Jiangsu, Suzhou 215000, China Applicant after: Suzhou Chisheng Information Technology Co., Ltd. Address before: 215021, Jiangsu Province, Suzhou Industrial Park, No. 258 benevolence Road, alone lake library, C106 Applicant before: Suzhou Speech Information Technology Co., Ltd. |
|
| C53 | Correction of patent for invention or patent application | ||
| CB03 | Change of inventor or designer information |
Inventor after: Zhou Weida Inventor after: Lin Yuandong Inventor after: Yu Kai Inventor after: Wang Huanliang Inventor after: Xue Feng Inventor after: Zhu Hongjie Inventor after: Zhang Li Inventor before: Zhou Weida Inventor before: Yu Kai Inventor before: Wang Huanliang Inventor before: Xue Feng Inventor before: Zhu Hongjie Inventor before: Zhang Li |
|
| COR | Change of bibliographic data |
Free format text: CORRECT: INVENTOR; FROM: ZHOU WEIDA YU KAI WANG HUANLIANG XUE FENG ZHU HONGJIE ZHANG LI TO: ZHOU WEIDA LIN YUANDONG YU KAI WANG HUANLIANG XUE FENG ZHU HONGJIE ZHANG LI |
|
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |