CN109408638A - Calibrate set update method and device - Google Patents
Calibrate set update method and device Download PDFInfo
- Publication number
- CN109408638A CN109408638A CN201811230610.5A CN201811230610A CN109408638A CN 109408638 A CN109408638 A CN 109408638A CN 201811230610 A CN201811230610 A CN 201811230610A CN 109408638 A CN109408638 A CN 109408638A
- Authority
- CN
- China
- Prior art keywords
- examination paper
- scoring
- scorer
- target sample
- sample examination
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 69
- 239000013598 vector Substances 0.000 claims abstract description 90
- 238000012360 testing method Methods 0.000 claims abstract description 28
- 239000013604 expression vector Substances 0.000 claims abstract description 8
- 238000007689 inspection Methods 0.000 claims description 53
- 238000004891 communication Methods 0.000 claims description 7
- 238000006243 chemical reaction Methods 0.000 claims description 5
- 230000001052 transient effect Effects 0.000 claims description 5
- 239000012141 concentrate Substances 0.000 abstract description 8
- 238000013473 artificial intelligence Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 10
- 238000012216 screening Methods 0.000 description 9
- 239000011159 matrix material Substances 0.000 description 6
- 238000012549 training Methods 0.000 description 6
- 238000004458 analytical method Methods 0.000 description 5
- 238000013136 deep learning model Methods 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 239000000203 mixture Substances 0.000 description 3
- 238000004590 computer program Methods 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000011835 investigation Methods 0.000 description 2
- 241000208340 Araliaceae Species 0.000 description 1
- 235000005035 Panax pseudoginseng ssp. pseudoginseng Nutrition 0.000 description 1
- 235000003140 Panax quinquefolius Nutrition 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000000151 deposition Methods 0.000 description 1
- 235000008434 ginseng Nutrition 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- SBNFWQZLDJGRLK-UHFFFAOYSA-N phenothrin Chemical compound CC1(C)C(C=C(C)C)C1C(=O)OCC1=CC=CC(OC=2C=CC=CC=2)=C1 SBNFWQZLDJGRLK-UHFFFAOYSA-N 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
- 230000003442 weekly effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06393—Score-carding, benchmarking or key performance indicator [KPI] analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/20—Education
- G06Q50/205—Education administration or guidance
Landscapes
- Business, Economics & Management (AREA)
- Human Resources & Organizations (AREA)
- Engineering & Computer Science (AREA)
- Strategic Management (AREA)
- Educational Administration (AREA)
- Economics (AREA)
- Tourism & Hospitality (AREA)
- General Physics & Mathematics (AREA)
- Educational Technology (AREA)
- Marketing (AREA)
- Theoretical Computer Science (AREA)
- Development Economics (AREA)
- Entrepreneurship & Innovation (AREA)
- Physics & Mathematics (AREA)
- General Business, Economics & Management (AREA)
- Quality & Reliability (AREA)
- Operations Research (AREA)
- Game Theory and Decision Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Primary Health Care (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the present invention provides a kind of calibration set update method and device, belongs to field of artificial intelligence.This method comprises: obtaining the examination paper of current test, converting higher-dimension for the content of text in each examination paper indicates vector, and concentrates each sample examination paper to be converted into higher-dimension expression vector calibration；Vector, which clusters, to be indicated to all higher-dimensions, according to the distance between the corresponding cluster result of each examination paper and calibration collection, calculates the corresponding target range of each examination paper；The examination paper that target range is greater than preset threshold updates calibration collection according to target sample examination paper as target sample examination paper.The examination paper that not scaled collection is covered in all examination papers due to can determine current test, and can be according to the scaled examination paper update calibration collection for collecting and covering, so that appraisal result is more accurate when the subsequent collection progress machine scoring according to calibration.
Description
Technical field
The present embodiments relate to field of artificial intelligence more particularly to a kind of calibration set update methods and device.
Background technique
Due to the error for comparing labor intensive cost and being easy to appear subjectivity of manually marking examination papers, so that nowadays computer is commented
It rolls up more more and more universal.It scores at present generally by paper points-scoring system, wherein paper points-scoring system is based on calibration collection
What training obtained, it includes multiple groups calibration data that calibration, which is concentrated, and every group of calibration data include the scoring of sample paper and sample paper.
In practical application, paper content, which is input to paper points-scoring system, can be obtained the scoring of the paper.Due to for training paper
The sample paper type of points-scoring system usually compares limitation, cannot cover all paper types when actually sentencing volume, to lead
Cause appraisal result inaccuracy.
Summary of the invention
To solve the above-mentioned problems, the embodiment of the present invention provides one kind and overcomes the above problem or at least be partially solved
State the calibration set update method and device of problem.
According to a first aspect of the embodiments of the present invention, a kind of calibration set update method is provided, comprising:
The examination paper for obtaining current test, converting higher-dimension for the content of text in each examination paper indicates vector, and will calibration
Concentrating each sample examination paper to be converted into higher-dimension indicates vector；
To all higher-dimensions indicate vector cluster, according to the corresponding cluster result of each examination paper and calibration collection between away from
From calculating the corresponding target range of each examination paper；
The examination paper that target range is greater than preset threshold is updated according to target sample examination paper and is calibrated as target sample examination paper
Collection.
Method provided in an embodiment of the present invention, by obtaining the examination paper of current test, by the content of text in each examination paper
Being converted into higher-dimension indicates vector, and concentrates each sample examination paper to be converted into higher-dimension expression vector calibration.All higher-dimensions are indicated
Vector is clustered, and according to the distance between the corresponding cluster result of each examination paper and calibration collection, it is corresponding to calculate each examination paper
Target range.The examination paper that target range is greater than preset threshold updates according to target sample examination paper and determines as target sample examination paper
Mark collection.The examination paper that not scaled collection is covered in all examination papers due to can determine current test, and can be contained according to not scaled collection
The examination paper update calibration collection of lid, so that appraisal result is more accurate when the subsequent collection progress machine scoring according to calibration.
According to a second aspect of the embodiments of the present invention, a kind of calibration collection updating device is provided, comprising:
Conversion module, for obtaining the examination paper of current test, converting higher-dimension for the content of text in each examination paper is indicated
Vector, and concentrate each sample examination paper to be converted into higher-dimension expression vector calibration；
Cluster module, for all higher-dimensions indicate vector cluster, according to the corresponding cluster result of each examination paper with
The distance between calibration collection, calculates the corresponding target range of each examination paper；
Update module, for the examination paper using target range greater than preset threshold as target sample examination paper, according to target sample
This examination paper updates calibration collection.
According to a third aspect of the embodiments of the present invention, a kind of electronic equipment is provided, comprising:
At least one processor；And
At least one processor being connect with processor communication, in which:
Memory is stored with the program instruction that can be executed by processor, and the instruction of processor caller is able to carry out first party
Set update method is calibrated in the various possible implementations in face provided by any possible implementation.
According to the fourth aspect of the invention, a kind of non-transient computer readable storage medium, non-transient computer are provided
Readable storage medium storing program for executing stores computer instruction, and computer instruction makes the various possible implementations of computer execution first aspect
In set update method is calibrated provided by any possible implementation.
It should be understood that above general description and following detailed description be it is exemplary and explanatory, can not
Limit the embodiment of the present invention.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is this hair
Bright some embodiments for those of ordinary skill in the art without creative efforts, can be with root
Other attached drawings are obtained according to these attached drawings.
Fig. 1 is a kind of flow diagram of calibration data update method provided in an embodiment of the present invention；
Fig. 2 is a kind of flow diagram of calibration data update method provided in an embodiment of the present invention；
Fig. 3 is a kind of flow diagram of calibration data update method provided in an embodiment of the present invention；
Fig. 4 is a kind of flow diagram of calibration data update method provided in an embodiment of the present invention；
Fig. 5 is a kind of flow diagram of calibration data update method provided in an embodiment of the present invention；
Fig. 6 is a kind of flow diagram of calibration data update method provided in an embodiment of the present invention；
Fig. 7 is a kind of flow diagram of calibration data update method provided in an embodiment of the present invention；
Fig. 8 is a kind of structural schematic diagram of calibration data updating device provided in an embodiment of the present invention；
Fig. 9 is the block diagram of a kind of electronic equipment provided in an embodiment of the present invention.
Specific embodiment
In order to make the object, technical scheme and advantages of the embodiment of the invention clearer, below in conjunction with the embodiment of the present invention
In attached drawing, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described embodiment is
A part of the embodiment of the present invention, instead of all the embodiments.Based on the embodiments of the present invention, those of ordinary skill in the art
Every other embodiment obtained without creative efforts, shall fall within the protection scope of the present invention.
Due to the error for comparing labor intensive cost and being easy to appear subjectivity of manually marking examination papers, so that nowadays computer is commented
It rolls up more more and more universal.It scores at present generally by paper points-scoring system, wherein paper points-scoring system is based on calibration collection
What training obtained, it includes multiple groups calibration data that calibration, which is concentrated, and every group of calibration data include the scoring of sample paper and sample paper.
In practical application, paper content, which is input to paper points-scoring system, can be obtained the scoring of the paper.Due to for training paper
The sample paper type of points-scoring system usually compares limitation, cannot cover all paper types when actually sentencing volume, to lead
Cause appraisal result inaccuracy.In addition, scorer is possible to have certain fluctuation on scoring scale even over time
Error leads to scoring inaccuracy.
For example, concentrating the examination paper being not covered with composition data by taking the type of examination paper is composition as an example for calibration, leading
Cause the appraisal result inaccuracy of the examination paper.In the case where actually using scene, the horizontal level that examinee answers is uneven, and content of answering is a variety of
Multiplicity causes the scale of the calibration collection actually used to be restricted, and the sample examination paper for calibrating concentration is difficult to cover and all answer.
For said circumstances, the embodiment of the invention provides a kind of calibration data update methods.It should be noted that the party
Method is mainly to be updated to calibration collection, and it can be composition that the content of sample examination paper is concentrated in calibration, can also be with the answer of question-and-answer problem
Deng the present invention is not especially limit this.The executing subject of this method can be the computer with function of marking examination papers, or
The equipment that person is independently arranged function of marking examination papers, the embodiment of the present invention are also not especially limited this.Specifically, referring to Fig. 1, this method
Include:
101, the examination paper for obtaining current test, converting higher-dimension for the content of text in each examination paper indicates vector, and will
Calibration, which concentrates each sample examination paper to be converted into higher-dimension, indicates vector.
In 101, the examination paper of current test is the paper to score to scorer.Higher-dimension indicates that vector is referred to examination paper
After content of text is indicated with vector, the semantic vector of examination paper semanteme is indicated in higher dimensional space.Wherein, between different examination papers
Semantic difference is bigger, then the higher-dimension of different examination papers indicates that distance of the vector in higher dimensional space is also remoter.For any examination paper, originally
Inventive embodiments do not limit the mode for converting higher-dimension expression vector for the content of text in the examination paper specifically, including but not
It is limited to: converts term vector matrix for the content of text in each examination paper, extremely by the corresponding term vector Input matrix of each examination paper
Chapter grade deep learning model, the higher-dimension for exporting each examination paper indicate vector.
Wherein it is possible to using common skip-gram or cbow algorithm, by words all under current test be mapped as to
Amount, then the vector after mapping is spliced into term vector matrix, the present invention is not especially limit this.For example, if examination paper
Content of text be " I am a boy ", map the term vector of acquisition are as follows: I=[0.1 0.2 ... 0.1], am=[0.2 0.3
... 0.4], above-mentioned term vector can be then spliced into one by a=[0.5 0.2 ... 0.6], boy=[0.7 0.1 ... 0.9]
Term vector matrix.
It should be noted that chapter grade deep learning model, can carry out different thicknesses for the chapter text information in examination paper
The traversal of granularity, and from the context the vector combined weighted of word level can be obtained into the semantic vector of full piece level, namely
Higher-dimension indicates vector, thus can avoid merely calculating word registration and the problem of bring " not synonymous with word ".Wherein, a piece
The type of chapter grade deep learning model can be convolutional neural networks CNN, or length Memory Neural Networks LSTM, this hair
Bright embodiment is not especially limited this.
102, vector, which clusters, to be indicated to all higher-dimensions, according between the corresponding cluster result of each examination paper and calibration collection
Distance, calculate the corresponding target range of each examination paper.
By above-mentioned 101, by the way that the vector of word level in examination paper each under current test to be weighted, can be obtained every
The higher-dimension of one examination paper indicates vector.Similarly, by concentrating the vector of word level in each sample examination paper to be weighted calibration,
The higher-dimension that each sample examination paper can be obtained indicates vector.In the case where obtaining current test, the higher-dimension of examination paper indicates vector and calibration
After concentrating the higher-dimension of sample examination paper to indicate vector, it can indicate that vector is clustered to all higher-dimensions.Wherein, each higher-dimension indicate to
Amount can be gathered for certain one kind.Namely in higher dimensional space, each higher-dimension indicates that vector can be by a type of cluster result
Included.
Since examination paper and higher-dimension indicate vector, there are one-to-one relationships, to indicate that vector is corresponding according to each higher-dimension
The distance between cluster result and calibration collection, calculating each higher-dimension indicates the corresponding target range of vector.Specifically, due to calibration
The corresponding higher-dimension of sample examination paper is concentrated to indicate that vector equally passes through above-mentioned cluster process and clustered, so as to according to each high
Dimension table shows the distance between the corresponding cluster result of vector other cluster results corresponding with calibration collection, and calculating each higher-dimension indicates
The corresponding target range of vector.Wherein, calibration collects other corresponding cluster results and refers to concentrating sample examination paper pair comprising calibration
The higher-dimension answered indicates the cluster result of vector.The above process is equivalent to, and for each examination paper under current test, is examined according to each
The distance between corresponding cluster result and calibration collection are rolled up, the corresponding target range of each examination paper is calculated.
The embodiment of the present invention is not to according to the distance between the corresponding cluster result of each examination paper and other cluster results, meter
The mode for calculating the corresponding target range of each examination paper specifically limits, including but not limited to: for any examination paper, according to the examination paper
The distance between corresponding cluster result and calibration collection, calculate the average value of all distance values, and examine using the average value as this
Roll up corresponding target range.Certainly, maximum distance can also be chosen from all distance values as target during actual implementation
Distance, the present invention is not especially limit this.In addition, k-means algorithm can be used in clustering algorithm, the present invention is implemented
Example is not especially limited this.
103, the examination paper that target range is greater than preset threshold is updated as target sample examination paper according to target sample examination paper
Calibration collection.
As shown in the above, the semantic difference between different examination papers is bigger, then the higher-dimension of different examination papers indicates that vector exists
Distance in higher dimensional space is also remoter.In this step, target range is greater than the examination paper of preset threshold, it can thinks that these are examined
The semantic information of volume is concentrated in current calibration and is not included, so as to using these examination papers as target sample examination paper, with right
Calibration collection is updated.Concentrating mainly include due to calibration is sample examination paper and its corresponding scoring, thus can when updating
Target sample examination paper and its corresponding scoring are added to calibration collection.Wherein, the corresponding scoring of target sample examination paper can be by scoring
Member is commented.
Method provided in an embodiment of the present invention, by obtaining the examination paper of current test, by the content of text in each examination paper
Being converted into higher-dimension indicates vector, and concentrates each sample examination paper to be converted into higher-dimension expression vector calibration.All higher-dimensions are indicated
Vector is clustered, and according to the distance between the corresponding cluster result of each examination paper and calibration collection, it is corresponding to calculate each examination paper
Target range.The examination paper that target range is greater than preset threshold updates according to target sample examination paper and determines as target sample examination paper
Mark collection.The examination paper that not scaled collection is covered in all examination papers due to can determine current test, and can be contained according to not scaled collection
The examination paper update calibration collection of lid, so that appraisal result is more accurate when the subsequent collection progress machine scoring according to calibration.
Since when updating calibration collection, the scoring of target sample examination paper is commented by scorer, and what scorer was commented
It is not necessarily accurate to score, to need to screen scorer, avoids the target sample examination paper of scoring inaccuracy from being added to fixed
Mark is concentrated.Content based on the demand and above-described embodiment is wrapped in all target sample examination papers as a kind of alternative embodiment
It is rolled up containing preset quantity quality inspection；Correspondingly, before updating calibration collection according to target sample examination paper, further includes: according to default
Quantity quality inspection volume, screens out in all target sample examination papers and is unsatisfactory for preset condition target sample examination paper.
Wherein, quality inspection volume can be chosen from the examination paper of current test in advance.Quality inspection volume can be by selection of specialists of marking examination papers, also
It can be chosen, such as be chosen according to the score gradient of examination paper, so that subsequent can pass through different gradients using other selection standards
The examination paper of lower practical score, the scoring level to measure subsequent scorer, the present invention is not especially limit this.
Since each target sample examination paper is scored by a scorer, so that quality inspection volume equally can the person's of being scored scoring.
Correspondingly, the embodiment of the present invention not to according to preset quantity quality inspection roll up, screened out in all target sample examination papers be unsatisfactory for it is pre-
If the mode of conditions object sample examination paper specifically limits, including but not limited to: according to each scorer to preset quantity matter
The scoring characteristic parameter for examining volume, screens out the target sample examination paper for being unsatisfactory for preset condition in all target sample examination papers；Wherein,
Scoring mass parameter that scoring characteristic parameter, which includes each scorer, rolls up preset quantity quality inspection and/or each scorer are to pre-
If the scoring process parameter of quantity quality inspection volume.
In actually distribution quality inspection volume, preset quantity quality inspection volume can be distributed to each scorer and scored, thus
Facilitate subsequent progress quality inspection investigation.The embodiment of the present invention is not special to the scoring rolled up according to each scorer to preset quantity quality inspection
Parameter is levied, is screened out in all target sample examination papers and is unsatisfactory for the mode of the target sample examination paper of preset condition and specifically limits,
Including but not limited to: for any scorer, calculating the flat of the scoring characteristic parameter that the scorer rolls up preset quantity quality inspection
Mean value；If average value not in pre-set interval, screens out the target sample that the scorer is commented in all target sample examination papers
Examination paper.
It should be noted that the quantity that controllable quality inspection is rolled up during actual implementation, to avoid normal scoring process is influenced.
Wherein, the quantity of quality inspection volume cannot be very few, otherwise cannot measure the scoring quality of scorer well.The quantity of quality inspection volume is not yet
It excessively otherwise can will increase additional workload, influence normally to score.In addition, scoring characteristic parameter may include scoring quality ginseng
Several and/or scoring process parameter.
Method provided in an embodiment of the present invention passes through the scoring feature rolled up according to each scorer to preset quantity quality inspection
Parameter screens out the target sample examination paper for being unsatisfactory for preset condition in all target sample examination papers.Since the side of quality inspection can be passed through
Formula investigates the scoring situation of each scorer, and can screen out according to quality inspection result and be unsatisfactory for the mesh that the scorer of preset condition is commented
This examination paper of standard specimen, so that appraisal result is more accurate when the subsequent collection progress machine scoring according to calibration.
Content based on the above embodiment, as a kind of alternative embodiment, the characteristic parameter that scores includes each scorer pair
The scoring mass parameter of preset quantity quality inspection volume；Correspondingly, the embodiment of the present invention not to according to each scorer to present count
A scoring characteristic parameter for quality inspection volume is measured, the target sample examination paper for being unsatisfactory for preset condition is screened out in all target sample examination papers
Mode specifically limit.Referring to fig. 2, including but not limited to:
201, preset quantity quality inspection is rolled up according to the scale of preset quantity quality inspection volume and each scorer
Scoring, calculates the scoring mass parameter of each scorer.
Wherein, scale can be expert analysis mode.For any scorer, the scoring mass parameter of the scorer can be with
For the close degree coefficient between the scoring and expert analysis mode of the scorer, the present invention is not especially limit this.Tool
Body, the expert analysis mode that can roll up preset quantity quality inspection rolls up preset quantity quality inspection as a vector, by the scorer
Scoring as another vector, to calculate the distance between two vectors, using as close degree coefficient, and this is close
Degree coefficient is as the scoring mass parameter to the scorer.For example, with standards of grading use 5 points system and quality inspection volume quantity for
For 6, if the expert analysis mode of preset quantity quality inspection volume is respectively 0,1,2,3,4,5, and the scoring of preset quantity quality inspection volume
Member's scoring is 1,2,3,4,5,4, then above-mentioned two groups of scorings can be used as two vectors, by calculating the distance between two vectors,
So as to regard the distance between two vectors as close degree coefficient.
202, the scoring mass parameter of scorer is less than preset threshold if it exists, then screens out in all target sample examination papers
Scoring mass parameter is less than the target sample examination paper that the scorer of preset threshold is commented.
If the scoring mass parameter of the scorer is less than preset threshold, show that the scoring level of the scorer can't reach
To demand, it is less than the mesh that the scorer of preset threshold is commented so as to screen out scoring mass parameter in all target sample examination papers
This examination paper of standard specimen.By above-mentioned steps it is found that scoring mass parameter can be indicated by close degree coefficient, and close degree coefficient
Value be 0 to 1, so that the value of preset threshold may be 0 to 1, such as 0.8, the embodiment of the present invention does not limit this specifically
It is fixed.
Method provided in an embodiment of the present invention passes through the scale rolled up according to preset quantity quality inspection and each scoring
Scoring of the member to preset quantity quality inspection volume, calculates the scoring mass parameter of each scorer.The scoring matter of scorer if it exists
It measures parameter and is less than preset threshold, then screen out the scorer that scoring mass parameter is less than preset threshold in all target sample examination papers
The target sample examination paper commented.The target sample that the scorer of preset requirement is commented is not achieved since scoring mass parameter can be screened out
Examination paper, so that appraisal result is more accurate when the subsequent collection progress machine scoring according to calibration.
Content based on the above embodiment, as a kind of alternative embodiment, the characteristic parameter that scores includes each scorer pair
The scoring process parameter of preset quantity quality inspection volume, scoring process parameter are the scoring degree of correlation；Correspondingly, the embodiment of the present invention is not
To the scoring characteristic parameter rolled up according to each scorer to preset quantity quality inspection, screened out in all target sample examination papers discontented
The mode of the target sample examination paper of sufficient preset condition specifically limits.Referring to Fig. 3, including but not limited to:
301, preset quantity quality inspection is rolled up according to the machine scoring of preset quantity quality inspection volume and each scorer
Scoring, calculates the scoring degree of correlation between each scorer and machine appraisal result.
Wherein, the scoring degree of correlation can be the decimal between 0 to 1, for example Pearson correlation coefficient, the embodiment of the present invention pair
This is not especially limited.For example, if the machine scoring of preset quantity quality inspection volume is respectively 0,1,2,3,4,5.And for any
Scorer, the scoring which rolls up preset quantity quality inspection are respectively 1,3,4,5,4.Calculate Pearson came phase between the two
Relationship number is 0.8944.Wherein, machine scoring refers to after quality inspection volume is input to paper points-scoring system, by paper points-scoring system
Output obtains.
302, the corresponding scoring degree of correlation of scorer is less than preset threshold if it exists, then sieves in all target sample examination papers
Except the scoring degree of correlation is less than the target sample examination paper that the scorer of preset threshold is commented.
If the corresponding scoring degree of correlation of the scorer is less than preset threshold, show that the scoring of the scorer and machine are scored
Between concordance rate it is poor, so as in all target sample examination papers score the degree of correlation be less than preset threshold scorer commented
Target sample examination paper.It should be noted that the preset threshold that above and below is mentioned can be different specific values, it is practical
Value can be specifically set according to demand in implementation process, and the present invention is not especially limit this.
Method provided in an embodiment of the present invention passes through the machine scoring rolled up according to preset quantity quality inspection and each scoring
Scoring of the member to preset quantity quality inspection volume, calculates the scoring degree of correlation between each scorer and machine appraisal result.If depositing
It is less than preset threshold in the corresponding scoring degree of correlation of scorer, then screens out the scoring degree of correlation in all target sample examination papers and be less than
The target sample examination paper that the scorer of preset threshold is commented.The scoring that preset requirement is not achieved since scoring mass parameter can be screened out
The target sample examination paper that member is commented, so that appraisal result is more accurate when the subsequent collection progress machine scoring according to calibration.
Due to the consistent degree between the standards of grading of scorer and the standards of grading of expert, subsequent screen out equally can be used as
The basis of target sample examination paper, thus content based on the above embodiment, as a kind of alternative embodiment, according to target sample
Before examination paper updates calibration collection, target sample examination paper can also be further screened.The embodiment of the present invention is not to screening target sample
The method of examination paper makees specific limit.By taking any scorer as an example, referring to fig. 4, which includes but is not limited to:
401, specified target sample examination paper is determined in all target sample examination papers that the scorer is commented, and is concentrated in calibration
Determine specified sample examination paper；Wherein, the corresponding higher-dimension of target sample examination paper is specified to indicate that vector is corresponding with specified sample examination paper
Higher-dimension indicates the distance between vector recently.
Wherein, target sample examination paper and the corresponding higher-dimension of sample examination paper indicate that the calculation of vector can refer to above-mentioned implementation
The content of example, details are not described herein again.For the scorer, in the corresponding higher-dimension of target sample examination paper for obtaining the scorer and being commented
After indicating vector, can calculate the corresponding higher-dimension of each target sample examination paper indicates vector high dimension table corresponding with each sample examination paper
Show the distance between vector.Wherein, the distance the close, shows semantic closer.By comparing apart from size, one finally can determine
Examination paper namely specified target sample examination paper and specified sample examination paper, the two are examined in all target sample examination papers with all samples
In combination between volume, higher-dimension indicates that the distance between vector is nearest.
It should be noted that since the embodiment of the present invention is the scoring mark by standards of grading and expert based on scorer
Consistent degree between standard, to carry out screening out for target sample examination paper, and the judgement of standards of grading consistent degree, it needs with the two
Commented examination paper is premised on same or similar in semantic and content, so that the above process is to find the specified mesh for meeting the premise
This examination paper of standard specimen and specified sample examination paper.
If 402, the scorer to the scoring of specified target sample examination paper scale corresponding with specified sample examination paper it
Between difference be greater than preset threshold, then the target sample examination paper that the scorer is commented is screened out in all target sample examination papers.
Wherein, preset threshold can be percentage shared by score difference value, such as 10% or 20%, the embodiment of the present invention is to this
It is not especially limited.
Method provided in an embodiment of the present invention passes through all target samples commented in the scorer for any scorer
Specified target sample examination paper is determined in this examination paper, is concentrated in calibration and is determined specified sample examination paper.If the scorer is to specified target
Difference between the scoring scale corresponding with specified sample examination paper of sample examination paper is greater than preset threshold, then in all targets
The target sample examination paper that any scorer is commented is screened out in sample examination paper.Due to that can screen out and expert analysis mode standard is inconsistent comments
The target sample examination paper for dividing member to be commented, so that appraisal result is more accurate when the subsequent collection progress machine scoring according to calibration.
Since language proficiency is a content for needing to examine in examination paper scoring, and scorer is to the language embodied in examination paper
Say horizontal auditing capabilities and an investigation target, thus content based on the above embodiment, as a kind of optional implementation
Example can also further screen target sample examination paper before updating calibration collection according to target sample examination paper.The embodiment of the present invention
Specific limit is not made to the method for screening target sample examination paper.By taking any scorer as an example, referring to Fig. 5, above-mentioned screening technique packet
It includes but is not limited to:
501, the target sample examination paper commented the scorer re-starts language proficiency scoring.
Wherein, language proficiency scores and refers to excluding other examination items, the language water that only examination examination paper embodies emphatically
The scoring of examination paper when flat ability.The side of language proficiency scoring is re-started about the target sample examination paper commented the scorer
Formula, the present invention is not especially limit this, including but not limited to: being lower than either objective sample examination paper, by the target sample
The horizontal points-scoring system of this examination paper input language exports the language proficiency scoring of the target sample examination paper.Wherein, language proficiency scores
System is to be got based on calibration training, but lay particular emphasis on the relevant feature of concern language proficiency in training process (such as vocabulary makes
Link up degree and the degree of syntax error etc. between difficulty level, sentence smoothness degree, sentence), and ignore examination content phase
The feature (such as chapter length and theme) of pass.
If 502, the scoring degree of correlation between the scoring that language proficiency scoring and the scorer are commented is less than preset threshold,
The target sample examination paper that the scorer is commented is screened out in all target sample examination papers.
In 501, the language proficiency scoring for all target sample examination papers that the scorer is commented can be calculated.And the scoring
All target sample examination papers for being commented of member be it is known, so as to calculate the scoring degree of correlation between the two.Wherein, scoring is related
Degree can respectively be scored by the two forms the expression of the distance between vector, and the present invention is not especially limit this.If commenting
Divide the degree of correlation to be less than preset threshold, then shows that the standards of grading of the scorer do not meet the language proficiency standard generally recognized, from
And need to screen out target sample examination paper and scoring that the scorer is commented.It should be noted that marking examination papers for participating in current test
Scorer, each scorer is both needed to carry out that it marks examination papers screens out as procedure described above, and details are not described herein again.
Method provided in an embodiment of the present invention examines any scorer by the target sample commented the scorer
Volume re-starts language proficiency scoring.If the scoring degree of correlation between the scoring that language proficiency scoring and the scorer are commented is less than
Preset threshold then screens out the target sample examination paper that the scorer is commented in all target sample examination papers.Due to can be according to language
Level standard screens out target sample examination paper, so that appraisal result is more quasi- when the subsequent collection progress machine scoring according to calibration
Really.
Content based on the above embodiment is updating calibration collection according to target sample examination paper as a kind of alternative embodiment
Before, target sample examination paper can also further be screened.The embodiment of the present invention does not have the method work of screening target sample examination paper
Body limits.By taking any scorer as an example, referring to Fig. 6, above-mentioned screening technique includes but is not limited to:
601, the grading parameters of the scorer are determined.
Wherein, grading parameters can be a parameter, can also be divided into multiple parameters.Grading parameters can include at least with
Any one in lower four kinds of parameters, following four parameter are respectively score speed, the average mark of all scorings, all scorings
Mean square deviation and the scoring kurtosis of all scorings.
If 602, the grading parameters of the scorer are not in pre-set interval, this is screened out in all target sample examination papers and is commented
The target sample examination paper for dividing member to be commented.
It should be noted that if the scoring speed of the scorer then illustrates the scoring of the scorer not in pre-set interval
Standard has been likely to occur relatively large deviation, so as to screen out the target sample examination paper that the scorer is commented.In addition, examination paper is random point
Match, its average mark of the score of general the commented examination paper of scorer is relatively more fixed.If average mark is not in pre-set interval, same explanation
The standards of grading of scorer have been likely to occur relatively large deviation, and the mean square deviation and scoring kurtosis of all scorings are also as foundation is screened out
Similarly.
It should also be noted that, what above-mentioned grading parameters were directed to is all current test.And calibrating collection may require at any time
It updates, so that grading parameters are carried out in addition to the data according to current test when screening out target sample examination paper according to grading parameters
Except determination, it can also be determined according to the data of a period of time, the present invention is not especially limit this.For example,
It by taking the speed that scores as an example, can count under current test, the scoring speed of the scorer.The scoring of a period of time can also be counted
Speed, scoring speed such as daily, monthly or weekly.
Method provided in an embodiment of the present invention, for any scorer, by the grading parameters for determining the scorer.If should
The grading parameters of scorer then screen out the target sample that the scorer is commented not in pre-set interval in all target sample examination papers
This examination paper.Since target sample examination paper can be screened out according to grading parameters, so that subsequent collect progress machine scoring according to calibration
When, appraisal result is more accurate.
Since when the examination paper under to current test scores, the scoring distribution of each scorer is usually regular,
So as to according to scoring distribution there is regular this point further to screen target sample examination paper.Content based on the above embodiment,
Target sample can also be further screened before updating calibration collection according to target sample examination paper as a kind of alternative embodiment
Examination paper.The embodiment of the present invention does not make specific limit to the method for screening target sample examination paper.By taking any scorer as an example, referring to figure
7, above-mentioned screening technique includes but is not limited to:
701, it determines the scoring distribution of the commented scoring of the scorer, and determines the whole scoring of all commented scorings of scorer
Distribution.
With full marks for 5 points, for score value value range is 0 to 5, if the scorer scores 100 parts altogether, distribution of scoring can
To indicate as follows: 0 point 10 parts, 1 point 15 parts, 2 points 33 parts, 3 points 37 parts, 4 points 4 parts, 5 points 1 part.Scoring is distributed as
[0.1 0.15 0.33 0.37 0.04 0.01]
If 702, preset condition is unsatisfactory between scoring distribution and whole scoring distribution, in all target sample examination papers
Screen out the target sample examination paper that the scorer is commented.
Wherein, the measure of criterions such as Euclidean distance or KL distance can be used between scoring distribution and whole scoring distribution, preset
Condition can be value in pre-set interval, and the present invention is not especially limit this.It is with Measure Indexes between the two
For Euclidean distance, if Euclidean distance between the two not in pre-set interval, can sieve in all target sample examination papers
The target sample examination paper commented except the scorer.
Method provided in an embodiment of the present invention, for any scorer, by the grading parameters for determining the scorer.If should
The grading parameters of scorer then screen out the target sample that the scorer is commented not in pre-set interval in all target sample examination papers
This examination paper.Since target sample examination paper can be screened out according to grading parameters, so that subsequent collect progress machine scoring according to calibration
When, appraisal result is more accurate.
It should be noted that above-mentioned all alternative embodiments, can form optional implementation of the invention using any combination
Example, this is no longer going to repeat them.
Content based on the above embodiment, the embodiment of the invention provides a kind of calibrations to collect updating device, which is used for
Execute the calibration set update method in above method embodiment.Referring to Fig. 8, which includes: conversion module 801, cluster module
802 and update module 803；Wherein,
Conversion module 801 converts high dimension table for the content of text in each examination paper for obtaining the examination paper of current test
Show vector, and concentrate each sample examination paper to be converted into higher-dimension calibration to indicate vector；
Cluster module 802, for indicating that vector clusters to all higher-dimensions, according to the corresponding cluster result of each examination paper
The distance between calibration collection, calculates the corresponding target range of each examination paper；
Update module 803, for the examination paper using target range greater than preset threshold as target sample examination paper, according to target
Sample examination paper updates calibration collection.
Conversion module 801 is the paper to score to scorer in the examination paper for obtaining current test, the examination paper of current test.
Higher-dimension indicates that vector refers to after indicating examination paper content of text with vector, and the semanteme of examination paper semanteme is indicated in higher dimensional space
Vector.Wherein, the semantic difference between different examination papers is bigger, then the higher-dimension of different examination papers indicate vector in higher dimensional space away from
It is remoter from also.For any examination paper, the embodiment of the present invention is not converted into higher-dimension expression vector to by the content of text in the examination paper
Mode specifically limit, including but not limited to: converting term vector matrix for the content of text in each examination paper, examined each
Corresponding term vector Input matrix is rolled up to chapter grade deep learning model, the higher-dimension for exporting each examination paper indicates vector.
Through cluster module 802 after indicating that vector clusters to all higher-dimensions, each higher-dimension indicates that vector can be by
Gather for certain one kind.Namely in higher dimensional space, each higher-dimension indicates that vector can include by a type of cluster result.
Since the semantic difference between different examination papers is bigger, the higher-dimensions of different examination papers indicate vector in higher dimensional space away from
It is remoter from also, so that target range is greater than the examination paper of preset threshold, it can think that the semantic information of these examination papers is fixed currently
Mark is concentrated and is not included, and update module 803 can be using these examination papers as target sample examination paper, to carry out more to calibration collection
Newly.Concentrating mainly include due to calibration is sample examination paper and its corresponding scoring, to can examine target sample when updating
Volume and its corresponding scoring are added to calibration collection.Wherein, the corresponding scoring of target sample examination paper can be commented by scorer.
Device provided in an embodiment of the present invention, by obtaining the examination paper of current test, by the content of text in each examination paper
Being converted into higher-dimension indicates vector, and concentrates each sample examination paper to be converted into higher-dimension expression vector calibration.All higher-dimensions are indicated
Vector is clustered, and according to the distance between the corresponding cluster result of each examination paper and calibration collection, it is corresponding to calculate each examination paper
Target range.The examination paper that target range is greater than preset threshold updates according to target sample examination paper and determines as target sample examination paper
Mark collection.The examination paper that not scaled collection is covered in all examination papers due to can determine current test, and can be contained according to not scaled collection
The examination paper update calibration collection of lid, so that appraisal result is more accurate when the subsequent collection progress machine scoring according to calibration.
Fig. 9 illustrates the entity structure schematic diagram of a kind of electronic equipment, as shown in figure 9, the electronic equipment may include: place
Manage device (processor) 910, communication interface (Communications Interface) 920,930 He of memory (memory)
Communication bus 940, wherein processor 910, communication interface 920, memory 930 complete mutual lead to by communication bus 940
Letter.Processor 910 can call the logical order in memory 930, to execute following method: the examination paper of current test is obtained,
Converting higher-dimension for the content of text in each examination paper indicates vector, and concentrates each sample examination paper to be converted into high dimension table calibration
Show vector；To all higher-dimensions indicate vector cluster, according to the corresponding cluster result of each examination paper and calibration collection between away from
From calculating the corresponding target range of each examination paper；Target range is greater than the examination paper of preset threshold as target sample examination paper, root
Calibration collection is updated according to target sample examination paper.
In addition, the logical order in above-mentioned memory 930 can be realized by way of SFU software functional unit and conduct
Independent product when selling or using, can store in a computer readable storage medium.Based on this understanding, originally
Substantially the part of the part that contributes to existing technology or the technical solution can be in other words for the technical solution of invention
The form of software product embodies, which is stored in a storage medium, including some instructions to
So that a computer equipment (can be personal computer, electronic equipment or the network equipment etc.) executes each reality of the present invention
Apply all or part of the steps of a method.And storage medium above-mentioned include: USB flash disk, mobile hard disk, read-only memory (ROM,
Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic or disk etc. it is various
It can store the medium of program code.
The embodiment of the present invention also provides a kind of non-transient computer readable storage medium, is stored thereon with computer program,
The computer program is implemented to carry out the various embodiments described above offer method when being executed by processor, for example, obtain current
The examination paper of examination, converting higher-dimension for the content of text in each examination paper indicates vector, and each sample examination paper is concentrated in calibration
Being converted into higher-dimension indicates vector；Vector, which clusters, to be indicated to all higher-dimensions, according to the corresponding cluster result of each examination paper and is determined
The distance between mark collection, calculates the corresponding target range of each examination paper；Target range is greater than the examination paper of preset threshold as mesh
This examination paper of standard specimen updates calibration collection according to target sample examination paper.
The apparatus embodiments described above are merely exemplary, wherein described, unit can as illustrated by the separation member
It is physically separated with being or may not be, component shown as a unit may or may not be physics list
Member, it can it is in one place, or may be distributed over multiple network units.It can be selected according to the actual needs
In some or all of the modules achieve the purpose of the solution of this embodiment.Those of ordinary skill in the art are not paying creativeness
Labour in the case where, it can understand and implement.
Through the above description of the embodiments, those skilled in the art can be understood that each embodiment can
It realizes by means of software and necessary general hardware platform, naturally it is also possible to pass through hardware.Based on this understanding, on
Stating technical solution, substantially the part that contributes to existing technology can be embodied in the form of software products in other words, should
Computer software product may be stored in a computer readable storage medium, such as ROM/RAM, magnetic disk, CD, including several fingers
It enables and using so that a computer equipment (can be personal computer, server or the network equipment etc.) executes each implementation
Method described in certain parts of example or embodiment.
Finally, it should be noted that the above embodiments are merely illustrative of the technical solutions of the present invention, rather than its limitations；Although
Present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that: it still may be used
To modify the technical solutions described in the foregoing embodiments or equivalent replacement of some of the technical features；
And these are modified or replaceed, technical solution of various embodiments of the present invention that it does not separate the essence of the corresponding technical solution spirit and
Range.
Claims (13)
1. a kind of calibration set update method characterized by comprising
The examination paper for obtaining current test, converting higher-dimension for the content of text in each examination paper indicates vector, and calibration is concentrated
Each sample examination paper, which is converted into higher-dimension, indicates vector；
To all higher-dimensions indicate vector cluster, according to the corresponding cluster result of each examination paper and it is described calibration collection between away from
From calculating the corresponding target range of each examination paper；
Examination paper using target range greater than preset threshold is as target sample examination paper, according to target sample examination paper update
Calibration collection.
2. the method according to claim 1, wherein including preset quantity matter in all target sample examination papers
Inspection volume, the higher-dimension indicate that vector is the semantic vector of full piece level；It is described described fixed according to target sample examination paper update
Before mark collection, further includes:
It is rolled up according to the preset quantity quality inspection, the target sample for being unsatisfactory for preset condition is screened out in all target sample examination papers
Examination paper.
3. according to the method described in claim 2, it is characterized in that, each target sample examination paper is scored by a scorer；
Correspondingly, described to be rolled up according to the preset quantity quality inspection, it is screened out in all target sample examination papers and is unsatisfactory for preset condition mesh
This examination paper of standard specimen, comprising:
According to the scoring characteristic parameter that each scorer rolls up the preset quantity quality inspection, sieved in all target sample examination papers
Except the target sample examination paper for being unsatisfactory for preset condition；Wherein, the scoring characteristic parameter includes each scorer to described default
The scoring mass parameter of quantity quality inspection volume and/or each scorer join the scoring process that the preset quantity quality inspection is rolled up
Number.
4. according to the method described in claim 3, it is characterized in that, the scoring characteristic parameter includes each scorer to described
The scoring mass parameter of preset quantity quality inspection volume；Correspondingly, it is described according to each scorer to the preset quantity quality inspection
The scoring characteristic parameter of volume screens out the target sample examination paper for being unsatisfactory for preset condition in all target sample examination papers, comprising:
The preset quantity quality inspection is rolled up according to the scale of preset quantity quality inspection volume and each scorer
Scoring, calculates the scoring mass parameter of each scorer；
The scoring mass parameter of scorer is less than preset threshold if it exists, then screens out scoring quality in all target sample examination papers
Parameter is less than the target sample examination paper that the scorer of the preset threshold is commented.
5. according to the method described in claim 3, it is characterized in that, the scoring characteristic parameter includes each scorer to described
The scoring process parameter of preset quantity quality inspection volume, the scoring process parameter are the scoring degree of correlation；Correspondingly, the basis is every
The scoring characteristic parameter that one scorer rolls up the preset quantity quality inspection, screened out in all target sample examination papers be unsatisfactory for it is pre-
If the target sample examination paper of condition, comprising:
The preset quantity quality inspection is rolled up according to the machine scoring of preset quantity quality inspection volume and each scorer
Scoring, calculates the scoring degree of correlation between each scorer and machine appraisal result；
The corresponding scoring degree of correlation of scorer is less than preset threshold if it exists, then screens out scoring phase in all target sample examination papers
Guan Du is less than the target sample examination paper that the scorer of the preset threshold is commented.
6. the method according to claim 1, wherein each target sample examination paper is scored by a scorer,
Before the calibration collection according to target sample examination paper update, further includes:
For any scorer, determine that specified target sample is examined in all target sample examination papers that any scorer is commented
Volume is concentrated in the calibration and determines specified sample examination paper；Wherein, the corresponding higher-dimension of the specified target sample examination paper indicates vector
Higher-dimension corresponding with the specified sample examination paper indicates the distance between vector recently；
If any scorer is to the scoring of specified target sample examination paper standard corresponding with the specified sample examination paper
Difference between scoring is greater than preset threshold, then the target that any scorer is commented is screened out in all target sample examination papers
Sample examination paper.
7. the method according to claim 1, wherein each target sample examination paper is scored by a scorer,
Before the calibration collection according to target sample examination paper update, further includes:
For any scorer, the target sample examination paper commented any scorer re-starts language proficiency scoring；
If the scoring degree of correlation between the scoring that the language proficiency scoring and any scorer are commented is less than preset threshold,
The target sample examination paper that any scorer is commented then is screened out in all target sample examination papers.
8. the method according to claim 1, wherein each target sample examination paper is scored by a scorer,
Before the calibration collection according to target sample examination paper update, further includes:
For any scorer, the grading parameters of any scorer are determined；
If the grading parameters of any scorer not in pre-set interval, screen out described appoint in all target sample examination papers
The target sample examination paper that one scorer is commented.
9. according to the method described in claim 8, it is characterized in that, the grading parameters include at least in following four parameter
Any one, the following four parameter is respectively score speed, the average mark of all scorings, the mean square deviation of all scorings and institute
There is the scoring kurtosis of scoring.
10. the method according to claim 1, wherein each target sample examination paper is scored by a scorer,
Before the calibration collection according to target sample examination paper update, further includes:
For any scorer, the scoring distribution of the commented scoring of any scorer is determined, and determine that all scorers are commented
The whole scoring distribution of scoring；If preset condition is unsatisfactory between the scoring distribution and the whole scoring distribution, in institute
Have and screens out the target sample examination paper that any scorer is commented in target sample examination paper.
11. a kind of calibration collects updating device characterized by comprising
Conversion module, for obtaining the examination paper of current test, converting higher-dimension for the content of text in each examination paper indicates vector,
And each sample examination paper is concentrated to be converted into higher-dimension expression vector calibration；
Cluster module, for indicating that vector clusters to all higher-dimensions, according to the corresponding cluster result of each examination paper and calibration
The distance between collection, calculates the corresponding target range of each examination paper；
Update module, for the examination paper using target range greater than preset threshold as target sample examination paper, according to the target sample
This examination paper updates the calibration collection.
12. a kind of electronic equipment characterized by comprising
At least one processor；And
At least one processor being connect with the processor communication, in which:
The memory is stored with the program instruction that can be executed by the processor, and the processor calls described program to instruct energy
Enough methods executed as described in claims 1 to 10 is any.
13. a kind of non-transient computer readable storage medium, which is characterized in that the non-transient computer readable storage medium is deposited
Computer instruction is stored up, the computer instruction makes the computer execute the method as described in claims 1 to 10 is any.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811230610.5A CN109408638B (en) | 2018-10-22 | 2018-10-22 | Calibration set updating method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811230610.5A CN109408638B (en) | 2018-10-22 | 2018-10-22 | Calibration set updating method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109408638A true CN109408638A (en) | 2019-03-01 |
| CN109408638B CN109408638B (en) | 2021-04-30 |
Family
ID=65468701
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811230610.5A Active CN109408638B (en) | 2018-10-22 | 2018-10-22 | Calibration set updating method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109408638B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111931486A (en) * | 2020-08-12 | 2020-11-13 | 广东讯飞启明科技发展有限公司 | Method and device for screening sample rolls for manual grading |
| CN117437094A (en) * | 2023-08-04 | 2024-01-23 | 北京信诺软通信息技术有限公司 | Answer sheet classification method, electronic equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104331498A (en) * | 2014-11-19 | 2015-02-04 | 亚信科技（南京）有限公司 | Method for automatically classifying webpage content visited by Internet users |
| CN107679069A (en) * | 2017-08-18 | 2018-02-09 | 国家计算机网络与信息安全管理中心 | Method is found based on a kind of special group of news data and related commentary information |
| CN107992596A (en) * | 2017-12-12 | 2018-05-04 | 百度在线网络技术（北京）有限公司 | A kind of Text Clustering Method, device, server and storage medium |
| CN108446277A (en) * | 2018-03-27 | 2018-08-24 | 北京大前科技有限责任公司 | The method and device of simulation learning |
-
2018
- 2018-10-22 CN CN201811230610.5A patent/CN109408638B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104331498A (en) * | 2014-11-19 | 2015-02-04 | 亚信科技（南京）有限公司 | Method for automatically classifying webpage content visited by Internet users |
| CN107679069A (en) * | 2017-08-18 | 2018-02-09 | 国家计算机网络与信息安全管理中心 | Method is found based on a kind of special group of news data and related commentary information |
| CN107992596A (en) * | 2017-12-12 | 2018-05-04 | 百度在线网络技术（北京）有限公司 | A kind of Text Clustering Method, device, server and storage medium |
| CN108446277A (en) * | 2018-03-27 | 2018-08-24 | 北京大前科技有限责任公司 | The method and device of simulation learning |
Non-Patent Citations (1)
| Title |
|---|
| 李学俊 西南科技大学: "基于人工智能的试卷自动评分系统研究与实现", 《基于人工智能的试卷自动评分系统研究与实现》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111931486A (en) * | 2020-08-12 | 2020-11-13 | 广东讯飞启明科技发展有限公司 | Method and device for screening sample rolls for manual grading |
| CN117437094A (en) * | 2023-08-04 | 2024-01-23 | 北京信诺软通信息技术有限公司 | Answer sheet classification method, electronic equipment and storage medium |
| CN117437094B (en) * | 2023-08-04 | 2024-05-07 | 北京信诺软通信息技术有限公司 | Answer sheet classification method, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109408638B (en) | 2021-04-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105606499B (en) | Suspended particulate matter mass concentration real-time detection device, and measuring method | |
| CN104166731B (en) | A kind of overlapping community discovery system and method for social networks | |
| CN111383210B (en) | Fundus image classification model training method and device | |
| CN112948453B (en) | System for paper-out based on question difficulty | |
| CN115063653B (en) | Method and system for measuring dominant color of built environment based on brain sensitive data of image | |
| CN108648766A (en) | Speech evaluating method and system | |
| CN111174370A (en) | Fault detection method and device, storage medium and electronic device | |
| CN109408638A (en) | Calibrate set update method and device | |
| CN110110610A (en) | A kind of event detecting method for short-sighted frequency | |
| CN107819810A (en) | Adaptive planning formulates system | |
| CN111476307A (en) | Lithium battery surface defect detection method based on depth field adaptation | |
| CN113269737A (en) | Method and system for calculating diameter of artery and vein of retina | |
| CN110082106B (en) | Bearing fault diagnosis method based on Yu norm deep measurement learning | |
| CN115563484A (en) | Street greening quality detection method based on physiological awakening identification | |
| CN109948569A (en) | A kind of three-dimensional hybrid expression recognition method using particle filter frame | |
| CN113392851A (en) | Intelligent discrimination method and device for black tea fermentation degree | |
| CN108734221A (en) | The improvement density peaks clustering method and system automatically determined based on subclass numbers | |
| CN117348831A (en) | Picture adjustment method and system for liquid crystal display screen | |
| CN112202630A (en) | Network quality abnormity detection method and device based on unsupervised model | |
| CN115169751B (en) | Method and system for improving maintenance quality of traditional Chinese medicine decoction pieces | |
| CN110706808A (en) | Aneurysm rupture state prediction method and device | |
| CN110321787A (en) | Disease recognition method, system and the storage medium indicated based on joint sparse | |
| CN113889274B (en) | Method and device for constructing risk prediction model of autism spectrum disorder | |
| CN109978406A (en) | A kind of method and system of security downside risks assessment diagnosis | |
| CN109816028A (en) | A kind of unbalanced dataset classification Model Fusion method of Partial Feature migration |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |