WO2022213466A1 - 识别结果的智能校对方法、装置、设备及存储介质 - Google Patents
识别结果的智能校对方法、装置、设备及存储介质 Download PDFInfo
- Publication number
- WO2022213466A1 WO2022213466A1 PCT/CN2021/097162 CN2021097162W WO2022213466A1 WO 2022213466 A1 WO2022213466 A1 WO 2022213466A1 CN 2021097162 W CN2021097162 W CN 2021097162W WO 2022213466 A1 WO2022213466 A1 WO 2022213466A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- proofreading
- result
- recognition result
- recognition
- correct
- Prior art date
Links
- 230000001915 proofreading effect Effects 0.000 title claims abstract description 393
- 238000000034 method Methods 0.000 title claims abstract description 39
- 238000004590 computer program Methods 0.000 claims description 11
- 238000012795 verification Methods 0.000 claims description 10
- 230000009471 action Effects 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 3
- 230000006399 behavior Effects 0.000 description 7
- 238000013473 artificial intelligence Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000008447 perception Effects 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000010408 sweeping Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
Definitions
- the present application belongs to the technical field of artificial intelligence recognition, and in particular relates to an intelligent proofreading method, device, device and storage medium for recognition results.
- AI artificial intelligence Intelligence
- AI technology is widely used in the recognition of various scenarios, such as image text recognition, intelligent translation, intent recognition and other scenarios.
- the proofreading system distributes the content identified by AI to users for manual verification, and the manual judges whether the content identified by AI is accurate; if the content identified by AI is judged to be accurate by manual verification, it will be directly submitted to the proofreading system; if the content identified by AI is judged by manual verification If it is not accurate, the recognition results are manually modified and submitted to the proofreading system.
- the inventor found that there will be cheating loopholes in this way. Some users do not manually check the content recognized by the AI received, and they default to "correct” and directly submit “brainless”; The modified AI-identified content is directly submitted to the proofreading system without modification, and the proofreading system cannot detect such cheating behavior, which eventually leads to a low proofreading accuracy rate of the overall proofreading system.
- the purpose of the present application is to provide an intelligent proofreading method, device, device and storage medium for identification results, which can detect cheating users and effectively improve the accuracy of intelligent proofreading of identification results.
- the present application provides an intelligent proofreading method for recognition results, including:
- the proofreading system takes the recognition result as a proofreading task and sends it to the client for the client to proofread the recognition result;
- the target recognition result includes the first recognition result and the second recognition result; wherein, all the The second recognition result is a preset wrong recognition result, and the corresponding correct recognition result is stored in the proofreading system;
- the proofreading system receives the proofreading result of the target recognition result from the client, and judges that the proofreading result belongs to The proofreading result of the first recognition result is still the proofreading result of the second recognition result; if the proofreading result belongs to the proofreading result of the second recognition result, then the proofreading result is compared with the corresponding correct recognition result stored in advance, and the proofreading is judged. Whether the result is consistent with the correct identification result; if consistent, the proofreading result of the second identification result is used as the final AI identification result; if inconsistent, a warning message is sent to the client and/or a penalty operation is performed.
- the present application provides an intelligent proofreading device for recognition results, including:
- a sending module is used to obtain a target recognition result, and send the recognition result as a proofreading task to the client, so that the client can proofread the recognition result;
- the target recognition result includes a first recognition result and a second recognition result;
- the second recognition result is a preset wrong recognition result, and the corresponding correct recognition result is stored in the sending module;
- the receiving and judging module is used for receiving the proofreading result of the target recognition result from the client, And judge that the proofreading result belongs to the proofreading result of the first recognition result or belongs to the proofreading result of the second recognition result; if the proofreading result belongs to the proofreading result of the second recognition result, then the proofreading result and the pre-stored corresponding correct identification
- the results are compared to determine whether the proofreading result is consistent with the correct recognition result; if it is consistent, the proofreading result of the second recognition result is used as the final AI recognition result; if it is inconsistent, a warning message and/or a warning message is sent to the client. Perform punishment action.
- the present application provides an electronic device, the electronic device includes a processor and a memory, and the processor is configured to execute a computer program stored in the memory to implement the intelligent proofreading method for the recognition result.
- the present application provides a computer-readable storage medium, where the computer-readable storage medium stores at least one instruction, and when the at least one instruction is executed by a processor, implements the intelligent proofreading method for the recognition result.
- the present application provides an intelligent proofreading method, device, equipment and storage medium for recognition results, which are used to intelligently proofread the recognition results of artificial intelligence recognition; the present application first obtains the target recognition results, takes the recognition results as proofreading tasks, and sends to the client, so that the client can check the recognition result; the target recognition result includes a first recognition result and a second recognition result; wherein, the second recognition result is a preset wrong recognition result, and the proofreading system The corresponding correct recognition result is stored; through the proofreading result of the second recognition result returned from the client, it can be judged whether the client is submitting "brainless", and when the proofreading result submitted by the client for the second recognition result is inconsistent , the proofreading system will send warning information to the client and/or execute punishment operation to remind the client and take relative punishment measures to improve the accuracy of the client’s proofreading; the second recognition result of this application is the same as the real first recognition result , the client will not perceive that it is being monitored, and the proofreading system can recognize the cheating behavior of the client without perception,
- Fig. 1 is the flow chart of the intelligent proofreading method of a kind of recognition result of the application
- FIG. 2 is a structural block diagram of an intelligent proofreading device for a recognition result of the application
- FIG. 3 is a structural block diagram of an electronic device of the present application.
- the present application provides an intelligent proofreading method for recognition results, including:
- the proofreading system uses the recognition result as a proofreading task and sends it to the client, so that the client can proofread the recognition result;
- the target recognition result includes the first recognition result and the second recognition result; wherein , the second recognition result is a preset wrong recognition result, and the corresponding correct recognition result is stored in the proofreading system;
- the target recognition result is the AI recognition result, including the identity information of the recognized object and the result information of the AI recognition; the recognized object is sound, picture, text, translated text, etc.; the proofreading task includes the first recognition result and the second recognition result.
- the first recognition result is a normal proofreading task, and the proofreading system does not know whether the result information of the recognition result is correct or not, and the client needs to perform real proofreading.
- the second identification result is an error identification result distributed to the user in order to verify whether the user has actually participated in the verification. If the user has verified the error information, it means that the user has participated in the verification work. If the user has not verified the error information, it means The user may not participate in the proofreading work, but simply skip the proofreading.
- the proofreading system receives the proofreading result of the target recognition result from the client, and judges that the proofreading result belongs to the proofreading result of the first recognition result or the proofreading result of the second recognition result;
- the first identification result includes first identification information representing its type, and the second identification result includes second identification information indicating its type; the proofreading result of the first identification result includes the first identification information, and the proofreading result of the second identification result includes the first identification information.
- Including second identification information The first identification information and the second identification information are invisible to the client, and the proofreading system can know the first identification information and the second identification information, and use this to judge whether the received proofreading result belongs to the proofreading result of the first identification result or belongs to the The proofreading result of the second recognition result.
- the proofreading system calls the next second recognition result and sends it to the client for proofreading again, until the proofreading system determines that the received proofreading result of the second recognition result fed back by the client is the same as the previous proofreading result.
- the stored correct recognition results are consistent, and the correct proofreading result is used as the final recognition result, and the next proofreading task is performed in sequence.
- the proofreading system determines that the proofreading result of the second recognition result submitted by the client is inconsistent with the stored correct recognition result, it will randomly select another second recognition result from the second recognition result library and distribute it to the client for continuous proofreading until proofreading.
- the system determines that the proofreading result of the second recognition result fed back by the client is consistent with the stored correct recognition result; If it is inconsistent, the proofreading system will suspend the distribution of proofreading tasks to the client.
- the correct recognition result of the second recognition result is pre-stored in the proofreading system, so from the proofreading result returned by the client, it can be judged whether the client is submitting "brainless", and the proofreading result of the second recognition result submitted by the client is correct and stored.
- the proofreading system will issue an alarm message to remind the client and take relative punishment measures to improve the accuracy of the client’s proofreading;
- the second identification result of this application is the same as the real first identification result, the client It does not perceive that it is being monitored, and the proofreading system can identify the cheating behavior of the client without perception, and through this method, the client can be kept in a state of serious proofreading, so that the proofreading system can obtain valuable proofreading results to the greatest extent, effectively The improvement improves the intelligent proofreading accuracy of the recognition results.
- the present application provides an intelligent proofreading method for recognition results, including:
- the proofreading system uses the recognition result as a proofreading task and sends it to the client for the client to proofread the recognition result;
- the target recognition result includes the first recognition result and the second recognition result;
- the second recognition result is a preset wrong recognition result, and the corresponding correct recognition result is stored in the proofreading system;
- the target recognition result is the AI recognition result, including the identity information of the recognized object and the result information of the AI recognition; wherein , the recognition objects are sounds, pictures, words, translated texts, etc.;
- the proofreading task includes the first recognition result and the second recognition result.
- the first recognition result is a normal proofreading task, and the proofreading system does not know whether the result information of the recognition result is correct or not, and the client needs to perform real proofreading.
- the second identification result is an error identification result distributed to the user in order to verify whether the user has actually participated in the verification. If the user has verified the error information, it means that the user has participated in the verification work. If the user has not verified the error information, it means The user may not participate in the proofreading work, but simply skip the proofreading.
- the second recognition result (“fishing” topic): select clear and identifiable pictures, and fill in the algorithm recognition results and correct recognition results for the pictures.
- the algorithm recognition results are the recognition results that the client can see.
- the correct identification results are consistent with the picture information and stored in the proofreading system.
- the recognition object in the second recognition result is a red light picture
- the correct AI recognition result information is "red light” and stored in the proofreading system
- the AI recognition result information that is deliberately set to be wrong in the second recognition result is "green light” ”, sent to the client, forcing the client to proofread and modify the AI recognition result information.
- the proofreading system is known to be correct, so the results submitted by the client will not be cross-checked. If the proofreading system judges that the proofreading result of the second recognition result submitted by the client is inconsistent with the stored correct recognition result (the most common way is to submit it directly), the proofreading system will send a warning message to the client and/or execute a penalty operation, At the same time, it is believed that the user is suspected of cheating. On the contrary, as long as the proofreading system judges that the proofreading result of the second recognition result submitted by the client is consistent with the stored correct recognition result, corresponding reward points will be obtained.
- the first area in the client terminal displays the identity information of the recognition object, and the second area displays the result information of AI recognition; and adjusts the result information displayed in the second area according to the identity information of the recognition object displayed in the first area to obtain Proofreading the result; and sending the proofreading result to the proofreading system; the proofreading system receives the proofreading result of the target recognition result from the client, and judges that the proofreading result belongs to the proofreading result of the first recognition result or belongs to the second recognition result proofreading results.
- the client determines that the result information displayed in the second area is correct, it will be submitted directly without adjustment; at this time, the proofreading result is consistent with the proofreading task content;
- the client determines that the result information displayed in the second area is incorrect, it adjusts the result information in the second display area, and saves and obtains the proofreading result; at this time, the proofreading result is different from the original proofreading task content, including the identity information of the identification object and the adjusted result information.
- the proofreading system judges that the received proofreading result belongs to the proofreading result of the first recognition result, the proofreading result is regarded as the final AI recognition result, and the points are recorded, and then the next proofreading task is sent to the client; if the proofreading system judges The received proofreading result belongs to the proofreading result of the second recognition result, then compare the proofreading result with the corresponding correct recognition result stored in advance, and judge whether the proofreading result is consistent with the described correct recognition result; The proofreading result of the second recognition result is recorded as the final AI recognition result, and the next proofreading task is issued to the client; if it is inconsistent, a warning message is sent to the client and the next second proofreading task is issued to the client until proofreading When the system determines that the proofreading result of the second recognition result received from the client is consistent with the correct recognition result, the proofreading result is regarded as the final AI recognition result, the points are recorded, and the next proofreading task is issued to the client.
- the present application provides an intelligent proofreading method for identification results, which, on the basis of Embodiment 1 or 2, further includes the step of establishing a second identification result library.
- the proofreading system selects some target recognition results from the target recognition results to be proofread, and modifies the recognition results so that the recognition results are obviously wrong, and forms the second recognition results; all the second recognition results are stored in the second recognition results.
- the library it is used for proofreading system calls.
- the proofreading system can design the number of second identification results distributed therein according to the total number of target identification results to form a second identification result library; for details, see Table 1:
- the proofreading system determines that the proofreading result of the second recognition result submitted by the client is inconsistent with the stored correct recognition result, it will randomly select another second recognition result from the second recognition result library and distribute it to the client for continuous proofreading until proofreading.
- the system determines that the proofreading result of the second recognition result fed back by the client is consistent with the stored correct recognition result; If it is inconsistent, the proofreading system will suspend the distribution of proofreading tasks to the client.
- the correct recognition result of the second recognition result is pre-stored in the proofreading system, so from the proofreading result returned by the client, it can be judged whether the client is submitting "brainless", and the proofreading result of the second recognition result submitted by the client is correct and stored.
- the proofreading system will issue an alarm message to remind the client and take relative punishment measures to improve the accuracy of the client’s proofreading;
- the second identification result of this application is the same as the real first identification result, the client It does not perceive that it is being monitored, and the proofreading system can identify the cheating behavior of the client without perception, and through this method, the client can be kept in a state of serious proofreading, so that the proofreading system can obtain valuable proofreading results to the greatest extent, effectively The improvement improves the intelligent proofreading accuracy of the recognition results.
- the present application provides an intelligent proofreading method for a recognition result, which, on the basis of Embodiment 1 or 2, further includes the steps of judging and punishing the second recognition result.
- the proofreading system counts the number of errors in which the proofreading result of the second recognition result is inconsistent with the correct recognition result, as well as the consecutive number values that are inconsistent with each other, as a condition for punishing the operation;
- the error number value is 1, and the continuous number value is 0;
- the continuous quantity value is incremented by 1 in turn, and at the same time the continuous quantity value is incremented by 1; until when the proofreading result of the second recognition result is consistent with the correct recognition result, the wrong quantity value remains unchanged, and the continuous quantity value is 0.
- the proofreading system judges that the proofreading result of the second recognition result submitted by the client is inconsistent with the stored correct recognition result
- the rule is: after the proofreading system compares the proofreading result of the second recognition result with the stored correct recognition result each time, it calculates the incorrect quantity value K and the continuous quantity value Q of the client; when the proofreading system judges that the comparison is inconsistent, the incorrect quantity value Add 1 to K, and add 1 to the continuous quantity value Q; when the proofreading system judges that the comparison is consistent, the error quantity value K remains unchanged, and the continuous quantity value Q is cleared to zero.
- the two dimensions constitute the record of the client's scoring behavior, which is used to distinguish the machine scoring behavior from the artificial scoring behavior, so as to exclude bad clients from the proofreading system in the shortest time and give different punishment measures accordingly.
- the proofreading system compares the proofreading result of any second recognition result of the client with the stored correct recognition result, the next proofreading task issued by the proofreading system will also be the second recognition result. At this time:
- the proofreading system calculates the error quantity value K and the continuous quantity value Q of the client terminal each time after comparing the proofreading result of the second recognition result of the client with the stored correct recognition result.
- the size of the corresponding first penalty coefficient is set according to the size of the final error quantity value, and the size of the corresponding second penalty coefficient is set according to the size of the continuous quantity value;
- the size of the first penalty coefficient and the second penalty coefficient, the larger of the two is used as the effective value to calculate the penalty multiple; when the first penalty coefficient and/or the second penalty coefficient reaches the preset threshold, according to the preset additional penalty Rules penalize users.
- the present application provides an intelligent proofreading device for recognition results, including:
- the sending module is used to obtain the target recognition result, and the proofreading system takes the recognition result as a proofreading task and sends it to the client, so that the client can proofread the recognition result;
- the target recognition result includes the first recognition result and the second recognition result Result; wherein, the second recognition result is a preset wrong recognition result, and a corresponding correct recognition result is stored in the proofreading system;
- Receiving and judging module used for receiving the proofreading result of the target recognition result from the client, and judging whether the proofreading result belongs to the proofreading result of the first recognition result or the proofreading result of the second recognition result;
- the result belongs to the proofreading result of the second recognition result, then compare the proofreading result with the corresponding correct recognition result stored in advance, and judge whether the proofreading result is consistent with the correct recognition result;
- the proofreading result is used as the final AI recognition result; if it is inconsistent, a warning message will be sent to the client and/or a penalty action will be performed.
- the receiving and judging module is also used for: calculating the error quantity value K and the continuous quantity value Q of the client terminal after each time comparing the proofreading result of the second second identification result of the client terminal with the stored correct identification result; 2.
- the error quantity value K is incremented by 1
- the continuous quantity value Q is incremented by 1
- the quantity value Q is cleared to zero;
- the present application further provides an electronic device 100 for an intelligent proofreading method for recognition results;
- the electronic device 100 includes a memory 101, at least one processor 102, which is stored in the memory 101 and can be accessed at any location.
- the computer program 103 running on the at least one processor 102 and the at least one communication bus 104.
- the memory 101 can be used to store the computer program 103, and the processor 102 implements any one of Embodiments 1 to 4 by running or executing the computer program stored in the memory 101 and calling the data stored in the memory 101 The method steps of the intelligent proofreading method of the recognition result.
- the memory 101 may mainly include a stored program area and a stored data area, wherein the stored program area may store an operating system, an application program required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the storage data area may Data such as audio data and the like created according to the use of the electronic device 100 are stored.
- the memory 101 may include non-volatile memory such as hard disk, internal memory, plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card (Flash Card), At least one disk storage device, flash memory device, or other non-volatile solid state storage device, may also be volatile.
- non-volatile memory such as hard disk, internal memory, plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card (Flash Card), At least one disk storage device, flash memory device, or other non-volatile solid state storage device, may also be volatile.
- the at least one processor 102 may be a central processing unit (Central Processing Unit, CPU), and may also be other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application-specific integrated circuits (Application Specific Integrated Circuit, ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
- the processor 102 can be a microprocessor or the processor 102 can also be any conventional processor, etc.
- the processor 102 is the control center of the electronic device 100, and uses various interfaces and lines to connect the entire electronic device 100. various parts.
- the memory 101 in the electronic device 100 stores multiple instructions to implement a coupled multi-task feature extraction method, and the processor 102 can execute the multiple instructions to implement:
- the proofreading system takes the recognition result as a proofreading task and sends it to the client for the client to proofread the recognition result;
- the target recognition result includes the first recognition result and the second recognition result; wherein, all the The second recognition result is a preset wrong recognition result, and the corresponding correct recognition result is stored in the proofreading system;
- the proofreading system receives the proofreading result of the target recognition result from the client, and judges that the proofreading result belongs to the proofreading result of the first recognition result or the proofreading result of the second recognition result;
- proofreading result belongs to the proofreading result of the second recognition result, then compare the proofreading result with the corresponding correct recognition result stored in advance, and judge whether the proofreading result is consistent with the correct recognition result;
- the proofreading result of the second recognition result is used as the final AI recognition result
- modules/units integrated in the electronic device 100 are implemented in the form of software functional units and sold or used as independent products, they may be stored in a computer-readable storage medium.
- the present application can implement all or part of the processes in the methods of the above embodiments, and can also be completed by instructing the relevant hardware through a computer program.
- the computer program can be stored in a computer-readable storage medium, and the computer When the program is executed by the processor, the steps of the foregoing method embodiments can be implemented.
- the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file or some intermediate form, and the like.
- the computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, removable hard disk, magnetic disk, optical disk, computer memory and read-only memory (ROM, Read-Only Memory) .
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
一种识别结果的智能校对方法、装置、设备及存储介质，所述方法包括：获取目标识别结果，校对系统将所述识别结果作为校对任务，发送给客户端，以供客户端对识别结果进行校对；所述目标识别结果包括第一识别结果和第二识别结果；校对系统接收来自所述客户端的对所述目标识别结果的校对结果，并判断；若所述校对结果属于第二识别结果的校对结果，则将该校对结果与预先存储的对应正确识别结果进行比对；若一致，则将该第二识别结果的校对结果作为最终的AI识别结果；若不一致，则向客户端发送警示信息和/或执行惩罚操作。所述方法能够无感知地识别客户端的作弊行为，提升识别结果的校对准确率。
Description
本申请要求2021年4月8日提交中国专利局、申请号为202110379808.5，发明名称为“识别结果的智能校对方法、装置、设备及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
本申请属于人工智能识别技术领域，特别涉及一种识别结果的智能校对方法、装置、设备及存储介质。
人工智能（Artificial
Intelligence），英文缩写为AI。它是研究、开发用于模拟、延伸和扩展人的智能的理论、方法、技术及应用系统的一门新的技术科学。
如今AI技术被广泛应用在各类场景的识别当中，如图片文字识别、智能翻译、意图识别等场景。发明人发现，AI识别出来的结果在准确性上仍有所不足，所以在一些对准确率要求高的场景下需要专门的校对人员及校对系统，对AI识别的内容进行人工核查与校对。
校对系统将AI识别的内容分发给用户进行人工核查，由人工进行判断AI识别的内容是否准确；如果人工核查判断AI识别的内容准确，则直接提交给校对系统；如果人工核查判断AI识别的内容不准确，则人工对识别结果进行修改后提交给校对系统。发明人发现，这样就会存在作弊漏洞，有些用户不对收到的AI识别的内容进行人工核对，都默认“正确”直接“无脑”进行提交；这种作弊行为，就会将大量原本需要进行修改的AI识别内容未修改直接提交给校对系统，而校对系统无法检测出这种作弊行为，最终导致校对系统整体的校对准确率不高。
本申请的目的在于提供一种识别结果的智能校对方法、装置、设备及存储介质，能够发现作弊用户，有效的提升识别结果的智能校对准确率。
为了实现上述目的，本申请采用如下技术方案：
第一方面，本申请提供一种识别结果的智能校对方法，包括：
获取目标识别结果，校对系统将所述识别结果作为校对任务，发送给客户端，以供客户端对识别结果进行校对；所述目标识别结果包括第一识别结果和第二识别结果；其中，所述第二识别结果为预设的错误识别结果，且校对系统中存储有对应的正确识别结果；校对系统接收来自所述客户端的对所述目标识别结果的校对结果，并判断所述校对结果属于第一识别结果的校对结果还是属于第二识别结果的校对结果；若所述校对结果属于第二识别结果的校对结果，则将该校对结果与预先存储的对应正确识别结果进行比对，判断校对结果是否与所述正确识别结果一致；若一致，则将该第二识别结果的校对结果作为最终的AI识别结果；若不一致，则向客户端发送警示信息和/或执行惩罚操作。
第二方面，本申请提供一种识别结果的智能校对装置，包括：
发送模块，用于获取目标识别结果，将所述识别结果作为校对任务，发送给客户端，以供客户端对识别结果进行校对；所述目标识别结果包括第一识别结果和第二识别结果；其中，所述第二识别结果为预设的错误识别结果，且发送模块中存储有对应的正确识别结果；接收判断模块，用于接收来自所述客户端的对所述目标识别结果的校对结果，并判断所述校对结果属于第一识别结果的校对结果还是属于第二识别结果的校对结果；若所述校对结果属于第二识别结果的校对结果，则将该校对结果与预先存储的对应正确识别结果进行比对，判断校对结果是否与所述正确识别结果一致；若一致，则将该第二识别结果的校对结果作为最终的AI识别结果；若不一致，则向客户端发送警示信息和/或执行惩罚操作。
第三方面，本申请提供一种电子设备，所述电子设备包括处理器和存储器，所述处理器用于执行存储器中存储的计算机程序以实现所述的识别结果的智能校对方法。
第四方面，本申请提供一种计算机可读存储介质，所述计算机可读存储介质存储有至少一个指令，所述至少一个指令被处理器执行时实现所述的识别结果的智能校对方法。
本申请提供一种识别结果的智能校对方法、装置、设备及存储介质，用于对人工智能识别的识别结果进行智能校对；本申请首先获取目标识别结果，将所述识别结果作为校对任务，发送给客户端，以供客户端对识别结果进行校对；所述目标识别结果包括第一识别结果和第二识别结果；其中，所述第二识别结果为预设的错误识别结果，且校对系统中存储有对应的正确识别结果；通过从客户端返回的第二识别结果的校对结果，就能够判断出客户端是否在“无脑”提交，客户端提交第二识别结果的校对结果比对不一致时，校对系统会向客户端发送警示信息和/或执行惩罚操作，提醒客户端并有相对的惩罚措施，提高客户端校对的准确性；本申请的第二识别结果与真实的第一识别结果一样，客户端不会感知在被监控，校对系统能够无感知地识别客户端的作弊行为，并通过此方法持续让客户端处于认真校对的状态之中，使校对系统能够最大程度地获取有价值的校对结果，有效的提提升了识别结果的智能校对准确率。
图1为本申请一种识别结果的智能校对方法的流程图；
图2为本申请一种识别结果的智能校对装置的结构框图；
图3为本申请一种电子设备的结构框图。
本申请的实施方式
实施例1
请参阅图1所示，本申请提供一种识别结果的智能校对方法，包括：
S1、获取目标识别结果，校对系统将所述识别结果作为校对任务，发送给客户端，以供客户端对识别结果进行校对；所述目标识别结果包括第一识别结果和第二识别结果；其中，所述第二识别结果为预设的错误识别结果，且校对系统中存储有对应的正确识别结果；
目标识别结果为AI识别结果，包括识别对象的身份信息以及AI识别的结果信息；其中，识别对象为声音、图片、文字、翻译文本等；校对任务包括第一识别结果和第二识别结果。
第一识别结果为正常的校对任务，校对系统对该识别结果的结果信息正确与否未知，需要客户端进行真正校对。
第二识别结果为是为了校验用户是否有真正参与校验而给用户派发的错误识别结果，用户校验出错误信息则表示用户有参与校对工作，若用户未校验出错误信息，则表示用户可能并未参与校对工作，而是直接跳过校验。
S2、校对系统接收来自所述客户端的对所述目标识别结果的校对结果，并判断所述校对结果属于第一识别结果的校对结果还是属于第二识别结果的校对结果；
第一识别结果包括表征其类型的第一标识信息，第二识别结果包括表征其类型的第二标识信息；第一识别结果的校对结果中包括第一标识信息，第二识别结果的校对结果中包括第二标识信息。第一标识信息和第二标识信息对客户端不可见，校对系统能够获知第一标识信息和第二标识信息，并以此来判断收到的校对结果是属于第一识别结果的校对结果还是属于第二识别结果的校对结果。
S3、若所述校对结果属于第二识别结果的校对结果，则将该校对结果与预先存储的对应正确识别结果进行比对，判断校对结果是否与所述正确识别结果一致；若一致，则将该第二识别结果的校对结果作为最终的AI识别结果；若不一致，则向客户端发送警示信息和/或执行惩罚操作。
其中，若不一致时，校对系统调取下一个第二识别结果发送给所述客户端再次进行校对，直至所述校对系统判定接收到的所述客户端反馈的第二识别结果的校对结果与预先存储的正确识别结果一致，将正确的校对结果作为最终的识别结果，并依次执行下一个校对任务。
校对系统一旦判断客户端提交的第二识别结果的校对结果与存储的正确识别结果比对不一致，将从第二识别结果库随机抽取另外一个第二识别结果派发给客户端进行持续校对，直至校对系统判定收到客户端反馈的第二识别结果的校对结果与存储的正确识别结果比对一致为止；如果客户端连续设定次数提交的第二识别结果的校对结果与存储的正确识别结果比对不一致，校对系统将暂停派发校对任务给该客户端。
第二识别结果的正确识别结果预存于校对系统中，所以从客户端返回的校对结果就能够判断出客户端是否在“无脑”提交，客户端提交第二识别结果的校对结果与存储的正确识别结果比对不一致时，校对系统会发出警报信息，提醒客户端并有相对的惩罚措施，提高客户端校对的准确性；本申请的第二识别结果与真实的第一识别结果一样，客户端不会感知在被监控，校对系统能够无感知地识别客户端的作弊行为，并通过此方法持续让客户端处于认真校对的状态之中，使校对系统能够最大程度地获取有价值的校对结果，有效的提提升了识别结果的智能校对准确率。
实施例2
请参阅图1所示，本申请提供一种识别结果的智能校对方法，包括：
1）、获取目标识别结果，校对系统将所述识别结果作为校对任务，发送给客户端，以供客户端对识别结果进行校对；所述目标识别结果包括第一识别结果和第二识别结果；其中，所述第二识别结果为预设的错误识别结果，且校对系统中存储有对应的正确识别结果；目标识别结果为AI识别结果，包括识别对象的身份信息以及AI识别的结果信息；其中，识别对象为声音、图片、文字、翻译文本等；校对任务包括第一识别结果和第二识别结果。
第一识别结果为正常的校对任务，校对系统对该识别结果的结果信息正确与否未知，需要客户端进行真正校对。
第二识别结果为是为了校验用户是否有真正参与校验而给用户派发的错误识别结果，用户校验出错误信息则表示用户有参与校对工作，若用户未校验出错误信息，则表示用户可能并未参与校对工作，而是直接跳过校验。
以AI图片识别情景为例，通过第二识别结果（“钓鱼”题目）来进行智能校对。设计第二识别结果（“钓鱼”题目）：挑选清晰可辨的图片，并对图片进行算法识别结果和正确识别结果的填写，其中，算法识别结果是客户端能够看到的识别结果，故意设置为错误的识别结果，正确识别结果则与图片信息保持一致，存储于校对系统中。例如，第二识别结果中识别对象为红灯图片，正确的AI识别的结果信息为“红灯”存储于校对系统中，在第二识别结果中故意设置为错误的AI识别结果信息为“绿灯”，发送给客户端，强制客户对AI识别结果信息进行校对修改。对于这些第二识别结果（“钓鱼”题目），校对系统是已知正确识别结果，所以客户端提交的结果不会进行交叉校验程序。如果校对系统判断客户端提交的第二识别结果的校对结果与存储的正确识别结果比对不一致（最常见的方式是直接提交），校对系统会向客户端发送警示信息和/或执行惩罚操作，同时认为该用户存在作弊的嫌疑。反之，只要校对系统判断客户端提交的第二识别结果的校对结果与存储的正确识别结果比对一致，会获得相应的积分奖励。
2）、客户端中第一区域显示识别对象的身份信息，第二区域显示AI识别的结果信息；并根据第一区域中显示的识别对象的身份信息调整第二区域中显示的结果信息，获得校对结果；并将校对结果发送给校对系统；校对系统接收来自所述客户端的对所述目标识别结果的校对结果，并判断所述校对结果属于第一识别结果的校对结果还是属于第二识别结果的校对结果。
其中，如果客户端判断第二区域中显示的结果信息正确，则直接提交，无需调整；此时，校对结果与校对任务内容一致；
如果客户端判断第二区域中显示的结果信息有误，则在第二显示区域中调整结果信息，并保存获得校对结果；此时校对结果与原始校对任务内容有区别，包括识别对象的身份信息以及调整后的结果信息。
3）、如果校对系统判断收到的校对结果属于第一识别结果的校对结果，将校对结果作为最终的AI识别结果，并记录积分，然后下发下一个校对任务给客户端；如果校对系统判断收到的校对结果属于第二识别结果的校对结果，则将该校对结果与预先存储的对应正确识别结果进行比对，判断校对结果是否与所述正确识别结果一致；若一致，则将该第二识别结果的校对结果作为最终的AI识别结果记录积分，并下发下一个校对任务给客户端；若不一致，向客户端发送警示信息并下发下一个第二校对任务给客户端，直至校对系统判定收到客户端反馈的第二识别结果的校对结果与正确识别结果一致时，将校对结果作为最终的AI识别结果，记录积分，并下发下一个校对任务给客户端。
实施例3
本申请提供一种识别结果的智能校对方法，在实施例1或2的基础上，还包括第二识别结果库的建立步骤。
本申请中校对系统在待校对的目标识别结果中选择部分目标识别结果，对其识别结果进行修改，使其识别结果明显错误，形成第二识别结果；所有第二识别结果存储于第二识别结果库中，供校对系统调用。
本申请中，校对系统可以根据总目标识别结果数设计其中分布的第二识别结果数量，形成第二识别结果库；具体参见表1所示：
表1总目标识别结果中第二识别结果的数量
| 总目标识别结果数为N | 第二识别结果分布 |
| N=[0-15] | 随机分布1个第二识别结果 |
| N=[16-40] | 随机分布5个第二识别结果（共5个第二识别结果） |
| N=[41-100] | 每15个校对任务分布一个第二识别结果（共4个第二识别结果） |
| N=[101-200] | 每25个校对任务分布一个第二识别结果（共4个第二识别结果） |
| N=[201-400] | 每20个校对任务分布一个第二识别结果（共10个第二识别结果） |
| N>400 | 每20个校对任务分布一个第二识别结果 |
校对系统一旦判断客户端提交的第二识别结果的校对结果与存储的正确识别结果比对不一致，将从第二识别结果库随机抽取另外一个第二识别结果派发给客户端进行持续校对，直至校对系统判定收到客户端反馈的第二识别结果的校对结果与存储的正确识别结果比对一致为止；如果客户端连续设定次数提交的第二识别结果的校对结果与存储的正确识别结果比对不一致，校对系统将暂停派发校对任务给该客户端。
第二识别结果的正确识别结果预存于校对系统中，所以从客户端返回的校对结果就能够判断出客户端是否在“无脑”提交，客户端提交第二识别结果的校对结果与存储的正确识别结果比对不一致时，校对系统会发出警报信息，提醒客户端并有相对的惩罚措施，提高客户端校对的准确性；本申请的第二识别结果与真实的第一识别结果一样，客户端不会感知在被监控，校对系统能够无感知地识别客户端的作弊行为，并通过此方法持续让客户端处于认真校对的状态之中，使校对系统能够最大程度地获取有价值的校对结果，有效的提提升了识别结果的智能校对准确率。
实施例4
本申请提供一种识别结果的智能校对方法，在实施例1或2的基础上，还包括进行第二识别结果的判定与惩罚的步骤。
1、作弊计数
校对系统统计第二识别结果的校对结果与所述正确识别结果不一致的错误数量值，以及连续不一致的连续数量值，以此作为惩罚操作的条件；第一次识别出所述第二识别结果的校对结果与所述正确识别结果不一致时，所述错误数量值为1，所述连续数量值为0；后续当所述第二识别结果的校对结果与所述正确识别结果不一致时，所述错误数量值依次加1，同时所述连续数量值加1；直至当所述第二识别结果的校对结果与所述正确识别结果一致时，所述错误数量值保持不变，所述连续数量值为0。
具体的，校对系统判断客户端提交的第二识别结果的校对结果与存储的正确识别结果比对不一致时，计算有两个维度：错误数量值K以及连续数量值Q，其中错误数量值K={0，1，2，3，4，5}，连续数量值Q={0，1，2，3，4}。规则是：校对系统每次对第二识别结果的校对结果与存储的正确识别结果比对完成后，计算客户端的错误数量值K以及连续数量值Q；校对系统判断比对不一致时，错误数量值K加1，连续数量值Q加1；校对系统判断比对一致时，错误数量值K不变，连续数量值Q清零。
两个维度构成了客户端的刷分行为记录，用以区别机器刷分行为与人为刷分，实现在最短的时间内将不良客户端摒除在校对系统之外并对应给出不同力度的惩罚措施。
校对系统比对客户端第一个第二识别结果的校对结果不一致时，标记错误数量值K=1，连续数量值Q=1（错误数量值K、连续数量值Q的初始值均为0）；
校对系统比对客户端任意一个第二识别结果的校对结果与存储的正确识别结果不一致时，校对系统下发的下一个校对任务也会是第二识别结果，此时：
若校对系统比对客户端第二个第二识别结果的校对结果与存储的正确识别结果一致时（即校对结果未连续比对不一致），错误数量值K的计数不变，连续数量值Q重置为0，即此时错误数量值K=1，连续数量值Q=0；
若校对系统比对客户端第二个第二识别结果的校对结果与存储的正确识别结果继续不一致时（即连续两次比对不一致），同时标记错误数量值K=K+1，连续数量值Q=Q+1，即此时错误数量值K=2，连续数量值Q=2；
按照以上方法，校对系统每次对比对客户端第二识别结果的校对结果与存储的正确识别结果后，计算客户端的错误数量值K以及连续数量值Q。
2、惩罚系数
校对系统判断所有校对任务全部校对完成后，根据最终的错误数量值的大小设置对应的第一惩罚系数的大小，根据连续数量值的大小设置对应的第二惩罚系数的大小；校对系统比较所述第一惩罚系数和第二惩罚系数的大小，将两者中较大者作为有效值计算惩罚倍数；当第一惩罚系数和/或第二惩罚系数达到预设阈值时，按照预设的附加惩罚规则对用户进行惩罚操作。
错误数量值K的不同计数分别对应不同的第一惩罚系数DK值，具体对应关系如下：
表2错误数量值K与第一惩罚系数DK值对应表
| K | 0 | 1 | 2 | 3 | 4 | 5 |
| DK | 0 | 0 | 0 | 0.4 | 0.8 | 1 |
连续数量值Q的不同计数分别对应不同的第二惩罚系数DQ值，具体对应关系如下：
表3 连续数量值Q与第二惩罚系数DQ值对应表
| Q | 0 | 1 | 2 | 3 | 4 |
| DQ | 0 | 0 | 0.2 | 0.8 | 1 |
第一惩罚系数DK，第二惩罚系数DQ的分值代表校对系统将扣除客户端当天的任务积分的比例，取两者的最大值作为惩罚系数的当前有效值。例如：当第一惩罚系数DK=0.4，第二惩罚系数DQ=0.8，校对系统将取0.8为有效值，扣除客户端当天80%的任务积分作为惩罚。此外，当第一惩罚系数DK=1时，校对系统除了扣除客户端当天所做的所有任务积分之外，还会将客户端今天的任务完成单数清零，重新计数；当第二惩罚系数DQ=1时，校对系统除了清零任务单数与任务积分之外，会在一个小时之内暂停对该刷分客户端的任务派发。
实施例5
请参阅图2所示，本申请提供一种识别结果的智能校对装置，包括：
发送模块，用于获取目标识别结果，校对系统将所述识别结果作为校对任务，发送给客户端，以供客户端对识别结果进行校对；所述目标识别结果包括第一识别结果和第二识别结果；其中，所述第二识别结果为预设的错误识别结果，且校对系统中存储有对应的正确识别结果；
接收判断模块，用于接收来自所述客户端的对所述目标识别结果的校对结果，并判断所述校对结果属于第一识别结果的校对结果还是属于第二识别结果的校对结果；若所述校对结果属于第二识别结果的校对结果，则将该校对结果与预先存储的对应正确识别结果进行比对，判断校对结果是否与所述正确识别结果一致；若一致，则将该第二识别结果的校对结果作为最终的AI识别结果；若不一致，则向客户端发送警示信息和/或执行惩罚操作。
接收判断模块，还用于：每次比对客户端第二个第二识别结果的校对结果与存储的正确识别结果后，计算客户端的错误数量值K以及连续数量值Q；比对客户端第二识别结果的校对结果与存储的正确识别结果不一致时，错误数量值K加1，连续数量值Q加1；比对客户端第二识别结果的校对结果与存储的正确识别结果一致时，连续数量值Q清零；接收判断模块判断客户端校对的校对任务总数达到设定的总校对任务数N时，根据错误数量值K计算第一惩罚系数DK，根据连续数量值Q计算第二惩罚系数DQ；比较第一惩罚系数DK和第二惩罚系数DQ的大小，两者中较大者作为有效值X；扣除客户端当天X倍的任务积分；当第一惩罚系数DK=1时，除了扣除客户端当天所做的所有任务积分之外，还将客户端今天的任务完成单数清零，重新计数；当第二惩罚系数DQ=1时，除了清零任务单数与任务积分之外，在预设时间之内暂停对该客户端的任务派发。
实施例6
请参阅图3所示，本申请还提供一种识别结果的智能校对方法的电子设备100；所述电子设备100包括存储器101、至少一个处理器102、存储在所述存储器101中并可在所述至少一个处理器102上运行的计算机程序103及至少一条通讯总线104。
存储器101可用于存储所述计算机程序103，所述处理器102通过运行或执行存储在所述存储器101内的计算机程序，以及调用存储在存储器101内的数据，实现实施例1至4中任一个所述的识别结果的智能校对方法的方法步骤。所述存储器101可主要包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需的应用程序(比如声音播放功能、图像播放功能等)等；存储数据区可存储根据电子设备100的使用所创建的数据(比如音频数据)等。此外，存储器101可以包括非易失性存储器，例如硬盘、内存、插接式硬盘，智能存储卡(Smart Media Card，SMC)，安全数字(Secure Digital，SD)卡，闪存卡(Flash Card)、至少一个磁盘存储器件、闪存器件、或其他非易失性固态存储器件，也可以是易失性的。
所述至少一个处理器102可以是中央处理单元(Central Processing Unit，CPU)，还可以是其他通用处理器、数字信号处理器（Digital Signal Processor，DSP）、专用集成电路（Application
Specific Integrated Circuit，ASIC）、现场可编程门阵列（Field-Programmable Gate Array，FPGA）或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。处理器102可以是微处理器或者该处理器102也可以是任何常规的处理器等，所述处理器102是所述电子设备100的控制中心，利用各种接口和线路连接整个电子设备100的各个部分。
所述电子设备100中的所述存储器101存储多个指令以实现一种耦合的多任务特征提取方法，所述处理器102可执行所述多个指令从而实现：
获取目标识别结果，校对系统将所述识别结果作为校对任务，发送给客户端，以供客户端对识别结果进行校对；所述目标识别结果包括第一识别结果和第二识别结果；其中，所述第二识别结果为预设的错误识别结果，且校对系统中存储有对应的正确识别结果；
校对系统接收来自所述客户端的对所述目标识别结果的校对结果，并判断所述校对结果属于第一识别结果的校对结果还是属于第二识别结果的校对结果；
若所述校对结果属于第二识别结果的校对结果，则将该校对结果与预先存储的对应正确识别结果进行比对，判断校对结果是否与所述正确识别结果一致；
若一致，则将该第二识别结果的校对结果作为最终的AI识别结果；
若不一致，则向客户端发送警示信息和/或执行惩罚操作。
具体地，所述处理器102对上述指令的具体实现方法可参考实施例1中相关步骤的描述，在此不赘述。
实施例7
所述电子设备100集成的模块/单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。
基于这样的理解，本申请实现上述实施例方法中的全部或部分流程，也可以通过计算机程序来指令相关的硬件来完成，所述的计算机程序可存储于一计算机可读存储介质中，该计算机程序在被处理器执行时，可实现上述各个方法实施例的步骤。其中，所述计算机程序包括计算机程序代码，所述计算机程序代码可以为源代码形式、对象代码形式、可执行文件或某些中间形式等。所述计算机可读介质可以包括：能够携带所述计算机程序代码的任何实体或装置、记录介质、U盘、移动硬盘、磁碟、光盘、计算机存储器及只读存储器(ROM，Read-Only Memory)。
最后应当说明的是：以上实施例仅用以说明本申请的技术方案而非对其限制，尽管参照上述实施例对本申请进行了详细的说明，所属领域的普通技术人员应当理解：依然可以对本申请的具体实施方式进行修改或者等同替换，而未脱离本申请精神和范围的任何修改或者等同替换，其均应涵盖在本申请的权利要求保护范围之内。
Claims (20)
- 识别结果的智能校对方法，包括：获取目标识别结果，校对系统将所述识别结果作为校对任务，发送给客户端，以供客户端对识别结果进行校对；所述目标识别结果包括第一识别结果和第二识别结果；其中，所述第二识别结果为预设的错误识别结果，且校对系统中存储有对应的正确识别结果；校对系统接收来自所述客户端的对所述目标识别结果的校对结果，并判断所述校对结果属于第一识别结果的校对结果还是属于第二识别结果的校对结果；若所述校对结果属于第二识别结果的校对结果，则将该校对结果与预先存储的对应正确识别结果进行比对，判断校对结果是否与所述正确识别结果一致；若一致，则将该第二识别结果的校对结果作为最终的AI识别结果；若不一致，则向客户端发送警示信息和/或执行惩罚操作。
- 根据权利要求1所述的识别结果的智能校对方法，所述若所述校对结果属于第二识别结果的校对结果，则将该校对结果与预先存储的对应正确识别结果进行比对，判断校对结果是否与所述正确识别结果一致，还包括：若不一致，所述校对系统调取下一个第二识别结果发送给所述客户端再次进行校对，直至所述校对系统判定接收到的所述客户端反馈的第二识别结果的校对结果与预先存储的正确识别结果一致，将正确的校对结果作为最终的识别结果，并依次执行下一个校对任务。
- 根据权利要求2所述的识别结果的智能校对方法，所述若不一致，所述校对系统调取下一个第二识别结果发送给所述客户端再次进行校对，直至所述校对系统判定接收到的所述客户端反馈的第二识别结果的校对结果与预先存储的正确识别结果一致，将正确的校对结果作为最终的识别结果，并依次执行下一个校对任务之后，还包括：统计所述第二识别结果的校对结果与所述正确识别结果不一致的错误数量值，以及连续不一致的连续数量值，以此作为惩罚操作的条件。
- 根据权利要求3所述的识别结果的智能校对方法，所述统计所述第二识别结果的校对结果与所述正确识别结果不一致的错误数量值，以及连续不一致的连续数量值，以此作为惩罚操作的条件包括：第一次识别出所述第二识别结果的校对结果与所述正确识别结果不一致时，所述错误数量值为1，所述连续数量值为0；后续当所述第二识别结果的校对结果与所述正确识别结果不一致时，所述错误数量值依次加1，同时所述连续数量值加1；直至当所述第二识别结果的校对结果与所述正确识别结果一致时，所述错误数量值保持不变，所述连续数量值为0。
- 根据权利要求4所述的识别结果的智能校对方法，所述统计所述第二识别结果的校对结果与所述正确识别结果不一致的错误数量值，以及连续不一致的连续数量值，以此作为惩罚操作的条件包括：校对系统判断所有校对任务全部校对完成后，根据最终的错误数量值计算第一惩罚系数，根据连续数量值计算第二惩罚系数；校对系统比较所述第一惩罚系数和第二惩罚系数的大小，将两者中较大者作为有效值计算惩罚倍数；当第一惩罚系数和/或第二惩罚系数达到预设阈值时，按照预设的附加惩罚规则对用户进行惩罚操作。
- 根据权利要求5述的识别结果的智能校对方法，所述根据最终错误数量值计算第一惩罚系数的方法包括：根据所述错误数量值的大小设置对应的第一惩罚系数的大小；所述根据连续数量值计算第二惩罚系数的方法包括：根据所述连续数量值的大小设置对应的第二惩罚系数的大小。
- 根据权利要求1述的识别结果的智能校对方法，所述第一识别结果包括表征其类型的第一标识信息，所述第二识别结果包括表征其类型的第二标识信息；所述第一识别结果的校对结果中包括所述第一标识信息，所述第二识别结果的校对结果中包括所述第二标识信息。
- 识别结果的智能校对装置，包括：发送模块，用于获取目标识别结果，将所述识别结果作为校对任务，发送给客户端，以供客户端对识别结果进行校对；所述目标识别结果包括第一识别结果和第二识别结果；其中，所述第二识别结果为预设的错误识别结果，且发送模块中存储有对应的正确识别结果；接收判断模块，用于接收来自所述客户端的对所述目标识别结果的校对结果，并判断所述校对结果属于第一识别结果的校对结果还是属于第二识别结果的校对结果；若所述校对结果属于第二识别结果的校对结果，则将该校对结果与预先存储的对应正确识别结果进行比对，判断校对结果是否与所述正确识别结果一致；若一致，则将该第二识别结果的校对结果作为最终的AI识别结果；若不一致，则向客户端发送警示信息和/或执行惩罚操作。
- 根据权利要求8所述的识别结果的智能校对装置，所述接收判断模块包括：比较判断模块，用于将该校对结果与预先存储的对应正确识别结果进行比对，判断校对结果是否与所述正确识别结果一致；若不一致，所述校对系统调取下一个第二识别结果发送给所述客户端再次进行校对，直至所述校对系统判定接收到的所述客户端反馈的第二识别结果的校对结果与预先存储的正确识别结果一致，将正确的校对结果作为最终的识别结果，并依次执行下一个校对任务。
- 根据权利要求9所述的识别结果的智能校对装置，所述接收判断模块包括：统计模块，用于统计所述第二识别结果的校对结果与所述正确识别结果不一致的错误数量值，以及连续不一致的连续数量值，以此作为惩罚操作的条件。
- 根据权利要求10所述的识别结果的智能校对装置，所述统计模块包括：第一统计子模块，用于在第一次识别出所述第二识别结果的校对结果与所述正确识别结果不一致时，使错误数量值为1，连续数量值为0；第二统计子模块，用于后续当所述第二识别结果的校对结果与所述正确识别结果不一致时，使错误数量值依次加1，同时所述连续数量值加1；第三统计子模块，用于直至当所述第二识别结果的校对结果与所述正确识别结果一致时，使错误数量值保持不变，所述连续数量值为0。
- 根据权利要求11所述的识别结果的智能校对装置，所述统计模块包括：第二惩罚系数计算模块，用于校对系统判断所有校对任务全部校对完成后，根据最终的错误数量值计算第一惩罚系数，根据连续数量值计算第二惩罚系数；惩罚倍数计算模块，用于校对系统比较所述第一惩罚系数和第二惩罚系数的大小，将两者中较大者作为有效值计算惩罚倍数；惩罚模块，用于当第一惩罚系数和/或第二惩罚系数达到预设阈值时，按照预设的附加惩罚规则对用户进行惩罚操作。
- 一种电子设备，所述电子设备包括处理器和存储器，所述处理器用于执行存储器中存储的计算机程序实现如下步骤：获取目标识别结果，校对系统将所述识别结果作为校对任务，发送给客户端，以供客户端对识别结果进行校对；所述目标识别结果包括第一识别结果和第二识别结果；其中，所述第二识别结果为预设的错误识别结果，且校对系统中存储有对应的正确识别结果；校对系统接收来自所述客户端的对所述目标识别结果的校对结果，并判断所述校对结果属于第一识别结果的校对结果还是属于第二识别结果的校对结果；若所述校对结果属于第二识别结果的校对结果，则将该校对结果与预先存储的对应正确识别结果进行比对，判断校对结果是否与所述正确识别结果一致；若一致，则将该第二识别结果的校对结果作为最终的AI识别结果；若不一致，则向客户端发送警示信息和/或执行惩罚操作。
- 根据权利要求13所述的一种电子设备，所述若所述校对结果属于第二识别结果的校对结果，则将该校对结果与预先存储的对应正确识别结果进行比对，判断校对结果是否与所述正确识别结果一致，还包括：若不一致，所述校对系统调取下一个第二识别结果发送给所述客户端再次进行校对，直至所述校对系统判定接收到的所述客户端反馈的第二识别结果的校对结果与预先存储的正确识别结果一致，将正确的校对结果作为最终的识别结果，并依次执行下一个校对任务。
- 根据权利要求14所述的一种电子设备，所述若不一致，所述校对系统调取下一个第二识别结果发送给所述客户端再次进行校对，直至所述校对系统判定接收到的所述客户端反馈的第二识别结果的校对结果与预先存储的正确识别结果一致，将正确的校对结果作为最终的识别结果，并依次执行下一个校对任务之后，还包括：统计所述第二识别结果的校对结果与所述正确识别结果不一致的错误数量值，以及连续不一致的连续数量值，以此作为惩罚操作的条件。
- 根据权利要求15所述的一种电子设备，所述统计所述第二识别结果的校对结果与所述正确识别结果不一致的错误数量值，以及连续不一致的连续数量值，以此作为惩罚操作的条件包括：第一次识别出所述第二识别结果的校对结果与所述正确识别结果不一致时，所述错误数量值为1，所述连续数量值为0；后续当所述第二识别结果的校对结果与所述正确识别结果不一致时，所述错误数量值依次加1，同时所述连续数量值加1；直至当所述第二识别结果的校对结果与所述正确识别结果一致时，所述错误数量值保持不变，所述连续数量值为0。
- 根据权利要求16所述的一种电子设备，所述统计所述第二识别结果的校对结果与所述正确识别结果不一致的错误数量值，以及连续不一致的连续数量值，以此作为惩罚操作的条件包括：校对系统判断所有校对任务全部校对完成后，根据最终的错误数量值计算第一惩罚系数，根据连续数量值计算第二惩罚系数；校对系统比较所述第一惩罚系数和第二惩罚系数的大小，将两者中较大者作为有效值计算惩罚倍数；当第一惩罚系数和/或第二惩罚系数达到预设阈值时，按照预设的附加惩罚规则对用户进行惩罚操作。
- 一种计算机可读存储介质，所述计算机可读存储介质存储有至少一个指令，所述至少一个指令被处理器执行时实现如下步骤：获取目标识别结果，校对系统将所述识别结果作为校对任务，发送给客户端，以供客户端对识别结果进行校对；所述目标识别结果包括第一识别结果和第二识别结果；其中，所述第二识别结果为预设的错误识别结果，且校对系统中存储有对应的正确识别结果；校对系统接收来自所述客户端的对所述目标识别结果的校对结果，并判断所述校对结果属于第一识别结果的校对结果还是属于第二识别结果的校对结果；若所述校对结果属于第二识别结果的校对结果，则将该校对结果与预先存储的对应正确识别结果进行比对，判断校对结果是否与所述正确识别结果一致；若一致，则将该第二识别结果的校对结果作为最终的AI识别结果；若不一致，则向客户端发送警示信息和/或执行惩罚操作。
- 根据权利要求18所述的一种计算机可读存储介质，所述若所述校对结果属于第二识别结果的校对结果，则将该校对结果与预先存储的对应正确识别结果进行比对，判断校对结果是否与所述正确识别结果一致，还包括：若不一致，所述校对系统调取下一个第二识别结果发送给所述客户端再次进行校对，直至所述校对系统判定接收到的所述客户端反馈的第二识别结果的校对结果与预先存储的正确识别结果一致，将正确的校对结果作为最终的识别结果，并依次执行下一个校对任务。
- 根据权利要求19所述的一种计算机可读存储介质，所述若不一致，所述校对系统调取下一个第二识别结果发送给所述客户端再次进行校对，直至所述校对系统判定接收到的所述客户端反馈的第二识别结果的校对结果与预先存储的正确识别结果一致，将正确的校对结果作为最终的识别结果，并依次执行下一个校对任务之后，还包括：统计所述第二识别结果的校对结果与所述正确识别结果不一致的错误数量值，以及连续不一致的连续数量值，以此作为惩罚操作的条件。
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110379808.5 | 2021-04-08 | ||
| CN202110379808.5A CN113032426A (zh) | 2021-04-08 | 2021-04-08 | 识别结果的智能校对方法、装置、设备及存储介质 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2022213466A1 true WO2022213466A1 (zh) | 2022-10-13 |
Family
ID=76456035
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2021/097162 WO2022213466A1 (zh) | 2021-04-08 | 2021-05-31 | 识别结果的智能校对方法、装置、设备及存储介质 |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN113032426A (zh) |
| WO (1) | WO2022213466A1 (zh) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116129731A (zh) * | 2022-12-29 | 2023-05-16 | 北京布局未来教育科技有限公司 | 人工智能模拟教学系统与方法 |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105808197A (zh) * | 2014-12-30 | 2016-07-27 | 联想(北京)有限公司 | 一种信息处理方法和电子设备 |
| JP2017174172A (ja) * | 2016-03-24 | 2017-09-28 | 株式会社アドバンスト・メディア | 表示処理装置及び表示処理プログラム |
| CN108734196A (zh) * | 2018-04-17 | 2018-11-02 | 平安科技（深圳）有限公司 | 众包平台的作业方法、装置、计算机设备和存储介质 |
| CN112559369A (zh) * | 2020-12-23 | 2021-03-26 | 上海眼控科技股份有限公司 | 一种自动化测试方法、设备和存储介质 |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6697943B1 (en) * | 1999-12-17 | 2004-02-24 | Cisco Technology, Inc. | Use of cyclic redundancy checking for segregating control traffic |
| CN105184653A (zh) * | 2015-09-08 | 2015-12-23 | 苏州大学 | 一种面向社交网络的基于信任的众包工人筛选方法 |
-
2021
- 2021-04-08 CN CN202110379808.5A patent/CN113032426A/zh active Pending
- 2021-05-31 WO PCT/CN2021/097162 patent/WO2022213466A1/zh active Application Filing
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105808197A (zh) * | 2014-12-30 | 2016-07-27 | 联想(北京)有限公司 | 一种信息处理方法和电子设备 |
| JP2017174172A (ja) * | 2016-03-24 | 2017-09-28 | 株式会社アドバンスト・メディア | 表示処理装置及び表示処理プログラム |
| CN108734196A (zh) * | 2018-04-17 | 2018-11-02 | 平安科技（深圳）有限公司 | 众包平台的作业方法、装置、计算机设备和存储介质 |
| CN112559369A (zh) * | 2020-12-23 | 2021-03-26 | 上海眼控科技股份有限公司 | 一种自动化测试方法、设备和存储介质 |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116129731A (zh) * | 2022-12-29 | 2023-05-16 | 北京布局未来教育科技有限公司 | 人工智能模拟教学系统与方法 |
| CN116129731B (zh) * | 2022-12-29 | 2023-09-15 | 北京布局未来教育科技有限公司 | 人工智能模拟教学系统与方法 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113032426A (zh) | 2021-06-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108259936B (zh) | 基于直播技术的问答方法、服务器及存储介质 | |
| US20160366475A1 (en) | Method for controlling network media information publication, apparatus, and server | |
| CN107622197B (zh) | 设备识别方法及装置、用于设备识别的权重计算方法及装置 | |
| CN112668453B (zh) | 视频识别方法及相关设备 | |
| WO2022213466A1 (zh) | 识别结果的智能校对方法、装置、设备及存储介质 | |
| WO2022193071A1 (zh) | 一种基于直播的送礼方法及装置、设备以及存储介质 | |
| CN116416190A (zh) | 瑕疵检测方法及装置、电子设备及存储介质 | |
| CN112988896B (zh) | 一种基于区块链的同步共识方法及系统及装置及介质 | |
| CN111582405B (zh) | 一种数据标注方法及装置 | |
| CN110599233A (zh) | 一种基于点击验证的广告反作弊方法及装置 | |
| CN110708333B (zh) | 一种位置验证方法以及相关设备 | |
| CN111128233B (zh) | 录音检测方法、装置、电子设备及存储介质 | |
| CN111784173A (zh) | Ab实验数据处理方法、装置、服务器及介质 | |
| JP7485271B2 (ja) | パルス信号ベースの表示方法及び装置、電子デバイス、並びに媒体 | |
| CN113301017B (zh) | 基于联邦学习的攻击检测与防御方法、装置及存储介质 | |
| CN112286780B (zh) | 识别算法的测试方法、装置、设备及存储介质 | |
| CN112491625A (zh) | 基于即时通讯平台的运维告警方法、装置及设备 | |
| WO2019042249A1 (zh) | 一种翻页方法、装置和存储介质 | |
| CN112698808B (zh) | 调整音频设备音量的方法、装置、电子装置及介质 | |
| CN110837562A (zh) | 案件的处理方法、装置和系统 | |
| CN110874567B (zh) | 颜值判定方法、装置、电子设备及存储介质 | |
| TWI738131B (zh) | 影像系統及檢測方法 | |
| CN114845142A (zh) | 数据记录方法、装置、电子设备及存储介质 | |
| CN113032278A (zh) | 应用程序的运行方式、终端设备的等级确认方法及装置 | |
| CN113035240A (zh) | 语音播报方法、装置、设备及存储介质 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 21935687 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 21935687 Country of ref document: EP Kind code of ref document: A1 |