CN114154611A - Man-machine confrontation system supporting Turing test mode and intelligent agent test method - Google Patents
Man-machine confrontation system supporting Turing test mode and intelligent agent test method Download PDFInfo
- Publication number
- CN114154611A CN114154611A CN202111328333.3A CN202111328333A CN114154611A CN 114154611 A CN114154611 A CN 114154611A CN 202111328333 A CN202111328333 A CN 202111328333A CN 114154611 A CN114154611 A CN 114154611A
- Authority
- CN
- China
- Prior art keywords
- confrontation
- data
- man
- machine
- human
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012360 testing method Methods 0.000 title claims abstract description 72
- 238000010998 test method Methods 0.000 title abstract description 8
- 238000011156 evaluation Methods 0.000 claims abstract description 57
- 238000007405 data analysis Methods 0.000 claims abstract description 8
- 230000009471 action Effects 0.000 claims description 80
- 238000000034 method Methods 0.000 claims description 73
- 230000008569 process Effects 0.000 claims description 51
- 230000008447 perception Effects 0.000 claims description 18
- 238000004088 simulation Methods 0.000 claims description 17
- 230000006870 function Effects 0.000 claims description 10
- 238000004458 analytical method Methods 0.000 claims description 8
- 238000001914 filtration Methods 0.000 claims description 8
- 125000004122 cyclic group Chemical group 0.000 claims description 7
- 230000001351 cycling effect Effects 0.000 claims description 2
- 239000003795 chemical substances by application Substances 0.000 description 176
- 238000007726 management method Methods 0.000 description 20
- 230000008859 change Effects 0.000 description 10
- 230000000875 corresponding effect Effects 0.000 description 10
- 230000001149 cognitive effect Effects 0.000 description 9
- 230000010485 coping Effects 0.000 description 8
- 238000013439 planning Methods 0.000 description 8
- 238000004590 computer program Methods 0.000 description 6
- 230000019771 cognition Effects 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000003993 interaction Effects 0.000 description 4
- 230000006399 behavior Effects 0.000 description 3
- 230000002079 cooperative effect Effects 0.000 description 2
- 238000013523 data management Methods 0.000 description 2
- 230000000873 masking effect Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000007619 statistical method Methods 0.000 description 2
- 241000282412 Homo Species 0.000 description 1
- 241000196171 Hydrodictyon reticulatum Species 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 150000001875 compounds Chemical class 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011056 performance test Methods 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000011158 quantitative evaluation Methods 0.000 description 1
- 230000003014 reinforcing effect Effects 0.000 description 1
- 238000012827 research and development Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000009897 systematic effect Effects 0.000 description 1
- 238000012876 topography Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/004—Artificial life, i.e. computing arrangements simulating life
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/211—Schema design and management
- G06F16/212—Schema design and management with details for data modelling support
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/248—Presentation of query results
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Artificial Intelligence (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention provides a man-machine confrontation system supporting a Turing test mode and an intelligent agent test method, wherein the system comprises: the man-machine confrontation module is used for finishing man-machine confrontation; wherein, human-computer confrontation is completed by human confronters and intelligent agents in a double-blind environment; the confrontation data acquisition module is used for acquiring a man-machine confrontation result and countermark confrontation data of man-machine confrontation; the confrontation data analysis module is used for acquiring the efficiency data of the intelligent agent according to the counteroffer data; the intelligent agent comprises a smart questionnaire module, a database module and a database module, wherein the smart questionnaire module is used for acquiring external evaluation information of the intelligent agent in a smart questionnaire form; and the confrontation result management module is used for receiving and storing the man-machine confrontation result, the external evaluation information of the intelligent agent and the efficiency data of the intelligent agent, and acquiring the capability test result of the intelligent agent according to the man-machine confrontation result, the external evaluation information of the intelligent agent and the efficiency data of the intelligent agent. The invention realizes the comprehensive and effective test of the decision-making capability of the intelligent agent.
Description
Technical Field
The invention relates to the technical field of man-machine confrontation, in particular to a man-machine confrontation system supporting a Turing test mode and an intelligent body test method.
Background
Artificial intelligence technology is undergoing the evolution from perceptual intelligence to cognitive intelligence, and is working on machines developing from being able to understand data to being able to understand the real world, and from being able to hear, say, see, develop as being able to think like humans. The perception intelligence usually takes machine learning as a technical route, the feature representation which can be obviously fitted with an expected target is learned from a large-scale data set, the final output of the feature representation is usually instant and clear, and the feature representation can be matched and compared with known standard answers, so that the quantitative evaluation of indexes such as accuracy and error values is convenient to use. In the field of cognitive intelligence, an agent is often in various flexible application scenes, and a certain action purpose is achieved through a continuous cognitive decision process of perception-judgment-decision-action. The cognitive decision process is continuously and dynamically changed, uncertainty is filled, and sometimes it is difficult to obtain a Reward (Reward) of a short-term decision behavior, a traditional test method usually defines a final objective index (such as a decision score) for evaluation, but the evaluation of the decision of the middle process of the intelligent agent is ignored by the method, the capability level of the intelligent agent in each aspect cannot be comprehensively reflected, and a profit "drill a space" behavior with the score being improved as the main is also easily brought, so that the accuracy of the decision capability test result of the intelligent agent cannot be ensured. Therefore, how to comprehensively and effectively test the decision-making capability of the cognitive and decision-making type intelligent agents is an urgent problem in the cognitive intelligence era.
Allen-turing has proposed a "turing test" in 1950, which is a test to determine whether a machine has human intelligence, to determine whether a machine has intelligence by countermeasures and analogies. "Turing test" definition: in the case where the tester is separated from the testee (one person and one machine), the testee is asked questions at will by some means. After multiple tests, a machine is considered to have human intelligence if it has more than 30% false positives on average per participant.
However, there is no man-machine countermeasure system supporting the turing test mode in the prior art, and it is necessary to provide a man-machine countermeasure system supporting the turing test mode to acquire performance test data of the agent and to comprehensively and effectively test the cognitive and decision-making ability of the agent according to the test data.
Disclosure of Invention
The invention provides a man-machine confrontation system supporting a Turing test mode and an intelligent agent test method, which are used for solving the defect that the capability level of an intelligent agent in each aspect cannot be reflected in all aspects in the prior art and realizing comprehensive and effective test on the cognitive and decision-making capability of the intelligent agent.
The invention provides a man-machine confrontation system supporting a Turing test mode, which comprises:
the man-machine confrontation module is used for finishing man-machine confrontation; wherein the human-machine confrontation is completed by a human confronter and an agent in a double-blind environment;
the confrontation data acquisition module is used for acquiring a man-machine confrontation result and countermark confrontation data of man-machine confrontation;
the confrontation data analysis module is used for acquiring the efficiency data of the intelligent agent according to the counteroffer data of the multi-disk;
the intelligent agent comprises a smart questionnaire module, a database module and a database module, wherein the smart questionnaire module is used for acquiring external evaluation information of the intelligent agent in a smart questionnaire form;
and the confrontation result management module is used for receiving and storing the man-machine confrontation result, the external evaluation information of the intelligent agent and the efficiency data of the intelligent agent, and acquiring the capability test result of the intelligent agent according to the man-machine confrontation result, the external evaluation information of the intelligent agent and the efficiency data of the intelligent agent.
According to the man-machine confrontation system supporting the Turing test mode, the man-machine confrontation module comprises:
the confrontation course generating module is used for generating a confrontation circular course and coding the system codes of the human confronters and the intelligent agents; wherein encoding system codes of the human competitor and the agent is used to create the double-blind environment;
the confrontation scheduling management module is used for automatically scheduling the man-machine confrontation process according to the circular race course;
and the man-machine confrontation function module is used for completing the man-machine confrontation in a simulated environment through the human confronter and the intelligent body under the process scheduling of the man-machine confrontation.
According to the man-machine confrontation system supporting the Turing test mode, the man-machine confrontation function module comprises:
a confrontational environment simulation module for generating the simulated environment;
the engine kernel module is used for receiving action instructions sent by the human opponents and the intelligent agents, updating the environment state and the battlefield situation of the simulation environment according to the action instructions and generating real-time deduction data;
the UI module is used for analyzing and displaying the real-time deduction data to the human opponent, sending an action instruction sent by the human opponent to the engine kernel module, and filtering identity information of the human opponent;
and the AI module is used for analyzing the real-time deduction data to the intelligent agent, sending the action instruction sent by the intelligent agent to the engine kernel module, and filtering the identity information of the intelligent agent.
According to the man-machine confrontation system supporting the Turing test mode, the confrontation data analysis module obtains the efficiency data of the intelligent agent according to the countersignature data, and the system comprises the following components:
according to the countercheck data, acquiring operation data of the intelligent agent in the man-machine countercheck process, and respectively acquiring performance data of the intelligent agent from a plurality of different aspects according to the operation data; wherein the operational data of the agent comprises: the own resource application class data, the own strategy class data, the enemy perception class data and the man-machine distinguishing characteristic class data.
The man-machine confrontation system supporting the Turing test mode further comprises a confrontation and sightseeing module;
the confrontation data acquisition module is also used for acquiring real-time confrontation data in the man-machine confrontation process;
the countercheck viewing module is used for analyzing the real-time countercheck data and/or the countercheck data, and performing anonymous 2D or 3D countercheck graphic display and switching display of different battlefield situations according to the analysis result of the real-time countercheck data and/or the countercheck data.
The invention also provides an intelligent agent testing method, which comprises the following steps:
acquiring a man-machine confrontation result and countercheck data of man-machine confrontation; wherein the human-machine confrontation is completed by a human confronter and an agent in a double-blind environment;
acquiring efficiency data of the intelligent agent according to the countermeasures data, and acquiring external evaluation information of the intelligent agent in the form of a Tuoling questionnaire;
and acquiring a capability test result of the intelligent agent according to the man-machine confrontation result, the external evaluation information of the intelligent agent and the efficiency data of the intelligent agent.
According to the intelligent agent testing method provided by the invention, the human-computer confrontation completed by the human confronter and the intelligent agent in a double-blind environment comprises the following steps:
generating a cyclic course of confrontations and encoding the system codes of the human confronters and the agents; wherein encoding system codes of the human competitor and the agent is used to create the double-blind environment;
and automatically scheduling the process of the man-machine confrontation according to the circular competition process, and finishing the man-machine confrontation in a simulated environment through the human confronter and the intelligent agent under the scheduling of the process of the man-machine confrontation.
According to the invention, the intelligent agent testing method is provided, wherein the human-computer confrontation completed by the human confronter and the intelligent agent in a simulated environment comprises the following steps:
an environment generation step: generating the simulated environment;
a countermeasure step: receiving action instructions sent by the human opponents and the intelligent agents, updating the environment state and the battlefield situation of the simulation environment according to the action instructions, and generating real-time deduction data;
cycling the step of confrontation until the human-machine confrontation is completed;
wherein, before the countermeasure step, the method further comprises: filtering identity information of the human competitor and the agent.
According to the intelligent agent testing method provided by the invention, the step of acquiring the efficiency data of the intelligent agent according to the countercheck data comprises the following steps:
according to the countercheck data, acquiring operation data of the intelligent agent in the man-machine countercheck process, and respectively acquiring performance data of the intelligent agent from a plurality of different aspects according to the operation data; wherein the operational data of the agent comprises: the own resource application class data, the own strategy class data, the enemy perception class data and the man-machine distinguishing characteristic class data.
According to the intelligent agent testing method provided by the invention, in the process of the man-machine confrontation and/or after the man-machine confrontation is completed, the intelligent agent testing method further comprises the following steps:
acquiring real-time confrontation data in the man-machine confrontation process;
and analyzing the real-time countermeasure data and/or the countersignature data, and performing anonymous 2D or 3D countersignature graphic display and switching display of different battlefield situations according to the analysis result of the real-time countermeasure data and/or the countersignature data.
The invention provides a man-machine confrontation system supporting a Turing test mode and an intelligent body test method.A Turing questionnaire module is creatively introduced into the man-machine confrontation system, external evaluation information of an intelligent body is acquired through the Turing questionnaire module, effectiveness data of the intelligent body is obtained through analysis of confrontation data in the man-machine confrontation process, and the decision-making capability of the intelligent body is tested by combining a man-machine confrontation result, the external evaluation information of the intelligent body and the effectiveness data of the intelligent body, so that comprehensive and effective test of the cognition and decision-making capability of the intelligent body is realized; meanwhile, the man-machine confrontation process is completed in a double-blind environment, so that the fairness of data in the cognitive and decision-making capability test process of the intelligent agent is further ensured, and the accuracy of the decision-making capability test result of the intelligent agent is improved.
Drawings
In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
FIG. 1 is a schematic structural diagram of a man-machine confrontation system supporting Turing test mode according to the present invention;
FIG. 2 is a schematic flow diagram of the method for testing an agent according to the present invention;
fig. 3 is a schematic structural diagram of an electronic device provided in the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The man-machine confrontation system supporting the turing test mode of the invention is described below with reference to fig. 1, and the man-machine confrontation system supporting the turing test mode is shown in fig. 1 and comprises:
a man-machine confrontation module 710 for completing man-machine confrontation; wherein, human-computer confrontation is completed by human confronters and intelligent agents in a double-blind environment;
the confrontation data acquisition module 720 is used for acquiring a man-machine confrontation result and counteroffer data of man-machine confrontation;
the confrontation data analysis module 730 is used for acquiring the efficiency data of the intelligent agent according to the counteroffer data of the multiple disks;
the Turing questionnaire module 740 is used for acquiring external evaluation information of the intelligent agent in the form of Turing questionnaire; that is, the turing questionnaire module 740 collects the guessing and evaluation information of the external world to the agent in the man-machine interaction mode of the electronic questionnaire;
the confrontation result management module 750 is used for receiving and storing the man-machine confrontation result, the external evaluation information of the intelligent agent and the efficiency data of the intelligent agent, and acquiring the capability test result of the intelligent agent according to the man-machine confrontation result, the external evaluation information of the intelligent agent and the efficiency data of the intelligent agent.
As an alternative, the man-machine confrontation system supporting the Turing test mode further comprises a confrontation and sightseeing module; the confrontation data acquisition module 720 is further used for acquiring real-time confrontation data in the man-machine confrontation process; the anti-sightseeing module is used for analyzing the real-time anti-data and/or the multi-disc anti-data, and performing anonymous 2D or 3D anti-graph display and switching display of different battlefield situations according to the analysis result of the real-time anti-data and/or the multi-disc anti-data. The module is mainly characterized in that identity information of both parties of the battle is screened, real man-machine conditions and user information of the confrontation players cannot be known during observation, and only anonymous system codes of the confrontation players can be known. The confrontation and sightseeing module comprises a real-time sightseeing unit and a re-disk sightseeing unit; the real-time fighting unit is used for analyzing real-time fighting data in the man-machine fighting process and providing a real-time graphic display interface according to the fighting process; the countersignature watching unit is used for analyzing countersignature data of the man-machine countersignature and providing a graphic display interface after the countersignature is finished; the double-disk fighting unit also has flexible progress bar dragging and speed multiplying playing functions.
In the present invention, the human-machine confrontation module 710 includes:
the competition course generating module 711 is used for generating a competition cycle course and realizing a man-machine hybrid double-blind competition system through the generated cycle course; in this embodiment, in the generated cyclic course, the contestants compete in turn with each other to ensure the fairness and the rationality of the competition scheme. During the generation process of the cycle course, cycle confrontation competitions in different modes are formulated according to the number of players, for example, when the number of players is small, a single cycle competition system or a double cycle competition system is adopted; when the number of players is excessive, a grouping and circulating competition system is adopted. After the cycle course is formulated, the confrontation course generating module 711 is further configured to encode the system codes of the human confronters and the agents, for example, to name the human confronters and the agents uniformly; through the coding, only the system knows the identity of the competitor, different competitors cannot mutually know the identity information of the other party, and the spectator cannot know the identity information of the competitor. There are various methods of unifying naming, for example, m1 human competitors and m2 agents participating in human-computer competition are mixed by the competition course generation module 711 and given anonymous system codes "1 to (m1+ m2) players", respectively, so that the front-end interface can only display the system codes of the players.
The confrontation scheduling management module 712 is configured to automatically schedule a human-computer confrontation process according to a cyclic course, that is, the confrontation scheduling management module 712 automatically generates a plurality of confrontation rooms according to the course arrangement, and schedules players to reach the operation interfaces of the seats corresponding to the corresponding rooms, so as to enter the confrontation process, and implement conventional operations such as selecting scenes, opponents, seats and the like without intervention of confrontation personnel, thereby effectively preventing the players from obtaining some operation information of the opponents in the course operation, assisting in judging the identity of the opponents, ensuring the conditions of double-blind confrontation, and further improving the accuracy and effectiveness of the external evaluation information of the intelligent objects collected by the turing questionnaire module 740. For example, in a chess deduction system, the form of the confrontation is a game confrontation which must decide two seats of red and blue, so as to plan for a scene of summarizing the confrontation. Specific conditions for confrontation are pointed out, including map selection, combat types, positions of control points, force configuration and initial positions of two parties and the like, and some common chess deductions are given as follows: mountain passage seizing control warfare, water net paddy field warfare, moderate-lodging land warfare and the like. For example, according to the course arrangement, if player No. 1 and player No. 2 are to develop a confrontation under the assumption of mountain channel racing, player No. 1 executes the red square and player No. 2 executes the blue square, the confrontation scheduling management module 712 generates a confrontation room under the assumption, and directly schedules player No. 1 to enter the operation interface of the red square seat in the room and player No. 2 to enter the operation interface of the blue square seat in the room.
The man-machine confrontation function module 713 is used for completing man-machine confrontation in a simulation environment through a human confronter and an intelligent agent under the process scheduling of the man-machine confrontation; the man-machine confrontation function module 713 is a basic module of the man-machine confrontation system supporting the Turing test mode, and is used for game confrontation of human confronters and intelligent agents for developing cognition and decision-making in a simulation environment. The human-machine confrontation function module 713 includes:
a confrontational environment simulation module 714 for generating a simulated environment; for example, in a chess push system, the simulation environment is embodied as a simulation map area with different terrain and topography, and a plurality of confrontation environments can be operated in parallel in the chess push system.
The engine kernel module 715 is used for receiving action instructions sent by human opponents and intelligent agents, updating the environment state and battlefield situation of the simulation environment according to the action instructions, and generating real-time deduction data; the engine kernel module 715 is connected to a rule knowledge base and a decision knowledge base, and updates the environment state and the battlefield situation of the simulation environment according to the rule knowledge base and the decision knowledge base, and calculates and generates real-time deduction data.
The UI module 716 is configured to perform interaction and information exchange between the confrontation environment and the human confronter, parse and display real-time deduction data to the human confronter, and send an action instruction sent by the human confronter to the engine kernel module 715; the UI module 716 displays the analyzed real-time deduction data through an image interface; meanwhile, the UI module 716 is further configured to filter identity information of the human confronter, so that the human confrontation function module 713 cannot acquire real identity information of the player, and anonymity of human confrontation is ensured. The UI module 716 includes a first countermeasure unit and a first environment unit, where the first countermeasure unit is used for pre-battle deployment, action generation, and resource clearing to implement a pre-battle deployment action list of human competitors, receive battlefield situation and output action instructions during the countermeasure, and clear a full countermeasure flow of data resources after the countermeasure; the first environment unit is used for operating the confrontation environment so as to realize resetting, pushing, saving and restoring of the environment.
The AI module 717 is used for interacting and exchanging information with the agent against the environment, analyzing real-time deduction data to the agent, and sending an action instruction sent by the agent to the engine kernel module 715; meanwhile, the AI module 717 is further configured to filter the identity information of the agent, so that the man-machine confrontation function module 713 cannot acquire the real identity information of the player, and the anonymity of the man-machine confrontation is ensured. The AI module 717 comprises a second countermeasure unit and a second environment unit, wherein the second countermeasure unit is used for initializing, deploying before battle, generating actions and clearing resources, so as to realize the full countermeasure flow of initializing countermeasure basic information of the agent, deploying an action list before countermeasure, receiving battlefield situation and outputting action instructions during countermeasure and clearing resources such as data and models after countermeasure; the second environment unit is used for operating the countermeasure environment to realize the resetting, pushing, saving and restoring of the environment.
The UI module 716 and the AI module 717 filter out the identity information of human competitors and agents by using a masking technique.
The confrontation data analysis module 730 obtains the performance data of the agent according to the counteroffer data of the multiple disks, including:
acquiring operation data of the intelligent agent in a man-machine countermeasure process according to the counteroffer data, and respectively acquiring efficiency data of the intelligent agent from a plurality of different aspects according to the operation data; wherein the plurality of different aspects include: available confrontation resource application capacity, confrontation deployment and planning capacity, awareness and coping capacity for enemy actions and situation changes and intelligent agent remarkable characteristics; the operation data of the agents corresponding to all aspects are respectively as follows: the method comprises the following steps of self resource application class data (for example, in a military chess deduction system, the number of actions of an intelligent body for operating different operators and the number of different weapons in all attack actions of the intelligent body), self strategy class data (for example, in the military chess deduction system, the motion track and the heat degree of track positions of the operators operated by the intelligent body, the action number of the intelligent body for cooperatively attacking firepower among the operators), enemy perception class data (for example, in the military chess deduction system, the average time interval between the actions of the intelligent body and the actions of an opponent and the action number of the intelligent body in unit time), and human-machine resolution characteristic class data (for example, in the military chess deduction system, the number of illegal operations of the intelligent body).
The turing questionnaire module 740 includes a confrontation person questionnaire unit and a spectator person questionnaire unit; wherein, the confrontation personnel questionnaire unit collects guessing and evaluating information of the human confronters to the opponents through the UI module 716 in a man-machine interaction mode of an electronic questionnaire after each opponent is ended; the interviewer questionnaire unit collects guessing and evaluation information of the interviewer to the interviewer in a man-machine interaction mode of an electronic questionnaire through the interview module (the real identity information of the interviewer is shielded) after the interview of each opponent is finished. As an alternative, the design scheme of the turing questionnaire is as follows: the Turing questionnaire comprises guesses of the appraiser on the identity of the competitor and scores of various aspects of the competitor performance; guessing the opponent's identity is: guessing whether the competitor is a human or an agent; the various scores for the performance of the competitors are: within the standard score range of 0-P, the available countermeasure resource application capacity, the countermeasure deployment and planning capacity, the perception and coping capacity of the enemy action and situation change and the obvious characteristics of the intelligent agent of the competitor are scored and evaluated.
The confrontation achievement management module 750 comprises a system adjudication score management unit, an efficiency data management unit, a Tuoling questionnaire result management unit and a statistical unit; the system adjudication score management unit is used for storing an artificial confrontation result, namely a system adjudication score, and the artificial confrontation result comprises the following components: attack score, seize score, remaining operator score, total score and net win score; the efficiency data management unit is used for storing the efficiency data of the intelligent agent; the picture questionnaire result management unit is used for storing external evaluation information of human confronters and intelligent agents; the statistical unit is used for carrying out statistical analysis on the human-computer confrontation result, the external evaluation information of the intelligent body and the efficiency data of the intelligent body, and obtaining the capability test result of the intelligent body according to the statistical analysis result.
As an alternative, the man-machine confrontation system supporting the turing test mode further comprises a confrontation database module, wherein the confrontation database module is used for storing and managing countervailing data of the compound disk; the countermeasure database module generates a file after each countermeasure node is ended, the file records the whole process of the whole countermeasure, the file comprises a plurality of frame data, each frame data comprises a plurality of fields and is divided into a basic field and an action field, the basic field records the basic information at the time step, and the action field records the action executing condition at the time step. For example, in a war game deduction system, the countercheck data is in the form of json files, and each game of countercheck is finished and then a json file is generated to record the whole process of the whole game of countercheck. The multi-disc confrontation data stores the deduction situation information according to the time sequence, one frame of data is recorded every 1 second, and each frame of data comprises a plurality of fields which are divided into a basic field and an action field. Basic fields record basic information under the time step, such as operator information, time information, control point stealing information, system arbitration scores and other fields; the action field records the condition of executing the action at the time step, and is related to specific actions, such as aiming point information, shooting arbitration and other fields. If no action occurs at this time step, the action field in the frame is empty, and only the base field is reserved.
The following describes the testing method of the smart agent provided by the present invention with reference to fig. 2, and the testing method of the smart agent described below and the above-described man-machine confrontation system supporting the turing test mode can be referred to correspondingly. The intelligent agent testing method is shown in fig. 2 and comprises the following steps:
s100, acquiring a man-machine confrontation result and multi-disc confrontation data of man-machine confrontation; wherein, human-computer confrontation is completed by human confronters and intelligent agents in a double-blind environment; in the step, the human-computer confrontation is completed by the human confronter and the intelligent body in a double-blind environment, and the step comprises the following steps:
s110, generating a cyclic competition course of the confrontation, and coding system codes of human confronters and intelligent agents; wherein, the system code of the human confronter and the intelligent agent is coded for creating a double-blind environment; before generating a cyclic competition course of the confrontation, determining human confronters and agents participating in man-machine confrontation; in this embodiment, players whose ability levels are known through various events are selected and have different ability levels so as to be analogized as the ability of the agent; specifically, the selected human competitors have higher confrontation level and rich confrontation experience in the confrontation application scene, so that effective comparison evaluation with the intelligent agent can be carried out, and the confrontation application scene can be obtained by means of inviting of elite players. In this embodiment, the human opponent system sends invitations to m human elite players ahead according to the total score of the user's historical oppositions, thereby determining the human opponents participating in the human opponent. Meanwhile, the selected intelligent agent also needs to have a higher deduction level so as to enable the testing process of the intelligent agent to be more challenging, and more comprehensive and more valuable evaluation on the intelligent agent can be obtained. In this embodiment, the selection of the agents adopts a mechanical-mechanical confrontation mode, and it is assumed that the number of agents who enter into the evaluation is large, so that the mechanical confrontation in the two stages of the election game and the promotion game is adopted to obtain the agents that finally enter into the mechanical confrontation. The method comprises the following specific steps:
s111, developing a grouped circulating competition of the mechanical countermeasures in a countermeasure environment with the initial configuration determined to be unchanged, and entering the promotion competition by N intelligent agents before selecting according to the integral;
in this embodiment, in the election competition stage of step S111, the mechanical countermeasure is performed under the initial assumption that is not changed by using the grouping cycle point competition system. In order to reduce the influence of the machine on the chance and the game asymmetry, a plurality of battles are specified between the same opponent, each battle has two hands, and after one hand is finished, the two hands change and exchange seats, for example, when the No. 1 player in the first game carries on the basketball side, the No. 2 player carries on the red side, and after the second game carries on the basketball side, the No. 1 player carries on the red side, and the No. 2 player carries on the basketball side. Therefore, each player and the opponent develop multiple countermeasures, and finally N agents before being drawn out are selected to enter the promotion competition of the second stage.
S112, a double-circulation match of the mechanical-mechanical countermeasures is developed in the initially configured and dynamically adjusted countermeasures environment, the former N intelligent body promotion Turing test man-machine countermeasures playground matches are selected according to the points, and N is smaller than N.
In this embodiment, in the promotion competition stage of step S112, a two-cycle point competition system is adopted to perform the mechanical competition under the dynamically adjusted online assumption. And the same opponent is used for fighting a plurality of fields, each field is used for fighting two rounds, and after one round is finished, the opponents exchange seats to carry out the confrontation. Finally, the top n strong promotion levels are selected to carry out the man-machine confrontation final match with the man-machine confronters. The opportunistic thought in this stage is to increase the emergency situation at random on the basis of the initial thought, such as randomly setting roadblocks, changing positions of robbing control points, changing initial positions of military forces, reinforcing the military forces in the confrontation process and the like. The opportunistic setting can provide flexible scene selection for confrontation implementation, increase the confrontation difficulty without losing reasonableness, test the autonomous strain capacity of the intelligent agent more and provide higher requirements for the research and development of the intelligent agent.
The process of generating the cyclic competition course of the confrontation comprises the following steps: the method comprises the following steps of (1) compiling a mixture of human competitors and agents without distinguishing identities into a large group, setting a competition scene through a competition course generation module 711, and generating a competition cycle course; for example, taking a war act deduction as an example: the specific scenes of the confrontation are summarized by imagination and divided into initial imagination and opportunistic imagination. The initial plan refers to that various configurations in each confrontation condition are fixed, and the opportunistic plan refers to that certain random configurations can appear in each confrontation condition, such as increasing roadblocks with random positions, changing positions of control-taking points and the like. In this embodiment, the form of the game is determined as a circular confrontation, and multiple rounds of confrontations are played, so that all players can play one confrontation with each other. Multiple games are simultaneously carried out in each round, two games are played between different players, each game adopts one game and two games, and after the game is finished, the two players change seats. Specific contents of the course include a confrontational scene, a confrontational round, and an opponent's schedule for each round. In this embodiment, the system administrator sets the confrontation scene through the confrontation course generating module 711, for example, the confrontation scene can be set as mountain passage fighting envisaged by the opponent. The competition course generating module 711 then automatically determines the competition rounds and the arrangement of the opponents in each round according to the two-cycle competition system, and generates a round course of the competition.
There are various methods for encoding the system code numbers of human competitors and agents, for example, uniformly naming human competitors and agents; through the coding, only the system knows the identity of the competitor, different competitors cannot mutually know the identity information of the other party, and the spectator cannot know the identity information of the competitor. There are various methods of unifying naming, for example, m1 human competitors and m2 agents participating in human-computer competition are mixed by the competition course generation module 711 and given anonymous system codes "1 to (m1+ m2) players", respectively, so that the front-end interface can only display the system codes of the players.
And S120, automatically scheduling a man-machine confrontation process according to the circular competition process, and completing the man-machine confrontation in a simulated environment through the human confronters and the intelligent bodies under the scheduling of the man-machine confrontation process. In this step, under the condition that the previous work ensures the man-machine double-blind circulation, the confrontation scheduling management module 712 centrally schedules the whole confrontation process according to the arrangement of the race course. The confrontation scheduling management module 712 starts the engine kernel module 715 according to the specific conditions of the confrontation to generate a confrontation room meeting the conditions, matches the corresponding seats of all the players entering the corresponding room according to the course arrangement, and schedules the players to directly enter the confrontation process until all rounds of confrontation are finished.
Accomplishing human-machine confrontation in a simulated environment by human confrontants and agents includes:
s121, generating a simulation environment through the confrontation environment simulation module 714;
s122, receiving action instructions sent by human confronters and intelligent agents, updating the environment state and the battlefield situation of the simulation environment according to the action instructions, generating real-time deduction data, and completing man-machine confrontation; the method specifically comprises the following steps: after receiving the signals that the preparation of the players of the two parties is finished, the system starts to time and formally enters a confrontation state; both players perceive and judge the current environmental state and battlefield situation, select in the selectable action set according to the perception result, and make action decision; the engine kernel module 715 receives the action instruction of the player, updates the environment state and the battlefield situation of the simulation environment according to the action instruction, and feeds back the updated environment state and the battlefield situation to the user terminal through the UI module 716 and the AI module 717. Repeating the step S122 until meeting the confrontation ending condition, and ending the local confrontation; after the opposing competition of the local bureau ends, the opposing competition scheduling management module 712 schedules the exchanging seats of the players of both parties, enters the appointed competition process to carry out the hand-over competition, and repeats the steps S121 to S122 until the competition of one (two bureaus) is completed.
Wherein, before the man-machine confrontation begins, still include: filtering the identity information of the human competitor and the agent through the UI module 716 and the AI module 717, respectively, so that the competitor's identity information cannot be known to the contestants participating in the competition; the UI module 716 and AI module 717 employ masking techniques to filter out identity information of human competitors and agents.
S130, after the confrontation is finished, the man-machine confrontation result is transmitted to the confrontation result management module 750 through the engine kernel module 715, and meanwhile, the countermark data is transmitted to the confrontation database module for persistent storage. In this embodiment, after the competition is completed, the engine kernel module 715 sends the man-machine competition result, that is, the system adjudication score obtained by the player to the competition result management module 750 for unified storage management, where the system adjudication score includes: attack score, seize score, remaining operator score, total score and net win score; the statistic unit in the confrontation achievement management module 750 ranks the achievements of human confronters and intelligent agents according to the system adjudication score, which specifically comprises the following steps: the statistic unit is used for counting the large scores and the small scores of the players in all the participating games according to the system adjudication scores obtained by the players in each game, and then ranking all the players according to the scores. Wherein, the large score is the number of the player winning in all the participating competitions, and the small score is the sum of the systematic adjudication net scores of the player in all the participating competitions.
S200, acquiring efficiency data of the intelligent agent according to the confrontation data of the real copy disc, and acquiring external evaluation information of the intelligent agent in the form of a Tuoling questionnaire;
in this step, obtaining performance data of the agent according to the countermeasures data includes:
the countercheck data stored in the countercheck database module is read by the countercheck data analysis module 730, the operation data of the intelligent agent in the man-machine countercheck process is obtained according to the countercheck data, and the efficiency data of the intelligent agent is respectively obtained from a plurality of different aspects according to the operation data; alternatively, the acquired operation data is mapped to a score range of 0 to P in degree as an evaluation score S of the player in the corresponding dimensionj0 denotes the lowest level of player expression in the dimension, P denotes the highest level of player expression in the dimension, and P denotes>0. Several different aspects include: the system can be used for resisting resource application capacity, resisting deployment and planning capacity, sensing and coping capacity for enemy action and situation change and remarkable characteristics of an intelligent agent. The operation data of the agents corresponding to each aspect are respectively as follows: the own resource application class data, the own strategy class data, the enemy perception class data and the man-machine distinguishing characteristic class data.
The acquisition of performance data in different aspects comprises:
obtaining the evaluation score of the available countermeasure resource application capability of the intelligent agent according to the resource application class data of the intelligent agent; for example, in a war and chess deduction system, the application capacity of the available countermeasure resources of an intelligent agent is embodied in two aspects of operator comprehensive application capacity and weapon comprehensive application capacity; the comprehensive operator application capacity is obtained according to the number of actions of the intelligent agent for operating different operators, and the more balanced the action proportion among the different operators is, the stronger the comprehensive operator application capacity is; for example, the combat operators operated by players include various types such as infantry, chariot, tank, patrol missile, artillery and the like, the action quantity of different types of operators in the overall countermeasure is counted, the proportion of the action quantity is used as quantitative data, and the more balanced the proportion is, the stronger the comprehensive application capacity of the operators is; the comprehensive utilization capacity of the weapons is obtained according to the use number of different weapons in all the striking actions of the intelligent agent, and the more balanced the use proportion among the weapons, the stronger the comprehensive utilization capacity of the weapons is; for example, the shooting weapons are divided into various types such as direct-fired guns, rapid-fired guns, missiles, infantry light weapons and vehicle-mounted light weapons, the number of different weapons used by all shooting actions of the player in the whole confrontation is counted, the ratio of the using number of the different weapons is used as quantitative data, and the more balanced the ratio is, the stronger the comprehensive utilization capacity of the weapons is.
Obtaining the evaluation score of the confrontation deployment and planning capacity of the intelligent agent according to the policy class data of the intelligent agent; for example, in a war game deduction system, the confrontation deployment and planning capacity of an intelligent body is embodied in two aspects of tactical novelty and action cooperative control capacity; tactical novelty is obtained according to the heat of the motion track and the track position of an operator operated by the intelligent agent, the heat of the motion track and the track position of the operator operated by the intelligent agent is compared with the heat of a historical track, and the lower the matching degree is, the more novel the tactical is; for example, movement is the basis for operators to execute high-level actions, and a movement path is the most intuitive embodiment of a player tactical strategy, so that the movement locus of all operators belonging to the player on the whole map and the heat (measured by the total time length) of each position on the locus are counted, and the longer the operator is at a certain position, the higher the heat of the position is; after statistics is finished, comparing the track heat table of the local countermeasure with the track heat of all historical countermeasures under the same countermeasure condition, wherein the lower the matching degree is, the more novel the battle of the field countermeasure is; the action cooperative control capability is obtained according to the action number of cooperative fire striking of the intelligent agent between operators, and the more cooperative actions, the stronger the action cooperative control capability; for example, the cooperative fire striking action is defined as: the attack behaviors sent by two (and more) operators have respective fire shooting action occurrence time intervals not exceeding a threshold value T, and the distance between targets shot by two adjacent fire shots not exceeding a threshold value D, the number of cooperative fire striking actions executed by players in the whole confrontation is counted, and the more cooperative actions, the stronger the cooperative control capability of the actions.
Obtaining the evaluation score of the perception and the coping ability of the intelligent agent on the action and situation change of the enemy according to the perception class data of the intelligent agent on the enemy; for example, in a war game deduction system, the perception and coping ability of an intelligent body on the action and situation change of an enemy is embodied in two aspects of the perception judgment ability of an opponent and the telepresence strain ability; the opponent perception judging capability is obtained according to the average time interval between the intelligent body action and the opponent action, and the smaller the time interval is, the more the player can perceive the change of the opponent action and timely respond; specifically, an enemy operator action list in the visual field range of the player and a my operator action list following the corresponding action of the enemy are counted, the time interval of the action at the corresponding index of the two action lists is calculated, and then the average value of all the intervals is obtained; the smaller the average time interval is, the more the enemy can discover the action of the enemy and respond, namely the stronger the adversary perception judgment ability is; the opportunistic strain capacity is obtained according to the action quantity of the agent in unit time, and the greater the action density is, the stronger the strain capacity is; for example, the opportunistic responses are expressed in various action types such as shooting, robbing control and moving, and whether a player can make a reasonable and timely action decision under the current situation is an important factor for measuring the strain capacity.
Obtaining evaluation scores of the remarkable characteristics of the intelligent agent according to the man-machine distinguishing characteristic class data of the intelligent agent; for example, in a war and chess deduction system, an evaluation score of the remarkable characteristics of an intelligent body is obtained according to the number of illegal operations of the intelligent body, if an action violating the human operation principle occurs in the countermeasure process, punishment is made on the evaluation score, and the more unreasonable types of conditions, the greater the punishment degree is; for example, in a war game pursuit, if a player has an ideal situation in which the manual operation speed of a human being is difficult to achieve, such as a plurality of operators shooting at the same time or an infantry and a cruise missile operator getting off at the same time, the player can be significantly judged as an agent. And counting the number of the types of unreasonable conditions in the whole fight, wherein the larger the number is, the larger the fraction punishment degree is.
Thus, the performance data for the agent obtained from 4 different aspects is expressed as S1,S2,S3,S4]。
The method for acquiring the external evaluation information of the intelligent agent in the form of the Tuoling questionnaire comprises the following steps:
after each confrontation is completed, guessing and evaluating information of the confrontation personnel to the opponent and the spectator to the confrontation personnel is collected through the picture questionnaire module 740. The Turing questionnaire comprises guesses of the appraiser on the identity of the competitor and scores of various aspects of the competitor performance; guessing the opponent's identity is: guessing whether the competitor is a human or an agent; the various scores for the performance of the competitors are: within the standard score range of 0-P, the available countermeasure resource application capacity, the countermeasure deployment and planning capacity, the perception and coping capacity of the enemy action and situation change and the obvious characteristics of the intelligent agent of the competitor are scored and evaluated. Wherein, the confrontation achievement management module 750 reads and stores the guess and evaluation information from the turing questionnaire module 740, and then the statistics unit counts the times that each player is guessed as human and guessed as machine in all the turing questionnaires, and calculates the man-machine guess misjudgment rate of the intelligent agent; the man-machine guess misjudgment rate is calculated according to the times of guessing as human and the times of guessing as machine of the player in all the Tuoling questionnaires, and specifically comprises the following steps:
wherein, PMisjudgmentGuessing the misjudgment rate t for humanHuman beingNumber of times guessed as human, tMachine with a rotatable shaftThe number of times it is guessed to be a machine.
S300, obtaining a capability test result of the intelligent agent according to the man-machine confrontation result, the external evaluation information of the intelligent agent and the efficiency data of the intelligent agent. In the step, the capability test result of the intelligent agent comprises a comprehensive evaluation score, a man-machine guess misjudgment rate and a system score; the comprehensive evaluation score is obtained according to the efficiency data of the agent and the external evaluation information of the agent, for example, a score sum of the efficiency data of the agent in the four aspects of the available countermeasure resource application capacity, the countermeasure deployment and planning capacity, the perception and coping capacity for the action and situation change of the enemy and the remarkable feature of the agent is obtained as an efficiency score, a score sum of the external evaluation information of the agent in the four aspects of the available countermeasure resource application capacity, the countermeasure deployment and planning capacity, the perception and coping capacity for the action and situation change of the enemy and the remarkable feature of the agent is obtained as an external evaluation score, and a sum or an average of the efficiency score and the external evaluation score is obtained as the comprehensive evaluation score. The system score is obtained according to a human-computer countermeasure result, namely a system adjudication score, and comprises the following steps: the system adjudges the total and average score, total points, victory arenas, battle-defeat arenas, victory, heat and ELO (ELO) scores of the scores; wherein, the total integral is the winning times in each fight, and the heat is the fight in which the user participates in the total.
During and/or after the man-machine competition, the method further comprises the following steps:
acquiring real-time confrontation data in a man-machine confrontation process;
and analyzing the real-time countermeasure data and/or the multi-disc countermeasure data, and performing anonymous 2D or 3D countermeasure graphic display and switching display of different battlefield situations according to the analysis result of the real-time countermeasure data and/or the multi-disc countermeasure data.
The exhibition of the confrontation image and the exhibition of the switching of different battlefield situations are convenient for observers to observe and massage, wherein the external evaluation information of the intelligent body comprises evaluation information made by human confronters and evaluation information made by observers.
The invention provides a man-machine confrontation system supporting a Turing test mode and an intelligent body test method.A Turing questionnaire module is creatively introduced into the man-machine confrontation system, external evaluation information of an intelligent body is acquired through the Turing questionnaire module, effectiveness data of the intelligent body is obtained through analysis of confrontation data in the man-machine confrontation process, and the decision-making capability of the intelligent body is tested by combining a man-machine confrontation result, the external evaluation information of the intelligent body and the effectiveness data of the intelligent body, so that comprehensive and effective test of the cognition and decision-making capability of the intelligent body is realized; meanwhile, the system center scheduling is carried out through the countermeasure scheduling management module, the front-end anonymous processing is completed through unified coding of the player system codes, the rear-end interface information filtering is carried out through the UI module and the AI module, the man-machine double-blind countermeasure environment is realized, the man-machine countermeasure process is ensured to be completed in the double-blind environment, and therefore the fairness of data in the intelligent agent cognition and decision-making capability test process is ensured; in addition, the fairness of data in the process of the intelligent agent cognition and decision-making ability test is further improved through anonymous observation (the observation interface displays the system code and does not display the true identity of an opponent), and therefore the accuracy of the test result of the intelligent agent decision-making ability is improved.
Fig. 3 illustrates a physical structure diagram of an electronic device, which may include, as shown in fig. 3: a processor (processor)810, a communication Interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication Interface 820 and the memory 830 communicate with each other via the communication bus 840. Processor 810 may invoke logic instructions in memory 830 to perform an agent testing method comprising: acquiring a man-machine confrontation result and countercheck data of man-machine confrontation; wherein, human-computer confrontation is completed by human confronters and intelligent agents in a double-blind environment;
acquiring efficiency data of the intelligent agent according to the counterwork data of the multiple disks, and acquiring external evaluation information of the intelligent agent in the form of a map questionnaire;
and acquiring a capability test result of the intelligent agent according to the man-machine confrontation result, the external evaluation information of the intelligent agent and the efficiency data of the intelligent agent.
In addition, the logic instructions in the memory 830 may be implemented in software functional units and stored in a computer readable storage medium when the logic instructions are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product, the computer program product comprising a computer program, the computer program being stored on a non-transitory computer-readable storage medium, wherein when the computer program is executed by a processor, the computer is capable of executing the method for testing an agent provided by the above methods, the method comprising: acquiring a man-machine confrontation result and countercheck data of man-machine confrontation; wherein, human-computer confrontation is completed by human confronters and intelligent agents in a double-blind environment;
acquiring efficiency data of the intelligent agent according to the counterwork data of the multiple disks, and acquiring external evaluation information of the intelligent agent in the form of a map questionnaire;
and acquiring a capability test result of the intelligent agent according to the man-machine confrontation result, the external evaluation information of the intelligent agent and the efficiency data of the intelligent agent.
In yet another aspect, the present invention also provides a non-transitory computer-readable storage medium having stored thereon a computer program, which when executed by a processor, implements an agent testing method provided by the above methods, the method comprising: acquiring a man-machine confrontation result and countercheck data of man-machine confrontation; wherein, human-computer confrontation is completed by human confronters and intelligent agents in a double-blind environment;
acquiring efficiency data of the intelligent agent according to the counterwork data of the multiple disks, and acquiring external evaluation information of the intelligent agent in the form of a map questionnaire;
and acquiring a capability test result of the intelligent agent according to the man-machine confrontation result, the external evaluation information of the intelligent agent and the efficiency data of the intelligent agent.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A human-machine confrontation system supporting a turing test mode, comprising:
the man-machine confrontation module is used for finishing man-machine confrontation; wherein the human-machine confrontation is completed by a human confronter and an agent in a double-blind environment;
the confrontation data acquisition module is used for acquiring a man-machine confrontation result and countermark confrontation data of man-machine confrontation;
the confrontation data analysis module is used for acquiring the efficiency data of the intelligent agent according to the counteroffer data of the multi-disk;
the intelligent agent comprises a smart questionnaire module, a database module and a database module, wherein the smart questionnaire module is used for acquiring external evaluation information of the intelligent agent in a smart questionnaire form;
and the confrontation result management module is used for receiving and storing the man-machine confrontation result, the external evaluation information of the intelligent agent and the efficiency data of the intelligent agent, and acquiring the capability test result of the intelligent agent according to the man-machine confrontation result, the external evaluation information of the intelligent agent and the efficiency data of the intelligent agent.
2. The system of claim 1, wherein the warrior countermeasure module comprises:
the confrontation course generating module is used for generating a confrontation circular course and coding the system codes of the human confronters and the intelligent agents; wherein encoding system codes of the human competitor and the agent is used to create the double-blind environment;
the confrontation scheduling management module is used for automatically scheduling the man-machine confrontation process according to the circular race course;
and the man-machine confrontation function module is used for completing the man-machine confrontation in a simulated environment through the human confronter and the intelligent body under the process scheduling of the man-machine confrontation.
3. The system of claim 2, wherein the human-machine confrontation function module comprises:
a confrontational environment simulation module for generating the simulated environment;
the engine kernel module is used for receiving action instructions sent by the human opponents and the intelligent agents, updating the environment state and the battlefield situation of the simulation environment according to the action instructions and generating real-time deduction data;
the UI module is used for analyzing and displaying the real-time deduction data to the human opponent, sending an action instruction sent by the human opponent to the engine kernel module, and filtering identity information of the human opponent;
and the AI module is used for analyzing the real-time deduction data to the intelligent agent, sending the action instruction sent by the intelligent agent to the engine kernel module, and filtering the identity information of the intelligent agent.
4. The system of claim 1, wherein the confrontation data analysis module obtains performance data of the agent according to the countermeasures data comprises:
according to the countercheck data, acquiring operation data of the intelligent agent in the man-machine countercheck process, and respectively acquiring performance data of the intelligent agent from a plurality of different aspects according to the operation data; wherein the operational data of the agent comprises: the own resource application class data, the own strategy class data, the enemy perception class data and the man-machine distinguishing characteristic class data.
5. The system of claim 1, further comprising a confrontation and massage module;
the confrontation data acquisition module is also used for acquiring real-time confrontation data in the man-machine confrontation process;
the countercheck viewing module is used for analyzing the real-time countercheck data and/or the countercheck data, and performing anonymous 2D or 3D countercheck graphic display and switching display of different battlefield situations according to the analysis result of the real-time countercheck data and/or the countercheck data.
6. An agent testing method, comprising the steps of:
acquiring a man-machine confrontation result and countercheck data of man-machine confrontation; wherein the human-machine confrontation is completed by a human confronter and an agent in a double-blind environment;
acquiring efficiency data of the intelligent agent according to the countermeasures data, and acquiring external evaluation information of the intelligent agent in the form of a Tuoling questionnaire;
and acquiring a capability test result of the intelligent agent according to the man-machine confrontation result, the external evaluation information of the intelligent agent and the efficiency data of the intelligent agent.
7. The agent testing method of claim 6, wherein completing the human-machine confrontation in a double-blind environment by a human confronter and an agent comprises:
generating a cyclic course of confrontations and encoding the system codes of the human confronters and the agents; wherein encoding system codes of the human competitor and the agent is used to create the double-blind environment;
and automatically scheduling the process of the man-machine confrontation according to the circular competition process, and finishing the man-machine confrontation in a simulated environment through the human confronter and the intelligent agent under the scheduling of the process of the man-machine confrontation.
8. The agent testing method of claim 7, wherein completing the human-machine confrontation in a simulated environment by the human confronter and the agent comprises:
an environment generation step: generating the simulated environment;
a countermeasure step: receiving action instructions sent by the human opponents and the intelligent agents, updating the environment state and the battlefield situation of the simulation environment according to the action instructions, and generating real-time deduction data;
cycling the step of confrontation until the human-machine confrontation is completed;
wherein, before the countermeasure step, the method further comprises: filtering identity information of the human competitor and the agent.
9. The agent testing method of claim 6, wherein obtaining performance data of the agent based on the countermeasures data comprises:
according to the countercheck data, acquiring operation data of the intelligent agent in the man-machine countercheck process, and respectively acquiring performance data of the intelligent agent from a plurality of different aspects according to the operation data; wherein the operational data of the agent comprises: the own resource application class data, the own strategy class data, the enemy perception class data and the man-machine distinguishing characteristic class data.
10. The agent testing method of claim 6, further comprising, during and/or after the human-machine confrontation,:
acquiring real-time confrontation data in the man-machine confrontation process;
and analyzing the real-time countermeasure data and/or the countersignature data, and performing anonymous 2D or 3D countersignature graphic display and switching display of different battlefield situations according to the analysis result of the real-time countermeasure data and/or the countersignature data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111328333.3A CN114154611B (en) | 2021-11-10 | 2021-11-10 | Man-machine confrontation system supporting Turing test mode and intelligent agent test method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111328333.3A CN114154611B (en) | 2021-11-10 | 2021-11-10 | Man-machine confrontation system supporting Turing test mode and intelligent agent test method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114154611A true CN114154611A (en) | 2022-03-08 |
| CN114154611B CN114154611B (en) | 2022-12-06 |
Family
ID=80459412
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111328333.3A Active CN114154611B (en) | 2021-11-10 | 2021-11-10 | Man-machine confrontation system supporting Turing test mode and intelligent agent test method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114154611B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114706795A (en) * | 2022-06-07 | 2022-07-05 | 湖南智擎科技有限公司 | Turing test method, device and system for SaaS artificial intelligence application |
| CN114882755A (en) * | 2022-04-27 | 2022-08-09 | 中国人民解放军军事科学院战略评估咨询中心 | Multi-scheme autonomous switching method for intelligent confrontation |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101964019A (en) * | 2010-09-10 | 2011-02-02 | 北京航空航天大学 | Against behavior modeling simulation platform and method based on Agent technology |
| US20110191820A1 (en) * | 2010-01-29 | 2011-08-04 | Christopher Liam Ivey | System and Method for Restricting Access to a Computer System to Live Persons by Means of Semantic Association of Images |
| US20130236878A1 (en) * | 2012-03-12 | 2013-09-12 | Alexey Saltanov | Method for Testing and Developing Intelligence |
| CN104516939A (en) * | 2013-10-04 | 2015-04-15 | 高霆科技股份有限公司 | Parallel hardware search system for constructing artificial intelligent computer |
| CN109636699A (en) * | 2018-11-06 | 2019-04-16 | 中国电子科技集团公司第五十二研究所 | A kind of unsupervised intellectualized battle deduction system based on deeply study |
| CN109692431A (en) * | 2019-01-23 | 2019-04-30 | 郑州大学 | The double interactive balanced ability of human body evaluation and test of one kind and training system |
| US20190258254A1 (en) * | 2018-02-22 | 2019-08-22 | Alan M. Kadin | System and method for conscious machines |
| CN110694256A (en) * | 2019-09-18 | 2020-01-17 | 徐磊 | Novel emergency computer war game deduction system and method |
| CN112215328A (en) * | 2020-10-29 | 2021-01-12 | 腾讯科技（深圳）有限公司 | Training of intelligent agent, and action control method and device based on intelligent agent |
| CN112295229A (en) * | 2020-10-28 | 2021-02-02 | 中国电子科技集团公司第二十八研究所 | Intelligent game confrontation platform |
| CN112329348A (en) * | 2020-11-06 | 2021-02-05 | 东北大学 | Intelligent decision-making method for military countermeasure game under incomplete information condition |
| CN112364500A (en) * | 2020-11-09 | 2021-02-12 | 中国科学院自动化研究所 | Multi-concurrency real-time countermeasure system oriented to reinforcement learning training and evaluation |
| CN113379054A (en) * | 2021-05-28 | 2021-09-10 | 中国科学院自动化研究所 | Open type intelligent game ecological platform |
-
2021
- 2021-11-10 CN CN202111328333.3A patent/CN114154611B/en active Active
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110191820A1 (en) * | 2010-01-29 | 2011-08-04 | Christopher Liam Ivey | System and Method for Restricting Access to a Computer System to Live Persons by Means of Semantic Association of Images |
| CN101964019A (en) * | 2010-09-10 | 2011-02-02 | 北京航空航天大学 | Against behavior modeling simulation platform and method based on Agent technology |
| US20130236878A1 (en) * | 2012-03-12 | 2013-09-12 | Alexey Saltanov | Method for Testing and Developing Intelligence |
| CN104516939A (en) * | 2013-10-04 | 2015-04-15 | 高霆科技股份有限公司 | Parallel hardware search system for constructing artificial intelligent computer |
| US20190258254A1 (en) * | 2018-02-22 | 2019-08-22 | Alan M. Kadin | System and method for conscious machines |
| CN109636699A (en) * | 2018-11-06 | 2019-04-16 | 中国电子科技集团公司第五十二研究所 | A kind of unsupervised intellectualized battle deduction system based on deeply study |
| CN109692431A (en) * | 2019-01-23 | 2019-04-30 | 郑州大学 | The double interactive balanced ability of human body evaluation and test of one kind and training system |
| CN110694256A (en) * | 2019-09-18 | 2020-01-17 | 徐磊 | Novel emergency computer war game deduction system and method |
| CN112295229A (en) * | 2020-10-28 | 2021-02-02 | 中国电子科技集团公司第二十八研究所 | Intelligent game confrontation platform |
| CN112215328A (en) * | 2020-10-29 | 2021-01-12 | 腾讯科技（深圳）有限公司 | Training of intelligent agent, and action control method and device based on intelligent agent |
| CN112329348A (en) * | 2020-11-06 | 2021-02-05 | 东北大学 | Intelligent decision-making method for military countermeasure game under incomplete information condition |
| CN112364500A (en) * | 2020-11-09 | 2021-02-12 | 中国科学院自动化研究所 | Multi-concurrency real-time countermeasure system oriented to reinforcement learning training and evaluation |
| CN113379054A (en) * | 2021-05-28 | 2021-09-10 | 中国科学院自动化研究所 | Open type intelligent game ecological platform |
Non-Patent Citations (7)
| Title |
|---|
| ZHINAN PENG 等: "Understanding the mechanism of human–computer game: a distributed reinforcement learning perspective", 《INTERNATIONAL JOURNAL OF SYSTEMS SCIENCE》 * |
| 中国科学院自动化研究所: "2021届"庙算杯"人机对抗测试赛成功举行", 《HTTP://WWW.IA.CAS.CN/XWZX/TTXW/202108/T20210809_6155347.HTML》 * |
| 方伟 等: "航空兵智能决策模型的评估方法", 《兵器装备工程学报》 * |
| 李琛 等: "Actor-Critic框架下的多智能体决策方法及其在兵棋上的应用", 《系统工程与电子技术》 * |
| 韩超: "作战推演中智能博弈对抗算法水平评估模型研究", 《舰船电子工程》 * |
| 黄凯奇 等: "人机对抗智能技术", 《中国科学》 * |
| 黄凯奇 等: "视觉图灵：从人机对抗看计算机视觉下一步发展", 《图学学报》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114882755A (en) * | 2022-04-27 | 2022-08-09 | 中国人民解放军军事科学院战略评估咨询中心 | Multi-scheme autonomous switching method for intelligent confrontation |
| CN114706795A (en) * | 2022-06-07 | 2022-07-05 | 湖南智擎科技有限公司 | Turing test method, device and system for SaaS artificial intelligence application |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114154611B (en) | 2022-12-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Clarke et al. | How computer gamers experience the game situation: a behavioral study | |
| Tastan et al. | Learning policies for first person shooter games using inverse reinforcement learning | |
| CN114154611B (en) | Man-machine confrontation system supporting Turing test mode and intelligent agent test method | |
| CN114239228A (en) | Efficiency evaluation method based on modeling and analysis of massive countermeasure simulation deduction data | |
| Kim et al. | Performance evaluation gaps in a real-time strategy game between human and artificial intelligence players | |
| CN113262488B (en) | Control method, device, equipment and storage medium for virtual objects in virtual scene | |
| Lee et al. | Showdown AI competition | |
| Debus | Metagames: on the ontology of games outside of games | |
| Conroy et al. | Modeling player-like behavior for game AI design | |
| WO2022095672A1 (en) | Screen display method and apparatus, device and storage medium | |
| Dahlskog et al. | Mapping the game landscape: Locating genres using functional classification | |
| Tekofsky et al. | Play style: Showing your age | |
| US8834246B1 (en) | Methods and techniques for incorporating game play in a tournament format to develop thinking skills | |
| Bakkes | Rapid adaptation of video game AI | |
| Hagen et al. | Agent Participation in First Person Shooter Games Using Reinforcement Learning and Behaviour Cloning | |
| Xu et al. | MiaoSuan Wargame: A Multi-Mode Integrated Platform for Imperfect Information Game | |
| Duarte | Review of Recent Work on Computational Intelligence in Games | |
| McCabe | Community Perception of Gunplay inModern FPS games: within Call of Duty: Warzone and Rainbow Six: Siege | |
| Korostelyov et al. | Software Platform for Designing and Running Artificial Intelligence Competitions with a Visualization Subsystem | |
| Schoenborn et al. | Towards Case-Based Reasoning in Real-Time Strategy Environments with SEASALT. | |
| Bowen et al. | Adjutant bot: An evaluation of unit micromanagement tactics | |
| Zhou | Where Am I: Players’ Memory of 2D Games’ Maps | |
| Fox | The Development of a Framework for Weapon Balancing in Multiplayer First-Person Shooter Games | |
| Hudson | Social presence in team-based digital games | |
| Chen et al. | AI, virtual worlds, and massively multiplayer online games |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |