CN107945499B - Evaluation method and equipment for teleoperation system - Google Patents
Evaluation method and equipment for teleoperation system Download PDFInfo
- Publication number
- CN107945499B CN107945499B CN201711236258.1A CN201711236258A CN107945499B CN 107945499 B CN107945499 B CN 107945499B CN 201711236258 A CN201711236258 A CN 201711236258A CN 107945499 B CN107945499 B CN 107945499B
- Authority
- CN
- China
- Prior art keywords
- teleoperation
- task
- testing
- teleoperation system
- data corresponding
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08C—TRANSMISSION SYSTEMS FOR MEASURED VALUES, CONTROL OR SIMILAR SIGNALS
- G08C25/00—Arrangements for preventing or correcting errors; Monitoring arrangements
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Supply And Distribution Of Alternating Current (AREA)
- Testing Or Calibration Of Command Recording Devices (AREA)
Abstract
The embodiment of the invention provides a method and equipment for evaluating a teleoperation system, wherein the method comprises the following steps: obtaining control data for at least one teleoperational task, wherein the teleoperational task comprises: controlling the operation of the external equipment through the teleoperation system; determining test data corresponding to the at least one teleoperation task according to the control data; determining a test result of the teleoperation system according to the test data; and displaying the test result and an evaluation result output according to the test result. The method can accurately evaluate the quality of each capability of the teleoperation system and can also provide guidance for the teleoperation system.
Description
Technical Field
The embodiment of the invention relates to the field of teleoperation system evaluation, in particular to a teleoperation system evaluation method and equipment.
Background
Teleoperation is a new development of the traditional remote measuring and controlling technology on the basis of the full combination of the information age, the control technology, the network technology and the simulation technology, is an indispensable key technology for the unmanned, remote and intelligent application of robots, and particularly is along with the continuous development of human space activities and the increasing expansion of the application requirements of complex spaces. .
Teleoperation is developed to the present day, and is abundant and developed, and the operation mode is numerous, and the application field is wider, and various teleoperation systems lay endlessly, and each teleoperation system's each ability is uneven, and each teleoperation system probably has the advantage in many ability, but inevitable also has the limitation in other ability aspect, and any ability limitation is the factor that limits teleoperation system to stronger development.
However, according to different tasks, the capabilities of the teleoperation system and the content of evaluating the teleoperation system are not consistent, and evaluating the capabilities and the comprehensive capabilities of the teleoperation system is an urgent problem in the teleoperation technical field.
Disclosure of Invention
The embodiment of the invention provides an evaluation method and equipment of a teleoperation system, which are used for solving the problem of evaluating various capabilities and comprehensive capabilities of the teleoperation system.
In a first aspect, an embodiment of the present invention provides an evaluation method for a teleoperation system, where the method includes:
obtaining control data for at least one teleoperational task, wherein the teleoperational task comprises: controlling the operation of the external equipment through the teleoperation system; determining test data corresponding to the at least one teleoperation task according to the control data; determining a test result of the teleoperation system according to the test data; and displaying the test result and an evaluation result output according to the test result.
In one possible embodiment, the method further comprises: performing feature classification on the teleoperation system;
wherein the characteristics include at least one of: a remote sharing feature, a teleoperation feature, a telepresence feature, or a remote system feature.
In one possible embodiment, the method further comprises: and establishing a teleoperation task step library corresponding to the operation according to the operation of the teleoperation system for controlling the external equipment.
In one possible embodiment, the determining test data corresponding to the at least one teleoperational task according to the control data includes: testing control data corresponding to a single teleoperation task to obtain test data corresponding to the single teleoperation task; and determining test data corresponding to at least one teleoperation task according to the test data corresponding to the single teleoperation task.
In one possible embodiment, the evaluation result includes at least one of: testing the coverage capability of the teleoperation task; testing the coverage capability of the teleoperation task; testing the safety protection capability of the teleoperation system; testing the autonomous ability and the intelligence of the teleoperation system; testing the real-time processing capability of the teleoperation system; testing the communication capacity of the teleoperation system; testing the man-machine efficacy, the electromechanics and the electrical property of the teleoperation system; testing the delay impact reduction capability of the teleoperation system; the remote operation system carries out testing on the backup, analysis, reproduction and time synchronization capabilities; and testing the synchronicity of the shared operation or the difference tolerance of the shared operation.
In one possible embodiment, the evaluation result is displayed by one of the following ways:
documents, tables, or pictures.
In a possible embodiment, before determining the evaluation data corresponding to the at least one teleoperation task according to the experimental data, the method further includes:
and checking the control data.
In a possible embodiment, after determining the evaluation data corresponding to the at least one teleoperation task according to the experimental data, the method further includes:
and checking the control data.
In a second aspect, an embodiment of the present invention provides an evaluation apparatus for a teleoperation system, including:
an obtaining module, configured to obtain control data of at least one teleoperation task, where the teleoperation task includes: controlling the operation of the external equipment through the teleoperation system;
a first determining module, configured to determine, according to the control data, test data corresponding to the at least one teleoperation task;
the second determining module is used for determining a test result of the teleoperation system according to the test data;
and the display module is used for displaying the test result and the evaluation result output according to the test result.
In one possible embodiment, the apparatus further includes:
the classification module is used for carrying out feature classification on the teleoperation system;
wherein the characteristics include at least one of:
a remote sharing feature, a teleoperation feature, a telepresence feature, or a remote system feature.
In one possible embodiment, the apparatus further includes:
and the establishing module is used for establishing a teleoperation task step library corresponding to the operation according to the operation of the teleoperation system control external equipment.
In a possible embodiment, the obtaining module is configured to test control data corresponding to a single teleoperation task, and obtain test data corresponding to the single teleoperation task; and determining test data corresponding to at least one teleoperation task according to the test data corresponding to the single teleoperation task.
In one possible embodiment, the evaluation result includes at least one of:
testing the coverage capability of the teleoperation task; testing the coverage capability of the teleoperation task; testing the safety protection capability of the teleoperation system; testing the autonomous ability and the intelligence of the teleoperation system; testing the real-time processing capability of the teleoperation system; testing the communication capacity of the teleoperation system; testing the man-machine efficacy, the electromechanics and the electrical property of the teleoperation system; testing the delay impact reduction capability of the teleoperation system; the remote operation system carries out testing on the backup, analysis, reproduction and time synchronization capabilities; and testing the synchronicity of the shared operation or the difference tolerance of the shared operation.
In one possible embodiment, the evaluation result is displayed by one of the following ways:
documents, tables, or pictures.
In one possible embodiment, the apparatus further comprises:
and the checking module is used for checking the test data.
In a possible implementation manner, the proofreading module is further configured to proofread the test data.
According to the evaluation method and the evaluation device for the teleoperation system, the control data of at least one teleoperation task are obtained, and the test data corresponding to the teleoperation task are determined according to the control data; determining a test result of the teleoperation system according to the test data; and displaying the test result and the evaluation result output according to the test result, so that the advantages and disadvantages of various capabilities of the teleoperation system can be accurately evaluated, and guidance can be provided for the teleoperation system.
Drawings
Fig. 1 is a schematic flowchart illustrating an evaluation method for a teleoperation system according to an embodiment of the present invention;
fig. 2 is a schematic diagram illustrating the coverage capability of a teleoperation task according to an embodiment of the present invention;
fig. 3 is a schematic diagram illustrating security protection capability of a remote operating system according to an embodiment of the present invention;
fig. 4 is a schematic diagram of the autonomous capability and intelligence of the teleoperation system provided by the embodiment of the present invention;
fig. 5 is a schematic diagram of real-time processing capability of a teleoperation system according to an embodiment of the present invention;
fig. 6 is a schematic diagram of the communication capability of the remote operation system according to the embodiment of the present invention;
fig. 7 is a schematic diagram of the human-machine efficiency, the electromechanical properties and the electrical properties of the teleoperation system provided by the embodiment of the invention;
fig. 8 is a schematic diagram of a delay impact reduction capability of a teleoperation system according to an embodiment of the present invention;
FIG. 9 is a diagram illustrating the backup, analysis, replication, and time synchronization capabilities of a teleoperation system according to an embodiment of the present invention;
FIG. 10 is a diagram illustrating synchronization of a sharing operation according to an embodiment of the present invention;
FIG. 11 is a schematic diagram of the difference tolerance provided by the embodiments of the present invention;
fig. 12 is a schematic structural diagram of an evaluation device of a teleoperation system according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
For the convenience of understanding of the embodiments of the present invention, the following description will be further explained with reference to specific embodiments, which are not to be construed as limiting the embodiments of the present invention.
Fig. 1 is a schematic flow chart of an evaluation method of a teleoperation system according to an embodiment of the present invention, and with reference to fig. 1, the method specifically includes the following steps:
s101, control data of at least one teleoperation task are obtained.
The teleoperation system involved in the embodiments of the present invention is generally used for remotely controlling external devices, and the external devices may include, but are not limited to: for example, a robot, an unmanned aerial vehicle, and other devices control the robot in a teleoperation manner based on a teleoperation system.
Before S101 is executed, feature classification needs to be performed on the teleoperation system, for example, feature classification needs to be performed on the teleoperation system according to four features, namely, a telesharing feature, a teleoperation feature, a telepresence feature, and a telesystem feature, and the feature classification aims to evaluate the teleoperation system through the four features.
Optionally, as an embodiment of the present invention, a teleoperation task step library corresponding to an operation may be established according to the operation of the teleoperation system controlling the external device, where the teleoperation task step library stores a plurality of operations of controlling the external device through the teleoperation system, for example, a teleoperation task a may control the robot to perform an operation a, and a teleoperation task B may control the robot to perform an operation B, where the teleoperation task includes: the operation of the external device is controlled by the teleoperation system.
After the teleoperation system is subjected to feature classification and the teleoperation task step library is established, all aspects of the teleoperation system can be evaluated, one teleoperation task in the teleoperation task step library is selected to control the external equipment, and experimental data fed back by the external equipment is received.
It should be noted that, in order to ensure the accuracy of the evaluation, at least one teleoperation task in the teleoperation task step library may be teleoperation task within a set time period, so as to obtain experimental data of the teleoperation task.
Optionally, as an embodiment of the present invention, when acquiring the experimental data, there may be human or machine errors, and therefore, the experimental data may also be subjected to a calibration, where the calibration may be an error analysis, or a threshold is set for verification, so as to ensure validity of the evaluation.
And S102, determining test data corresponding to at least one teleoperation task according to the control data.
In this embodiment, the step of evaluating the experimental data corresponding to a single teleoperation task to obtain the evaluation data, and determining the test data corresponding to at least one teleoperation task according to the test data corresponding to the single teleoperation task to specifically obtain the evaluation data may be obtained by:
and S1021, testing the coverage capability of the teleoperation task through the control data.
Referring to fig. 2, the covering capability of the operation mode for the teleoperation task specifically includes: shared teleoperation capability of the teleoperation system, operational mode coverage, operational mode initiation, operational mode termination, operational mode switching, etc.; the main evaluation indexes of the step are as follows: whether the operating mode of the teleoperation system covers the operating mode required by the teleoperation task or not is mainly reflected by the following capacity when the uncertain teleoperation task is faced: which operation modes the teleoperation system has; whether the operation mode can be started, stopped and switched according to the requirement of the teleoperation task; further embodying the time it takes to start, stop and switch the operating mode
And S1022, testing the safety protection capability of the teleoperation system through the control data.
Referring to fig. 3, the security protection capability of the teleoperation system is mainly embodied in its state detection/monitoring capability, misoperation prevention capability, error code checking and fault tolerance capability, prevention simulation capability, quick recovery capability, emergency intervention capability, hardware protection capability, and the like.
The missed detection rate in the state detection/monitoring capability evaluation can be obtained by the following formula:
obtaining the average value of abnormal state detection result parameters under various abnormal modes, considering the difference of abnormal state detection capabilities under different abnormal conditions, and adopting a conservative evaluation mode, wherein the abnormal state detection capability evaluation result of the teleoperation system is as follows:
the error instruction sending rate and the tolerable telemetry data abnormal rate instruction sending rate in error code checking and fault tolerance capability evaluation can be obtained by the following formula:
and S1023, testing the autonomous ability and intelligence of the teleoperation system through the control data.
Referring to fig. 4, the autonomous capability and intelligence of the teleoperation system are mainly embodied in the following five items of autonomy: the method comprises the steps of instruction generation autonomy, planning autonomy, telemetry data processing autonomy and delay influence reduction autonomy. The autonomy and the intelligent capability of the teleoperation system can be evaluated through the five autonomy items.
And S1024, testing the real-time processing capacity of the teleoperation system through the control data.
Referring to fig. 5, the real-time processing capability of the teleoperation system is mainly embodied in the following seven real-times: the method comprises the steps of real-time scheduling of sharing operation, real-time telemetry data processing, real-time sending of planning instructions, real-time safety protection, real-time backup, real-time reduction of time delay influence and real-time simulation environment of a remote site.
The scheduling real-time capability assessment of the sharing operation can be calculated by the following formula:
operating environment data refresh rate T0To T1Number of operating environment data refreshes/T over a period of time1-T0
Operation authority data refresh rate T0To T1Number of operation authority data refreshes/T within time period1-T0
Operating space data refresh rate T0To T1Number of operating space data refreshes/T over a period of time1-T0
The telemetry data processing real-time capability assessment can be calculated by the following formula:
remote data refresh frequency T0To T1Number of telemetry data refreshes/T over time period1-T0
Telemetric image refresh frequency T0To T1Number of telemetry image refreshes/T over a period of time1-T0
The safety protection real-time capability evaluation can be calculated by the following formula:
and (3) backup real-time capability evaluation:
and S1025, testing the communication capability of the teleoperation system through the control data.
Referring to fig. 6, the teleoperation system communication capability includes the teleoperation system communication with the external system and the teleoperation system internal unit communication, and the direct representation of the capability includes: bandwidth of internal/external communication channel, limit code rate of internal/external communication, bit error rate, packet loss rate, etc.
The code rate, the bit error rate and the packet loss rate can be obtained by calculating according to the following formulas:
and S1026, testing man-machine efficacy, electromechanical and electrical performance parts of the teleoperation system through the control data.
Referring to fig. 7, the human-machine efficiency, electromechanical and electrical performance of several teleoperational systems shown in fig. 7 can be tested.
S1027, testing the delay influence eliminating capability of the teleoperation system through the control data.
Referring to fig. 8, it can be calculated by the following formula:
s1028, testing the backup, analysis, reproduction and time synchronization capabilities of the teleoperation system through the control data.
Referring to fig. 9, several of the teleoperating system backup, analysis, replication, time synchronization capabilities of fig. 9 may be evaluated.
S1029, testing the synchronism of the sharing operation through the control data.
With reference to figure 10 in which,
the single-person multi-machine operation response synchronism is as follows:
single-person multi-machine operation response time difference is object i (i-2, 3.) response time-object 1 response time
The response synchronism of the multi-user multi-computer operation is as follows:
response time difference between multi-user multi-machine operation and object i (i-2, 3.) -response time and object 1 response time
The time synchronism of the sharing end is as follows:
shared end time synchronization difference (operation end i (i ═ 2, 3.) data receiving time-operation end 1 data sending time) -corresponding time delay
The object side time synchronism is as follows:
the object-side time synchronization difference (i) 2, 3. -) data receiving time-data sending time of the operation end 1) -corresponding time delay
The time synchronism of the intermediate service node is as follows:
the time synchronization difference of the intermediate service node is (the time when the intermediate service node receives the data-the time when the operation end 1 sends the data) -corresponding time delay
Multi-person single-machine operation forecast synchronism: consider the start time difference and the end time difference, respectively, as:
the forecast time difference (operation end i (i ═ 2, 3.) pre-simulation start time-operation end 1 pre-simulation start time) -corresponding time delay
The forecast time difference (i ═ 2, 3.)) pre-simulation end time-operation end 1 pre-simulation end time) -corresponding time delay
Multi-person multi-machine operation forecast synchronism: also consider the start time difference and the end time difference, respectively, as:
the forecast time difference (operation end i (i ═ 2, 3.) pre-simulation start time-operation end 1 pre-simulation start time) -corresponding time delay
The forecast time difference (i ═ 2, 3.)) pre-simulation end time-operation end 1 pre-simulation end time) -corresponding time delay
The single-person multi-machine operation response synchronism is as follows:
single-person multi-machine operation response time difference is object i (i-2, 3.) response time-object 1 response time
The response synchronism of the multi-user multi-computer operation is as follows:
response time difference between multi-user multi-machine operation and object i (i-2, 3.) -response time and object 1 response time
S10210, testing tolerance of the sharing operation by the control data.
Referring to fig. 11, the tolerance to differences between the operations shared by several teleoperational systems shown in fig. 11 can be evaluated.
Ten capabilities of the teleoperation system can be evaluated according to the above S1021-S10210, so as to obtain evaluation data of experimental data corresponding to a single teleoperation task.
Optionally, as an embodiment of the present invention, after the evaluation data is obtained, the evaluation data may be checked to ensure validity of the evaluation data.
In the above, how to obtain the evaluation data of the experimental data corresponding to a single teleoperation task is introduced, and the evaluation data corresponding to at least one teleoperation task can be obtained by selecting a plurality of operation tasks in the teleoperation task step library.
After all the evaluation data are obtained, the evaluation data can be displayed in the form of a table as shown in the following table:
TABLE 1
TABLE 2
TABLE 3
TABLE 4
S103, determining a test result of the teleoperation system according to the test data.
Optionally, the evaluation result includes at least one of:
the covering capability of the operation mode on teleoperation tasks, the safety protection capability of a teleoperation system, the autonomous capability and intelligence of the teleoperation system, the real-time processing capability of the teleoperation system, the man-machine efficacy of the teleoperation system, the electromechanics, the electricities, the delay influence reduction capability of the teleoperation system, the backup, the analysis, the recurrence, the time synchronization capability of the teleoperation system, the synchronism of shared operation or the difference tolerance of the shared operation.
Determining an evaluation result of the teleoperation system according to the evaluation data may specifically be obtained by the following sub-steps:
and S1031, operation flow test statistics.
Specifically, the record number, the use frequency, the number of times of unsatisfied conditions and the unsatisfied probability of all the task steps in all teleoperation experiments are counted. Meanwhile, when the number of steps which do not satisfy the condition is i (i is 0,1,2,3,4), the corresponding number of teleoperation experiments, the corresponding number of experimental average steps, and the number of single experimental steps are distributed (the step length is 5), wherein the statistical calculation method comprises the following steps:
the task step record number is the total times of the task step appearing in all teleoperation experiments
The number of times that the task step does not meet the condition is the total number of times that the task step does not meet the condition and all teleoperation experiments occur
And S1032, carrying out task execution capacity test statistics.
Specifically, the 6 contents of all the evaluation items are counted, including: the number of experiments satisfying the term, the probability of satisfying the term, the average number of steps of experiments satisfying the term, the number of experiments not satisfying the term, the probability of not satisfying the term, the average number of steps of experiments not satisfying the term. Meanwhile, the relation between 4 indexes of 'data volume', 'joint angle absolute error range', 'average time delay' and 'maximum time delay' of all the evaluation items and the experiment is subjected to correlation statistics. The statistical calculation method comprises the following steps:
the number of experiments satisfying this term is the total number of experiments that all teleoperation experiments satisfy this term
The number of experiments not satisfying this term-the total number of teleoperation experiments-the number of experiments satisfying this term
And S1033, data sufficiency and backup capability test statistics.
The statistical calculation of the 6 contents of all the evaluation items is consistent with the statistical calculation of the 6 contents of the task execution capability evaluation in step S1032, and the statistical calculation method is the same and is not described herein again.
And S1034, carrying out real-time data processing and time synchronization capability test statistics. The statistics total 6 items including: time length distribution statistics, data volume distribution statistics, joint angle absolute error range distribution statistics, joint angle relative error range distribution statistics, average delay distribution statistics, maximum delay distribution statistics, and delay range distribution statistics. The statistical calculation formula is as follows:
the time length is the command value starting time-command value ending time of the teleoperation experiment
Length of time distribution step 500s
Max { number of instruction values, number of predicted values, number of field measured values, number of delayed reception values }
Step size of data size distribution is 10000
Joints angle absolute error is | joint angle field measured value-joint angle predicted value-
Time delay-the time of the actually measured value on the spot corresponding to the time of delaying the received value
Max { all delays }; min (all delays) minimum delay
The step length of time delay distribution is equal to the step length of average time delay distribution and equal to the step length of maximum time delay distribution and equal to 2s
And S1035, overall testing.
Specifically, the following 10 capability indexes are used, including "operation flow of a specific task", "operation mode", "operation support means", "system autonomy", "operation time sufficiency", "telemetry data sufficiency", "time delay reduction data sufficiency", and "time synchronization capability" real-time processing capability ". Each index has two statistics, that is, "a statistical rate that completely satisfies a condition" and "a statistical rate that partially satisfies a condition", and two statistical methods for different abilities are given below.
1) "task-specific operational flow": performing statistical analysis on all tasks-steps (system existing steps and user new input steps) based on all input experiment contents, wherein a certain step appears in all experiments, namely the step is considered to completely meet the conditions; if a step occurs only in part of the experiment, i.e. if the step is considered to partially satisfy the condition:
2) "operation mode", "operation support means", "system autonomy", "operating time sufficiency", "telemetry data sufficiency", "latency reduction data sufficiency": each experiment meets a certain evaluation item, namely the evaluation item is considered to completely meet the condition; if part of the experiment satisfies a certain evaluation item, namely the evaluation item is considered to partially satisfy the condition:
3) "time synchronization capability": measuring the time synchronization capacity of the system by using data density (instruction value data density and predicted value data density), and regulating that the two data density values of a certain experiment are both more than 10Hz, namely the experiment is considered to completely meet the time synchronization capacity; only one data density value of a certain experiment is larger than 10Hz, namely the experiment part is considered to meet the time synchronization capability, then:
4) "real-time processing capability": measuring the real-time processing capacity of the system by using the error range of the joint angle, and regulating the error range of all experiments to be less than 0.015 degrees, namely, considering the statistical rate of completely meeting the conditions to be 100 percent and the statistical rate of partially meeting the conditions to be 0; the error range of only part of the experiments is less than 0.015 degrees, namely the statistical rate of completely meeting the conditions is considered to be 0, and the statistical rate of partially meeting the conditions is 100 percent.
After the 4 steps are executed, determining an evaluation result of the teleoperation system, wherein the evaluation result can be generated in a form of a statistical report, such as a document, a table or a picture, and the like, such as a Word document or an Excel table.
It should be noted that, the above steps S101 to S103 may perform an evaluation on a quantifiable index quantity, and for the evaluation on an unquantizable index quantity, the evaluation may be performed in a judgment form (for example, experimental data of a teleoperation task for a single time or a certain time is judged), and when performing the statistical evaluation in S103, the quantitative evaluation is performed on the unquantizable index quantity in a statistical manner, and specific contents may refer to the above table, which is not described herein again.
And S104, displaying the test result and outputting an evaluation result according to the test result.
Optionally, the evaluation result is displayed by one of the following ways: documents, tables, or pictures.
It should be noted that, while the test result is displayed, the evaluation result may also be output, and the evaluation result may be output in the form of a Word document or an Excel table.
According to the evaluation method of the teleoperation system provided by the embodiment of the invention, experimental data of at least one teleoperation task is obtained, and evaluation data corresponding to the teleoperation task is determined according to the experimental data; determining an evaluation result of the teleoperation system according to the evaluation data; and displaying the evaluation result, so that the advantages and disadvantages of various capabilities of the teleoperation system can be accurately evaluated, and guidance can be provided for the teleoperation system.
The above description mainly introduces the solution of the embodiment of the present invention from the perspective of a teleoperation system. It is to be understood that the teleoperation system and the like includes hardware structures and/or software modules corresponding to the respective functions for realizing the above-described functions. Those of skill in the art will readily appreciate that the present invention can be implemented in hardware or a combination of hardware and computer software, with the exemplary elements and algorithm steps described in connection with the embodiments disclosed herein. Whether a function is performed as hardware or computer software drives hardware depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the embodiment of the present invention, the teleoperation system and the like may be divided into functional units according to the above method, for example, each functional unit may be divided corresponding to each function, or two or more functions may be integrated into one processing unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit. It should be noted that the division of the unit in the embodiment of the present invention is schematic, and is only a logic function division, and there may be another division manner in actual implementation.
Under the condition of adopting an integrated unit, fig. 12 is a schematic structural diagram of an evaluation device of a teleoperation system according to an embodiment of the present invention, where the device specifically includes:
an obtaining module 1201, configured to obtain control data of at least one teleoperation task, where the teleoperation task includes: controlling the operation of the external equipment through the teleoperation system;
a first determining module 1202, configured to determine, according to the control data, test data corresponding to the at least one teleoperation task;
a second determining module 1203, configured to determine a test result of the teleoperation system according to the test data;
a display module 1204, configured to display the test result and an evaluation result output according to the test result.
Optionally, the apparatus further comprises: a classification module 1205 for performing feature classification on the teleoperation system;
wherein the characteristics include at least one of: a remote sharing feature, a teleoperation feature, a telepresence feature, or a remote system feature.
Optionally, the apparatus further comprises: an establishing module 1206, configured to establish a teleoperation task step library corresponding to the operation according to the operation of the teleoperation system controlling the external device.
Optionally, the obtaining module 1201 is configured to test control data corresponding to a single teleoperation task, and obtain test data corresponding to the single teleoperation task; and determining test data corresponding to at least one teleoperation task according to the test data corresponding to the single teleoperation task.
Optionally, the evaluation result includes at least one of: testing the coverage capability of the teleoperation task; testing the coverage capability of the teleoperation task; testing the safety protection capability of the teleoperation system; testing the autonomous ability and the intelligence of the teleoperation system; testing the real-time processing capability of the teleoperation system; testing the communication capacity of the teleoperation system; testing the man-machine efficacy, the electromechanics and the electrical property of the teleoperation system; testing the delay impact reduction capability of the teleoperation system; the remote operation system carries out testing on the backup, analysis, reproduction and time synchronization capabilities; and testing the synchronicity of the shared operation or the difference tolerance of the shared operation.
Optionally, the evaluation result is displayed by one of the following ways: documents, tables, or pictures.
Optionally, the apparatus further comprises: and the checking module 1207 is configured to check the test data.
Optionally, the proofreading module 1207 is further configured to perform proofreading on the test data.
The evaluation device of the teleoperation system provided in the embodiment of the present invention may be used as an execution main body of the evaluation method of the teleoperation system shown in fig. 1, and executes each step of the method shown in fig. 1, thereby achieving the technical effect of the method shown in fig. 1, and for brevity, reference may be made to the above description on fig. 1, which is not described herein again.
Those of skill would further appreciate that the various illustrative components and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied in hardware, a software module executed by a processor, or a combination of the two. A software module may reside in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
The above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention in further detail, and it should be understood that the above-mentioned embodiments are merely exemplary embodiments of the present invention, and are not intended to limit the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (10)
1. An evaluation method of a teleoperation system, comprising:
performing feature classification on the teleoperation system, wherein the features comprise at least one of: a remote share feature, a remote operation feature, a remote site feature, or a remote system feature;
establishing a teleoperation task step library corresponding to the operation according to the operation of the teleoperation system for controlling the external equipment;
obtaining control data for at least one teleoperational task, wherein the teleoperational task comprises: controlling the operation of the external equipment through the teleoperation system;
determining test data corresponding to at least one teleoperation task according to the control data, specifically comprising: testing control data corresponding to a single teleoperation task to obtain test data corresponding to the single teleoperation task, and determining test data corresponding to at least one teleoperation task according to the test data corresponding to the single teleoperation task;
determining a test result of the teleoperation system according to the test data;
and displaying the test result and an evaluation result output according to the test result.
2. The method of claim 1, wherein the testing of the control data corresponding to the single teleoperation task comprises at least one of:
testing the coverage capability of the teleoperation task;
testing the safety protection capability of the teleoperation system;
testing the autonomous ability and the intelligence of the teleoperation system;
testing the real-time processing capability of the teleoperation system;
testing the communication capacity of the teleoperation system;
testing the man-machine efficacy, the electromechanical performance and the electrical performance of the teleoperation system;
testing the delay impact reduction capability of the teleoperation system;
testing the backup, analysis, reproduction and time synchronization capabilities of the teleoperation system;
and testing the synchronicity of the shared operation or the difference tolerance of the shared operation.
3. The method of claim 1, wherein the test results are displayed by one of:
documents, tables, or pictures.
4. The method according to any of claims 1-3, further comprising, prior to said determining test data corresponding to said at least one teleoperational task from said control data:
and checking the control data.
5. The method of claim 4, further comprising, after said determining test data corresponding to said at least one teleoperational task from said control data:
and checking the test data.
6. An evaluation apparatus of a teleoperation system, comprising:
a classification module configured to classify characteristics of the teleoperation system, wherein the characteristics include at least one of: a remote share feature, a remote operation feature, a remote site feature, or a remote system feature;
the establishment module is used for establishing a teleoperation task step library corresponding to the operation according to the operation of the teleoperation system for controlling the external equipment;
an obtaining module, configured to obtain control data of at least one teleoperation task, where the teleoperation task includes: controlling the operation of the external equipment through the teleoperation system;
the first determining module is used for determining test data corresponding to at least one teleoperation task according to the control data, and specifically comprises: testing control data corresponding to a single teleoperation task to obtain test data corresponding to the single teleoperation task, and determining test data corresponding to at least one teleoperation task according to the test data corresponding to the single teleoperation task;
the second determining module is used for determining a test result of the teleoperation system according to the test data;
and the display module is used for displaying the test result and the evaluation result output according to the test result.
7. The device according to claim 6, wherein the first determining module is specifically configured to:
testing the coverage capability of the teleoperation task;
testing the safety protection capability of the teleoperation system;
testing the autonomous ability and the intelligence of the teleoperation system;
testing the real-time processing capability of the teleoperation system;
testing the communication capacity of the teleoperation system;
testing the man-machine efficacy, the electromechanical performance and the electrical performance of the teleoperation system;
testing the delay impact reduction capability of the teleoperation system;
testing the backup, analysis, reproduction and time synchronization capabilities of the teleoperation system;
and testing the synchronicity of the shared operation or the difference tolerance of the shared operation.
8. The apparatus of claim 7, wherein the evaluation result is displayed by one of:
documents, tables, or pictures.
9. The apparatus according to any one of claims 6-8, wherein the apparatus further comprises:
and the checking module is used for checking the control data.
10. The apparatus of claim 9, wherein the collation module is further configured to collate the test data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711236258.1A CN107945499B (en) | 2017-11-30 | 2017-11-30 | Evaluation method and equipment for teleoperation system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711236258.1A CN107945499B (en) | 2017-11-30 | 2017-11-30 | Evaluation method and equipment for teleoperation system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107945499A CN107945499A (en) | 2018-04-20 |
| CN107945499B true CN107945499B (en) | 2021-04-20 |
Family
ID=61947964
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711236258.1A Active CN107945499B (en) | 2017-11-30 | 2017-11-30 | Evaluation method and equipment for teleoperation system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107945499B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108828935B (en) * | 2018-05-07 | 2022-05-13 | 中国科学院力学研究所 | Intelligent auxiliary operation method and system for remote operation |
| CN108830453B (en) * | 2018-05-07 | 2021-10-08 | 中国科学院力学研究所 | Method and system for evaluating remote operation effect |
| CN108762060B (en) * | 2018-05-07 | 2021-02-02 | 中国科学院力学研究所 | Optimization method and system for remote operation task formulation |
| CN108656109B (en) * | 2018-05-07 | 2021-03-02 | 中国科学院力学研究所 | Remote operation training method and system |
| CN110187705B (en) * | 2019-05-27 | 2020-03-31 | 武昌理工学院 | Robot remote control delay elimination control system and method |
| CN110411506B (en) * | 2019-07-08 | 2021-09-14 | 中国航天员科研训练中心 | Fine operation capability test system facing remote operation |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103117816A (en) * | 2013-01-04 | 2013-05-22 | 西北工业大学 | Measuring method of space teleoperation command time delay |
| CN104015190A (en) * | 2014-05-13 | 2014-09-03 | 中国科学院力学研究所 | Robot remote control method and system under indeterminate bidirectional time delay condition |
| CN104778872A (en) * | 2015-04-03 | 2015-07-15 | 上海交通大学 | Simulation testing and training platform for mechanical arm teleoperation human-computer interaction study |
| US20170063997A1 (en) * | 2015-08-26 | 2017-03-02 | Hitachi, Ltd. | Evaluation apparatus, evaluation system, device, and evaluation method |
| CN107213990A (en) * | 2017-05-08 | 2017-09-29 | 浙江大学 | Electric dust removing system Performance Evaluation and operation optimizing system |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5720398B2 (en) * | 2011-04-25 | 2015-05-20 | ソニー株式会社 | Evaluation apparatus and method, service providing system, and computer program |
| CN103336740B (en) * | 2013-06-28 | 2016-12-28 | 广东电网公司电力科学研究院 | The operating system integrated test facility of electric power secondary system and device |
| CN103722554B (en) * | 2014-01-15 | 2015-12-09 | 西北工业大学 | A kind of Space teleoperation method |
| CN106527177B (en) * | 2016-10-26 | 2019-07-12 | 北京控制工程研究所 | The multi-functional one-stop remote operating control design case of one kind and analogue system and method |
-
2017
- 2017-11-30 CN CN201711236258.1A patent/CN107945499B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103117816A (en) * | 2013-01-04 | 2013-05-22 | 西北工业大学 | Measuring method of space teleoperation command time delay |
| CN104015190A (en) * | 2014-05-13 | 2014-09-03 | 中国科学院力学研究所 | Robot remote control method and system under indeterminate bidirectional time delay condition |
| CN104778872A (en) * | 2015-04-03 | 2015-07-15 | 上海交通大学 | Simulation testing and training platform for mechanical arm teleoperation human-computer interaction study |
| US20170063997A1 (en) * | 2015-08-26 | 2017-03-02 | Hitachi, Ltd. | Evaluation apparatus, evaluation system, device, and evaluation method |
| CN107213990A (en) * | 2017-05-08 | 2017-09-29 | 浙江大学 | Electric dust removing system Performance Evaluation and operation optimizing system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107945499A (en) | 2018-04-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107945499B (en) | Evaluation method and equipment for teleoperation system | |
| US10235231B2 (en) | Anomaly fusion on temporal casualty graphs | |
| KR20200123454A (en) | Cyber-attack detection, positioning, and neutralization of unmanned aerial vehicles | |
| Huang et al. | Active fault tolerant control systems by the semi‐Markov model approach | |
| CN109614318A (en) | Automated testing method, device, electronic equipment and computer-readable medium | |
| CN107370806A (en) | HTTP conditional codes monitoring method, device, storage medium and electronic equipment | |
| US20140297856A1 (en) | Diagnosis for goose communication | |
| Gillies et al. | Probabilistic approaches to estimating the quality of information in military sensor networks | |
| Al-Dabbagh et al. | Reliability modeling of networked control systems using dynamic flowgraph methodology | |
| CN112040506A (en) | Reliability evaluation method and device of wireless sensor network and storage medium | |
| Liu et al. | Resilience assessment and importance measure for interdependent critical infrastructures | |
| KR101475571B1 (en) | Apparatus and method for data processing in SCADA System | |
| Braun et al. | Hierarchical attack identification for distributed robust nonlinear control | |
| CN109947728B (en) | Log file processing method and device | |
| Goloskokov et al. | Features of Assessing the Quality of Functioning of Telecommunication Systems in Transport | |
| Jones-Wyatt et al. | A reliability-based measurement of interoperability for systems of systems | |
| CN104914853A (en) | Measurement method and system for switching time of main and auxiliary controllers | |
| Long et al. | Robust fault detection for networked control systems with packet dropouts: An average dwell time method | |
| JP6142878B2 (en) | Information system performance evaluation apparatus, method and program | |
| Chabane et al. | Fault tolerant control approach based on multiple models and set-membership state estimation | |
| Malere | Application of linear programming to optimize the cost-benefit of an ivhm system | |
| KR102627063B1 (en) | Apparatus for predicting ai based error vulerable element in information infra | |
| Abramov et al. | Distribution law evaluation for demand service time in information system with multiple access to the data link. | |
| Kolisnyk et al. | Markov model for availability assessment of PLC in Industrial IoT considering subsystems failures | |
| KR20230099504A (en) | Apparatus for automatic controlling error free information infra platform |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |