WO2021016807A1

WO2021016807A1 - Context awareness device simulation method, device, and system

Info

Publication number: WO2021016807A1
Application number: PCT/CN2019/098188
Authority: WO
Inventors: 李婧; 徐蔚峰; 李明; 卢超
Original assignee: 西门子股份公司; 西门子（中国）有限公司
Priority date: 2019-07-29
Filing date: 2019-07-29
Publication date: 2021-02-04
Also published as: CN113874844A

Abstract

Provided are a context awareness device simulation method, device, and system, comprising the following steps: S1: modeling the environment of a context awareness device on the basis of customer requirements, and traversing the environment model to generate a test case; S2: performing action planning and action simulation on the context awareness device on the basis of said test case. By means of the described method, device, and system, it is possible to better support the simulation of a context awareness device. In the described modeling process, the environment of the context awareness device and a target object in the environment and their relationships and attributes are classified and described, and the modeling process is more clear and precise, providing support for a subsequent-step test scenario generation by means of establishing a semantic model. It is also possible to retrieve the graphical structure of the environment model and generate a test case for each traversal, the context awareness system having transformed into a list of test cases to fulfill customer requirements. The test data can be mapped to a simulator which interacts with the context awareness device to automatically generate a simulated 3-D scene.

Description

Simulation method, device and system of context sensing device

Technical field

The present invention relates to the field of simulation, in particular to a simulation method, device and system of a context sensing device.

Background technique

Context sensing devices are also called autonomous systems, such as smart vision devices or smart robots. The context sensing device can perceive the environment or the situation, recognize the target that needs to be interacted, and then perform actions accordingly. More and more context-aware devices are used in the industry, such as intelligent transportation, advanced manufacturing or building technology.

Unlike traditional automatic systems, the larger and complex input space makes it difficult to verify context-aware devices. Current simulation tools are good at evaluating event-driven and deterministic devices. For example, some industrial simulation software allows users to define events and 3D scenes to simulate the inputs and actions of equipment in production.

However, context sensing devices need to be evaluated in different contexts, so it is difficult to manually generate all possible contexts in a simulator. On the other hand, generating test cases in a specific way can lead to simulation blind spots or extreme cases. In order to better support the simulation of context-aware devices, current simulators need to add features to model the equipment context and to automatically generate test cases.

In order to solve the above-mentioned problems, the prior art provides two solutions. The first solution provides a simulation platform for the autonomous driving system, which is limited to the evaluation of the autonomous driving field. UML language is used to model the system scenario, but the raw data of the synthetic sensor is generated based on a custom algorithm, which cannot be used for Other cases.

The second solution provides robot training in a simulation environment. It obtains a comprehensive demonstration of packing to form an algorithm management in the simulator, and then it learns strategies through synthetic data from the simulator, which is also based on the simulation The acquired data trains a robot controller. Although this solution can simulate intelligent robots and obtain simulation data, they are not concerned about simulators that have the ability to automatically evaluate and verify intelligent robots.

Summary of the invention

The first aspect of the present invention provides a simulation method of a context sensing device, which includes the following steps: S1, modeling the environment of the context sensing device based on customer needs, and traversing the environment model to generate test cases; S2, based on testing The example performs action planning and action simulation on the emotion perception device.

Further, the step S1 also includes the following steps: S11, describing the basic objects and their relationships in the environment of the context-aware device by using ontology to model the environment of the context-aware device and based on the customer Need to mark the environment model; S12, traverse the environment model based on the test case, and extract data from the environment model to generate a test case.

Further, the environmental model includes objects, and the objects include scenario targets, a system under test, and test cases.

Further, the scene target also includes target attributes and noise, and the target attributes include the material, color, shape, and position of the target.

Further, the context perception device includes a grasping robot, and the system under test based on the grasping robot includes a vision system, a robotic arm, and a gripper, wherein the vision system includes visual noise, and the robotic arm includes Mechanical noise.

Further, the context sensing device is a grasping robot, wherein, the step S2 includes the following steps: step S22, planning a motion trajectory for the grasping robot for the scene target that the grasping robot needs to grasp, wherein, The motion trajectory includes a posture representing the grasping robot, and the posture includes a rotation angle of a plurality of joints of the mechanical arm of the grasping robot in a time sequence. In step S23, the grasping robot is driven to move along the motion trajectory planned by the motion planning module 222 and returns the simulation result based on the simulation requirement.

Further, the step S2 includes step S21: mapping the data of the test instance to the simulation scene, and identifying the scene target object and its position information in the simulation scene based on a visual algorithm.

Further, after the step S2, the method further includes a step S3: performing a simulation on the context sensing device based on the test example.

The second aspect of the present invention provides a simulation device of a context perception device, which includes: an environment management device that models the environment of the context perception device based on customer needs, and traverses the environment model to generate test cases; a test simulation device, which Perform action planning and action simulation on the emotion perception device based on test cases.

Further, the environment management device further includes: a modeling device, which describes basic objects and their relationships in the environment of the context perception device using ontology to model the environment of the context perception device, and Annotate the environmental model based on customer requirements; a test data generation device that traverses the environmental model based on the test case, and extracts data from the environmental model to generate a test case.

Further, the context sensing device is a grasping robot, wherein the test simulation device further includes: a motion planning module, which plans a motion trajectory for the grasping robot according to the situation target that the grasping robot needs to grasp, Wherein, the motion trajectory includes the posture representing the grasping robot, and the posture includes the rotation angles of the multiple joints of the robot arm of the grasping robot in a time series; the motion simulation module drives the grasping robot to Move along the motion trajectory and return simulation results based on simulation requirements.

Further, the test simulation device further includes a perception module, which maps the data of the test instance to the simulation scene, and recognizes the scene target object and its position information in the simulation scene based on a visual algorithm.

Further, the simulation device of the context sensing device further includes a test evaluation device that performs simulation on the context sensing device based on the test example.

A third aspect provides a simulation system of a context-aware device, which includes: a processor; and a memory coupled with the processor, the memory having instructions stored therein, and the instructions cause all of them when executed by the processor. The electronic device executes actions, and the actions include: S1, modeling the environment of the context sensing device based on customer needs, and traversing the environment model to generate test cases; S2, performing action planning on the emotion sensing device based on the test cases, and Action simulation.

Further, the action S1 also includes: S11, describing the basic objects and their relationships in the environment of the context sensing device using ontology to model the environment of the context sensing device, and labeling based on customer needs The environment model; S12, traverse the environment model based on the test case, and extract data from the environment model to generate a test case.

Further, the context sensing device is a grasping robot, wherein the action S2 includes the following actions: S22, planning a motion trajectory for the grasping robot according to the situation target that the grasping robot needs to grasp, wherein the movement The trajectory includes a posture representing the grasping robot, and the posture includes a rotation angle of a plurality of joints of the robotic arm of the grasping robot in a time series. S23: Drive the grasping robot to move along the motion track and return a simulation result based on simulation requirements.

Further, the action S22 also includes an action S21 before the action S22: mapping the data of the test instance to the simulation scene, and identifying the scene target object and its position information in the simulation scene based on a visual algorithm.

Further, after the action S2, an action S3 is further included: performing a simulation on the context sensing device based on the test example.

The fourth aspect of the present invention provides a computer program product, which is tangibly stored on a computer-readable medium and includes computer-executable instructions, which when executed, cause at least one processor to execute The method described in the first aspect of the present invention.

The fifth aspect of the present invention provides a computer-readable medium on which computer-executable instructions are stored, and when executed, the computer-executable instructions cause at least one processor to perform the method according to the first aspect of the present invention.

The present invention can better support the simulation of the context sensing device. The modeling process provided by the present invention categorizes and describes the environment of the context sensing device and the target objects in the environment and their relationships and attributes. The modeling process provided by the present invention is simpler, and the next test scenario is tested by establishing a semantic model. Generate support. The present invention can also retrieve the graphical structure of the environment model and generate test cases for each traversal. In this process, the context-aware system is converted into a test case list to advance customer requirements. The present invention can also map test data to a simulator that interacts with the context sensing device, and can automatically generate a simulated stereo scene.

Description of the drawings

Figure 1 is a system framework diagram of a simulation device of a context sensing device according to a specific embodiment of the present invention;

2 is a schematic structural diagram of the environment model of the context sensing device in the simulation mechanism of the context sensing device according to a specific embodiment of the present invention;

3 is a schematic diagram of the target attribute structure of the contextual target of the simulation mechanism of the contextual sensing device according to a specific embodiment of the present invention;

Fig. 4 is a schematic diagram of a simulation of a grabbing robot grabbing parts from a box according to the simulation mechanism of the context sensing device according to a specific embodiment of the present invention;

Fig. 5 is a graph showing the posture of the robot arm of the grasping robot B of the simulation mechanism of the context sensing device according to a specific embodiment of the present invention.

Detailed ways

The specific embodiments of the present invention will be described below with reference to the accompanying drawings.

The invention provides a simulation mechanism of a situational perception device, the simulation function is extended, the simulation efficiency is higher and more accurate, and it does not rely on manpower. The present invention provides environmental management functions and test scenario generation functions. The present invention is suitable for scene sensing devices, such as automatic driving, intelligent robots and unmanned aerial vehicles, etc. The present invention is also especially suitable for visual grasping robots.

As shown in FIG. 1, the simulation device of the context perception device provided by the present invention includes an environment management device 100 and a simulation device 200, wherein the simulation device 200 further includes a scenario generation device 210, a test simulation device 220 and a test evaluation device 230. Wherein, the environmental management device 100 generates a test case based on the input customer requirements, and sends the test case to the scenario generation device 210. The test simulation device 220 is used to map the test case data to a simulation scene in the simulation device 200, for example, to generate 3D objects and environments in a virtual scene. Finally, the test evaluation device 230 simulates the context sensing device based on the input scene.

The first aspect of the present invention provides a simulation method of a situational device.

Firstly, step S1 is performed. The environment management device 100 models the environment of the context-aware device based on customer requirements, and traverses the environment model to generate test cases. Among them, the environment management device 100 is used to generate context data for the context perception device based on customer needs. Specifically, the environment management device 100 includes an environment modeling device 110 and a test data generating device 120, which can automatically generate test cases using parameter values provided by the simulation device 200, and then execute and evaluate them. The environmental management device 100 can also generate noise factors for different modules of the system under test (SUT, System Under Test).

Specifically, as shown in Fig. 1, for example, in this embodiment, customer requirement A is to verify the ability of visual grasping robot B to grasp multiple targets in a box, where the number of multiple targets is 10~ 200, the probability of generating a cylinder in multiple targets is m, and the probability of generating a cube in multiple targets is n.

Among them, the environment modeling device 110 is used to describe the environment of the context sensing device in a formal language/method. The environment refers to the basic objects that the context sensing device may "see". In this embodiment, it refers to the basic objects that the visual grasping robot B can "see". That is, the environment modeling device 110 is used to describe the basic objects that can be "seen" by the visual grasping robot B and their relationships with formal language/methods.

Further, the step S1 further includes sub-step S11 and sub-step S12.

In sub-step S11, the environment modeling device 110 uses the ontology to describe the basic objects and their relationships in the environment of the context perception device, so as to model the environment of the context perception device, and mark all objects based on customer needs. The environmental model.

As shown in FIG. 2, according to a preferred embodiment of the present invention, the environment modeling device 110 describes the basic objects and their relationships in the environment of the visual grasping robot B using an ontology. Among them, the environmental model includes objects, and the objects include scenario targets, systems under test, and test cases. The scene target also includes target attributes and noise, and the target attributes include the material, color, shape, and location of the target. For example, in this embodiment, the scene target includes boxes and parts. The box includes a cube box, which has parts. Parts include cube parts, cylindrical parts and spherical parts. Among them, the relationship between the box and the component is that the position of the component is inside the box. Noise includes visual noise and mechanical noise.

In addition, the context perception device includes a grasping robot, and the system under test based on the grasping robot includes a vision system, a robotic arm, and a gripper, wherein the vision system includes visual noise, and the robotic arm includes a mechanical noise. The relationship between the visual system and visual noise is that the visual system has noise, and the relationship between the mechanical arm and mechanical noise is that the mechanical arm has noise.

And, as shown in Figure 2, the test case includes test case 1. Test case 1 is the test case corresponding to customer requirement A. And, based on customer demand A, the environmental model shown in Figure 2 is labeled, for example, the number of 10 to 200 is labeled on the component, the generation probability m is labeled on the cylindrical component, and the generation probability n is labeled on the spherical component. .

It should be noted that using ontology knowledge to model the environment of the context sensing device is a preferred embodiment of the present invention, and other implementation methods are not excluded. Each category in the model shown in Figure 2 also includes multiple attributes and has a data range.

In sub-step S12, the test data generating device 120 traverses the environmental model based on the test case, and extracts data from the environmental model to generate a test case. Specifically, the test data generating device 120 automatically generates the test case 1 based on the environmental model shown in FIG. 2, and the environmental model uses ontology knowledge and serves as the input of the test data generating device 120. Moreover, in order to derive details from the test scenario and data, the test data generating device 120 provides an algorithm based on specific rules to traverse the ontology of the environment model shown in FIG. 2. The test data generating device 120 can also generate noise factors for different environmental models.

The test data generating device 120 implements the traversal process through algorithms, and outputs test cases and saves the test cases in JSON format for further analysis.

Specifically, in this embodiment, customer requirement A is to verify the ability of the visual grasping robot B to grasp multiple targets in a box, where the number of multiple targets is 10 to 200, and the number of multiple targets is cylindrical The generation probability of a volume is m, and the generation probability of a cube in multiple targets is n. The test data generating device 120 first reads the type in the test case generated based on the customer demand A, and then detects all connected types, and for each type, randomly initializes the attributes of the type (for example, including color) within the value range defined by the ontology. , Size, location). After that, determine whether each type meets the quantity requirements. Among them, the quantity requirement here includes "the number of multiple targets is 10 to 200". If the result returned at this time is 5, so the quantity requirement is not met, then it is further judged whether the secondary type (its subcategory) has generation Probability attribute, if it has a generation probability attribute, an instance of this subclass is generated according to the generation probability. If the secondary type (its sub-category) does not generate a probability attribute, an instance of the sub-category is directly generated, and it is judged whether it meets the quantity requirement of "the number of multiple targets is 10 to 200". When the judgment result is yes, then judge whether the test instance meets the constraint relationship (for example, whether the position generated by the target component is in the box) rule, if the constraint relationship rule is satisfied, output the test instance, if not, reset part of the instance value. After that, the constraint relationship is judged again until the constraint relationship is satisfied.

Finally, step S2 is executed, and the simulation device 200 performs action planning and action simulation on the emotion perception device based on the test example. Wherein, the simulation device 200 includes a scene generation device 210, a test simulation device 220, and a test evaluation device 230.

The scenario generating device 210 is used to map test case data to a simulation scenario. The input of the scenario generating device 210 is a test case script, which maps test case data values to a target of a specific simulator to generate and propose a 3D target in simulation, and the output of the scenario generating device 210 is a simulation scenario described by the test case script.

In this embodiment, customer requirement A is: to verify the ability of the visual grasping robot B to grasp multiple targets in a box, where the number of multiple targets is 10 to 200, and the generation of cylinders in multiple targets The probability is m, and the probability of generating cubes in multiple targets is n. Fig. 3 is a schematic diagram of the target attribute structure of the context target of the simulation mechanism of the context sensing device according to a specific embodiment of the present invention. The present invention defines the target attributes of each scene target, and the target attributes of the scene target include but are not limited to material, shape, position, color, center point, length, width, and height. As shown in Figure 3, specifically, the target attributes of the box include material, shape, color, length, width, and height. The material of the box is paper, the shape of the box is cube, the center point of the box is (x11, y11, z11), the color of the box is white, the length of the box is l1, the width of the box is w1, and the height of the box is h1 . The target attributes of the cube part include material, shape, position, color, length, width, and height. Among them, the material of the cube part is rubber, the shape of the cube part is a cube, the center point of the cube part is (x21, y21, z21), the color of the cube part is blue, the length of the cube part is l2, and the width of the cube part is w2 , The height of the cube part is h2. The target properties of the spherical part include material, shape, color, center point, radius, and height. Among them, the material of the sphere part is rubber, the shape of the sphere part is a sphere, the color of the sphere part is red, the center point of the sphere part is (x32, y32, z32), the radius of the sphere part is r3, and the height of the sphere part is h3 .

It should be noted that the ability of the grasping robot B to grasp multiple targets in a box, where the number of multiple targets is 10 to 200, the generation probability of a cylinder in multiple targets is m, and the number of multiple targets is m. The probability of cube generation is n. Therefore, each cube part and each sphere part has target attributes. After the traversal of step S22 is performed and the scene data including the target attributes are collected, the scene generating device 210 is based on the target of each cube part and each sphere part. The attribute maps the data of test case 1 to a simulation scene, such as a 3D scene.

The test simulation device 220 is used to simulate the action of the context sensing device and provide an input scene. The test simulation device 220 includes a perception module 221, a motion planning module 222, and a motion simulation module 223. Among them, the motion planning module 222 and the motion simulation module 223 are necessary modules, and the perception module 221 is an optional module.

The step S2 includes sub-step S21, sub-step S22 and sub-step S23.

In the sub-step S21, the perception module 221 maps the data of the test instance to the simulation scene, and recognizes the scene target object and its position information in the simulation scene based on the visual algorithm.

Among them, in order to detect one or more scene objects and their details, the input of the perception module 221 is a synthetic scene. Whether the sensing module 221 is necessary depends on the specific implementation, and it may be integrated in the test simulation device 220 or independently exist outside the test simulation device 220. The perception module 221 uses the image of the synthesized scene for analysis, that is, the 3D scene output above. The perception module 221 uses multiple vision algorithms to recognize the target objects in the scene, namely, boxes, cube parts and sphere parts. Among them, vision algorithms include traditional vision algorithms or vision algorithms based on deep learning. In this embodiment, for example, the scene target object is a cube part in a box, and the grasping robot B grasps the cube part in a box. The output of the perception module 221 is the recognized component and includes its posture information, for example, in the following format:

<posture>x, y, z, roll, pitch, yaw</posture>

Among them, x, y, z are the position coordinates of the part, roll is the angle between the part and the x axis, pitch is the angle between the part and the y axis, and yaw is the angle between the part and z.

Among them, the sensing module 221 is not a necessary module, and step S21 is not a necessary step. The sensing module 221 and step S21 are optional.

If there is no perception module 221, the scene generating device 210 can also output one or more scene target objects and their posture information for the grabbing robot B to grab in a box. The difference is that the scene target object and its posture information are used to serve as the motion planning module 222.

In sub-step S22, the motion planning module 222 plans a motion trajectory for the grasping robot according to the scene target that the grasping robot needs to grasp, where the motion track includes the posture of the grasping robot, and the posture includes The rotation angle of the multiple joints of the robot arm of the grasping robot in a time sequence.

Wherein, the motion planning module 222 automatically plans a motion trajectory for the identified scene target, which automatically generates a motion trajectory, and specifies a starting position for the scene target plan posture and the robot arm of the grasping robot B. In the process of generating the motion trajectory, the motion planning module 222 uses a planning algorithm to define several points. The output of the motion planning module 222 is the joint angle matrix of the robotic arm of the grasping robot B:

Among them, j represents attitude, m represents time series, and n is a natural number. The posture j represents the angle of the robot arm of the grasping robot B at the time point m. For example, the grasping robot B shown in FIG. 4 has three joints, namely a first joint j ₁ , a second joint j ₂ and a third joint j ₃ . Therefore, the motion planning module 222 can plan the rotation angle of the first joint j ₁ , the second joint j ₂ and the third joint j ₃ at a specific time point, and output a joint angle matrix M of the robot arm of the grasping robot B _j .

Fig. 5 is a graph showing the posture of the manipulator arm of the grasping robot B of the simulation mechanism of the context sensing device according to a specific embodiment of the present invention, wherein the abscissa represents time and the ordinate represents angle. In the embodiment shown in Figure 5, the robotic arm has 8 joints, where j ₁ represents the posture of the first joint, j ₂ represents the posture of the second joint, j ₃ represents the posture of the third joint, and j ₄ represents The posture of the fourth joint, j ₅ represents the posture of the fifth joint, j ₆ represents the posture of the sixth joint, j ₇ represents the posture of the seventh joint, and j ₈ represents the posture of the eighth joint. Therefore, in the curve shown in FIG. 5, the present invention can obtain the postures of the above-mentioned eight joints in a time series for simulating the grasping robot B. It should be noted that each joint has its ability to rotate, that is, the rotation angle has a numerical range, so the simulation of the above joints should be based on the ability of each joint.

In sub-step S23, the motion simulation module 223 drives the grasping robot to move along the motion trajectory planned by the motion planning module 222 and returns a simulation result based on the simulation requirements.

The motion simulation module 223 drives the context sensing device to move along the path planned by the motion planning module 222 and returns simulation results based on simulation requirements. FIG. 4 is a schematic diagram of a simulation diagram of a grasping robot grasping parts from a box according to a simulation mechanism of a scene sensing device according to a specific embodiment of the present invention. Therefore, the motion simulation module 223 performs a path planned by the grasping robot B based on the motion planning module 222 The process of moving to rotate the first joint j ₁ , the second joint j ₂ and the third joint j ₃ to grab multiple parts from the box B'is simulated in multiple possible 3D scenarios, including the first part p ₁ , The second part p ₂ and the third part p ₃ .

The results returned by the motion simulation module 223 may include grasping success and failure results, simulation time, collision detection, energy loss in various stages, and so on. For example, the simulation time at different stages is:

Simulation time=[t ₀ … t _i ]

Among them, i are different stages in the simulation.

Finally, step S3 is executed, and the test evaluation device 230 performs simulation on the context sensing device based on the test example.

Specifically, the test evaluation device 230 is used to summarize the simulation result based on the test case, the output of which is the simulation result, and generates evaluation results in different aspects, such as reliability, safety, efficiency, and robustness. For example, reliability is related to success rate, and the average success rate can be calculated as follows:

If the simulator's engine utilizes a physics engine, the execution of the simulation itself can find out whether the target object has been captured and successfully moved to the predetermined position. If the simulator's engine is a geometry engine, this metric cannot be evaluated.

Among them, safety is about arbitrary geometric collisions, which may occur during the operation of the gripping robot, for example, the gripper collides with the side of the box. For example, the collision check result can be expressed as follows:

Among them, Time refers to the time when the simulation collision occurs, Object_1 and Object_1 represent two target objects that collide with each other, and the distance between the two is less than the threshold, and Distance refers to the distance between Object_1 and Object_1. If Distance is less than 0 or negative, it means that two target objects have collided.

Among them, the efficiency represents the number of grabs per hour in the motor of the machine system and the average time and energy consumption of each grab when performing a movement. For example, the crawling efficiency per hour can be calculated as follows:

Crawling efficiency=average#fetch attempts × success rate

Among them, robustness is the sensitivity of performance standards under extreme conditions and extreme values. Among them, extreme values are automatically generated by algorithms. In addition, the content may include noise descriptions of system components. Therefore, the test can evaluate the quantitative impact on the above performance criteria when applying noise values to system components.

In addition to evaluating performance standards, the platform can also provide test coverage measurement methods, which represent the comprehensiveness of the coverage of automatically generated test cases. The test coverage measurement method can be evaluated based on the retrieval rules applied in the test data generating device 120. For example, scene-related coverage measures whether part of the context model and its combination are covered in the test.

Therefore, based on this, the present invention has a more comprehensive and objective method that can be quantified to automatically generate test cases.

The second aspect of the present invention provides a simulation device of a context sensing device, which includes:

The environmental management device 100 models the environment of the context-aware device based on customer needs, and traverses the environmental model to generate test cases;

The test simulation device 220 performs action planning and action simulation on the emotion perception device based on the test example.

Further, the environmental management device 100 further includes:

Modeling device 110, which describes basic objects and their relationships in the environment of the context-aware device using ontology to model the environment of the context-aware device, and annotates the environment model based on customer needs;

The test data generating device 120 traverses the environment model based on the test case, and extracts data from the environment model to generate a test case.

Further, the scene target also includes target attributes and noise, and the target attributes include the target's material, color, shape, position, generation probability, and the like.

Further, the context sensing device is a grasping robot, wherein the test simulation device 220 further includes:

A motion planning module 222, which plans a motion trajectory for the grasping robot according to the situational target that the grasping robot needs to grasp, wherein the motion track includes a posture representing the grasping robot, and the posture includes the grasping The rotation angles of the multiple joints of the robot arm in the time series;

The motion simulation module 223 drives the grasping robot to move along the motion trajectory and returns simulation results based on simulation requirements.

Further, the test simulation device 220 further includes a perception module 221, which maps the data of the test instance to the simulation scene, and recognizes the scene target object and its position information in the simulation scene based on a visual algorithm.

Further, the simulation device of the context sensing device further includes a test evaluation device 230, which performs simulation on the context sensing device based on the test example.

A third aspect provides a simulation system of a context-aware device, which includes: a processor; and a memory coupled with the processor, the memory having instructions stored therein, and the instructions cause all of them when executed by the processor. The electronic device performs actions, and the actions include: S1, modeling the environment of the context sensing device based on customer needs, and traversing the environment model to generate test cases; S2, performing action planning on the emotion perception device based on the test cases, Motion simulation and system evaluation.

Further, the scene target also includes target attributes and noise, and the target attributes include the target's material, color, shape, position, and generation probability.

The present invention can better support the simulation of the context sensing device. The modeling process provided by the present invention classifies and describes the environment of the context sensing device and the target objects in the environment and their relationships and attributes. The modeling process provided by the present invention is more clear and accurate, and avoids confusion and The next test scenario is generated to provide support. The present invention can also retrieve the graphical structure of the environment model and generate test cases for each traversal. In this process, the context-aware system is converted into a test case list to advance customer requirements. The present invention can also map test data to a simulator that interacts with the context sensing device, and can automatically generate a simulated stereo scene.

The present invention can better support the simulation of the context sensing device. The modeling process provided by the present invention classifies and describes the environment of the context sensing device and the target objects in the environment and their relationships and attributes. The modeling process provided by the present invention is simpler, and the next test scenario is tested by establishing a semantic model. Generate support. The present invention can also retrieve the graphical structure of the environment model and generate test cases for each traversal. In this process, the context-aware system is converted into a test case list to advance customer requirements. The present invention can also map the test data to the simulator that interacts with the context sensing device, and can automatically generate a simulated stereo scene.

Although the content of the present invention has been described in detail through the above preferred embodiments, it should be recognized that the above description should not be considered as limiting the present invention. After those skilled in the art have read the above content, various modifications and substitutions to the present invention will be apparent. Therefore, the protection scope of the present invention should be defined by the appended claims. In addition, any reference signs in the claims should not be regarded as limiting the involved claims; the word "comprising" does not exclude other claims or devices or steps not listed in the specification; "first", "section Words such as "two" are only used to indicate names, and do not indicate any specific order.

Claims

The simulation method of the context sensing device includes the following steps:

S1: Model the environment of the context-aware device based on customer requirements, and traverse the environment model to generate test cases;

S2: Perform action planning and action simulation on the emotion perception device based on the test example.
The simulation method of the context sensing device according to claim 1, wherein said step S1 further comprises the following steps:

S11, describing basic objects and their relationships in the environment of the context sensing device using ontology to model the environment of the context sensing device, and labeling the environment model based on customer needs;

S12: Traverse the environment model based on the test case, and extract data from the environment model to generate a test case.
The simulation method of the context sensing device according to claim 2, wherein the environment model includes objects, and the objects include a context target, a system under test, and a test case.
The simulation method of the context sensing device according to claim 3, wherein the context target further includes target attributes and noise, and the target attributes include material, color, shape, and position of the target.
The simulation method of the context perception device according to claim 3, wherein the context perception device comprises a grasping robot, and the system under test based on the grasping robot includes a vision system, a mechanical arm and a gripper, Wherein, the vision system includes visual noise, and the mechanical arm includes mechanical noise.
The simulation method of the context perception device according to claim 1, wherein the context perception device is a grasping robot, wherein the step S2 includes the following steps:

Step S22, planning a movement trajectory for the grabbing robot according to the scene target that the grabbing robot needs to grab, wherein the movement trajectory includes a posture representing the grabbing robot, and the posture includes the mechanical of the grabbing robot. The rotation angle of multiple joints of the arm in time series.

In step S23, the grasping robot is driven to move along the motion trajectory planned by the motion planning module 222 and returns the simulation result based on the simulation requirement.
The simulation method of the context sensing device according to claim 6, wherein the step S2 includes step S21: mapping the data of the test instance to the simulation scene, and identifying the scene target objects in the simulation scene based on a vision algorithm and Its location information.
The simulation method of the context sensing device according to claim 1, characterized in that, after the step S2, it further comprises a step S3: performing a simulation on the context sensing device based on the test example.
The simulation device of the context sensing device, including:

An environmental management device (100), which models the environment of the context-aware device based on customer needs, and traverses the environmental model to generate test cases;

A test simulation device (220), which performs action planning and action simulation on the emotion perception device based on a test example.
The simulation device of the context sensing device according to claim 9, characterized in that the environment management device (100) further comprises:

A modeling device (110), which describes the basic objects and their relationships in the environment of the context perception device using ontology to model the environment of the context perception device, and annotates the environment model based on customer needs ；

A test data generating device (120), which traverses the environmental model based on the test case, and extracts data from the environmental model to generate a test case.
The simulation device of the context sensing device according to claim 10, wherein the environment model includes objects, and the objects include a context target, a system under test, and a test case.
The simulation device of the context sensing device according to claim 11, wherein the context target further includes target attributes and noise, and the target attributes include the material, color, shape, and position of the target.
The simulation device of the context perception device according to claim 11, wherein the context perception device comprises a grasping robot, and the system under test based on the grasping robot includes a vision system, a mechanical arm and a gripper, Wherein, the visual system includes visual noise, and the mechanical arm includes mechanical noise.
The simulation device of the context perception device according to claim 9, wherein the context perception device is a grasping robot, wherein the test simulation device (220) further comprises:

A motion planning module (222), which plans a motion trajectory for the grasping robot according to the scene target that the grasping robot needs to grasp, wherein the motion track includes the posture of the grasping robot, and the posture includes the Grasp the rotation angle of multiple joints of the robotic arm of the robot in time series;

A motion simulation module (223), which drives the grabbing robot to move along the motion trajectory and returns a simulation result based on simulation requirements.
The simulation device of the context perception device according to claim 14, characterized in that the test simulation device (220) further comprises a perception module (221), which maps the data of the test instance to the simulation scene, and recognizes all data based on a visual algorithm. The scene target object and its position information in the simulation scene are described.
The simulation device of the context perception device according to claim 9, wherein the simulation device of the context perception device further comprises a test evaluation device (230), which performs simulation on the context perception device based on the test example.
The simulation system of the context sensing device, including:

Processor; and

A memory coupled to the processor, the memory having instructions stored therein, the instructions, when executed by the processor, cause the electronic device to perform actions, and the actions include:

S1: Model the environment of the context-aware device based on customer requirements, and traverse the environment model to generate test cases;

S2: Perform action planning and action simulation on the emotion perception device based on the test example.
A computer program product, which is tangibly stored on a computer-readable medium and includes computer-executable instructions, which when executed, cause at least one processor to execute any of claims 1 to 8 The method described in one item.
A computer-readable medium having computer-executable instructions stored thereon, the computer-executable instructions, when executed, cause at least one processor to execute the method according to any one of claims 1 to 8.