CN117245648A - A robot virtual modeling and tactile control method and device - Google Patents

Abstract

The application discloses a robot virtual modeling and touch control method and device, wherein the method comprises the following steps: establishing a virtual robot in a virtual environment according to physical entity data of the robot, and establishing connection between a force feedback controller and the virtual robot; the operator operates the force feedback controller and sends the tail end coordinate data of the force feedback controller to the virtual robot; the virtual robot calculates gesture data according to the terminal coordinate data, controls the virtual robot to perform gesture transformation according to the gesture data, acquires target interaction data interacted with the virtual environment in the gesture transformation process, and sends the target interaction data to the force feedback controller; the force feedback controller executes force feedback action according to the target interaction data; the method and the device realize the aim of providing bidirectional immersive tactile interaction for operators and the virtual robot, improve the reality of interaction with the virtual robot, and can be applied to robot control under virtual reality and augmented reality.

Description

A robot virtual modeling and tactile control method and device

Technical field

The present application relates to the field of robot simulation and control technology, and in particular to a robot virtual modeling and tactile control method and device.

Background technique

With the continuous development of the manufacturing and logistics industries, robots are widely used in assembly line production, assembly, logistics and transportation, and have become the core of smart factory construction. Robots can complete many tasks that are highly repetitive, dangerous, and require high precision, so high-precision control of robots is particularly important.

Robot virtual modeling is a technology that simulates the movement and control of robots. It simulates the movements and effects of robots through computers and is used in the design, layout and control of robots. In particular, with the introduction of technologies such as virtual reality, augmented reality, and mixed reality, real-time virtual modeling of robots can simulate, optimize, and correct the control of physical robots, which has become the key to supporting human-computer interaction and collaborative control.

During the interaction with the traditional robot virtual model, one can only rely on the head-mounted display and the handle for one-way visual and auditory operations, which will lead to a certain difference between the virtual model and the actual robot. That is to say, the traditional Robot virtual modeling and interaction have low authenticity and controllability, making it difficult to meet the needs of industrial applications.

Contents of the invention

In view of this, this application provides a robot virtual modeling and tactile control method and device, which helps to improve the authenticity of the interaction between the operator and the virtual robot.

According to one aspect of the present application, a robot virtual modeling and tactile control method is provided, which method includes:

Establish a virtual robot in the virtual environment based on the physical entity data of the robot, and establish a connection between the force feedback controller and the virtual robot;

The operator operates the force feedback controller and sends the end coordinate data of the force feedback controller to the virtual robot;

The virtual robot calculates posture data according to the terminal coordinate data, controls the virtual robot to perform posture transformation according to the posture data, obtains target interaction data that interacts with the virtual environment during the posture transformation process, and converts the The target interaction data is sent to the force feedback controller;

The force feedback controller performs force feedback actions according to the target interaction data.

Optionally, the physical entity data includes model data, mechanical size data, joint number data, and motion range data. The establishing a virtual robot in a virtual environment based on the physical entity data of the robot includes:

According to the model data, mechanical size data, joint number data and motion range data of the robot, a virtual model of each component of the virtual robot is established. According to the actual connection relationship of the robot, the virtual models of each component are combined to form the virtual model of each component. Describe a virtual three-dimensional model of the virtual robot, and bind the superior-subordinate relationship of each joint point in the virtual three-dimensional model;

A virtual robot is established according to the physical attributes of the robot and the bound virtual three-dimensional model. The physical attributes include mass, model size and surface material. The virtual models of each component include a base model, a connecting rod model, and a sensor model. and front-end tool models.

Optionally, binding the superior-subordinate relationship of each joint point in the virtual three-dimensional model includes:

Set the base model as a parent node, and set the first link model as a child node of the base model;

According to the preset sequence of the actual connection relationship of the robot, each link model is set as a child node of the previous link model in turn;

The sensor model and the front-end tool model are set as child nodes of the last connecting rod model.

Optionally, establishing a virtual robot based on the physical attributes of the robot and the bound virtual three-dimensional model includes:

Build a rigid body component, a collision body component and a surface material component for the bound virtual three-dimensional model according to the mass, model size and surface material of the robot to establish a virtual robot;

Wherein, the rigid body component is determined based on the mass of the robot, the collision body component is determined based on the model size of the robot, and the surface material component is determined based on the surface material properties of the robot.

Optionally, the virtual robot calculates posture data based on the terminal coordinate data, including:

Establishing a Cartesian coordinate space of the virtual robot, and establishing an independent coordinate system for each joint of the virtual robot in the Cartesian coordinate space according to the end coordinate data;

Determine the transformation matrix of adjacent joints of the virtual robot according to the independent coordinate system of each joint of the virtual robot;

According to the transformation matrix of each adjacent joint, the posture information of each joint of the virtual robot is calculated sequentially to obtain posture data.

Optionally, obtaining target interaction data for interacting with the virtual environment during the posture transformation process, and sending the target interaction data to the force feedback controller includes:

Detect whether the virtual robot comes into contact with the virtual environment during the posture transformation process. When contact occurs, obtain the collision contact position data and the collision virtual attribute data of the contacted virtual environment, and combine the collision contact position data and the collision virtual attribute data of the virtual environment. The collision virtual attribute data is used as the collision target interaction data, and the collision target interaction data is sent to the force feedback controller, wherein the virtual environment includes virtual objects;

If the operator presses the grasping button while the virtual robot is in contact with the virtual object, the virtual robot is controlled to perform a grasping action, the grasping contact position data and the grasping virtual attribute data are obtained, and all the objects are captured. The grabbing contact position data and the grabbing virtual attribute data are used as grabbing target interaction data, and the grabbing target interaction data is sent to the force feedback controller; if the operator releases the grabbing button, the control The virtual robot performs a release action, wherein the target interaction data includes collision target interaction data and grabbing target interaction data.

Optionally, the virtual attribute data includes rigid body component parameters, collision body component parameters and surface material component parameters;

Wherein, the rigid body component parameters are read from the rigid body component of the contacted virtual object and are determined by the quality of the contacted virtual object; the collider component parameters are read from the collider component of the contacted virtual object and are determined by the quality of the contacted virtual object. The model outline is determined; the surface material component parameters are read from the surface material component of the virtual object in contact, and determined by the hardness data, viscosity data and roughness data of the surface of the virtual object in contact.

Optionally, the force feedback action includes a gravity feedback action, an elastic force feedback action, a viscous force feedback action, and a friction force feedback action. The force feedback controller performs a force feedback action according to the target interaction data, including:

The force feedback controller performs the gravity feedback action according to the contact position data and the quality of the contacted virtual object;

The force feedback controller performs the elastic force feedback action according to the contact position data and the hardness data of the contacted virtual object;

The force feedback controller performs the viscous force feedback action according to the contact position data and the viscosity degree data of the contacted virtual object;

The force feedback controller performs the friction force feedback action according to the contact position data and the roughness data of the contacted virtual object.

According to another aspect of the present application, a robot virtual modeling and tactile control device is provided, which device includes:

A building module for establishing a virtual robot in a virtual environment based on the physical entity data of the robot, and establishing a connection between the force feedback controller and the virtual robot;

An operation module for the operator to operate the force feedback controller and send the end coordinate data of the force feedback controller to the virtual robot;

An interaction module, used for the virtual robot to calculate posture data according to the terminal coordinate data, control the virtual robot to perform posture transformation according to the posture data, and obtain the target interaction with the virtual environment during the posture transformation process. data, sending the target interaction data to the force feedback controller;

A feedback module is used for the force feedback controller to perform force feedback actions according to the target interaction data.

Optionally, the building module is also used to:

According to the model data, mechanical size data, joint number data and motion range data of the robot, a virtual model of each component of the virtual robot is established. According to the actual connection relationship of the robot, the virtual models of each component are combined to form the virtual model of each component. Describe a virtual three-dimensional model of the virtual robot, and bind the superior-subordinate relationship of each joint point in the virtual three-dimensional model;

A virtual robot is established according to the physical attributes of the robot and the bound virtual three-dimensional model. The physical attributes include mass, model size and surface material. The virtual models of each component include a base model, a connecting rod model, and a sensor model. and front-end tool models.

Optionally, the building module is also used to:

Set the base model as a parent node, and set the first link model as a child node of the base model;

According to the preset sequence of the actual connection relationship of the robot, each link model is set as a child node of the previous link model in turn;

The sensor model and the front-end tool model are set as child nodes of the last connecting rod model.

Optionally, the building module is also used to:

Build a rigid body component, a collision body component and a surface material component for the bound virtual three-dimensional model according to the mass, model size and surface material of the robot to establish a virtual robot;

Wherein, the rigid body component is determined based on the mass of the robot, the collision body component is determined based on the model size of the robot, and the surface material component is determined based on the surface material properties of the robot.

Optionally, the interactive module is also used to:

Establishing a Cartesian coordinate space of the virtual robot, and establishing an independent coordinate system for each joint of the virtual robot in the Cartesian coordinate space according to the end coordinate data;

Determine the transformation matrix of adjacent joints of the virtual robot according to the independent coordinate system of each joint of the virtual robot;

According to the transformation matrix of each adjacent joint, the posture information of each joint of the virtual robot is calculated sequentially to obtain posture data.

Optionally, the interactive module is also used to:

Detect whether the virtual robot comes into contact with the virtual environment during the posture transformation process. When contact occurs, obtain the collision contact position data and the collision virtual attribute data of the contacted virtual environment, and combine the collision contact position data and the collision virtual attribute data of the virtual environment. The collision virtual attribute data is used as the collision target interaction data, and the collision target interaction data is sent to the force feedback controller, wherein the virtual environment includes virtual objects;

If the operator presses the grasping button while the virtual robot is in contact with the virtual object, the virtual robot is controlled to perform a grasping action, the grasping contact position data and the grasping virtual attribute data are obtained, and all the objects are captured. The grabbing contact position data and the grabbing virtual attribute data are used as grabbing target interaction data, and the grabbing target interaction data is sent to the force feedback controller; if the operator releases the grabbing button, the control The virtual robot performs a release action, wherein the target interaction data includes collision target interaction data and grabbing target interaction data.

Optionally, the virtual attribute data includes rigid body component parameters, collision body component parameters and surface material component parameters;

Wherein, the rigid body component parameters are read from the rigid body component of the contacted virtual object and are determined by the quality of the contacted virtual object; the collider component parameters are read from the collider component of the contacted virtual object and are determined by the quality of the contacted virtual object. The model outline is determined; the surface material component parameters are read from the surface material component of the virtual object in contact, and determined by the hardness data, viscosity data and roughness data of the surface of the virtual object in contact.

Optionally, the feedback module is also used to:

The force feedback controller performs the gravity feedback action according to the contact position data and the quality of the contacted virtual object;

The force feedback controller performs the elastic force feedback action according to the contact position data and the hardness data of the contacted virtual object;

The force feedback controller performs the viscous force feedback action according to the contact position data and the viscosity degree data of the contacted virtual object;

The force feedback controller performs the friction force feedback action according to the contact position data and the roughness data of the contacted virtual object.

Through the above technical solution, this application provides a method and device for virtual modeling and tactile control of a robot, which accurately establishes a virtual robot through the physical entity data of the robot. The operator controls the force feedback controller to control the virtual robot according to the end coordinate data. Perform posture transformation to obtain target interaction data that interacts with the virtual environment during the posture transformation process. The force feedback controller performs force feedback actions based on the target interaction data, providing the operator with force-tactile perception and achieving two-way immersion between the operator and the virtual robot. The goal of tactile interaction is to improve the authenticity of interaction with virtual robots, and can be applied to robot control under virtual reality and augmented reality.

The above description is only an overview of the technical solutions of the present application. In order to have a clearer understanding of the technical means of the present application, they can be implemented according to the content of the description, and in order to make the above and other purposes, features and advantages of the present application more obvious and understandable. , the specific implementation methods of the present application are specifically listed below.

Description of drawings

The drawings described here are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation of the present application. In the attached picture:

Figure 1 shows a schematic flow chart of a robot virtual modeling and tactile control method provided by an embodiment of the present application;

Figure 2 shows a schematic flow chart of another robot virtual modeling and tactile control method provided by an embodiment of the present application;

Figure 3 shows a schematic structural diagram of a robot virtual modeling and tactile control device provided by an embodiment of the present application.

Detailed ways

The present application will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, as long as there is no conflict, the embodiments and features in the embodiments of this application can be combined with each other.

In the existing technology, virtual robots are mainly static presentations of physical robots and cannot simulate the dynamic behavior and properties of physical robots in the real world, such as collision and mass. This will lead to certain differences between virtual robots and physical robots. Due to differences, virtual robots cannot accurately simulate a series of behaviors of physical robots in the real world; during interaction with traditional virtual robots, they can only rely on head-mounted displays and handles for one-way visual and auditory operations, and it is difficult to perform two-way Immersive tactile interaction, that is to say, the creation and interaction of traditional virtual robots have low authenticity and controllability, and are difficult to meet the needs of industrial applications.

In this embodiment, a robot virtual modeling and tactile control method is provided. As shown in Figure 1, the method includes:

Step 101: Create a virtual robot in the virtual environment based on the physical entity data of the robot, and establish a connection between the force feedback controller and the virtual robot.

The embodiments of the present application can be applied to the process of interacting with a virtual robot to realize tactile interaction with the virtual robot, and can also be applied to robot control under virtual reality and augmented reality. First, establish a virtual robot in the virtual environment based on the physical entity data of the robot, and establish a connection between the force feedback controller and the above-mentioned virtual robot. For example, the above-mentioned virtual robot can be built in virtual reality glasses, and can be established through wireless networks, interfaces, etc. Establish a connection between the force feedback controller and the above-mentioned virtual robot. The virtual robot can be accurately established through the physical entity data of the robot, and establish a connection between the force feedback controller and the above-mentioned virtual robot to prepare for the subsequent tactile interaction. .

Step 102: The operator operates the force feedback controller and sends the end coordinate data of the force feedback controller to the virtual robot.

Next, the operator operates the above-mentioned force feedback controller and sends the terminal coordinate data of the above-mentioned force feedback controller to the above-mentioned virtual robot, thereby realizing one-way tactile interaction between the operator and the virtual robot, so that the virtual robot can operate as described End coordinate data for data interaction.

Step 103: The virtual robot calculates posture data according to the terminal coordinate data, controls the virtual robot to perform posture transformation according to the posture data, and obtains target interaction data for interacting with the virtual environment during the posture transformation process, Send the target interaction data to the force feedback controller.

Next, the above-mentioned virtual robot is controlled to simulate the posture corresponding to the above-mentioned terminal coordinate data, and posture transformation needs to be performed. Then the posture data needs to be calculated through the above-mentioned terminal coordinate data. The virtual robot then performs posture transformation based on the above-mentioned posture data, and obtains the posture during the above-mentioned posture transformation process. , the target interaction data of the virtual robot interacting with the virtual environment is used to complete the control and attitude change of the virtual robot, and the target interaction data is sent to the force feedback controller to facilitate subsequent tactile interaction.

Step 104: The force feedback controller performs a force feedback action according to the target interaction data.

Next, the above-mentioned force feedback controller performs force feedback actions according to the above-mentioned target interaction data to complete two-way tactile interaction. The above-mentioned force feedback controller can provide multi-degree-of-freedom force feedback information to give the operator force tactile perception.

By applying the technical solution of this embodiment, a virtual robot is accurately established through the physical entity data of the robot. The operator controls the force feedback controller, controls the virtual robot to perform posture transformation according to the terminal coordinate data, and obtains the target of the virtual environment during the posture transformation process. Interaction data, the force feedback controller performs force feedback actions according to the target interaction data, provides the operator with force tactile perception, achieves the goal of providing two-way immersive tactile interaction between the operator and the virtual robot, and improves the interaction with the virtual robot. authenticity.

Further, as a refinement and expansion of the specific implementation of the above embodiment, in order to completely explain the specific implementation process of this embodiment, the physical attributes include mass, model size and surface material, and the target interaction data includes collision target interaction data. Interaction data with the grasping target provides another method of robot virtual modeling and tactile control, as shown in Figure 2. This method includes:

Step 201: Establish a virtual model of each component of the virtual robot based on the robot's model data, mechanical size data, joint number data, and motion range data, and combine the virtual models of each component according to the actual connection relationship of the robot. , forming a virtual three-dimensional model of the virtual robot, and binding the superior-subordinate relationship of each joint point in the virtual three-dimensional model.

In the embodiment of the present application, firstly, a virtual model of each component of the above-mentioned virtual robot is established based on the model data, mechanical size data, joint number data and motion range data of the above-mentioned robot. The virtual model of each component includes a base model and a connecting rod. model, sensor model, front-end tool model, etc.; then, according to the actual connection relationship of the above-mentioned robot, the virtual models of the above-mentioned components are combined to form a virtual three-dimensional model of the above-mentioned virtual robot, and the superior-subordinate relationship of each joint point is bound .

Step 202: Build a rigid body component, a collision body component and a surface material component for the bound virtual three-dimensional model according to the mass, model size and surface material of the robot to create a virtual robot; establish a force feedback controller and the virtual Connections between robots.

Next, the above-mentioned rigid body component is determined according to the mass of the above-mentioned robot, the above-mentioned collision body component is determined according to the model size of the above-mentioned robot, the above-mentioned surface material component is determined according to the surface material of the above-mentioned robot, and a rigid body component is built for the above-mentioned virtual three-dimensional model based on the mass of the above-mentioned robot. , build a collision body component for the above-mentioned virtual three-dimensional model according to the model size of the above-mentioned robot, build a surface material component for the above-mentioned virtual three-dimensional model according to the surface material of the above-mentioned robot, establish a virtual robot, and establish a connection between the force feedback controller and the above-mentioned virtual robot. , by adding rigid body components, collision body components, and surface material components to give the virtual robot tactile interaction attributes, improving the authenticity of the interaction.

Step 203: The operator operates the force feedback controller and sends the end coordinate data of the force feedback controller to the virtual robot.

Next, the operator operates the force feedback controller and sends the terminal coordinate data of the force feedback controller to the virtual robot.

Step 204: Establish a Cartesian coordinate space of the virtual robot, and establish an independent coordinate system of each joint of the virtual robot in the Cartesian coordinate space according to the end coordinate data; according to the independent coordinate system of each joint of the virtual robot Determine the transformation matrices of adjacent joints of the virtual robot, calculate the posture information of each joint of the virtual robot in sequence according to the transformation matrix of each adjacent joint, and obtain posture data.

Next, the Cartesian coordinate space of the virtual robot is established according to the preset creation rules, and the independent coordinate system of each joint of the virtual robot is established in the Cartesian coordinate space according to the end coordinate data, wherein the right-hand rule is used Establish the independent coordinate system of each joint of the above-mentioned joints; determine the transformation matrix of the adjacent joints of the virtual robot according to the independent coordinate system of each joint of the virtual robot as:

Where, i represents the i-th joint in the virtual robot, represents the transformation matrix of the i-1th joint and the i-th joint, θ _i is the joint angle variable of the i-th joint, l _i-1 represents the transformation matrix between the i-1th joint and the i-th joint along the X-axis Distance, d _i represents the translation distance along the Z-axis between the i-1th joint and the i-th joint, α _i-1 represents the rotation angle around the X-axis between the i-1th joint and the i-th joint;

Then, based on the transformation matrix of each adjacent joint, the posture information of each joint of the virtual robot is calculated sequentially. For example, the posture information of the robot end relative to joint 1 can be calculated as Among them, T _i ⁱ⁺¹ represents the transformation matrix of the i-th and i+1-th two adjacent joints, and T ₁ ^End represents the transformation matrix between the first joint and the end position. The attitude data is obtained by calculating the phase. The transformation matrix of adjacent joints determines the posture data of the virtual robot that needs to be transformed to prepare for the subsequent completion of tactile interaction.

Step 205: Control the virtual robot to perform posture transformation according to the posture data, detect whether the virtual robot comes into contact with the virtual environment during the posture transformation process, and when contact occurs, obtain the collision contact position data and the contacted virtual environment. Collision virtual attribute data of the environment, the collision contact position data and the collision virtual attribute data are used as collision target interaction data, and the collision target interaction data is sent to the force feedback controller.

Next, the target interaction data includes collision target interaction data and grasping target interaction data. The virtual robot is controlled to perform posture transformation according to the above posture data, and whether the above virtual robot comes into contact with the above virtual environment during the posture transformation process is detected, wherein, The virtual environment includes virtual objects. The virtual robot may come into contact with the virtual environment or with virtual objects in the virtual environment during the posture transformation process; when it comes into contact with the virtual environment or virtual objects, the collision contact position data and the contact location data are obtained. The collision virtual attribute data of the virtual environment or the contacted virtual object, the above collision contact position data and the above collision virtual attribute data are used as the collision target interaction data, and the above collision target interaction data is sent to the force feedback controller. The target interaction data in the process is sent to the force feedback controller, which realizes data interaction with the virtual robot and facilitates the subsequent tactile interaction.

Step 206: If the operator presses the grabbing button while the virtual robot is in contact with the virtual environment, the virtual robot is controlled to perform a grabbing action, and the grabbing contact position data and the grabbing virtual attribute data are obtained. , use the grabbing contact position data and the grabbing virtual attribute data as grabbing target interaction data, and send the grabbing target interaction data to the force feedback controller; if the operator releases the grabbing button , then the virtual robot is controlled to perform the release action.

Next, the above-mentioned virtual environment includes virtual objects. When the operator presses the grabbing button while the virtual robot is in contact with the virtual object, the grabbing function is triggered, the virtual robot is controlled to perform the grabbing action, and the grabbing contact position data and grabbing Get the virtual attribute data, use the above-mentioned grasping contact position data and the above-mentioned grasping virtual attribute data as the grasping target interaction data, send the above-mentioned grasping target interaction data to the force feedback controller, and when the operator releases the button, the release function is triggered , controlling the virtual robot to perform a release action, in which the above-mentioned grabbing virtual attribute data can be obtained by adding a fixed joint component at the contact position. The fixed joint component is used to connect the virtual robot and the virtual object to simulate the grabbing effect, correspondingly , when the release function is triggered, delete the above-mentioned fixed joint components and the above-mentioned grabbing virtual attribute data, end the grabbing action, control the virtual robot to perform the action corresponding to the posture transformation through the operator's operation, obtain the target interaction data during transformation and send it to the force feedback control It can simulate the grasping effect by establishing fixed joint components and improve the authenticity of interaction.

Step 207: The force feedback controller performs a force feedback action according to the target interaction data.

Next, the above-mentioned force feedback controller performs a force feedback action according to the above-mentioned target interaction data, giving the operator force tactile perception and realizing tactile interaction with the virtual robot.

Optionally, in the above step 201, binding the superior-subordinate relationship of each joint point in the virtual three-dimensional model includes: setting the base model as the parent node, and setting the first link model as the Child nodes of the base model; according to the preset sequence of the actual connection relationship of the robot, each link model is set as a child node of the previous link model; the sensor model and the front-end tool model are set is the child node of the last connecting rod model.

In the above-mentioned embodiment of the present application, the connecting rod model includes multiple connecting rod models, and they are distinguished by a preset sequence. The above-mentioned base model is set as the parent node, and the first connecting rod model is set as the above-mentioned base model. sub-node, according to the preset order of the actual connection relationship of the above-mentioned robot, each link model is set as a child node of the previous link model in turn, and the above-mentioned sensor model and the above-mentioned front-end tool model are set as the last link The child nodes of the rod model complete the binding of the superior and subordinate relationships of each joint point to facilitate the realization of virtual robot control.

Optionally, the above force feedback action includes gravity feedback action, elastic force feedback action, viscous force feedback action and friction force feedback action; in the above step 207, "the force feedback controller executes the force feedback action according to the target interaction data" The method includes: the force feedback controller performs the gravity feedback action according to the contact position data and the quality of the contacted virtual object; the force feedback controller performs the gravity feedback action according to the contact position data and the contacted virtual object. The force feedback controller performs the viscous force feedback action based on the contact position data and the viscosity data of the contacted virtual object; the force feedback control The device performs the friction feedback action based on the contact position data and the roughness data of the contacted virtual object.

In the above-mentioned embodiment of the present application, the above-mentioned force feedback controller performs the above-mentioned gravity feedback action according to the above-mentioned contact position data and the above-mentioned mass of the contacted virtual object, for example, according to the above-mentioned contact position data and the above-mentioned mass of the contacted virtual object, the force feedback controller performs the above-mentioned gravity feedback action. Torque simulates the gravity feedback effect when lifting an object; the above-mentioned force feedback controller performs the above-mentioned elastic force feedback action based on the above-mentioned contact position data and the above-mentioned hardness data of the contacted virtual object. The hardness data of the object increases torque in the direction of contact, simulating the elastic force effect on the surface of the object; the above-mentioned force feedback controller performs the above-mentioned viscous force feedback action based on the above-mentioned contact position data and the above-mentioned viscosity data of the contacted virtual object, for example, according to The above-mentioned contact position data and the above-mentioned viscosity data of the contacted virtual object increase torque in the opposite direction of the contact direction to simulate the elastic force effect of the object surface; the above-mentioned force feedback controller is based on the above-mentioned contact position data and the above-mentioned roughness of the contacted virtual object. Degree data, perform the above-mentioned friction feedback action, for example, according to the above-mentioned contact position data and the above-mentioned roughness data of the contacted virtual object, add torque to the direction perpendicular to the contact, simulate the friction effect on the surface of the object, and enable the operator to control it through force feedback The device senses the tactile effect, improving the authenticity of interacting with the robot.

It should be noted that the above-mentioned virtual attribute data includes rigid body component parameters, collision body component parameters and surface material component parameters; the rigid body component parameters are read from the rigid body component of the contacted virtual object and are determined by the quality of the contacted virtual object; The collider component parameters are read from the collider component of the touched virtual object and determined by the model outline of the touched virtual object; the surface material component parameter parameters are read from the surface material component of the touched virtual object and determined by the touched virtual object. The hardness data, viscosity data and roughness data of the surface are determined.

In the above-mentioned embodiments of the present application, the above-mentioned rigid body component parameters are determined by the mass of the contacted virtual object and are read from the rigid body component of the contacted virtual object; the above-mentioned collision body component parameters are determined by the model outline of the contacted virtual object, and Read from the collider component of the virtual object in contact; the above surface material component parameters are determined by the hardness data, viscosity data and roughness data of the surface of the virtual object in contact.

By applying the technical solution of this embodiment, a virtual robot is created through a physical robot, and the virtual robot is given tactile interaction attributes by adding rigid body components, collision body components, and surface material components, and when the posture transformation of the virtual robot is controlled by real-time tactility, the tactile interaction properties are obtained. The target interaction data in the virtual environment provides force tactile feedback for the operator and provides two-way immersive tactile interaction between the operator and the virtual robot, improving the authenticity of the interaction.

Further, as a specific implementation of the method in Figure 1, an embodiment of the present application provides a robot virtual modeling and tactile control device. As shown in Figure 3, the device includes:

A building module for establishing a virtual robot in a virtual environment based on the physical entity data of the robot, and establishing a connection between the force feedback controller and the virtual robot;

An operation module for the operator to operate the force feedback controller and send the end coordinate data of the force feedback controller to the virtual robot;

An interaction module, used for the virtual robot to calculate posture data according to the terminal coordinate data, control the virtual robot to perform posture transformation according to the posture data, and obtain the target interaction with the virtual environment during the posture transformation process. data, sending the target interaction data to the force feedback controller;

A feedback module is used for the force feedback controller to perform force feedback actions according to the target interaction data.

Optionally, the building module is also used to:

According to the model data, mechanical size data, joint number data and motion range data of the robot, a virtual model of each component of the virtual robot is established. According to the actual connection relationship of the robot, the virtual models of each component are combined to form the virtual model of each component. Describe a virtual three-dimensional model of the virtual robot, and bind the superior-subordinate relationship of each joint point in the virtual three-dimensional model;

A virtual robot is established according to the physical attributes of the robot and the bound virtual three-dimensional model. The physical attributes include mass, model size and surface material. The virtual models of each component include a base model, a connecting rod model, and a sensor model. and front-end tool models.

Optionally, the building module is also used to:

Set the base model as a parent node, and set the first link model as a child node of the base model;

According to the preset sequence of the actual connection relationship of the robot, each link model is set as a child node of the previous link model in turn;

The sensor model and the front-end tool model are set as child nodes of the last connecting rod model.

Optionally, the building module is also used to:

Build a rigid body component, a collision body component and a surface material component for the bound virtual three-dimensional model according to the mass, model size and surface material of the robot to establish a virtual robot;

Wherein, the rigid body component is determined based on the mass of the robot, the collision body component is determined based on the model size of the robot, and the surface material component is determined based on the surface material properties of the robot.

Optionally, the interactive module is also used to:

Establishing a Cartesian coordinate space of the virtual robot, and establishing an independent coordinate system for each joint of the virtual robot in the Cartesian coordinate space according to the end coordinate data;

Determine the transformation matrix of adjacent joints of the virtual robot according to the independent coordinate system of each joint of the virtual robot;

According to the transformation matrix of each adjacent joint, the posture information of each joint of the virtual robot is calculated sequentially to obtain posture data.

Optionally, the interactive module is also used to:

Detect whether the virtual robot comes into contact with the virtual environment during the posture transformation process. When contact occurs, obtain the collision contact position data and the collision virtual attribute data of the contacted virtual environment, and combine the collision contact position data and the collision virtual attribute data of the virtual environment. The collision virtual attribute data is used as the collision target interaction data, and the collision target interaction data is sent to the force feedback controller, wherein the virtual environment includes virtual objects;

If the operator presses the grasping button while the virtual robot is in contact with the virtual object, the virtual robot is controlled to perform a grasping action, the grasping contact position data and the grasping virtual attribute data are obtained, and all the objects are captured. The grabbing contact position data and the grabbing virtual attribute data are used as grabbing target interaction data, and the grabbing target interaction data is sent to the force feedback controller; if the operator releases the grabbing button, the control The virtual robot performs a release action.

Optionally, the virtual attribute data includes rigid body component parameters, collision body component parameters and surface material component parameters;

Wherein, the rigid body component parameters are read from the rigid body component of the contacted virtual object and are determined by the quality of the contacted virtual object; the collider component parameters are read from the collider component of the contacted virtual object and are determined by the quality of the contacted virtual object. The model outline is determined; the surface material component parameters are read from the surface material component of the virtual object in contact, and determined by the hardness data, viscosity data and roughness data of the surface of the virtual object in contact.

Optionally, the feedback module is also used to:

The force feedback controller performs the gravity feedback action according to the contact position data and the quality of the contacted virtual object;

The force feedback controller performs the elastic force feedback action according to the contact position data and the hardness data of the contacted virtual object;

The force feedback controller performs the viscous force feedback action according to the contact position data and the viscosity degree data of the contacted virtual object;

The force feedback controller performs the friction force feedback action according to the contact position data and the roughness data of the contacted virtual object.

It should be noted that for other corresponding descriptions of the functional units involved in the robot virtual modeling and tactile control device provided by the embodiments of the present application, please refer to the corresponding descriptions in the method of Figures 1 to 2, and will not be described again here.

Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus the necessary general hardware platform, or can also be implemented by hardware, and the virtual entity can be accurately established through the physical entity data of the robot. For the robot, the operator controls the force feedback controller, controls the virtual robot to perform posture transformation according to the terminal coordinate data, and obtains the target interaction data with the virtual environment during the posture transformation process. The force feedback controller performs force feedback actions based on the target interaction data, giving The operator provides force tactile perception, achieving the goal of providing two-way immersive tactile interaction between the operator and the virtual robot, and improving the authenticity of the interaction with the virtual robot.

Those skilled in the art can understand that the accompanying drawing is only a schematic diagram of a preferred implementation scenario, and the modules or processes in the accompanying drawing are not necessarily necessary for implementing the present application. Those skilled in the art can understand that the modules in the devices in the implementation scenario can be distributed in the devices in the implementation scenario according to the description of the implementation scenario, or can be correspondingly changed and located in one or more devices different from the implementation scenario. The modules of the above implementation scenarios can be combined into one module or further split into multiple sub-modules.

The above serial numbers of this application are only for description and do not represent the advantages and disadvantages of the implementation scenarios. What is disclosed above are only a few specific implementation scenarios of the present application. However, the present application is not limited thereto. Any changes that can be thought of by those skilled in the art should fall within the protection scope of the present application.

Claims (9)

1. A robot virtual modeling and tactile control method, characterized in that the method includes: Establish a virtual robot in the virtual environment based on the physical entity data of the robot, and establish a connection between the force feedback controller and the virtual robot; The operator operates the force feedback controller and sends the end coordinate data of the force feedback controller to the virtual robot; The virtual robot calculates posture data according to the terminal coordinate data, controls the virtual robot to perform posture transformation according to the posture data, obtains target interaction data that interacts with the virtual environment during the posture transformation process, and converts the The target interaction data is sent to the force feedback controller; The force feedback controller performs force feedback actions according to the target interaction data. 2. The method according to claim 1, characterized in that the physical entity data includes model data, mechanical size data, joint number data and motion range data, and the virtual entity data is established in the virtual environment according to the physical entity data of the robot. Robots, including: According to the model data, mechanical size data, joint number data and motion range data of the robot, a virtual model of each component of the virtual robot is established. According to the actual connection relationship of the robot, the virtual models of each component are combined to form the virtual model of each component. Describe a virtual three-dimensional model of the virtual robot, and bind the superior-subordinate relationship of each joint point in the virtual three-dimensional model; A virtual robot is established according to the physical attributes of the robot and the bound virtual three-dimensional model. The physical attributes include mass, model size and surface material. The virtual models of each component include a base model, a connecting rod model, and a sensor model. and front-end tool models. 3. The method according to claim 2, wherein the binding of the superior-subordinate relationship of each joint point in the virtual three-dimensional model includes: Set the base model as a parent node, and set the first link model as a child node of the base model; According to the preset sequence of the actual connection relationship of the robot, each link model is set as a child node of the previous link model in turn; The sensor model and the front-end tool model are set as child nodes of the last connecting rod model. 4. The method of claim 2, wherein establishing a virtual robot based on the physical attributes of the robot and the bound virtual three-dimensional model includes: Build a rigid body component, a collision body component and a surface material component for the bound virtual three-dimensional model according to the mass, model size and surface material of the robot to establish a virtual robot; Wherein, the rigid body component is determined based on the mass of the robot, the collision body component is determined based on the model size of the robot, and the surface material component is determined based on the surface material properties of the robot. 5. The method according to claim 1, characterized in that the virtual robot calculates posture data according to the terminal coordinate data, including: Establishing a Cartesian coordinate space of the virtual robot, and establishing an independent coordinate system for each joint of the virtual robot in the Cartesian coordinate space according to the end coordinate data; Determine the transformation matrix of adjacent joints of the virtual robot according to the independent coordinate system of each joint of the virtual robot; According to the transformation matrix of each adjacent joint, the posture information of each joint of the virtual robot is calculated sequentially to obtain posture data. 6. The method according to claim 1, wherein the target interaction data includes collision target interaction data and grabbing target interaction data, and the acquisition of targets interacting with the virtual environment during the posture transformation process Interaction data, sending the target interaction data to the force feedback controller, including: Detect whether the virtual robot comes into contact with the virtual environment during the posture transformation process. When contact occurs, obtain the collision contact position data and the collision virtual attribute data of the contacted virtual environment, and combine the collision contact position data and the collision virtual attribute data of the virtual environment. The collision virtual attribute data is used as the collision target interaction data, and the collision target interaction data is sent to the force feedback controller, wherein the virtual environment includes virtual objects; If the operator presses the grasping button while the virtual robot is in contact with the virtual object, the virtual robot is controlled to perform a grasping action, the grasping contact position data and the grasping virtual attribute data are obtained, and all the objects are captured. The grabbing contact position data and the grabbing virtual attribute data are used as grabbing target interaction data, and the grabbing target interaction data is sent to the force feedback controller; if the operator releases the grabbing button, the control The virtual robot performs a release action. 7. The method according to claim 6, wherein the virtual attribute data includes rigid body component parameters, collision body component parameters and surface material component parameters; Wherein, the rigid body component parameters are read from the rigid body component of the contacted virtual object and are determined by the quality of the contacted virtual object; the collider component parameters are read from the collider component of the contacted virtual object and are determined by the quality of the contacted virtual object. The model outline is determined; the surface material component parameters are read from the surface material component of the virtual object in contact, and determined by the hardness data, viscosity data and roughness data of the surface of the virtual object in contact. 8. The method according to claim 7, wherein the force feedback action includes a gravity feedback action, an elastic force feedback action, a viscous force feedback action and a friction force feedback action, and the force feedback controller controls the force feedback action according to the target Interactive data execution force feedback actions, including: The force feedback controller performs the gravity feedback action according to the contact position data and the quality of the contacted virtual object; The force feedback controller performs the elastic force feedback action according to the contact position data and the hardness data of the contacted virtual object; The force feedback controller performs the viscous force feedback action according to the contact position data and the viscosity degree data of the contacted virtual object; The force feedback controller performs the friction force feedback action according to the contact position data and the roughness data of the contacted virtual object. 9. A robot virtual modeling and tactile control device, characterized in that the device includes: A building module for establishing a virtual robot in a virtual environment based on the physical entity data of the robot, and establishing a connection between the force feedback controller and the virtual robot; An operation module for the operator to operate the force feedback controller and send the end coordinate data of the force feedback controller to the virtual robot; An interaction module, used for the virtual robot to calculate posture data according to the terminal coordinate data, control the virtual robot to perform posture transformation according to the posture data, and obtain the target interaction with the virtual environment during the posture transformation process. data, sending the target interaction data to the force feedback controller; A feedback module is used for the force feedback controller to perform force feedback actions according to the target interaction data.