CN115329922A - Coal mine underground multifunctional four-legged robot digital twin system and operation method thereof - Google Patents
Coal mine underground multifunctional four-legged robot digital twin system and operation method thereof Download PDFInfo
- Publication number
- CN115329922A CN115329922A CN202211157311.XA CN202211157311A CN115329922A CN 115329922 A CN115329922 A CN 115329922A CN 202211157311 A CN202211157311 A CN 202211157311A CN 115329922 A CN115329922 A CN 115329922A
- Authority
- CN
- China
- Prior art keywords
- robot
- virtual
- subsystem
- planning
- coal mine
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/004—Artificial life, i.e. computing arrangements simulating life
- G06N3/008—Artificial life, i.e. computing arrangements simulating life based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/10—Geometric CAD
- G06F30/17—Mechanical parametric or variational design
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/20—Design optimisation, verification or simulation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/30—Creation or generation of source code
- G06F8/31—Programming languages or programming paradigms
- G06F8/315—Object-oriented languages
Abstract
The invention discloses a digital twinning system and an operation method of a multifunctional quadruped robot in a coal mine, and belongs to the technical field of digital twinning. The system comprises a robot virtual space and a robot physical entity, wherein the robot physical entity is of a four-footed structure and comprises an autonomous perception module, an autonomous decision module and an autonomous control module, and the robot virtual space comprises four subsystems of robot virtual simulation, robot virtual planning, robot virtual debugging and robot virtual monitoring. The robot virtual simulation, virtual planning and virtual debugging subsystem provides information and reference for the design and operation of the robot physical entity; the robot virtual monitoring subsystem provides a three-dimensional running state monitoring and remote visual man-machine interaction interface for the robot physical entity. Through the real-time bidirectional information interactive mapping of the physical space and the virtual space, a digital twin solution is provided for the design, operation and maintenance and monitoring of the underground coal mine robot, and the intelligent level of a coal mine is improved.
Description
Technical Field
The invention belongs to the technical field of digital twinning, and particularly relates to a digital twinning system and an operation method of a multifunctional quadruped robot in a coal mine.
Background
The intellectualization becomes the core technical support for the high-quality development of the coal industry, and is the future of the development of the coal industry. The super-sextupole coal mine personnel can effectively improve the safety of coal mine production by carrying out dangerous and heavy work and developing and applying the coal mine robot.
The technical difficulty of the research of the coal mine robot is high, the period is long, the cost is high, some key technologies are not solved, and the design and the application of the coal mine robot still have a plurality of defects which are mainly expressed in the following aspects:
(1) At present, the existing coal mine robot is only applied to fields related to detection, such as routing inspection, rescue and the like, and has no substantive progress for operation under complex conditions;
(2) Because the underground environment of the coal mine has complexity and unpredictability, compared with a ground robot, the coal mine robot has higher on-site debugging difficulty and stronger unknown property, and is easy to cause waste of manpower and material resources;
(3) At the present stage, the coal mine robot is difficult to replace a worker to realize complete autonomous operation, people and the robot still need to cooperate, and a human-computer interaction channel between an operator and the coal mine robot needs to be researched;
(4) The underground form of the coal mine is complicated and changeable, but the coal mine robot lacks an effective three-dimensional visual remote monitoring means, and the real-time pose state of the robot is difficult to acquire visually and comprehensively.
With the continuous progress and maturity of the digital twin technology and the combination of the wide application and the rapid development of the modern communication technology, a bridge spanning the physical world and the virtual world is built. The digital twin technology can map a digital model of a physical world in a virtual space, sense, diagnose and predict the state of a physical entity object in the virtual space in real time, and regulate and control the physical entity through optimization and instructions. The method also provides a brand new thought and an important reference for the design, operation, maintenance and monitoring of the coal mine robot.
In the prior art, a digital twin-driven inspection robot remote control system and method disclosed in patent document No. CN113050649A includes an inspection robot physical entity, a data sensing and transmission module, an inspection robot virtual entity, a control module and twin data; a virtual remote control platform of the inspection robot for the coal mine fully-mechanized mining face is established by using a digital twinning technology, and virtual and real synchronous motion, state monitoring and remote control of the inspection robot for the coal mine fully-mechanized mining face are realized. In a coal mine dispatching robot system based on an artificial intelligence technology disclosed in patent document with publication number CN114398773A, a digital twin technology is used to construct a dispatching twin model of a virtual space which is mapped with an underground physical space dispatching system, and real-time data obtained in the production process of a mine and a physical entity are fused and cooperated to realize three-dimensional visualization of coal mine production commanding and dispatching.
However, in the two schemes, the digital twin technology is only used for remote state monitoring and control scheduling of the coal mine robot, the powerful analysis and prediction capabilities of the digital twin technology in a virtual space are not fully exerted, the digital twin technology is not applied to preposed links of virtual simulation, virtual planning, virtual debugging and the like of the coal mine robot, and information and reference are provided for autonomous execution of a robot physical system. And does not relate to a human-computer interaction method for connecting a physical space and a virtual space in the process of robot state monitoring and remote control.
In the prior art, a test and evaluation method for an intelligent fully-mechanized mining robot production system disclosed in patent document No. CN113128109A is used to construct a virtual offline operation system of a fully-mechanized mining face, reproduce a virtual operation mining situation of the working face, determine simulation initial data and virtual scene operation data, construct an AI robot analysis system, input equipment and geological detection means according to parameters of future intelligent development operation, construct a fully-mechanized mining face operation evaluation system, simulate the future fully-mechanized mining robot operation, determine a development trend, and test the operation performance of the robot.
In the scheme, the system is mainly used for off-line simulation and the reproduction of the operation and mining conditions of the fully mechanized mining face, and the prediction analysis is carried out on the future operation state, so that the operation path of the equipment is optimized, and the working face can operate safely and efficiently. Although real-time operation data of equipment can also be accessed to synchronously perform virtual-real fusion monitoring in a virtual space, the real-time interaction between the off-line simulation and the on-line monitoring is not available, and the accuracy and the reliability of a mechanism model of an off-line simulation system cannot be verified through the real-time operation data.
In conclusion, the prior art does not fully utilize the characteristics and advantages of the digital twinning technology, only applies the digital twinning technology to a certain flow in the full life cycle of the coal mine robot, and has a single application scene; the method has the advantages of large field debugging difficulty, lack of effective three-dimensional visual remote monitoring means, imperfect digital twin model, further deepening of operation mechanisms such as real-time interaction, iterative optimization and dynamic correction of a physical space and a virtual space, and low system integration level.
Disclosure of Invention
The invention provides a digital twinning system of a multifunctional quadruped robot in a coal mine and an operation method thereof, which provide a digital twinning solution for design, operation, maintenance and monitoring of the robot in the coal mine through real-time bidirectional information interactive mapping of physical and virtual spaces, and improve the intelligent level of the coal mine.
In order to solve the above technical problems, according to one aspect of the present invention, there is provided a coal mine underground multifunctional quadruped robot digital twinning system, comprising a robot virtual space and a robot physical entity,
the robot virtual space comprises a robot virtual simulation subsystem, a robot virtual planning subsystem, a robot virtual debugging subsystem and a robot virtual monitoring subsystem;
the robot virtual simulation subsystem is developed by Unity3d and is used as a test verification platform for the design of a robot body and the kinematic modeling in the early stage;
the robot virtual planning subsystem is developed by Unity3d and is used for completing path planning of the coal mine underground multifunctional quadruped robot under a virtual environment;
the robot virtual debugging subsystem is developed by Webots and is used for testing and optimizing the robot virtual planning subsystem;
the robot virtual monitoring subsystem is developed by a Unity3d and comprises a robot virtual monitoring module and a teleoperation module;
the robot physical entity adopts a four-footed structure and comprises an autonomous perception module, an autonomous decision module and an autonomous control module; the robot physical entity and the robot virtual debugging system are communicated with a Unity3d upper computer of the robot virtual monitoring system through a wireless serial port communication module;
the autonomous perception module is used for perceiving the self state of the robot and the information of the environment where the robot is located;
the autonomous decision module is used for completing three kinds of self-adaptation of the robot body height, the body posture and the robot gait planning to unknown complex terrain;
the autonomous control module is used for receiving a control instruction from the Unity3d upper computer and converting the control instruction into a robot action.
Furthermore, in the robot virtual monitoring subsystem, the virtual monitoring module reads the angle information of each joint of the robot sent by the autonomous perception module through a serial port, disassembles the data of each joint according to the sending rule, assigns the data to each joint angle of the digital twin body of the robot, realizes the three-dimensional visual monitoring of the real-time running state of the robot, observes the pose state of the robot in real time in a virtual environment, can also read an environment map around the robot sent by the autonomous perception module, and observes the surrounding environment of the robot in real time in the virtual environment.
Further, in the robot virtual monitoring subsystem, the teleoperation module remotely controls a robot physical entity through a UGUI human-computer interaction panel in Unity3d or remotely controls a robot digital twin body in a robot virtual debugging system to complete uncertain complex tasks.
Furthermore, the autonomous sensing module comprises a three-dimensional laser radar, a monocular camera, a machine body attitude sensor, an angle sensor inside the steering engine and a foot-end touchdown sensor;
the three-dimensional laser radar, the monocular camera and the body attitude sensor are used for constructing an environment map in the virtual planning system of the robot, and meanwhile, the environment information is used for carrying out autonomous positioning, and the robustness and the accuracy of the system can be improved by fusing the three;
the angle sensor in the steering engine is used for feeding back angle information of each joint of the robot;
the foot end touchdown sensor is used for providing a feedback signal when the foot end of the robot touches the ground, so that the foot end stops falling to adapt to unknown complex ground.
Furthermore, the robot body attitude sensor is used for constructing an environment map and autonomously positioning, and is also used for sensing attitude information of the robot body in real time so as to keep the robot attitude stable.
According to another aspect of the invention, an operation method of the underground multifunctional quadruped robot digital twin system for the coal mine is provided, and the method comprises the following steps:
s1: the construction of the digital twin of the multifunctional robot under the coal mine in the Unity3d environment is completed in the robot virtual simulation subsystem, and the concrete process is as follows:
s101: designing the size of the robot, and establishing a robot model in modeling software;
s102: importing the robot model into three-dimensional rendering software, adjusting coordinate axes of all parts, rendering the model, and importing the rendered model into a robot virtual simulation subsystem; the coordinate axis of each part is adjusted to a proper position by adjusting the local coordinate system of each part of the robot model so as to ensure that the model led in the Unity3d does not rotate the local coordinate axis;
s103: establishing a parent-child relationship for a robot model in a robot virtual simulation subsystem, and adding a Transform component for each part to enable each part to have basic translation and rotation transformation capabilities;
s2: verifying and optimizing the structure, the kinematics model and the gait model of the robot in a virtual simulation subsystem of the robot, wherein the specific process comprises the following steps:
s201: establishing a coal mine underground multifunctional quadruped robot kinematics model, compiling a kinematics model C # script, and mounting the script on a robot digital twin;
s202: carrying out gait planning on the multifunctional quadruped robot in the coal mine well, compiling a planning result into a gait planning C # script, and mounting the script on a digital twin body of the robot;
s203: the robot digital twin body runs in the robot virtual simulation subsystem to test and optimize the structure, the kinematics model and the gait of the robot;
s3: leading the optimized digital twin of the underground multifunctional quadruped robot of the coal mine into a robot virtual planning subsystem, and building a robot physical entity according to the digital twin of the robot;
s4: the method comprises the following steps of establishing a digital twin body construction of a coal mine underground multifunctional quadruped robot under a Webots environment in a robot virtual debugging subsystem, wherein the specific process comprises the following steps:
s401: importing a robot model into Webots, and establishing main nodes of the robot model in a scene tree;
s402: adding an inertial unit (inertial unit), a position sensor (position sensor), a rotating motor (rotationmotor) and a contact sensor (TouchSensor) to a robot model, and setting appropriate parameters;
s403: establishing a main controller program and a kinematics program of the robot model, and establishing interfaces between the main controller and an inertial unit, a position sensor, a rotating motor and a contact sensor;
s5: respectively importing the coal mine underground offline map into the robot virtual planning subsystem and the robot virtual debugging subsystem;
s6: the coal mine underground multifunctional quadruped robot is debugged in the robot virtual debugging subsystem, the robot path is planned through the robot virtual planning subsystem, the planning result is executed in the robot virtual debugging subsystem, and the robot semi-autonomous control and manual control are carried out through the robot virtual monitoring subsystem, and the specific process is as follows:
s601: synchronizing the initial poses of the robot digital twin body in the robot virtual planning subsystem and the robot virtual debugging subsystem;
s602: establishing a coal mine underground multifunctional quadruped robot path planning model based on an A-x algorithm, compiling a path planning C # script, and mounting the path planning C # script on robot digital twin bodies in a robot virtual planning subsystem;
s603: planning a robot path in the robot virtual planning subsystem, synchronizing a planning result to the robot virtual debugging subsystem, and finishing walking according to the path planning result;
s604: the robot virtual monitoring subsystem is used for carrying out semi-autonomous control and manual control on the digital twin in the robot virtual debugging subsystem, so that the robot digital twin completes designated operation;
s7: comparing and analyzing a planning result of the robot virtual planning subsystem with a result executed by a robot digital twin in the robot virtual debugging subsystem, and optimizing the robot virtual planning subsystem until the robot virtual planning subsystem can accurately plan a path for the robot;
s8: the robot physical entity is arranged under a coal mine, and the underground multifunctional quadruped robot for the coal mine is capable of self-adaptive walking and complex operation through coordinated operation of the robot physical entity, the robot virtual planning subsystem and the robot virtual monitoring subsystem, and the specific process is as follows:
s801: an autonomous sensing module, an autonomous decision-making module and an autonomous control module are installed for a robot physical entity, a coal mine underground multifunctional quadruped robot software system is established, and the system is placed under an actual coal mine;
s802: the robot physical entity runs under a coal mine, an environment map around the robot is built in real time through the autonomous perception module, and the map is synchronized to the robot virtual planning subsystem;
s803: the method comprises the following steps of planning a robot path in a robot virtual planning subsystem, synchronizing a planning result to an autonomous control module of a robot physical entity, finishing walking according to the path planning result, and autonomously finishing repeated fixed tasks such as routing inspection and the like;
s804: semi-autonomous control and manual control are carried out on a robot physical entity through a teleoperation module in a robot virtual monitoring subsystem, so that the robot physical entity can complete complex tasks with uncertainty, and the real-time pose state and the surrounding environment of the robot are monitored in a virtual monitoring module;
s805: and verifying the accuracy and reliability of the kinematics model, the gait model and the path planning model in the virtual simulation system and the virtual planning system according to the real-time monitoring data of the virtual monitoring system, and performing iterative optimization and dynamic correction on the models.
Further, in step S201, the coal mine underground multifunctional quadruped robot kinematics model includes a robot forward kinematics model and a robot inverse kinematics model;
the positive kinematics model of the robot is used for calculating the position of the foot end according to the posture of the robot body, the length of a connecting rod and the rotation angle of each joint;
the robot inverse kinematics model is used for calculating the rotation angle of each joint according to the body posture, the length of a connecting rod and the position of a foot end of the robot;
further, in step S201, the kinematics model C # script includes a forward solution calculation script, an inverse solution calculation script, and an interpolation calculation script;
the positive kinematics model of the robot is programmed in the positive solution calculation script, and can be used for calculating the motion space of the foot end of the robot and judging whether the structure of the robot is reasonable or not;
the inverse solution calculation script is programmed with the inverse kinematics model of the robot, can calculate the angles of all joints and input the interpolation calculation script, and is the basis for realizing the control of the robot;
and calling a Mathf Lerp () function in an Update () function by the interpolation calculation script, acquiring a target joint corner of the robot from the inverse solution calculation script by each frame, and transferring the robot from the existing corner to the target corner.
Further, in the step S202, a static gait planning method is adopted for gait planning of the underground coal mine multifunctional quadruped robot, and the static gait planning method includes robot stepping sequence planning and robot gravity center trajectory planning;
the gait planning C # script is input into the movement speed, the attitude angle and the body height expected by the robot, and is output into the expected foot end coordinate.
Further, in step S5, the coal mine underground offline map specifically refers to a coal mine underground complete map which is pre-constructed based on geological exploration and three-dimensional mapping, and is used for testing the path planning function of the robot virtual planning subsystem.
Compared with the prior art, the digital twinning system and the operation method of the multifunctional quadruped robot in the coal mine have the following beneficial effects:
(1) The characteristics and advantages of the digital twinning technology are fully exerted, a perfect digital twinning system is constructed, the virtual space of the robot is refined into a virtual simulation system, a virtual planning system, a virtual debugging system and a virtual monitoring system of the robot, all the systems are divided into definite and highly integrated, and cost reduction and efficiency improvement can be realized for multiple links in the full life cycle of the multifunctional quadruped robot in the coal mine;
(2) The robot virtual simulation subsystem provides a platform for verifying and optimizing the structure, the kinematics model and the gait model of the multifunctional quadruped robot in the coal mine, can fully utilize the digital twin body of the robot to adjust and optimize the robot before the physical entity of the robot is put into production and manufacture, can enable the robot to reach an ideal level without repeated processing and manufacture, and shortens the development and iteration period of the robot;
(3) The path planning of the robot is completed in a Unity3 d-based robot virtual planning system, so that a large-scene dense three-dimensional point cloud map acquired by an autonomous perception module is prevented from being constructed by an autonomous decision module of a robot physical entity, and the large-scene dense three-dimensional point cloud map is processed by utilizing stronger computing capacity of a Unity3d upper computer, thereby reducing the computing pressure of the robot physical entity and avoiding the stagnation in the path planning process;
(4) The robot virtual planning subsystem is optimized by using the robot virtual debugging system, the process that a robot physical entity establishes an underground coal mine map in the robot virtual planning system based on the underground coal mine actual environment is effectively simulated by respectively leading in the underground coal mine off-line map in the robot virtual planning system and the virtual debugging system, the characteristic of high simulation of a Webots platform is effectively utilized, the robot virtual planning subsystem is optimized by comparing and analyzing a path planning result and a result executed by a robot digital twin organism, and the path is accurately planned;
(5) The robot virtual space is optimized before the robot physical entity goes into the well, the underground self-adaptive walking and complex operation of the underground multifunctional four-legged robot of the coal mine can be realized finally through the coordinated operation of the robot physical entity, the robot virtual planning subsystem and the robot virtual monitoring subsystem, and the verification and iterative optimization of the virtual simulation system and a physical model in the virtual planning system can be carried out based on the real-time monitoring data of the virtual monitoring system.
(6) The invention provides a relatively perfect solution for the design, manufacture, debugging and operation of the multifunctional quadruped robot in the coal mine, improves the application depth of the digital twin technology in the field of coal mine robots, and can also provide reference for digital twin of robots in other fields.
Drawings
The accompanying drawings are included to provide a further description of the invention and are incorporated in and constitute a part of this application, with the exemplary embodiments and description of the invention being given to illustrate and not limit the invention.
FIG. 1 is a digital twin system architecture diagram of a multifunctional quadruped robot in a coal mine;
FIG. 2 is a schematic flow chart of an operation method of a digital twin system of a multifunctional quadruped robot in a coal mine well, which is provided by the invention;
FIG. 3 is a diagram of a parent-child relationship of robot models in a robot virtual simulation subsystem;
FIG. 4 is a flowchart of forward solution computation script in a robot virtual simulation subsystem;
FIG. 5 is a flow chart of inverse solution computation scripts in the virtual simulation subsystem of the robot;
FIG. 6 is a structure diagram of a software system program file of the multifunctional quadruped robot under a coal mine well.
Detailed Description
An exemplary embodiment of the present invention provides a digital twin system of a quadruped robot in a coal mine, referring to fig. 1, the system includes a robot virtual space and a robot physical entity.
Robot virtual space
The robot virtual space comprises a robot virtual simulation subsystem, a robot virtual planning subsystem, a robot virtual debugging subsystem and a robot virtual monitoring subsystem.
The robot virtual simulation subsystem is developed by Unity3d and is used as a test verification platform for early robot body design and kinematics modeling. The robot can be designed in structure, gait, and motion sequence.
The robot virtual planning subsystem is developed by Unity3d and is used for completing path planning of the coal mine underground multifunctional quadruped robot under the virtual environment.
The robot virtual debugging subsystem is developed by Webots and is used for testing and optimizing the robot virtual planning subsystem.
The robot virtual debugging subsystem comprises an autonomous control mode, a semi-autonomous control mode and a manual control mode.
The autonomous control mode is used for fixed tasks with repeatability, such as robot routing inspection and the like, an operator sends a task instruction, the robot works autonomously, and manual interference is not needed in the process.
The semi-autonomous control mode is used for a specific area operation task, the task has uncertainty due to different maintenance operations, the robot is required to firstly reach an operation area, the instruction is sent by the system, and then an operator manually selects a specific task to complete the operation.
The manual control mode is used for uncertain tasks, an operator manually controls the robot according to the environment and the state of the robot, manually sets the posture and the foot end point of the robot, and sends a control instruction to the robot physical entity to complete control after the robot is confirmed to be correct.
The robot virtual monitoring subsystem is developed by Unity3d and comprises a robot virtual monitoring module and a teleoperation module.
The virtual monitoring module can read the angle information of each joint of the robot sent by the autonomous sensing module through a serial port, disassemble each joint data according to the sending rule, assign the data to each joint angle of the digital twin body of the robot, realize the three-dimensional visual monitoring of the real-time running state of the robot, observe the pose state of the robot in real time in a virtual environment, also read an environment map around the robot sent by the autonomous sensing module, and observe the surrounding environment of the robot in real time in the virtual environment.
The teleoperation module can remotely control the uncertain complex tasks of the robot physical entity or the robot digital twin in the robot virtual debugging system through the UGUI human-computer interaction panel in the Unity3 d.
Robot physical entity
The robot physical entity adopts a four-footed structure and comprises an autonomous perception module, an autonomous decision-making module and an autonomous control module, and the robot has multiple functions of routing inspection, picking, carrying, drilling, clicking and the like.
The robot physical entity and the robot virtual debugging system are communicated with the robot virtual monitoring system Unity3d upper computer through a wireless serial port communication module. Preferably, an HC-12 multi-channel embedded wireless data transmission module is adopted as the wireless serial port communication module.
The autonomous perception module is used for perceiving the state of the robot and the environment information of the robot.
The autonomous sensing module comprises a three-dimensional laser radar, a monocular camera, a machine body attitude sensor, an angle sensor inside the steering engine and a foot end touchdown sensor.
The LVI-SAM is adopted as an inertia/vision/laser radar combined SLAM algorithm which is provided with two modules of an inertia/vision milemeter and an inertia/laser radar milemeter and can construct a dense three-dimensional map with color information.
The three-dimensional laser radar, the monocular camera and the body attitude sensor are used for constructing an environment map in the virtual planning system of the robot, and meanwhile, the environment information is used for carrying out autonomous positioning, and the robustness and the accuracy of the system can be improved through the fusion of the three. The body attitude sensor is used for constructing an environment map and autonomously positioning, and is also used for sensing the attitude information of the robot body in real time so as to keep the stability of the robot attitude.
Relatively specifically, the three-dimensional laser radar is of a Velodyne Puck Hi-Res type; the monocular camera model is FLIR BFS-U3-04S2M-CS; robot attitude sensor adopts the MPU6050 chip, and this module can detect triaxial acceleration, triaxial angular velocity and module temperature simultaneously, and the data precision satisfies motion control's demand.
The LVI-SAM is used as an inertia/vision/laser radar combined SLAM algorithm which is provided with two modules of an inertia/vision odometer and an inertia/laser radar odometer and can construct a dense three-dimensional map with color information;
and the angle sensor in the steering engine is used for feeding back angle information of each joint of the robot.
The foot end touchdown sensor is used for providing a feedback signal when the foot end of the robot touches the ground, so that the foot end stops falling to adapt to unknown complex ground. Relatively specifically, an RP-C resistive membrane pressure sensor is employed as the foot-end touchdown sensor.
The autonomous decision module is used for completing three kinds of self-adaptation of robot body height self-adaptation, body posture self-adaptation and robot gait planning to unknown complex terrains.
The autonomous control module is used for receiving a control instruction from a Unity3d upper computer and converting the control instruction into a robot action. As a preferred scheme, the autonomous control module adopts a raspberry pi 4B as a control host.
In the system, the robot virtual simulation system and the robot virtual debugging system are respectively used for testing and optimizing a robot physical entity and a robot virtual planning system, and finally the parallel operation and real-time interaction of the robot virtual monitoring system, the robot virtual planning system and the robot physical entity are achieved;
the real-time monitoring data of the virtual monitoring system can verify the accuracy and reliability of the mechanism model in the virtual simulation system and the virtual planning system, and carry out iterative optimization and dynamic correction.
Another exemplary embodiment of the present invention provides an operation method of the digital twinning system of the quadruped robot in a coal mine, which is described above with reference to fig. 2 and includes the following steps.
S1: the construction of the digital twin of the multifunctional quadruped robot under the coal mine in the Unity3d environment is completed in a robot virtual simulation subsystem, and the concrete process is as follows:
s101: designing the size of the robot, establishing a robot model in modeling software, and storing the robot model in a specific format;
wherein, the modeling software is Solidworks, and the specific format is STEP.
S102: importing the model into three-dimensional rendering software, adjusting coordinate axes of all components, rendering the model, storing the model in a specific format, and importing the model into a Unity3d robot virtual simulation subsystem;
adjusting the coordinate axes of all parts specifically refers to adjusting the local coordinate systems of all parts of the robot model to proper positions so as to ensure that the model led into the Unity3d does not rotate the local coordinate axes;
the three-dimensional rendering software selects 3ds Max, and the specific format is FBX.
S103: a parent-child relationship shown in the attached figure 3 is established for a robot model in a Unity3d robot virtual simulation subsystem, and a Transform component is added to each part, so that each part has basic translation and rotation transformation capabilities. As shown in fig. 3, the body of the robot includes hip, thigh, calf and foot points numbered 1-4.
S2: the method is characterized in that the structure, the kinematics model and the gait model of the multifunctional quadruped robot in the underground coal mine are verified and optimized in a robot virtual simulation subsystem, and the specific process is as follows:
s201: establishing a coal mine underground multifunctional quadruped robot kinematics model, compiling a kinematics model C # script, and mounting the kinematics model C # script on a robot digital twin;
the coal mine underground multifunctional quadruped robot kinematics model comprises a robot forward kinematics model and a robot inverse kinematics model;
the positive kinematics model of the robot specifically means that the position of the foot end is calculated according to the posture of the robot body, the length of a connecting rod and the rotation angle of each joint;
the robot inverse kinematics model specifically calculates the rotation angle of each joint according to the body posture, the length of a connecting rod and the position of a foot end of the robot;
the kinematics model C # script comprises a forward solution calculation script, an inverse solution calculation script and an interpolation calculation script;
the positive solution calculation script is programmed with a robot positive kinematics model which can be used for resolving a robot foot end motion space and judging whether the robot structure is reasonable or not, and a flow chart is shown in the attached figure 4;
wherein, the inverse solution calculation script is programmed with the inverse kinematics model of the robot, can calculate the angle of each joint and input the interpolation calculation script, and is the basis for realizing the control of the robot, and the flow chart is shown in figure 5;
and the interpolation calculation script calls a Mathf Lerp () function in an Update () function, and each frame acquires a target joint corner of the robot from the inverse solution calculation script and enables the robot to be transferred to the target corner from the existing corner.
S202: carrying out gait planning on the multifunctional quadruped robot in the coal mine well, compiling a planning result into a gait planning C # script, and mounting the script on a digital twin body of the robot;
the gait planning of the underground multifunctional quadruped robot for the coal mine adopts a static gait planning method, and comprises robot stepping sequence planning and robot gravity center track planning;
and the gait planning C # script is input into the movement speed, attitude angle and body height expected by the robot and output into the expected foot end coordinates.
S203: the robot digital twin body runs in the robot virtual simulation subsystem to test and optimize the structure, the kinematics model and the gait of the robot.
S3: and leading the optimized digital twin of the underground multifunctional quadruped robot of the coal mine into a robot virtual planning subsystem, and building a robot physical entity according to the digital twin of the robot.
S4: the method comprises the following steps of establishing a digital twin body construction of a coal mine underground multifunctional quadruped robot under a Webots environment in a robot virtual debugging subsystem, wherein the specific process comprises the following steps:
s401: importing the robot model into Webots, and establishing main nodes of the robot model in a scene tree;
s402: adding an inertial unit (inertial Unit), a position sensor (Positionsensor), a rotating motor (rotationMotor) and a contact sensor (TouchSensor) for a robot model, and setting appropriate parameters;
s403: establishing a robot model main controller program and a kinematics program, and establishing interfaces between the main controller and an inertia unit, a position sensor, a rotating motor and a contact sensor;
the programming language of the main controller program and the kinematics program is Python.
S5: respectively importing the coal mine underground offline map into the robot virtual planning subsystem and the robot virtual debugging subsystem;
the coal mine underground off-line map specifically refers to a coal mine underground complete map which is constructed in advance based on geological exploration and three-dimensional mapping and is used for testing the path planning function of the robot virtual planning subsystem.
S6: the multifunctional quadruped robot for the underground coal mine is debugged in the robot virtual debugging subsystem, the robot path is planned through the robot virtual planning subsystem, the planning result is executed in the robot virtual debugging subsystem, the robot semi-autonomous control and manual control are carried out through the robot virtual monitoring subsystem, and the specific process is as follows:
s601: synchronizing the initial poses of the robot digital twin body in the robot virtual planning subsystem and the robot virtual debugging subsystem;
s602: establishing a coal mine underground multifunctional quadruped robot path planning model based on an A-x algorithm, compiling a path planning C # script, and mounting the path planning C # script on a robot digital twin in a robot virtual planning subsystem;
s603: planning a robot path in the robot virtual planning subsystem, synchronizing a planning result to the robot virtual debugging subsystem, and finishing walking according to the path planning result;
s604: and the robot virtual monitoring subsystem is used for carrying out semi-autonomous control and manual control on the digital twin body in the robot virtual debugging subsystem, so that the robot digital twin body finishes specified operation.
S7: and comparing and analyzing the planning result of the virtual robot planning subsystem with the result executed by the digital robot twin body in the virtual robot debugging subsystem, and optimizing the virtual robot planning subsystem until the virtual robot planning subsystem can accurately plan a path for the robot.
S8: the robot physical entity is arranged under a coal mine, and the underground multifunctional quadruped robot for the coal mine is capable of self-adaptive walking and complex operation through coordinated operation of the robot physical entity, the robot virtual planning subsystem and the robot virtual monitoring subsystem, and the specific process is as follows:
s801: an autonomous sensing module, an autonomous decision-making module and an autonomous control module are installed for a robot physical entity, a coal mine underground multifunctional quadruped robot software system is established and placed under an actual coal mine, and the program file structure of the robot software system is shown in an attached figure 6;
s802: the robot physical entity runs under a coal mine, an environment map around the robot is built in real time through the autonomous perception module, and the map is synchronized to the robot virtual planning subsystem;
s803: the method comprises the following steps of planning a robot path in a robot virtual planning subsystem, synchronizing a planning result to an autonomous control module of a robot physical entity, finishing walking according to the path planning result, and autonomously finishing repeated fixed tasks such as routing inspection and the like;
s804: semi-autonomous control and manual control are carried out on a robot physical entity through a teleoperation module in a robot virtual monitoring subsystem, so that the robot physical entity completes complex tasks with uncertainty, and the real-time pose state and the surrounding environment of the robot are monitored in a virtual monitoring module;
s805: and verifying the accuracy and reliability of the kinematics model, the gait model and the path planning model in the virtual simulation system and the virtual planning system according to the real-time monitoring data of the virtual monitoring system, and performing iterative optimization and dynamic correction on the models.
Claims (10)
1. The utility model provides a colliery is multi-functional four-footed robot digit twin system in pit, includes robot virtual space and robot physical entity, its characterized in that:
the robot virtual space comprises a robot virtual simulation subsystem, a robot virtual planning subsystem, a robot virtual debugging subsystem and a robot virtual monitoring subsystem;
the robot virtual simulation subsystem is developed by Unity3d and is used as a test verification platform for early robot body design and kinematics modeling;
the robot virtual planning subsystem is developed by Unity3d and is used for completing path planning of the coal mine underground multifunctional quadruped robot under a virtual environment;
the robot virtual debugging subsystem is developed by Webots and is used for testing and optimizing the robot virtual planning subsystem;
the robot virtual monitoring subsystem is developed by a Unity3d and comprises a robot virtual monitoring module and a teleoperation module;
the robot physical entity adopts a four-footed structure and comprises an autonomous perception module, an autonomous decision module and an autonomous control module; the robot physical entity and the robot virtual debugging system are communicated with a Unity3d upper computer of the robot virtual monitoring system through a wireless serial port communication module;
the autonomous perception module is used for perceiving the self state of the robot and the information of the environment where the robot is located;
the autonomous decision module is used for completing three kinds of self-adaptation of the robot body height, the body posture and the robot gait planning to unknown complex terrain;
the autonomous control module is used for receiving a control instruction from the Unity3d upper computer and converting the control instruction into a robot action.
2. The coal mine underground multifunctional quadruped robot digital twinning system as claimed in claim 1, which is characterized in that: in the robot virtual monitoring subsystem, a virtual monitoring module reads angle information of each joint of a robot sent by an autonomous sensing module through a serial port, disassembles data of each joint according to a sending rule, assigns the data to each joint angle of a digital twin body of the robot, realizes three-dimensional visual monitoring of a real-time running state of the robot, observes the pose state of the robot in real time in a virtual environment, can also read an environment map around the robot sent by the autonomous sensing module, and observes the surrounding environment of the robot in real time in the virtual environment.
3. The coal mine underground multifunctional quadruped robot digital twinning system as claimed in claim 2, which is characterized in that: in the virtual robot monitoring subsystem, the teleoperation module remotely controls a physical entity of the robot or a digital robot twin in the virtual robot debugging system to complete uncertain complex tasks through a UGUI (user generated user interface) man-machine interaction panel in the Unity3 d.
4. The coal mine underground multifunctional quadruped robot digital twinning system as claimed in claim 1 or 3, which is characterized in that: the autonomous sensing module comprises a three-dimensional laser radar, a monocular camera, a machine body attitude sensor, an internal steering engine angle sensor and a foot-end touchdown sensor;
the three-dimensional laser radar, the monocular camera and the body attitude sensor are used for constructing an environment map in the virtual planning system of the robot, and meanwhile, the environment information is used for carrying out autonomous positioning, and the robustness and the accuracy of the system can be improved by fusing the three;
the angle sensor inside the steering engine is used for feeding back angle information of each joint of the robot;
the foot end touchdown sensor is used for providing a feedback signal when the foot end of the robot touches down, so that the foot end stops falling to adapt to unknown complex ground.
5. The coal mine underground multifunctional quadruped robot digital twinning system as claimed in claim 4, wherein: the body attitude sensor is used for constructing an environment map and autonomously positioning, and is also used for sensing the attitude information of the robot body in real time so as to keep the stability of the robot attitude.
6. The operation method of the coal mine underground multifunctional quadruped robot digital twin system is characterized by comprising the following steps:
s1: the construction of the digital twin of the multifunctional robot under the coal mine in the Unity3d environment is completed in the robot virtual simulation subsystem, and the concrete process is as follows:
s101: designing the size of the robot, and establishing a robot model in modeling software;
s102: importing the robot model into three-dimensional rendering software, adjusting coordinate axes of all parts, rendering the model, and importing the rendered model into a robot virtual simulation subsystem; the coordinate axis of each part is adjusted to a proper position by adjusting the local coordinate system of each part of the robot model so as to ensure that the model led in the Unity3d does not rotate the local coordinate axis;
s103: establishing a parent-child relationship for a robot model in a robot virtual simulation subsystem, and adding a Transform component for each part to enable each part to have basic translation and rotation transformation capabilities;
s2: verifying and optimizing the structure, the kinematics model and the gait model of the robot in a virtual simulation subsystem of the robot, wherein the specific process comprises the following steps:
s201: establishing a coal mine underground multifunctional quadruped robot kinematics model, compiling a kinematics model C # script, and mounting the script on a robot digital twin;
s202: carrying out gait planning on the multifunctional quadruped robot in the coal mine well, compiling a planning result into a gait planning C # script, and mounting the script on a digital twin body of the robot;
s203: the robot digital twin body runs in the robot virtual simulation subsystem to test and optimize the structure, the kinematic model and the gait of the robot;
s3: leading the optimized digital twin of the underground multifunctional quadruped robot of the coal mine into a robot virtual planning subsystem, and building a robot physical entity according to the digital twin of the robot;
s4: the method comprises the following steps of establishing a digital twin body construction of a coal mine underground multifunctional quadruped robot under a Webots environment in a robot virtual debugging subsystem, wherein the specific process comprises the following steps:
s401: importing a robot model into Webots, and establishing main nodes of the robot model in a scene tree;
s402: adding an inertial unit (inertial Unit), a position sensor (Positionsensor), a rotating motor (rotationMotor) and a contact sensor (TouchSensor) for a robot model, and setting appropriate parameters;
s403: establishing a main controller program and a kinematics program of the robot model, and establishing interfaces between the main controller and an inertial unit, a position sensor, a rotating motor and a contact sensor;
s5: respectively importing the coal mine underground offline map into the robot virtual planning subsystem and the robot virtual debugging subsystem;
s6: the multifunctional quadruped robot for the underground coal mine is debugged in the robot virtual debugging subsystem, the robot path is planned through the robot virtual planning subsystem, the planning result is executed in the robot virtual debugging subsystem, the robot semi-autonomous control and manual control are carried out through the robot virtual monitoring subsystem, and the specific process is as follows:
s601: the initial pose of a robot digital twin body in the synchronous robot virtual planning subsystem and the robot virtual debugging subsystem;
s602: establishing a coal mine underground multifunctional quadruped robot path planning model based on an A-x algorithm, compiling a path planning C # script, and mounting the path planning C # script on a robot digital twin in a robot virtual planning subsystem;
s603: performing robot path planning in the robot virtual planning subsystem, synchronizing a planning result to the robot virtual debugging subsystem, and completing walking according to the path planning result;
s604: the robot virtual monitoring subsystem is used for carrying out semi-autonomous control and manual control on the digital twin in the robot virtual debugging subsystem, so that the robot digital twin completes designated operation;
s7: comparing and analyzing a planning result of the robot virtual planning subsystem with a result executed by a robot digital twin in the robot virtual debugging subsystem, and optimizing the robot virtual planning subsystem until the robot virtual planning subsystem can accurately plan a path for the robot;
s8: the robot physical entity is arranged underground the coal mine, and the underground self-adaptive walking and complex operation of the multifunctional quadruped robot for the coal mine are realized through the coordinated operation of the robot physical entity, the robot virtual planning subsystem and the robot virtual monitoring subsystem, and the specific process comprises the following steps:
s801: an autonomous sensing module, an autonomous decision-making module and an autonomous control module are installed for a robot physical entity, a coal mine underground multifunctional quadruped robot software system is established, and the system is placed under an actual coal mine;
s802: the robot physical entity runs under a coal mine, an environment map around the robot is built in real time through the autonomous perception module, and the map is synchronized to the robot virtual planning subsystem;
s803: the method comprises the following steps of planning a robot path in a robot virtual planning subsystem, synchronizing a planning result to an autonomous control module of a robot physical entity, finishing walking according to the path planning result, and autonomously finishing repeated fixed tasks such as routing inspection and the like;
s804: semi-autonomous control and manual control are carried out on a robot physical entity through a teleoperation module in a robot virtual monitoring subsystem, so that the robot physical entity completes complex tasks with uncertainty, and the real-time pose state and the surrounding environment of the robot are monitored in a virtual monitoring module;
s805: and verifying the accuracy and reliability of the kinematics model, the gait model and the path planning model in the virtual simulation system and the virtual planning system according to the real-time monitoring data of the virtual monitoring system, and performing iterative optimization and dynamic correction on the models.
7. The coal mine underground quadruped robot digital twinning system as claimed in claim 6, wherein:
in step S201, the coal mine underground multifunctional quadruped robot kinematics model comprises a robot positive kinematics model and a robot inverse kinematics model;
the positive kinematics model of the robot is used for calculating the position of the foot end according to the posture of the robot body, the length of a connecting rod and the rotation angle of each joint;
the robot inverse kinematics model is used for calculating the rotation angle of each joint according to the posture of the robot body, the length of a connecting rod and the position of a foot end.
8. The coal mine underground quadruped robot digital twinning system of claim 7, which is characterized in that:
in step S201, the kinematics model C # script includes a forward solution calculation script, an inverse solution calculation script, and an interpolation calculation script;
the positive kinematics model of the robot is programmed in the positive solution calculation script, and can be used for calculating the motion space of the foot end of the robot and judging whether the structure of the robot is reasonable or not;
the inverse solution calculation script is programmed with the inverse kinematics model of the robot, can calculate the angles of all joints and input the interpolation calculation script, and is the basis for realizing the control of the robot;
and the interpolation calculation script calls a Mathf Lerp () function in an Update () function, and each frame acquires a target joint corner of the robot from the inverse solution calculation script and enables the robot to transfer from the existing corner to the target corner.
9. The coal mine underground quadruped robot digital twinning system as claimed in claim 6 or 8, wherein:
in the step S202, a static gait planning method is adopted for gait planning of the underground multifunctional quadruped robot of the coal mine, and the static gait planning method comprises robot stepping sequence planning and robot gravity center track planning;
the gait planning C # script is input into the movement speed, the attitude angle and the body height expected by the robot, and is output into the expected foot end coordinate.
10. The coal mine underground quadruped robot digital twinning system of claim 9, which is characterized in that: in the step S5, the underground coal mine off-line map specifically refers to an underground coal mine complete map which is constructed in advance based on geological exploration and three-dimensional mapping and is used for testing the path planning function of the robot virtual planning subsystem.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211157311.XA CN115329922A (en) | 2022-09-22 | 2022-09-22 | Coal mine underground multifunctional four-legged robot digital twin system and operation method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211157311.XA CN115329922A (en) | 2022-09-22 | 2022-09-22 | Coal mine underground multifunctional four-legged robot digital twin system and operation method thereof |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115329922A true CN115329922A (en) | 2022-11-11 |
Family
ID=83914689
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211157311.XA Pending CN115329922A (en) | 2022-09-22 | 2022-09-22 | Coal mine underground multifunctional four-legged robot digital twin system and operation method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115329922A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117193288A (en) * | 2023-07-28 | 2023-12-08 | 安元科技股份有限公司 | Industrial safety intelligent inspection robot dog based on AI algorithm and inspection method |
-
2022
- 2022-09-22 CN CN202211157311.XA patent/CN115329922A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117193288A (en) * | 2023-07-28 | 2023-12-08 | 安元科技股份有限公司 | Industrial safety intelligent inspection robot dog based on AI algorithm and inspection method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112392485B (en) | Transparent digital twin self-adaptive mining system and method for fully mechanized coal mining face | |
| WO2021184614A1 (en) | Intelligent decision control method and system for fully-mechanized mining equipment used for working surface under complex condition | |
| CN111208759B (en) | Digital twin intelligent monitoring system for unmanned fully mechanized coal mining face of mine | |
| US11331803B2 (en) | Mixed reality assisted spatial programming of robotic systems | |
| CN111177942A (en) | Digital twin intelligent monitoring system for unmanned fully-mechanized excavation working face of mine | |
| CN107901039B (en) | Python-based desktop-level robot offline programming simulation system | |
| CN111633644A (en) | Industrial robot digital twin system combined with intelligent vision and operation method thereof | |
| CN103728972B (en) | Test platform and method for synchronous control over multiple mechanical arms | |
| CN108058758B (en) | Robot biped walking simulation evaluation system and evaluation method based on six-degree-of-freedom leg | |
| CN106444861A (en) | Space robot teleoperation system based on three-dimensional gestures | |
| CN112132359A (en) | Digital twin satellite pulsating assembly workshop design method and system | |
| CN109268010A (en) | A kind of long-range inspection interference method of virtual reality Mechanized | |
| CN113128109B (en) | Test and evaluation method for intelligent fully-mechanized mining robot production system | |
| CN101434066B (en) | Method and platform for predicating teleoperation of robot | |
| CN110321000B (en) | Virtual simulation system for complex tasks of intelligent system | |
| CN111152226B (en) | Robot working track planning method and system | |
| CN115329922A (en) | Coal mine underground multifunctional four-legged robot digital twin system and operation method thereof | |
| CN110154024A (en) | A kind of assembly control method based on shot and long term Memory Neural Networks incremental model | |
| Xie et al. | A virtual test and evaluation method for fully mechanized mining production system with different smart levels | |
| CN111273562A (en) | Method for realizing optimization of underground robot path based on membrane calculation | |
| Ramer et al. | A robot motion planner for 6-DOF industrial robots based on the cell decomposition of the workspace | |
| Xie et al. | A novel DT-based intelligent experiment method for complex industrial products | |
| CN117094056A (en) | Urban underground loop digital twin intelligent construction method and system | |
| Du et al. | Industrial robot digital twin system motion simulation and collision detection | |
| CN113051756B (en) | Virtual-real fusion coal mining machine memory cutting test system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |