CN112130555A - Self-propelled robot and system based on laser navigation radar and computer vision perception fusion - Google Patents

Abstract

The invention provides a self-propelled robot and system based on the fusion of laser navigation radar and computer vision perception. It is connected with the rotating pair of the panoramic camera module, and a vector angle sensor is arranged between the telescopic bracket and the panoramic camera module. The height of the group from the ground is greater than the height of the lidar from the ground. The telescopic bracket is used to automatically adjust the height of the panoramic camera module from the ground and provide a fixed reference for the vector angle sensor. The lidar is used to obtain the relationship between the robot body and its surrounding obstacles. The panoramic camera module is used to obtain the first color image of the obstacles around the robot body.

Description

Self-propelled robot based on the fusion of lidar navigation radar and computer vision perception and system

technical field

The invention relates to the field of robot technology, in particular to a self-propelled robot and a system based on fusion of laser navigation radar and computer vision perception.

Background technique

At present, self-propelled robots generally use magnetic stripe or lidar for navigation. Magnetic stripe navigation is not only expensive, but also limited by the environment and has many constraints. However, because lidar cannot collect information about things in the environment in more dimensions, it can only be used to measure the distance of some specific targets. Therefore, the self-propelled robot using this lidar can only navigate in some wide and simple terrain. Regional activities and application scenarios are greatly restricted.

For example, as shown in FIG. 1, a circular fence surrounded by a plurality of fixed shields 2 and a movable shield 3, wherein the color of the movable shield 3 is different from that of the fixed shield 2, or the Write the word "exit" etc. on the movable shield 3. At this time, if the lidar navigation self-propelled robot 1 is placed in it, the radar data obtained by it will be a circular fence, and it cannot obtain the data that the movable blocker 3 is the exit, which makes it unable to determine the correct action route. Get out of the fence.

For another example, as shown in FIG. 2 , a plurality of fixed shelters 2 are placed in different ways on two paths (path 1 and path 2) divided by the intermediate isolation belt 4 to form different passable paths. The manner of placing the fixed shelter 2 at the starting intersection of the path 1 and the path 2 is the same. At this time, if the LiDAR-navigated self-propelled robot 1 is placed at the intersection and let it pass through the road, the obstacles and difficulty levels it may encounter in front will be the same, and it will not be able to obtain the optimal path to pass through. , path two is better than path one.

SUMMARY OF THE INVENTION

Based on the above, a self-propelled robot suitable for more complex environments is provided, and a system suitable for the robot is provided.

In addition, a control method of the above self-propelled robot is also provided.

A self-propelled robot based on the fusion of laser navigation radar and computer vision perception, comprising a robot body for walking motion, a laser radar arranged on the robot body, a first telescopic robot vertically fixed on the robot body a bracket, and a first panoramic camera module arranged at the top of the first telescopic bracket; the first telescopic bracket is connected with the first panoramic camera module in a rotating pair; the first telescopic bracket is connected to the first A first vector angle sensor is arranged between the panoramic camera modules; the first vector angle sensor is used to obtain the rotation angle size and rotation direction of the first panoramic camera module relative to the first telescopic bracket, so as to Correct the deviation between the orientation of the object in the picture acquired by the first panoramic camera module and the orientation of the object acquired by the lidar due to the rotation of the first panoramic camera module relative to the first telescopic bracket, so that the two The orientation of the same object obtained by the user is the same; the height of the first panoramic camera module from the ground is greater than the height of the lidar from the ground; the first telescopic bracket is used to automatically adjust the distance of the first panoramic camera module The height of the ground is used to provide a first fixed reference for the first vector angle sensor; the lidar is used to obtain the distance between the robot body and its surrounding obstacles; the first panoramic camera module is used to A first color image of obstacles around the robot body is acquired to assist in judging the physical and/or chemical properties of the obstacles.

In one embodiment, the first panoramic camera module and the first telescopic bracket are connected by a first camera stabilizer; the first camera stabilizer and the first telescopic bracket are connected by a rotating pair connection; the first panoramic camera module includes a first lens module, a first lens support rod and a first gravity block; the first lens support rod is vertically fixed on the camera mount of the first camera stabilizer ; the first lens module is arranged on the top of the first lens support rod; the first gravity block is arranged on the back of the camera mount to lower the center of gravity of the first panoramic camera module and increase the The torque of the camera mount relative to the first camera stabilizer keeps the first lens support rod in a vertical state; the first vector angle sensor is arranged between the first telescopic bracket and the first Between a camera stabilizer to obtain the angle size and rotation direction of the first camera stabilizer when it rotates relative to the first telescopic bracket, so as to correct the objects in the picture obtained by the first panoramic camera module The azimuth and the azimuth of the object obtained by the laser radar are caused by the deviation caused by the rotation of the first panoramic camera module relative to the first telescopic bracket, so that the azimuth of the same object obtained by the two is the same.

In one embodiment, the first lens module includes a first wide-angle lens, a second wide-angle lens and a third wide-angle lens; the shooting angles of the first wide-angle lens, the second wide-angle lens and the third wide-angle lens are each 120 degrees , so as to obtain a 360-degree panoramic camera through stitching and synthesis.

In one of the embodiments, it also includes a second telescopic bracket and a second panoramic camera module disposed at the top of the second telescopic bracket; the second telescopic bracket is connected to the second panoramic camera module by a rotating pair A second vector angle sensor is arranged between the second telescopic bracket and the second panoramic camera module; the second vector angle sensor is used to obtain the relative relationship between the second panoramic camera module and the second The size of the corner and the direction of rotation when the telescopic bracket is rotated, so as to correct the orientation of the object in the picture acquired by the second panoramic camera module and the orientation of the object acquired by the lidar because the second panoramic camera module is relative to the The deviation caused by the rotation of the second telescopic support makes the orientation of the same object obtained by the two the same. The second telescopic bracket is vertically arranged and is fixedly connected to the robot body; the height of the second panoramic camera module from the ground is greater than the height of the lidar from the ground, and less than the height of the first panoramic camera module from the ground height; the second telescopic bracket is used to automatically adjust the height of the second panoramic camera module from the ground, and provide a second fixed reference for the second vector angle sensor; the second fixed reference and The first fixed reference is the same; the horizontal distance between the second panoramic camera module and the first panoramic camera module is greater than 0; the distance between the second telescopic bracket and the first telescopic bracket is The distance is greater than 0; the second panoramic camera module is used to obtain a second color image of obstacles around the robot body, and synthesize a three-dimensional image with the first color image. This method can synthesize up to 6 three-dimensional images, and can see up to 5 datum planes of things.

In one embodiment, the second panoramic camera module and the second telescopic bracket are connected by a second camera stabilizer; the second camera stabilizer and the second telescopic bracket are connected by a rotating pair connection; the second panoramic camera module includes a second lens module, a second lens support rod and a second gravity block; the second lens support rod is vertically fixed on the camera mount of the second camera stabilizer ; the second lens module is arranged on the top of the second lens support rod; the second gravity block is arranged on the back of the camera mount to lower the center of gravity of the second panoramic camera module, and increase the The torque of the camera mount relative to the second camera stabilizer keeps the second lens support rod in a vertical state; the second vector angle sensor is arranged between the second telescopic bracket and the first Between the two camera stabilizers, it is used to obtain the angle size and rotation direction of the second camera stabilizer when it rotates relative to the second telescopic bracket, so as to correct the objects in the picture obtained by the second panoramic camera module The azimuth and the azimuth of the object obtained by the laser radar are caused by the deviation caused by the rotation of the second panoramic camera module relative to the second telescopic bracket, so that the azimuth of the same object obtained by the two is the same.

In one embodiment, the second lens module includes a fourth wide-angle lens, a fifth wide-angle lens, and a sixth wide-angle lens; the shooting angles of the fourth wide-angle lens, the fifth wide-angle lens, and the sixth wide-angle lens are each 120°, In this way, a 360-degree panoramic camera is obtained by stitching and compositing.

In one of the embodiments, it further includes an audio receiver, a mechanical arm disposed on the robot body, and a hammer disposed at the end of the mechanical arm for knocking; the audio receiver is used to receive the hammer knocking The sound of hitting an object.

In one of the embodiments, it further includes an audio receiver, a terminal motor, a robotic arm, a robotic arm mounting seat and a third telescopic bracket provided on the robot body; the robotic arm is connected to the robotic arm mounting seat through the robotic arm mounting seat. the robot body is connected; the mechanical arm is a telescopic straight rod; the mechanical arm is connected with the rotating pair of the mechanical arm mounting seat; the mechanical arm is connected with the rotating pair of the third telescopic bracket, so that the third The extension and shortening of the telescopic bracket can drive the mechanical arm to rotate around the mechanical arm mounting seat. The end of the mechanical arm is provided with an electronically controlled ejection hammer module; the electronically controlled ejection hammer module is connected with the rotating pair at the end of the mechanical arm, and drives the electronically controlled ejection hammer module and the mechanical arm through the end motor The audio receiver is arranged on the electronically controlled ejection hammer module, and the audio receiving end faces the direction in which the electronically controlled ejection hammer module hits the object.

In one embodiment, an audio receiver and an ultrasonic transmitter are also included; the ultrasonic transmitter is used for transmitting ultrasonic waves; the audio receiver is used for receiving ultrasonic waves emitted by the ultrasonic transmitter and reflected back by obstacles, In order to realize the navigation function in weather conditions such as rainy and foggy.

In one of the embodiments, it also includes a first communication module and a second communication module; the first communication module is used to connect to the Internet to realize remote control and data maintenance; the second communication module is used for Real-time communication after pairing with other robots in the same area to realize the function of mutual learning between robots.

In one of the embodiments, it further includes an anemometer module disposed on the robot body; the anemometer module is used to measure the wind speed at the location of the robot body. Since the panoramic camera module in the solution of this application is erected on a retractable bracket, the position is usually high. If the wind speed is too high, the torsion arm will be too large. Therefore, the anemometer module is set. When the measured wind speed is greater than the set value When , the retractable bracket for erecting the panoramic camera module will be shortened to reduce the torsion arm.

In one of the embodiments, it further includes an anemometer module disposed on the robot body; the anemometer module is used to measure the wind direction at the location of the robot body. In this way, the posture of the robot body can be adjusted according to the wind direction, so as to operate with the minimum wind resistance.

In one embodiment, a first laser radar, a second laser radar, a third laser radar and a fourth laser radar are arranged around the robot body in sequence; the first laser radar, the second laser radar, the The third lidar and the fourth lidar are used to obtain distances between the robot and surrounding obstacles, respectively.

The self-propelled robot based on the fusion of laser navigation radar and computer vision perception provided above can obtain color information or text information that cannot be obtained by laser radar by placing a panoramic camera module on a vertically arranged retractable bracket. It can assist in judging the physical and/or chemical properties of obstacles around the robot, improve the self-propelled robot's understanding of the surrounding environment, and thus broaden the applicable scenarios of the self-propelled robot.

According to the above content, the present application also provides a self-propelled robot system based on the fusion of laser navigation radar and computer vision perception.

A self-propelled robot system based on the fusion of laser navigation radar and computer vision perception, comprising a central processing unit, an image data processing module, a laser navigation radar data processing module, a first panoramic camera module, a first vector angle sensor and a laser navigation radar module; the image data processing module and the laser navigation radar data processing module are connected to the central processing unit; the first panoramic camera module is connected to the image data processing module; the laser navigation radar module connected with the laser navigation radar data processing module; the first vector angle sensor is respectively connected with the first panoramic camera module and the central processing unit; the laser navigation radar module is used to obtain the robot body The distance between the obstacle and the surrounding obstacles; the first panoramic camera module is used to obtain the first color image of the obstacles around the robot body to assist in judging the physical and/or chemical properties of the surrounding obstacles; the image data The processing module is used to control the first panoramic camera module and perform first-level data processing on the first color image; the laser navigation radar data processing module is used to perform the first level data processing on the laser navigation radar module. control, and perform first-level processing on the data obtained by the laser navigation radar module; the first vector angle sensor is used to obtain the reference deviation of the first panoramic camera module relative to the laser navigation radar module The central processing unit performs second-level data processing on the data output by the image data processing module and the laser navigation radar data processing module through the data obtained by the first vector angle sensor, and corrects all The rotation of the first panoramic camera module causes a reference deviation relative to the laser navigation radar module, so that the orientation of the object in the first color image is the same as the orientation of the object obtained by the laser navigation radar.

In one embodiment, there are also a second panoramic camera module and a second vector angle sensor; the second panoramic camera module is connected to the image data processing module; the second vector angle sensor is respectively connected to the first Two panoramic camera modules are connected to the central processing unit; the second panoramic camera module is used to obtain a second color image of obstacles around the robot body, and synthesize a three-dimensional image with the first color image; the second The vector angle sensor is used to obtain the angular magnitude and direction of the reference deviation of the second panoramic camera module relative to the laser navigation radar module; the central processor uses the data obtained by the second vector angle sensor to The data output by the image data processing module and the laser navigation radar data processing module is subjected to second-level data processing, and the reference deviation of the second panoramic camera module relative to the laser navigation radar module caused by the rotation is corrected, so that the The orientation of the object in the second color image is the same as the orientation of the object obtained by the laser navigation radar; the central processing unit or the second panoramic camera module is also used to combine the first color image and the first color image. Two color images are combined into a three-dimensional image.

In one embodiment, the first panoramic camera module includes a first wide-angle lens, a second wide-angle lens, and a third wide-angle lens; the second panoramic camera module includes a fourth wide-angle lens, a fifth wide-angle lens, and a sixth wide-angle lens; all The first wide-angle lens and the sixth wide-angle lens are paired and connected to the image data processing module; the second wide-angle lens and the fifth wide-angle lens are paired and connected to the image data processing module; the third wide-angle lens After being paired and connected with the fourth wide-angle lens, it is connected with the image data processing module; the central processing unit is further configured to analyze the first panorama through the data of the first vector angle sensor and the second vector angle sensor The relative rotation angle of the camera module and the second panoramic camera module is corrected.

In one embodiment, the lidar navigation radar module includes a first lidar, a second lidar, a third lidar, and a fourth lidar; the first lidar, the second lidar, the The third laser radar and the fourth laser radar are used to be arranged around the robot in turn to obtain the distance between the robot and the surrounding obstacles; the laser navigation radar data processing module is also used to The data obtained by the second lidar, the third lidar and the fourth lidar are spliced into a complete radar image, and output to the central processor for further processing.

In one of the embodiments, it further includes an electronically controlled ejection hammer module and an audio receiver respectively connected to the central processing unit; the electronically controlled ejection hammer module is used to strike an object to make a sound; the audio receiver The device is used to receive the sound produced by the electronically controlled ejection hammer module striking the object; the central processing unit is also used to perform spectrum analysis on the sound produced by the electronically controlled ejection hammer module striking the object, and output the knocked The kind and basic physical and/or chemical properties of an object.

In one of the embodiments, it further includes an ultrasonic transmitter connected to the central processing unit; the ultrasonic transmitter is used for transmitting ultrasonic waves; the audio receiver is also used for receiving the ultrasonic wave transmitted by the ultrasonic transmitter and passing through obstacles The ultrasonic wave reflected from the object is used to realize the navigation function of the robot in weather conditions such as rain and fog.

In one of the embodiments, the ultrasonic transmitter is also connected with the audio receiver.

In one of the embodiments, it further includes a first communication module and a second communication module connected with the central processing unit; the first communication module is used for connecting with the Internet to realize remote control and data maintenance; The second communication module is used for real-time communication after pairing with other robots in the same region, so as to realize the function of mutual learning between the robots.

In one of the embodiments, it also includes an wind vane module connected to the central processing unit; the wind vane module is used to measure the wind direction at the location where the robot is located.

In one of the embodiments, it also includes an anemometer module connected to the central processing unit; the anemometer module is used to measure the wind speed at the location where the robot is located.

In one embodiment, it further includes a storage module connected to the central processing unit; the storage module stores a control program that can be executed by the central processing unit and attribute information of common objects, and is used to store various functions Information generated when the module runs.

The above-mentioned self-propelled robot system based on the fusion of laser navigation radar and computer vision perception can obtain color information or text information that cannot be obtained by laser radar by setting up a panoramic camera module, so as to assist in judging obstacles around the robot. The physical and/or chemical properties of the self-propelled robot can improve the awareness of the surrounding environment, thereby broadening the applicable scenarios of the self-propelled robot.

In addition, the present application also provides a control method of a self-propelled robot based on the fusion of laser navigation radar and computer vision perception.

A control method of a self-propelled robot based on laser navigation radar and computer vision perception fusion, the self-propelled robot is the self-propelled robot in any of the above-mentioned embodiments, and the method comprises: the laser navigation radar obtains obstacles around the robot object reflecting surface; the central processing unit determines whether the maximum distance between obstacles is less than the minimum passing distance of the robot, and if so, activates the first panoramic camera module.

In one embodiment, the method further includes: testing the maximum distance that the lidar navigation radar can obtain obstacles around the robot; judging whether the maximum distance is less than or equal to a first set value, and if so, starting the ultrasonic wave transmitter and the audio receiver.

The method provided above uses the preliminary judgment of the surrounding obstacle environment through the lidar navigation radar. When the laser radar navigation alone cannot meet the requirements, computer vision perception is activated to gain a deeper understanding of the surrounding environment, so as to find the best action path.

Description of drawings

1 is a schematic diagram of an obstacle fence set up for exposing the shortcomings of the existing robot in the background technology;

Figure 2 is a schematic diagram of a roadblock set in the background technology for exposing the shortcomings of the existing robot;

3 is a schematic structural diagram of a self-propelled robot provided with a panoramic camera module according to an embodiment;

4 is a schematic structural diagram of a self-propelled robot provided with two panoramic camera modules according to an embodiment;

FIG. 5 is a schematic structural diagram of a self-propelled robot system based on the fusion of laser navigation radar and computer vision perception according to an embodiment.

Description of reference numbers:

1. Lidar navigation self-propelled robot; 2. Fixed obstructions; 3. Movable obstructions; 4. Intermediate isolation belt;

10. Robot body; 21. First telescopic bracket; 22. First camera stabilizer; 23. Second telescopic bracket; 24. Second camera stabilizer; 31. Robot arm; 32. Robot arm mount; 33. Section Three telescopic brackets; 34. End motor;

110. The first lidar; 120. The second lidar; 130. The third lidar; 140. The fourth lidar; 210. The first panoramic camera module; 211. The first wide-angle lens; 212. The second wide-angle lens; 213 .The third wide-angle lens; 214. The first lens support rod; 215. The first gravity block; 220. The second panoramic camera module; 221. The fourth wide-angle lens; 222. The fifth wide-angle lens; 223. The sixth wide-angle lens; Two-lens support rod; 225. Second gravity block; 300. Electronically controlled ejection hammer module; 410. Audio receiver; 420. Ultrasonic transmitter; 510. First vector angle sensor; 520. Second vector angle sensor; 610 .First communication module; 620. Second communication module; 710. Anemometer module; 720. Anemometer module; 810. Central processing unit; 820. Image data processing module; 830. Laser navigation radar data processing module ; 840. Storage module.

Detailed ways

1-5, discussed below, and the various embodiments used to describe the principles or methods of the present disclosure in this patent document are by way of illustration only and should not be construed in any way to limit the scope of the disclosure. Those skilled in the art will understand that the principles or methods of the present disclosure may be implemented in any suitably arranged robot. Preferred embodiments of the present disclosure will be described hereinafter with reference to the accompanying drawings. In the following description, detailed descriptions of well-known functions or configurations are omitted so as not to obscure the subject matter of the present disclosure with unnecessary detail. Also, the terms used herein will be defined according to the functions of the present invention. Therefore, the terminology may vary according to the user's or operator's intention or usage. Therefore, the terms used in this document must be understood based on the descriptions made in this document.

A self-propelled robot based on the fusion of laser navigation radar and computer vision perception, as shown in FIG. 3, includes a robot body 10 for walking motion, a laser radar arranged on the robot body 10, and a robot body 10 vertically fixed on the robot body 10. The first telescopic bracket 21 on the upper part, and the first panoramic camera module 210 arranged on the top of the first telescopic bracket 21 . The first telescopic bracket 21 is rotatably connected with the first panoramic camera module 210 . A first vector angle sensor 510 is disposed between the first telescopic bracket 21 and the first panoramic camera module 210 . The first vector angle sensor 510 is used to obtain the angle size and rotation direction of the first panoramic camera module 210 when it rotates relative to the first telescopic bracket 21, so as to correct the orientation of the object and the laser beam in the picture obtained by the first panoramic camera module 210. The orientation of the object acquired by the radar is due to the deviation caused by the rotation of the first panoramic camera module 210 relative to the first telescopic bracket 21 , so that the orientation of the same object acquired by the two is the same. The height of the first panoramic camera module 210 from the ground is greater than the height of the lidar from the ground. The first telescopic bracket 21 is used to automatically adjust the height of the first panoramic camera module 210 from the ground, and provide a first fixed reference for the first vector angle sensor 510 . The lidar is used to obtain the distance between the robot body 10 and its surrounding obstacles. The first panoramic camera module 210 is used to obtain a first color image of obstacles around the robot body 10 to assist in judging the physical and/or chemical properties of the obstacles around.

In one embodiment, as shown in FIG. 3 , the first camera stabilizer 22 is connected between the first panoramic camera module 210 and the first telescopic bracket 21 . The first camera stabilizer 22 and the first telescopic bracket 21 are connected by a rotational pair. The first panoramic camera module 210 includes a first lens module, a first lens support rod 214 and a first gravity block 215 . The first lens support rod 214 is vertically fixed on the camera mount of the first camera stabilizer 22 . The first lens module is disposed on the top of the first lens support rod 214 . The first gravity block 215 is disposed on the back of the camera mount to lower the center of gravity of the first panoramic camera module 210 and increase the torque of the camera mount relative to the first camera stabilizer 22, so that the first lens support rod 214 remains vertical state. The first vector angle sensor 510 is disposed between the first telescopic bracket 21 and the first camera stabilizer 22 to obtain the size and direction of the rotation angle of the first camera stabilizer 22 when it rotates relative to the first telescopic bracket 21 for correction The deviation between the orientation of the object in the picture acquired by the first panoramic camera module 210 and the orientation of the object acquired by the lidar is caused by the rotation of the first panoramic camera module 210 relative to the first telescopic bracket 21, so that the two obtained are the same. The orientation of the objects is the same.

In one embodiment, as shown in FIG. 3 , the first lens module includes a first wide-angle lens 211 , a second wide-angle lens 212 and a third wide-angle lens 213 . The shooting angles of the first wide-angle lens 211 , the second wide-angle lens 212 , and the third wide-angle lens 213 are each 120, so that a 360-degree panoramic camera is obtained by splicing and synthesizing.

In one embodiment, as shown in FIG. 4 , it further includes a second telescopic bracket 23 and a second panoramic camera module 220 disposed at the top of the second telescopic bracket 23 . The second telescopic bracket 23 is rotatably connected with the second panoramic camera module 220 . A second vector angle sensor 520 is disposed between the second telescopic bracket 23 and the second panoramic camera module 220 . The second vector angle sensor 520 is used to obtain the size and direction of the rotation angle of the second panoramic camera module 220 when it rotates relative to the second telescopic bracket 23 , so as to correct the orientation of the object and the laser beam in the picture obtained by the second panoramic camera module 220 The orientation of the object acquired by the radar is due to the deviation caused by the rotation of the second panoramic camera module 220 relative to the second telescopic bracket 23, so that the orientation of the same object acquired by the two is the same. The second telescopic bracket 23 is vertically arranged and is fixedly connected with the robot body 10 . The height of the second panoramic camera module 220 from the ground is greater than the height of the lidar from the ground, and less than the height of the first panoramic camera module 210 from the ground. The second telescopic bracket 23 is used to automatically adjust the height of the second panoramic camera module 220 from the ground, and provide a second fixed reference for the second vector angle sensor 520 . The second fixed reference is the same as the first fixed reference. The horizontal distance between the second panoramic camera module 220 and the first panoramic camera module 210 is greater than zero. The distance between the second telescopic bracket 23 and the first telescopic bracket 21 is greater than zero. The second panoramic camera module 220 is used for acquiring a second color image of obstacles around the robot body 10, and synthesizing a three-dimensional image with the first color image. This method can synthesize up to 6 three-dimensional images, and can see up to 5 datum planes of things.

In one embodiment, as shown in FIG. 4 , the second camera stabilizer 24 is connected between the second panoramic camera module 220 and the second telescopic bracket 23 . The second camera stabilizer 24 and the second telescopic bracket 23 are connected by a rotational pair. The second panoramic camera module 220 includes a second lens module, a second lens support rod 224 and a second gravity block 225 . The second lens support rod 224 is vertically fixed on the camera mount of the second camera stabilizer 24 . The second lens module is disposed on the top of the second lens support rod 224 . The second gravity block 225 is disposed on the back of the camera mount to lower the center of gravity of the second panoramic camera module 220 and increase the torque of the camera mount relative to the second camera stabilizer 24 , so that the second lens support rod 224 remains vertical state. The second vector angle sensor 520 is disposed between the second telescopic bracket 23 and the second camera stabilizer 24, and is used to obtain the rotation angle and rotation direction of the second camera stabilizer 24 relative to the second telescopic bracket 23 for correction. The deviation of the orientation of the object in the picture obtained by the second panoramic camera module 220 and the orientation of the object obtained by the lidar due to the rotation of the second panoramic camera module 220 relative to the second telescopic bracket 23 makes the two obtained the same The orientation of the objects is the same.

In one embodiment, as shown in FIG. 4 , the second lens module includes a fourth wide-angle lens 221 , a fifth wide-angle lens 222 and a sixth wide-angle lens 223 . The shooting angles of the fourth wide-angle lens 221 , the fifth wide-angle lens 222 and the sixth wide-angle lens 223 are each 120°, so that a 360-degree panoramic camera is obtained by stitching and synthesizing.

In one of the embodiments, as shown in FIG. 3 or FIG. 4 , it further includes an audio receiver 410 , a mechanical arm 31 arranged on the 10 , and a hammer (which can be regarded as a hammer) arranged at the end of the mechanical arm 31 for striking. mark 300). The audio receiver 410 is used to receive the sound of the hammer hitting the object.

In one of the embodiments, as shown in FIG. 3 or FIG. 4 , an audio receiver 410 , an end motor 34 , and a robotic arm 31 , a robotic arm mount 32 and a third telescopic bracket 33 are provided on the robot body 10 . . The robot arm 31 is connected to the robot body 10 through the robot arm mounting seat 32 . The mechanical arm 31 is a telescopic straight rod. The manipulator 31 is connected to the manipulator mounting seat 32 for rotation. The mechanical arm 31 is rotationally connected with the third telescopic bracket 33 , so that the extension and shortening of the third telescopic bracket 33 can drive the mechanical arm 31 to rotate around the mechanical arm mounting seat 32 . The end of the mechanical arm 31 is provided with an electronically controlled ejection hammer module 300 . The electronically controlled ejection hammer module 300 is connected to the rotating pair at the end of the mechanical arm 31 , and drives the relative rotation between the electronically controlled ejection hammer module 300 and the mechanical arm 31 through the end motor 34 . The audio receiver 410 is disposed on the electronically controlled ejection hammer module 300 , so that the audio receiving end faces the direction in which the electronically controlled ejection hammer module 300 hits the object.

In one of the embodiments, as shown in FIG. 3 or FIG. 4 , an audio receiver 410 and an ultrasonic transmitter 420 are further included. The ultrasonic transmitter 420 is used to transmit ultrasonic waves. The audio receiver 410 is used to receive the ultrasonic waves emitted by the ultrasonic transmitter 420 and reflected back by obstacles, so as to realize the navigation function in weather conditions such as rainy and foggy conditions.

In one embodiment, as shown in FIG. 3 or FIG. 4 , a first communication module 610 and a second communication module 620 are further included. The first communication module 610 is used for connecting with the Internet to realize remote control and data maintenance. The second communication module 620 is used for real-time communication after pairing with other robots in the same area, so as to realize the function of mutual learning between the robots.

In one of the embodiments, as shown in FIG. 3 or FIG. 4 , an anemometer module 710 disposed on the robot body 10 is further included. The anemometer module 710 is used to measure the wind speed at the location where the robot body 10 is located. Since the panoramic camera module in the solution of the present application is erected on a retractable bracket, the position is usually high. If the wind speed is too high, the torsion arm will be too large. Therefore, the anemometer module 710 is set. When the measured wind speed is greater than the set value When the value is set, the retractable bracket on which the panoramic camera module is set will be shortened to reduce the torsion arm.

In one of the embodiments, as shown in FIG. 3 or FIG. 4 , a wind vane module 720 disposed on the robot body 10 is further included. The anemometer module 710 is used to measure the wind direction at the location of the robot body 10 . In this way, the attitude of 10 can be adjusted according to the wind direction, and the operation can be performed with the least wind resistance.

In one embodiment, as shown in FIG. 3 or FIG. 4 , a first laser radar 110 , a second laser radar 120 , a third laser radar 130 and a fourth laser radar 140 are arranged around the robot body 10 in sequence (in the figure not shown). The first laser radar 110 , the second laser radar 120 , the third laser radar 130 and the fourth laser radar 140 are used to obtain distances between the robot and surrounding obstacles, respectively.

The self-propelled robot based on the fusion of laser navigation radar and computer vision perception provided above can obtain color information or text information that cannot be obtained by laser radar by placing a panoramic camera module on a vertically arranged retractable bracket. It can assist in judging the physical and/or chemical properties of obstacles around the robot, improve the self-propelled robot's understanding of the surrounding environment, and thus broaden the applicable scenarios of the self-propelled robot.

According to the above content, the present application also provides a self-propelled robot system based on the fusion of laser navigation radar and computer vision perception.

A self-propelled robot system based on the fusion of laser navigation radar and computer vision perception, as shown in FIG. 5, includes a central processing unit 810, an image data processing module 820, a laser navigation radar data processing module 830, and a first panoramic camera module 210. , the first vector angle sensor 510 and the laser navigation radar module. The image data processing module 820 and the laser navigation radar data processing module 830 are connected to the central processing unit 810 . The first panoramic camera module 210 is connected to the image data processing module 820 . The laser navigation radar module is connected to the laser navigation radar data processing module 830 . The first vector angle sensor 510 is respectively connected with the first panoramic camera module 210 and the central processing unit 810 . The LiDAR module is used to obtain the distance between 10 and its surrounding obstacles. The first panoramic camera module 210 is used to obtain a first color image of obstacles around the robot body 10 to assist in judging the physical and/or chemical properties of the obstacles around. The image data processing module 820 is used to control the first panoramic camera module 210 and perform first-level data processing on the first color image. The laser navigation radar data processing module 830 is used to control the laser navigation radar module, and perform first-level processing on the data obtained by the laser navigation radar module. The first vector angle sensor 510 is used to obtain the angle magnitude and direction of the reference deviation of the first panoramic camera module 210 relative to the laser navigation radar module. The central processing unit 810 performs second-level data processing on the data output by the image data processing module 820 and the laser navigation radar data processing module 830 through the data obtained by the first vector angle sensor 510, and corrects the relative rotation of the first panoramic camera module 210. Based on the reference deviation of the laser navigation radar module, the orientation of the object obtained by the laser navigation radar in the first color image is the same.

In one embodiment, as shown in FIG. 5 , there are also a second panoramic camera module 220 and a second vector angle sensor 520 . The second panoramic camera module 220 is connected to the image data processing module 820 . The second vector angle sensor 520 is respectively connected to the second panoramic camera module 220 and the central processing unit 810 . The second panoramic camera module 220 is used for acquiring a second color image of obstacles around the robot body 10, and synthesizing a three-dimensional image with the first color image. The second vector angle sensor 520 is used to obtain the angular magnitude and direction of the reference deviation of the second panoramic camera module 220 relative to the laser navigation radar module. The central processing unit 810 performs second-level data processing on the data output by the image data processing module 820 and the laser navigation radar data processing module 830 through the data obtained by the second vector angle sensor 520, and corrects the relative rotation of the second panoramic camera module 220 due to rotation. Based on the reference deviation of the laser navigation radar module, the orientation of the object obtained by the laser navigation radar in the second color image is the same. The central processing unit 810 or the second panoramic camera module 220 is further configured to combine the first color image and the second color image into a three-dimensional image.

In one embodiment, as shown in FIG. 5 , the first panoramic camera module 210 includes a first wide-angle lens 211 , a second wide-angle lens 212 , and a third wide-angle lens 213 . The second panoramic camera module 220 includes a fourth wide-angle lens 221 , a fifth wide-angle lens 222 , and a sixth wide-angle lens 223 . The first wide-angle lens 211 is paired with the sixth wide-angle lens 223 and then connected to the image data processing module 820 . The second wide-angle lens 212 is paired with the fifth wide-angle lens 222 and then connected to the image data processing module 820 . The third wide-angle lens 213 is paired with the fourth wide-angle lens 221 and then connected to the image data processing module 820 . The central processing unit 810 is further configured to correct the relative rotation angle of the first panoramic camera module 210 and the second panoramic camera module 220 through the data of the first vector angle sensor 510 and the second vector angle sensor 520 .

In one embodiment, as shown in FIG. 5 , the laser navigation radar module includes a first laser radar 110 , a second laser radar 120 , a third laser radar 130 and a fourth laser radar 140 . The first laser radar 110 , the second laser radar 120 , the third laser radar 130 and the fourth laser radar 140 are sequentially arranged around the robot to obtain distances between the robot and surrounding obstacles, respectively. The laser navigation radar data processing module 830 is also used for splicing the data obtained by the first laser radar 110 , the second laser radar 120 , the third laser radar 130 and the fourth laser radar 140 into a complete radar image, and outputting it to the central processing device 810 for further processing.

In one embodiment, as shown in FIG. 5 , it further includes an electronically controlled ejector hammer module 300 and an audio receiver 410 respectively connected to the central processing unit 810 . The electronically controlled catapult hammer module 300 is used to knock objects to make sounds. The audio receiver 410 is used for receiving the sound produced by the electronically controlled catapult hammer module 300 striking the object. The central processing unit 810 is further configured to perform spectrum analysis on the sound emitted by the electronically controlled catapult hammer module 300 striking the object, and output the type and basic physical and/or chemical properties of the struck object.

In one of the embodiments, as shown in FIG. 5 , an ultrasonic transmitter 420 connected to the central processing unit 810 is further included. The ultrasonic transmitter 420 is used to transmit ultrasonic waves. The audio receiver 410 is also used to receive the ultrasonic waves emitted by the ultrasonic transmitter 420 and reflected back by obstacles, so as to realize the navigation function of the robot in weather conditions such as rainy and foggy conditions.

In one of the embodiments, as shown in FIG. 5 , the ultrasonic transmitter 420 is also connected with the audio receiver 410 .

In one embodiment, as shown in FIG. 5 , it further includes a first communication module 610 and a second communication module 620 connected to the central processing unit 810 . The first communication module 610 is used for connecting with the Internet to realize remote control and data maintenance. The second communication module 620 is used for real-time communication after pairing with other robots in the same area, so as to realize the function of mutual learning between the robots.

In one embodiment, as shown in FIG. 5 , the wind vane module 720 connected to the central processing unit 810 is further included. The wind vane module 720 is used to measure the wind direction at the location of the robot.

In one embodiment, as shown in FIG. 5 , an anemometer module 710 connected to the central processing unit 810 is further included. The anemometer module 710 is used to measure the wind speed at the location of the robot.

In one embodiment, as shown in FIG. 5 , a storage module 840 connected to the central processing unit 810 is further included. The storage module 840 stores a control program that can be run by the central processing unit 810 and attribute information of common objects, and is used to store information generated when each functional module runs.

The above-mentioned self-propelled robot system based on the fusion of laser navigation radar and computer vision perception can obtain color information or text information that cannot be obtained by laser radar by setting up a panoramic camera module, so as to assist in judging obstacles around the robot. The physical and/or chemical properties of the self-propelled robot can improve the awareness of the surrounding environment, thereby broadening the applicable scenarios of the self-propelled robot.

In addition, the present application also provides a control method of a self-propelled robot based on the fusion of laser navigation radar and computer vision perception.

A control method of a self-propelled robot based on laser navigation radar and computer vision perception fusion, the self-propelled robot is the self-propelled robot in any of the above-mentioned embodiments, and the method comprises steps S110-S140:

S110: The lidar navigation radar obtains the reflection surfaces of obstacles around the robot.

S120: The central processing unit 810 determines whether the maximum distance between obstacles is smaller than the minimum passing distance of the robot, and if so, executes step S130, otherwise executes step S140.

S130: Start the first panoramic camera module 210.

S140: Only start the LiDAR to continue running.

In one of the embodiments, the method further includes, steps S210-S240:

S210: Test the maximum distance that the lidar navigation radar can obtain the obstacles around the robot.

S220: Determine whether the maximum distance is less than or equal to the first set value, if so, execute step S230, otherwise execute step S240.

S230: Activate the ultrasonic transmitter 420 and the audio receiver 410.

S240: Only start the lidar navigation radar and continue to run.

The method provided above uses the preliminary judgment of the surrounding obstacle environment through the lidar navigation radar. When the laser radar navigation alone cannot meet the requirements, computer vision perception is activated to gain a deeper understanding of the surrounding environment, so as to find the best action path.

The technical features of the above embodiments can be combined arbitrarily. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features It is considered to be the range described in this specification.

The above examples only represent several embodiments of the present invention, and the descriptions thereof are specific and detailed, but should not be construed as a limitation on the scope of the invention patent. It should be pointed out that for those skilled in the art, without departing from the concept of the present invention, several modifications and improvements can be made, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the appended claims.

Claims (10)

1. A self-walking robot based on laser navigation radar and computer vision perception fusion is characterized by comprising a robot body for walking movement, a laser radar arranged on the robot body, a first telescopic bracket vertically and fixedly arranged on the robot body, and a first panoramic camera module arranged at the top end of the first telescopic bracket;

the first telescopic bracket is connected with the first panoramic camera module revolute pair;

a first vector angle sensor is arranged between the first telescopic bracket and the first panoramic camera module;

the first vector angle sensor is used for acquiring the rotation angle and the rotation direction of the first panoramic camera module when rotating relative to the first telescopic support;

the height of the first panoramic camera module from the ground is greater than that of the laser radar from the ground;

the first telescopic support is used for automatically adjusting the height of the first panoramic camera module from the ground and providing a first fixed reference for the first vector angle sensor;

the laser radar is used for acquiring the distance between the robot body and the surrounding obstacles;

the first panoramic camera shooting module is used for acquiring a first color image of the obstacle around the robot body so as to assist in judging the physical and/or chemical properties of the obstacle around the robot body.

2. The self-walking robot of claim 1,

the first panoramic camera shooting module is connected with the first telescopic bracket through a first camera stabilizer;

the first camera stabilizer is connected with the first telescopic bracket through a revolute pair;

the first panoramic camera shooting module comprises a first lens module, a first lens supporting rod and a first gravity block;

the first lens supporting rod is vertically and fixedly arranged on the camera mounting seat of the first camera stabilizer;

the first lens module is arranged at the top end of the first lens supporting rod;

the first gravity block is arranged on the back of the camera mounting seat so as to reduce the gravity center of the first panoramic camera shooting module and improve the torque of the camera mounting seat relative to the first camera stabilizer, so that the first lens supporting rod is kept in a vertical state;

the first vector angle sensor is arranged between the first telescopic support and the first camera stabilizer and used for acquiring the rotation angle size and the rotation direction of the first camera stabilizer relative to the first telescopic support during rotation.

3. The self-walking robot of claim 1,

the panoramic camera comprises a first telescopic support and a first panoramic camera module arranged at the top end of the first telescopic support;

the second telescopic bracket is connected with the second panoramic camera module revolute pair;

a second vector angle sensor is arranged between the second telescopic bracket and the second panoramic camera module;

the second vector angle sensor is used for acquiring the rotation angle and the rotation direction of the second panoramic camera module when rotating relative to the second telescopic support;

the second telescopic bracket is vertically arranged and is fixedly connected with the robot body;

the height of the second panoramic camera module from the ground is greater than that of the laser radar and less than that of the first panoramic camera module from the ground;

the second telescopic support is used for automatically adjusting the height of the second panoramic camera module from the ground and providing a second fixed reference for the second vector angle sensor;

the second fixed reference datum is the same as the first fixed reference datum;

the horizontal distance between the second panoramic camera module and the first panoramic camera module is greater than 0;

the distance between the second telescopic bracket and the first telescopic bracket is greater than 0;

the second panoramic camera shooting module is used for acquiring a second color image of obstacles around the robot body and synthesizing the second color image with the first color image into a three-dimensional image.

4. The self-walking robot of claim 1,

the robot further comprises an audio receiver, a mechanical arm arranged on the robot body and a hammer arranged at the tail end of the mechanical arm and used for knocking;

the audio receiver is used for receiving sound of the hammer hitting an object.

5. The self-walking robot of claim 1,

the robot further comprises an audio receiver, a tail end motor, a mechanical arm arranged on the robot body, a mechanical arm mounting seat and a third telescopic support;

the mechanical arm is connected with the robot body through the mechanical arm mounting seat;

the mechanical arm is a telescopic straight rod;

the mechanical arm is connected with the mechanical arm mounting seat revolute pair;

the mechanical arm is connected with the third telescopic bracket revolute pair;

an electric control ejection hammer module is arranged at the tail end of the mechanical arm;

the electric control ejection hammer module is connected with the mechanical arm tail end revolute pair and drives the electric control ejection hammer module to rotate relative to the mechanical arm through a tail end motor;

the audio receiver is arranged on the electronic control ejection hammer module, and enables the audio receiving end to face the direction in which the electronic control ejection hammer module strikes the object.

6. The self-walking robot of claim 1,

the device also comprises a first communication module and a second communication module;

the first communication module is used for being connected with the Internet so as to realize remote control and data maintenance;

the second communication module is used for real-time communication after being paired with other robots in the same area, so that the function of mutual learning among the robots is realized.

7. A self-walking robot system based on laser navigation radar and computer vision perception fusion is characterized in that,

the system comprises a central processing unit, an image data processing module, a laser navigation radar data processing module, a first panoramic camera module, a first vector angle sensor and a laser navigation radar module;

the image data processing module and the laser navigation radar data processing module are connected with the central processing unit;

the first panoramic camera module is connected with the image data processing module;

the laser navigation radar module is connected with the laser navigation radar data processing module;

the first vector angle sensor is respectively connected with the first panoramic camera module and the central processing unit;

the laser navigation radar module is used for acquiring the distance between the robot body and the surrounding obstacles;

the first panoramic camera module is used for acquiring a first color image of an obstacle around the robot body so as to assist in judging the physical and/or chemical properties of the obstacle around the robot body;

the image data processing module is used for controlling the first panoramic camera module and performing first-stage data processing on the first color image;

the laser navigation radar data processing module is used for controlling the laser navigation radar module and performing first-stage processing on data acquired by the laser navigation radar module;

the first vector angle sensor is used for acquiring the angle and the direction of the reference deviation of the first panoramic camera module relative to the laser navigation radar module;

and the central processing unit performs second-level data processing on the data output by the image data processing module and the laser navigation radar data processing module through the data acquired by the first vector angle sensor, and corrects the reference deviation of the first panoramic camera module relative to the laser navigation radar module caused by rotation, so that the object azimuth in the first color image is the same as the object azimuth acquired by the laser navigation radar.

8. The system of claim 7,

the electronic control ejection hammer module and the audio receiver are respectively connected with the central processing unit;

the electronic control ejection hammer module is used for knocking an object to make a sound;

the audio receiver is used for receiving sound generated by the electronic control ejection hammer module when the electronic control ejection hammer module strikes an object;

the central processing unit is also used for carrying out spectrum analysis on the sound emitted by the electric control ejection hammer module when the object is knocked, and outputting the type and basic physical and/or chemical properties of the knocked object.

9. The system of claim 8,

the ultrasonic transmitter is connected with the central processing unit;

the ultrasonic transmitter is used for transmitting ultrasonic waves;

the audio receiver is also used for receiving the ultrasonic waves which are emitted by the ultrasonic emitter and reflected back by the barrier so as to realize the navigation function of the robot under the weather conditions of overcast and rainy days, heavy fog days and the like.

10. The system of claims 7-9,

also comprises a first communication module and a second communication module connected with the central processing unit

The first communication module is used for being connected with the Internet so as to realize remote control and data maintenance;

the second communication module is used for real-time communication after being paired with other robots in the same area, so that the function of mutual learning among the robots is realized.

Images