CN112415998A - Obstacle classification and obstacle avoidance control system based on TOF camera - Google Patents
Obstacle classification and obstacle avoidance control system based on TOF camera Download PDFInfo
- Publication number
- CN112415998A CN112415998A CN202011159235.7A CN202011159235A CN112415998A CN 112415998 A CN112415998 A CN 112415998A CN 202011159235 A CN202011159235 A CN 202011159235A CN 112415998 A CN112415998 A CN 112415998A
- Authority
- CN
- China
- Prior art keywords
- obstacle
- mobile robot
- obstacles
- module
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000004888 barrier function Effects 0.000 claims description 47
- 238000004804 winding Methods 0.000 claims description 47
- 238000000034 method Methods 0.000 claims description 24
- 238000001514 detection method Methods 0.000 claims description 12
- 230000000007 visual effect Effects 0.000 claims description 10
- 230000008859 change Effects 0.000 claims description 8
- 230000003287 optical effect Effects 0.000 claims description 5
- 238000003384 imaging method Methods 0.000 claims description 4
- 238000009434 installation Methods 0.000 claims description 4
- 230000007175 bidirectional communication Effects 0.000 claims description 3
- 230000006854 communication Effects 0.000 claims description 3
- 238000004891 communication Methods 0.000 claims description 3
- 230000004044 response Effects 0.000 claims description 3
- 230000009471 action Effects 0.000 abstract description 4
- 238000012549 training Methods 0.000 abstract description 4
- 238000004519 manufacturing process Methods 0.000 abstract description 3
- 230000008569 process Effects 0.000 description 7
- 230000001960 triggered effect Effects 0.000 description 4
- 238000011065 in-situ storage Methods 0.000 description 3
- 238000013459 approach Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 239000004568 cement Substances 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 239000002023 wood Substances 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- WYTGDNHDOZPMIW-RCBQFDQVSA-N alstonine Natural products C1=CC2=C3C=CC=CC3=NC2=C2N1C[C@H]1[C@H](C)OC=C(C(=O)OC)[C@H]1C2 WYTGDNHDOZPMIW-RCBQFDQVSA-N 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000010408 sweeping Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
- G05D1/0214—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory in accordance with safety or protection criteria, e.g. avoiding hazardous areas
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
- G05D1/0246—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Aviation & Aerospace Engineering (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Electromagnetism (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
- Manipulator (AREA)
Abstract
The invention discloses an obstacle classifying and avoiding control system based on a TOF camera, which effectively classifies and identifies whether an indoor environment should be crossed and whether an indoor environment should be collided by adopting obstacle size information, position information and brightness information acquired by the TOF camera, and triggers a collision warning signal in time according to the type characteristics and the size information of the identified obstacle so as to propel a mobile robot to plan a passable area before moving to the corresponding obstacle. Compared with the prior art, the robot has the advantages that the multiple cameras or the multi-line laser heads are adopted, and excessive image feature point fitting and classification training is greatly executed, so that the production cost is reduced, and the real-time performance of obstacle avoidance actions of the robot is improved.
Description
Technical Field
The invention relates to the technical field of intelligent robots, in particular to an obstacle classification and obstacle avoidance control system based on a TOF camera.
Background
At present, SLAM robots based on inertial navigation, vision and laser are more and more popular, a family sweeping cleaning robot is relatively strong in representativeness, the indoor environment is positioned and a map is built in real time by combining the data of the vision, the laser, a gyroscope, acceleration and a wheel odometer, and then positioning navigation is realized according to the built map. However, at present, a robot often has movable obstacles such as toys and electric wires on the ground in a complex obstacle environment, when the robot collides with the obstacle of the type, the robot is either pushed or wound by the obstacle of the electric wire type, and a sofa-type obstacle still exists in a home environment, if the height below the bottom of the sofa is just lower than the height of the top surface of the robot, the robot is possibly blocked when entering the sofa-type obstacle.
Chinese patent CN110622085A applied for 12/27/2019 relates to acquiring a depth image of an obstacle by using at least one camera, and extracting and identifying the type of the obstacle by combining a second image feature of the depth image of the obstacle and a first image feature of a color image, but the identified obstacle is not effectively classified according to passable conditions in an actual moving environment in a robot room, and too many image feature point fitting classification training affects the real-time performance of the obstacle avoidance operation of the robot.
Disclosure of Invention
In order to solve the technical problem, the TOF camera module disclosed by the invention can acquire the depth information of the obstacles and the brightness information of the depth image in the range of the detectable visual angle, detect the relative positions of the obstacles and identify the effective type characteristics of the obstacles to distinguish whether the obstacles should be crossed or collided, so that when the robot approaches the obstacle to be collided, the robot can avoid the obstacles in advance, and a collision-free function can be realized or the robot can bypass the obstacles or walk along the edge. The specific technical scheme is as follows:
a barrier classification and obstacle avoidance control system based on a TOF camera is installed on a mobile robot and comprises a barrier classification module, a barrier positioning and marking module and an obstacle avoidance module; the barrier classification module comprises a TOF camera arranged in the advancing direction of the mobile robot and is used for calculating the relative position relation of at least one barrier and the size of the same barrier in the detection visual angle range of the TOF camera according to a depth image acquired by the TOF camera in real time, selecting brightness image data which is output in real time by combining the TOF camera and matched with the corresponding barrier, and identifying the type of the corresponding barrier; the obstacle positioning triggering module is used for receiving the relative position relation of the obstacles of the type currently identified by the obstacle classification module and the size of the same obstacle, and then deciding whether the obstacle of the corresponding identified type triggers a collision warning signal or not based on the data information so as to enable: the mobile robot plans a passable area before moving to a corresponding barrier; and the obstacle avoidance module is used for adjusting the current pose of the mobile robot according to the relative position relationship of the obstacles triggering the collision warning signals and the size of the same obstacle, which are transmitted by the obstacle positioning and marking module, and planning an obstacle avoidance path of the mobile robot according to the relative position relationship of the currently recognized type of obstacles and the size of the same obstacle, which are transmitted by the obstacle positioning and marking module, so that the mobile robot avoids the obstacles triggering the collision warning signals, bypasses the obstacles triggering the collision warning signals, or keeps the original pose to continue walking.
According to the technical scheme, the depth information, the size information and the brightness information of the obstacles acquired by the TOF camera are adopted, whether the obstacles should cross and collide in an indoor environment are effectively classified and identified, and collision warning signals are timely triggered according to the type characteristics, the relative position and the size information of the identified obstacles, so that the mobile robot is propelled to plan a passable area before moving to the corresponding obstacles. Compared with the prior art, the robot obstacle avoidance system has the advantages that the multiple cameras or the multi-line laser heads are adopted, excessive image feature points are executed, and fitting classification training is too large, so that the production cost of the system is reduced, and the real-time performance of obstacle avoidance actions of the robot is improved.
Further, in the obstacle location triggering module, the method of deciding whether the obstacle of the corresponding recognized type triggers the collision warning signal based on the data information at least includes: defining the obstacle of the identified type as a target obstacle; the types of the obstacles comprise winding obstacles, island obstacles, threshold capable of crossing and furniture capable of being crossed by the mobile robot; when the type of the target obstacle is a winding object, if the depth data of the depth image of the target obstacle shot by the TOF camera is smaller than a first safe depth threshold, controlling the obstacle positioning triggering module to trigger a collision warning signal; when the type of the target obstacle is an island obstacle, if the depth data of the depth image of the target obstacle shot by the TOF camera is smaller than a second safe depth threshold, controlling the obstacle positioning triggering module to trigger a collision warning signal; when the type of the target obstacle is a wall body, if the depth data of the depth image of the target obstacle shot by the TOF camera is smaller than a third safe depth threshold, controlling the obstacle positioning triggering module to trigger a collision warning signal; when the type of the target obstacle is a threshold capable of being crossed or furniture capable of being crossed by the mobile robot, controlling the obstacle positioning triggering module not to trigger a collision warning signal; the first safe depth threshold is larger than the second safe depth threshold, and the second safe depth threshold is larger than the third safe depth threshold. According to the technical scheme, the requirements of the matched type obstacles on collision and obstacle avoidance are met by setting different safe distances before the recognized target obstacles are approached within the visual angle range, so that the area where no obstacles can pass is judged in advance, and an effective obstacle avoidance path is conveniently planned in the follow-up process.
Further, in the obstacle avoidance module, the method for adjusting the current pose of the mobile robot according to the relative position relationship of the obstacle triggering the collision warning signal transmitted by the obstacle positioning and marking module and the size of the same obstacle at least includes: when the type of the target obstacle is a winding object and the obstacle positioning and marking module triggers the collision warning signal, controlling the mobile robot to reduce the real-time speed and change the current advancing direction according to the contour width of the target obstacle so that a straight path in the changed advancing direction avoids the target obstacle; when the type of the target obstacle is a threshold capable of being crossed or furniture capable of being crossed by the mobile robot, controlling the mobile robot to keep the current advancing direction; when the type of the target obstacle is an island obstacle and the obstacle positioning and marking module triggers the collision warning signal, controlling the mobile robot to change the current advancing direction so that the mobile robot starts to walk around the outline of the island obstacle; and when the type of the target barrier is a wall body and the barrier positioning and marking module triggers the collision warning signal, controlling the mobile robot to adjust the optimal edgewise direction so that the mobile robot starts to enter an edgewise mode. The method is beneficial to optimizing the current pose of the mobile robot, so that the current motion state (including normal linear walking, in-situ rotation, radian rotation and edgewise) of the mobile robot is optimized, and the probability of contact between the mobile robot and the identified type of barrier is reduced.
Further, in the obstacle avoidance module, the method for planning the obstacle avoidance path of the mobile robot according to the relative position relationship of the currently identified type of obstacle and the size of the same obstacle, which are transmitted by the obstacle positioning and marking module, at least includes: firstly, planning a passable area between the mobile robot and the obstacles according to the relative position relation of the obstacles of the currently identified type and the size of the same obstacle, and planning an obstacle avoidance path of the mobile robot in the passable area; the obstacles of the currently identified type transmitted by the obstacle positioning and marking module comprise obstacles triggering collision warning signals and other target obstacles in a TOF camera detection visual angle range; when the type of the target obstacle is a winding object and the obstacle positioning and marking module triggers the collision warning signal, planning an initial obstacle avoiding direction of the mobile robot according to the outline width of the target obstacle, and simultaneously adjusting the initial obstacle avoiding direction of the mobile robot in real time according to the position coordinates corresponding to each point on the identified outline of the winding object and the position coordinates of other target obstacles so as to avoid touching the identified winding object and keep adjusting the depth data corresponding to each point on the outline of the winding object acquired in real time to be larger than a first safety depth threshold; when the type of the target obstacle is a threshold capable of being crossed or furniture capable of being crossed by the mobile robot, controlling the mobile robot to walk in a straight line along the current advancing direction so as to cross or cross the target obstacle; when the type of the target obstacle is an island obstacle and the obstacle positioning and marking module triggers the collision warning signal, controlling the mobile robot to walk along the extension direction of the outline of the island obstacle, and keeping adjusting depth data corresponding to each point on the outline of the island obstacle collected and identified in real time to be larger than a second safety depth threshold so as to realize obstacle detouring walking of the mobile robot; and when the type of the target barrier is a wall body and the barrier positioning and marking module triggers the collision warning signal, controlling the mobile robot to adjust the optimal edgewise direction and starting to walk along the wall along the optimal edgewise direction.
Thereby, the following steps are achieved: can control mobile robot to stride across the threshold when TOF camera discerns the threshold, TOF camera control mobile robot gets into at the bottom of the sofa when discerning the sofa (when the height at the bottom of the sofa is less than the height of machine), TOF camera prohibits mobile robot collision toy but can walk around the barrier when discerning the toy, TOF camera allows mobile robot collision wall body in order to carry out the edgewise walking when discerning the wall, TOF camera forbids mobile robot to stride across winding objects such as electric wire and keep away the barrier with other routes when discerning the electric wire, and then plan out effectual obstacle-avoiding route.
Further, the TOF camera is mounted in front of the mobile robot, the mounting height of the TOF camera relative to the traveling plane is 6.7 centimeters, the optical axis of the TOF camera is at an inclination angle of 10 degrees relative to the top surface of the robot, and the driving wheel at the bottom of the mobile robot is in contact with the traveling plane. So that the brightness value of the target obstacle in the detection visual angle range is effective and the contour line of the target obstacle is complete to meet the depth positioning requirement.
Furthermore, the obstacle classification module and the obstacle avoidance module are respectively connected with the obstacle positioning trigger module through a bidirectional communication interface so as to establish a communication relation of receiving and sending responses between signal data. The real-time performance of the depth image information feedback of each obstacle is guaranteed, and the obstacle avoidance path planning efficiency is improved.
Further, the size of the same obstacle includes the height of each position point of the contour line of the corresponding obstacle within a preset horizontal width; the preset horizontal width is the length of a horizontal line segment in a pre-selected connectable domain corresponding to the surface of the obstacle, and horizontal projections of the contour line of the corresponding obstacle at each position point in the preset horizontal width are all on the horizontal line segment. The technical scheme is to identify the shape characteristics of the obstacle and the position points of the representative contour line which can be effectively sampled by the accessible characteristics.
Further, the method for selecting the brightness image data matched with the corresponding obstacle and output in real time in combination with the TOF camera to identify the type of the corresponding obstacle at least comprises the following steps: when the obstacle classification module judges that the height of each position point of the contour line of the corresponding obstacle in the preset horizontal width is in the threshold height range, the corresponding obstacle is identified as a threshold capable of being crossed; all heights in the threshold height range are smaller than the height of the body of the mobile robot; when the obstacle classification module judges that the heights of the lowest position points of the contour lines of the corresponding obstacles in the preset horizontal width are higher than the height of the body of the mobile robot, the corresponding obstacles are identified as furniture for the mobile robot to pass through; when the obstacle classification module judges that the contour lines of the corresponding obstacles accord with the rectangular characteristic condition and the depth values corresponding to each pixel point on the corresponding depth images are equal, and the heights of the position points of the contour lines in the preset horizontal width are higher than a preset passable height threshold value, identifying the corresponding obstacles as a wall body; the preset passable height threshold is higher than the height of the body of the mobile robot; when the obstacle classification module judges that the variance of the height of each position point of the contour line of the corresponding obstacle in the preset horizontal width meets a first winding condition and/or judges that the variance of the product of the depth of the pixel point of the depth image corresponding to each position point of the contour line of the corresponding obstacle in the preset horizontal width and the gray level of the pixel point of the matched brightness image meets a second winding condition, the corresponding obstacle is identified as a winding; when the obstacle classification module judges that the horizontal width of the space occupied by the corresponding obstacle is smaller than the preset island width and the vertical height of the space occupied by the corresponding obstacle is smaller than the preset island height, the corresponding obstacle is identified as an island obstacle; the preset horizontal width is larger than the width of the body of the mobile robot; presetting the height of the island to fall into the threshold height range; the preset island width is less than or equal to the preset horizontal width.
Thereby, the following steps are achieved: the lower threshold of high is discerned to the TOF camera, and the TOF camera discerns that sofa (the height is less than the furniture of the height of machine at the bottom of the sofa) control mobile robot gets into at the bottom of the sofa, and the TOF camera discerns the less toy of height, and the TOF camera discerns that the wall gallery, TOF camera are discerned highly little and have the electric wire of curve characteristic. Therefore, the method can effectively detect and identify large obstacles, small obstacles, obstacles capable of crossing and passing through and obstacles prohibited from contacting, and further plan an effective accessible area for the mobile robot to avoid obstacles.
Further, when the obstacle classification module judges that the brightness image data matched with a communicable surface area corresponding to the obstacle is in a first preset medium gray level threshold range, and/or when the obstacle classification module judges that the product of the depth of pixel points of the depth image corresponding to each position point of a communicable surface area corresponding to the obstacle and the gray level of the pixel points of the matched brightness image is in a second preset medium gray level threshold range, the obstacle classification module determines that the surface medium corresponding to the obstacle is a flat plane medium allowing the mobile robot to move without obstacle. According to the technical scheme, the brightness information of the light reflected by the surface of the obstacle and the depth data of the matched pixel points in the depth image are used for judging and identifying the surface medium of the obstacle, so that more specific obstacle types can be identified, and a traveling plane allowing the mobile robot to walk across without obstacles can be judged in advance.
Further, the brightness image data matched with the corresponding obstacle is light brightness information reflected back to an imaging plane of the TOF camera from the surface of the corresponding obstacle, and is in one-to-one correspondence with the depth of pixel points of the depth image of the corresponding obstacle acquired by the TOF camera. And the judgment effect of the first winding condition and the second winding condition is reduced by using fitting operation. The accuracy of obstacle type identification is improved.
Drawings
Fig. 1 is a block diagram of an obstacle classification and avoidance control system based on a TOF camera according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be described in detail below with reference to the accompanying drawings in the embodiments of the present invention. It should be noted that, in the present application, the whole text of chinese patent CN111624997A is introduced into the text of the present application, so as to complete the description of calculating the relative position information of the obstacle region from the depth image acquired by the TOF camera and the description of the map calibration marking method.
The embodiment of the invention discloses an obstacle classifying and avoiding control system based on a TOF camera, which is arranged on a mobile robot and comprises an obstacle classifying module, an obstacle positioning and marking module and an obstacle avoiding module, as shown in figure 1; the obstacle classification module comprises a TOF camera arranged on the advancing direction of the mobile robot, the TOF camera can be a 3d-TOF camera, the 3d-TOF camera is arranged in front of the robot in an unobstructed manner, the placement angle and the placement position of the 3d-TOF camera can be adjusted according to the actual environment, the obstacle classification module is used for calculating the relative position relation of at least one obstacle and the size of the same obstacle in the detection visual angle range of the TOF camera according to the depth image acquired by the TOF camera in real time, the relative position relation of the obstacle and the size of the same obstacle are calculated by combining the temperature calibration coefficient of a sensor and the related depth calibration coefficient in some implementation scenes so as to output more accurate depth data, and then the luminance image data which is output in real time by the TOF camera and is matched with the corresponding obstacle is selected, the type of the corresponding obstacle is identified.
It should be noted that TOF is an abbreviation of time of Flight (TOF) technology, that is, a sensor emits modulated near-infrared light, which is reflected after encountering an object, and the sensor converts the distance of a shot scene by calculating the time difference or phase difference between light emission and reflection to generate depth information, and in addition, the three-dimensional profile of the object can be presented in a manner that different colors represent topographic images of different distances by combining with the shooting of a traditional camera, so as to obtain a three-dimensional 3D model, and the TOF camera is a camera which adopts TOF technology to acquire data.
The depth image is also called a distance image, and refers to an image in which the distance between each pixel point of the depth image and the actual measurement point of the corresponding obstacle is taken as a pixel value. Wherein the deflection angle between each pixel point and the corresponding measurement point is determined based on the setting parameters of the imaging device. The depth image directly reflects the geometric shape of the visible surface of each obstacle in the shot physical scene, and the depth image can be converted into spatial point cloud data through coordinate conversion. And all the obstacles described by the depth data in the depth image can be used as images of the obstacles to be identified for subsequent processing. Wherein the obstacle shall be taken to broadly include an object temporarily placed on a traveling plane and an object that is not easily moved. The traveling plane includes, but is not limited to, cement floors, painted floors, floors with composite floors, floors with solid wood floors, floors with carpets, table tops, glass surfaces, etc. according to the actual application environment. Examples of the object temporarily placed on the traveling plane include a threshold (capable of crossing), a toy (collision prohibition), a wire (crossing prohibition), and the like; examples of the object which is not easy to move include a sofa (when the height of the sofa bottom is lower than that of the machine, the machine cannot be controlled to enter), a wall and the like.
The obstacle positioning triggering module is used for receiving the relative position relation (mainly three-dimensional coordinate positions of all position points on the contour line of the obstacle and angle information under a global coordinate system) of the type of obstacles currently identified by the obstacle classification module and the size of the same obstacle, and then deciding whether to trigger a collision warning signal or not on the basis of the data information, wherein some obstacles are allowed to cross and pass through by the mobile robot and do not need to trigger the collision warning signal, and for the obstacles which block the normal walking of the mobile robot, the collision warning signal needs to be triggered before the obstacles contact occurs, so that: the mobile robot plans a passable area before moving to a corresponding obstacle.
In addition, after the obstacle positioning triggering module receives the relative position relationship (mainly three-dimensional coordinate positions of all position points on the contour line of the obstacle and angle information under a global coordinate system) of the obstacle of the type currently identified by the obstacle classification module and the size of the same obstacle, the data are stored in real time and matched with analysis data in real time (simple geometric shape feature fitting is carried out, and a type indicating signal representing the shape feature of the obstacle is output on the premise of meeting the obstacle type identification condition), so that the received obstacle type is further corrected, and the identification result of the obstacle classification module is optimized. Then, the obstacle positioning triggering module marks the real-time constructed map of the mobile robot according to the relative position relationship of the obstacles calculated by the obstacle classification module, positions the recognized obstacles and the relative positions of the obstacles to be recognized (the obstacles to be recognized but having calculated the relative position relationship) within the view angle range of the recognized obstacles in the map, and conveniently plans an obstacle avoidance path.
And the obstacle avoidance module is used for adjusting the current pose of the mobile robot according to the relative position relationship of the obstacles triggering the collision warning signals and the size of the same obstacle transmitted by the obstacle positioning marking module after the collision warning signals are triggered by the obstacle positioning marking module, and planning an obstacle avoidance path of the mobile robot according to the relative position relationship of the currently identified type of obstacles and the size of the same obstacle transmitted by the obstacle positioning marking module so that the mobile robot avoids the obstacles triggering the collision warning signals or bypasses the obstacles triggering the collision warning signals. And the obstacle avoidance module is used for keeping the original pose to continue walking according to the type of the currently identified obstacle and the size information of the same obstacle transmitted by the obstacle positioning marking module when the obstacle positioning marking module does not send out a collision warning signal, so that the mobile robot can directly pass through the obstacle and cross the obstacle.
In this embodiment, the obstacle avoidance module plans an obstacle avoidance path closer to the actual environment according to the identified obstacle (including the obstacle triggering the collision warning signal) marked in the obstacle positioning and marking module, the relative position of the obstacle to be identified within the detection view angle range of the TOF camera, and the size of the same obstacle, so that the mobile robot can avoid the obstacle (including the obstacle identified and to be identified but having calculated the relative position relationship) within the view angle range of the TOF camera in time, or walk around the corresponding obstacle according to the type of the identified obstacle, or walk across the corresponding obstacle according to the type of the identified obstacle. In the specific implementation process, at least: the obstacle avoidance module determines to adjust the current pose of the mobile robot according to the current motion state (normal linear walking, in-situ rotation, radian rotation and edge) of the mobile robot and the type of the obstacle, so that the mobile robot can linearly cross the obstacle before passing through the obstacle and crossing the obstacle, or the mobile robot can walk around the obstacle or linearly avoid the obstacle without touching the obstacle before passing through a small obstacle (including a small winding object) by adjusting the current pose of the mobile robot, or the mobile robot can avoid the obstacle along the edge when approaching the wall by adjusting the current pose of the mobile robot. This is of course also relevant for the shape features of the recognized obstacle, which are composed or abstracted geometries, geometry combinations, etc. based on contour lines and/or feature points for matching the respective obstacle type. Wherein the geometry, combination of geometries may be based on the full outline or partial representation of the outline of the identified obstacle. For example, the shape features provided based on the island type include one or more combinations of circles, spheres, arcs, squares, cubes, pi-shapes, and the like. For example, the shoe shape features comprise a plurality of arc shapes which are connected end to end, and the chair shape features comprise a pi shape, an eight-claw shape and the like. The shape characteristics provided based on the type of wrap include at least one or more of a combination of curvilinear shapes, serpentine shapes, and the like. The shape features provided based on the space division type include at least one or more combinations of a straight line shape, a broken line shape, a rectangle shape, and the like.
In summary, the embodiment of the invention effectively classifies and identifies whether the obstacle should cross and whether the obstacle should collide in the indoor environment by using the obstacle depth information, the size information and the brightness information acquired by the TOF camera, and triggers the collision warning signal in time according to the type characteristics, the relative position and the size information of the identified obstacle, so as to propel the mobile robot to plan the passable area before moving to the corresponding obstacle. Compared with the prior art that a plurality of cameras or multi-line laser heads are adopted and excessive image feature point fitting and classifying training is executed, the system disclosed by the embodiment has the advantages that the production cost is reduced, and the real-time performance of obstacle avoidance actions of the robot is improved.
Preferably, the top surface carrier of the mobile robot provided by the embodiment of the invention is provided with a camera or a single-line laser head for map correction after the map of the robot slips; the mobile robot is also internally provided with a gyroscope for detecting the rotation angle, a mileometer for detecting the travel distance and a sensor capable of detecting the wall distance, wherein the sensor for detecting the wall distance can be an ultrasonic distance sensor, an infrared intensity detection sensor, an infrared distance sensor, a physical switch detection collision sensor, a capacitance or resistance change detection sensor and the like.
As an embodiment, in the obstacle location triggering module, the method for deciding whether the obstacle of the corresponding recognized type triggers the collision warning signal based on the data information at least comprises the following steps: defining the obstacle of the identified type as a target obstacle; the types of the obstacles comprise winding, island obstacles, traversable thresholds and furniture for the mobile robot to traverse, and the pixel point depth data of the depth image of part of the obstacles is continuous (wall) but the pixel point depth data of part of the obstacles is discontinuous (winding).
When the type of the target obstacle is a winding object, if the depth data of the depth image of the target obstacle shot by the TOF camera is smaller than a first safe depth threshold, controlling the obstacle positioning triggering module to trigger a collision warning signal; when the type of the target obstacle is an island obstacle, if the depth data of the depth image of the target obstacle shot by the TOF camera is smaller than a second safe depth threshold, controlling the obstacle positioning triggering module to trigger a collision warning signal; when the type of the target barrier is a wall body, if the depth data of the depth image of the target barrier shot by the TOF camera is smaller than a third safe depth threshold, controlling the barrier positioning triggering module to trigger a collision warning signal, although collision with the wall body is allowed in the subsequent process of controlling the mobile robot to walk along the wall; when the type of the target obstacle is a threshold capable of being crossed or furniture capable of being crossed by the mobile robot, the obstacle positioning triggering module is controlled not to trigger a collision warning signal, because the threshold crossed by the mobile robot or the furniture crossed by the mobile robot is controlled in an optimal planning mode to minimize the influence on the work traversing mode of the mobile robot, although the mobile robot can walk around the supporting part of the bottom of the furniture in the process; the first safety depth threshold is larger than the second safety depth threshold, the two safety depth thresholds are used for limiting the mobile robot not to touch the target obstacle in the process of decelerating to zero, the second safety depth threshold is larger than the third safety depth threshold, and the third safety depth threshold can limit the mobile robot to touch the obstacle characterized as a wall body before decelerating to zero. In the embodiment, before the identified target obstacle approaches the visual angle range, the requirements of the matched type obstacle on collision and obstacle avoidance are met by setting different safety distances, so that the area where no obstacle can pass is judged in advance, and an effective obstacle avoidance path is conveniently planned in the follow-up process.
As an embodiment, in the obstacle avoidance module, the method for adjusting the current pose of the mobile robot according to the relative position relationship of the obstacle and the size of the same obstacle, which are transmitted by the obstacle positioning and marking module and trigger the collision warning signal, at least includes: when the type of the target obstacle is a winding object and the obstacle positioning and marking module triggers the collision warning signal, the mobile robot is controlled to reduce the real-time speed and change the current advancing direction according to the contour width of the target obstacle, so that a straight line path in the changed advancing direction avoids the target obstacle, the mobile robot is favorable for directly avoiding the obstacle, and the mobile robot is not required to walk around the winding object and is easy to be clamped under the condition of mistakenly detecting the relative position of the winding object. When the type of the target obstacle is a threshold capable of being crossed or furniture for the mobile robot to pass through, the mobile robot is controlled to keep the current advancing direction, namely the collision warning signal is not required to be triggered, and obstacle avoidance action is not required to be executed, so that the influence degree of the target obstacle on the normal working behavior of the mobile robot is reduced to the minimum. When the type of the target obstacle is an island obstacle and the obstacle positioning and marking module triggers the collision warning signal, the mobile robot is controlled to change the current advancing direction so that the mobile robot starts to walk around the outline of the island obstacle, but the mobile robot cannot collide with the island obstacle. When the type of the target obstacle is a wall and the obstacle positioning and marking module triggers the collision warning signal, the mobile robot is controlled to adjust the optimal edgewise direction so that the mobile robot starts to enter an edgewise mode, but the mobile robot can collide with the wall in the process of walking edgewise. The embodiment is beneficial to optimizing the current pose of the mobile robot, so that the current motion state (including normal linear walking, in-situ rotation, radian rotation and edgewise) of the mobile robot is optimized, and the probability of contact between the mobile robot and the identified type of barrier is reduced.
On the basis of the above embodiment, in the obstacle avoidance module, the method for planning the obstacle avoidance path of the mobile robot according to the relative position relationship of the currently identified type of obstacle transmitted by the obstacle positioning and marking module and the size of the same obstacle, so that the mobile robot avoids the obstacle triggering the collision warning signal, bypasses the obstacle triggering the collision warning signal, or keeps the original pose and continues to walk at least includes: firstly, planning a passable area between the mobile robot and the obstacles according to the relative position relation of the obstacles of the currently identified type and the size of the same obstacle, and planning an obstacle avoidance path of the mobile robot in the passable area; when the contour line of the identified barrier is wider, the area of the planned passable area is larger, the obstacle avoidance path is longer, and otherwise, the area of the planned passable area is smaller; the currently identified type of obstacles transmitted by the obstacle positioning and marking module comprise an obstacle triggering a collision warning signal and other target obstacles in the detection view angle range of the TOF camera.
When the type of the target obstacle is a winding object and the obstacle positioning and marking module triggers the collision warning signal, planning an initial obstacle avoiding direction of the mobile robot according to the outline width of the target obstacle, and simultaneously adjusting the initial obstacle avoiding direction of the mobile robot in real time according to the position coordinates corresponding to each point on the identified outline of the winding object and the position coordinates of other target obstacles so as to avoid touching the identified winding object and keep adjusting the depth data corresponding to each point on the outline of the winding object acquired in real time to be larger than a first safety depth threshold; when the type of the target obstacle is a threshold capable of being crossed or furniture capable of being crossed by the mobile robot, controlling the mobile robot to walk in a straight line along the current advancing direction so as to cross or cross the target obstacle; when the type of the target barrier is an island barrier and the barrier positioning and marking module triggers the collision warning signal, controlling the mobile robot to walk along the extension direction of the outline of the island barrier, avoiding other barriers with recognized or marked coordinate positions, and keeping the depth data corresponding to each point on the outline line of the island barrier collected and recognized in real time larger than a second safety depth threshold so as to realize that the mobile robot walks around the barrier but cannot collide with the island barrier; and when the type of the target barrier is a wall body and the barrier positioning and marking module triggers the collision warning signal, controlling the mobile robot to adjust the optimal edgewise direction and starting to walk along the wall along the optimal edgewise direction. Thereby, the following steps are achieved: can control mobile robot to stride across the threshold when TOF camera discerns the threshold, TOF camera control mobile robot gets into at the bottom of the sofa when discerning the sofa (when the height at the bottom of the sofa is less than the height of machine), TOF camera prohibits mobile robot collision toy but can walk around the barrier when discerning the toy, TOF camera allows mobile robot collision wall body in order to carry out the edgewise walking when discerning the wall, TOF camera forbids mobile robot to stride across winding objects such as electric wire and keep away the barrier with other routes when discerning the electric wire, and then plan out effectual obstacle-avoiding route.
Preferably, the TOF camera is installed in front of the mobile robot, and the optical axis of the TOF camera is arranged obliquely downwards or horizontally relative to the top surface of the robot, so that the brightness value of the target obstacle within the detection visual angle range is effective and the contour line of the target obstacle is complete to meet the depth positioning requirement. Comprehensively considering the distance that the vertical height of the robot and the TOF camera need to detect, the optical axis of the TOF camera is obliquely arranged below the top surface of the mobile robot, the inclination angle of the optical axis of the TOF camera is 10 degrees, and meanwhile, the installation height of the TOF camera is 6.7cm relative to the traveling plane of the mobile robot, which is the optimal installation height and installation angle obtained through testing.
In the foregoing embodiment, the obstacle classification module and the obstacle avoidance module are respectively connected to the obstacle positioning trigger module through a bidirectional communication interface, so as to establish a communication relationship of the transceiving response between signal data. The real-time performance of the depth image information feedback of each obstacle is guaranteed, and the obstacle avoidance path planning efficiency is improved. As shown in fig. 1, the information transmitted by the obstacle classification module to the obstacle location triggering module includes the type of the obstacle identified by the TOF camera, the relative position and the size of the obstacle; the information transmitted to the obstacle avoidance module by the obstacle positioning triggering module comprises collision warning signals, the type of the obstacle acquired and identified by the TOF camera, the relative position of the obstacle and the size of the obstacle.
As an embodiment, the size of the same obstacle includes the height of each position point of the contour line of the corresponding obstacle within a preset horizontal width; the preset horizontal width is the length of a horizontal line segment in a preselected connectable domain corresponding to the surface of the obstacle, horizontal projections of all position points of the contour line of the corresponding obstacle in the preset horizontal width are all on the horizontal line segment, and the selected contour line is not necessarily complete but can cover the upper part or the lower part of the horizontal line segment. The present embodiment is to identify the shape feature of an obstacle and the position point of a representative contour line where reachable feature sampling is effective.
As an embodiment, the method for reselecting the brightness image data matched with the corresponding obstacle output in real time by combining with the TOF camera at least comprises the following steps:
when the obstacle classification module judges that the height of each position point of the contour line of the corresponding obstacle in the preset horizontal width is in the threshold height range, the corresponding obstacle is identified as a threshold capable of being crossed; all the heights in the threshold height range are smaller than the height of the body of the mobile robot, and the mobile robot can climb by all the heights in the threshold height range.
When the obstacle classification module judges that the heights of the lowest position points of the contour lines of the corresponding obstacles in the preset horizontal width are all higher than the body height of the mobile robot, the corresponding obstacles are identified as furniture which can be penetrated by the mobile robot, the contour lines of the obstacles are generally in a door frame shape and a trapezoid shape and can be sofa bottoms, bed bottoms and table and chair bottoms, the gap parts of the bottom shapes of the furniture are higher and wider, so that the height of the gap parts is larger than the body height of the mobile robot, and the width is larger than the body diameter of the mobile robot.
When the obstacle classification module judges that the contour lines of the corresponding obstacles accord with the rectangular characteristic condition and the depth values corresponding to each pixel point on the corresponding depth images are equal, and the heights of the position points of the contour lines in the preset horizontal width are higher than a preset passable height threshold value, identifying the corresponding obstacles as a wall body; because the surface of the wall body is relatively flat, the shape of the contour line of the wall body conforms to the rectangular characteristic, the contour lines are relatively long, the height of the wall body is higher than that of the body of the mobile robot, and the obstacle recognized as the wall body supports collision of the mobile robot, so that the recognition result of the type of the obstacle can be corrected after the mobile robot collides with the obstacle even if the obstacle is mistakenly judged as a hollow part at the bottom of furniture; wherein the preset passable height threshold is higher than the height of the body of the mobile robot.
When the obstacle classification module judges that the variance of the height of each position point of the contour line of the corresponding obstacle in the preset horizontal width meets a first winding condition and/or judges that the variance of the product of the depth of the pixel point of the depth image corresponding to each position point of the contour line of the corresponding obstacle in the preset horizontal width and the gray level of the pixel point of the matched brightness image meets a second winding condition, the corresponding obstacle is identified as a winding; the variance in the winding object judging method can be replaced by variables with statistical significance such as standard deviation, mean square deviation and the like, namely the statistical variables corresponding to the heights of the sampling position points on the contour line of the obstacle can effectively reflect the curve change characteristics of winding objects (such as wires and cables wound together in an indoor environment). When the variance of the height falls within a first preset range, determining that a first winding condition is met; and when the variance of the product falls within a second preset range, determining that the second winding condition is met. It should be noted that the height of these winding objects is relatively small, and generally smaller than the height of the body of the mobile robot, and may fall within the threshold height range, but in combination with the recognition result of the above winding object condition, the mobile robot is controlled not to cross this winding object, so as to avoid the occurrence of erroneous judgment.
When the obstacle classification module judges that the horizontal width of the space occupied by the corresponding obstacle is smaller than the preset island width and the vertical height of the space occupied by the corresponding obstacle is smaller than the preset island height, the corresponding obstacle is identified as an island obstacle; the preset horizontal width is larger than the width of a machine body of the mobile robot, and the width of the preset island is smaller than or equal to the preset horizontal width. In some implementation scenarios, the preset island height is smaller than the body height of the mobile robot, and even falls within the threshold height range, so that the mobile robot can be effectively prevented from crossing the island obstacle by identifying the obstacle with a lower height as the island obstacle rather than misjudging as the threshold, for example, the mobile robot is controlled not to collide and cross a small toy.
Thereby, the following steps are achieved: the lower threshold of high is discerned to the TOF camera, and the TOF camera discerns that sofa (the height is less than the furniture of the height of machine at the bottom of the sofa) control mobile robot gets into at the bottom of the sofa, and the TOF camera discerns the less toy of height, and the TOF camera discerns that the wall gallery, TOF camera are discerned highly little and have the electric wire of curve characteristic. Therefore, the method can effectively detect and identify large obstacles, small obstacles, obstacles capable of crossing and passing through and obstacles prohibited from contacting, and further pre-judge an effective accessible area for the mobile robot to avoid obstacles. And the function integration level of the identification obstacle avoidance algorithm of the obstacle classification obstacle avoidance control system is further improved.
In the above embodiment, when the obstacle classification module determines that the luminance image data matched with a communicable surface area corresponding to the obstacle is within a first preset medium grayscale threshold range, and/or when the obstacle classification module determines that the product of the depth of the pixel point of the depth image corresponding to each position point of a communicable surface area corresponding to the obstacle and the pixel grayscale of the matched luminance image is within a second preset medium grayscale threshold range, the obstacle classification module determines that the surface medium corresponding to the obstacle is a flat planar medium allowing the mobile robot to move without obstacle. According to the technical scheme, the surface medium of the obstacle is judged and recognized by utilizing the brightness information of the surface reflection light of the obstacle and the depth data of the matched pixel points in the depth image, so that more specific obstacle types (such as carpet surfaces, cement floors and wood boards) can be recognized, and a traveling plane allowing the mobile robot to walk across without obstacles can be judged in advance.
According to the characteristics of the depth image collected by the TOF camera, the brightness image data matched with the corresponding obstacle is light brightness information reflected back to the imaging plane of the TOF camera from the surface of the corresponding obstacle, and is matched with the depth of the pixel point of the depth image collected by the TOF camera in a one-to-one correspondence manner. And carrying out obstacle classification judgment by utilizing the matching relation between the features of the depth image and the features of the brightness image, so that the judgment effect of the first winding condition and the second winding condition is reduced, and the use of fitting operation is reduced. The accuracy of obstacle type identification is improved.
It should be understood that the above examples are only for clarity of illustration and are not intended to limit the embodiments. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. And are neither required nor exhaustive of all embodiments. And obvious variations or modifications therefrom are within the scope of the invention.
Claims (10)
1. A barrier classification and obstacle avoidance control system based on a TOF camera is characterized in that the barrier classification and obstacle avoidance control system is installed on a mobile robot and comprises a barrier classification module, a barrier positioning and marking module and an obstacle avoidance module;
the barrier classification module comprises a TOF camera arranged in the advancing direction of the mobile robot and is used for calculating the relative position relation of at least one barrier and the size of the same barrier in the detection visual angle range of the TOF camera according to a depth image acquired by the TOF camera in real time, selecting brightness image data which is output in real time by combining the TOF camera and matched with the corresponding barrier, and identifying the type of the corresponding barrier;
the obstacle positioning triggering module is used for receiving the relative position relation of the obstacles of the type currently identified by the obstacle classification module and the size of the same obstacle, and then deciding whether the obstacle of the corresponding identified type triggers a collision warning signal or not based on the data information so as to enable: the mobile robot plans a passable area before moving to a corresponding barrier;
and the obstacle avoidance module is used for adjusting the current pose of the mobile robot according to the relative position relationship of the obstacles triggering the collision warning signals and the size of the same obstacle, which are transmitted by the obstacle positioning and marking module, and planning an obstacle avoidance path of the mobile robot according to the relative position relationship of the currently recognized type of obstacles and the size of the same obstacle, which are transmitted by the obstacle positioning and marking module, so that the mobile robot avoids the obstacles triggering the collision warning signals, bypasses the obstacles triggering the collision warning signals, or keeps the original pose to continue walking.
2. The obstacle classification and avoidance control system according to claim 1, wherein in the obstacle location triggering module, the method for deciding whether the obstacle of the corresponding recognized type triggers the collision warning signal based on the data information at least comprises:
defining the obstacle of the identified type as a target obstacle; the types of the obstacles comprise winding obstacles, island obstacles, threshold capable of crossing and furniture capable of being crossed by the mobile robot;
when the type of the target obstacle is a winding object, if the depth data of the depth image of the target obstacle shot by the TOF camera is smaller than a first safe depth threshold, controlling the obstacle positioning triggering module to trigger a collision warning signal;
when the type of the target obstacle is an island obstacle, if the depth data of the depth image of the target obstacle shot by the TOF camera is smaller than a second safe depth threshold, controlling the obstacle positioning triggering module to trigger a collision warning signal;
when the type of the target obstacle is a wall body, if the depth data of the depth image of the target obstacle shot by the TOF camera is smaller than a third safe depth threshold, controlling the obstacle positioning triggering module to trigger a collision warning signal;
when the type of the target obstacle is a threshold capable of being crossed or furniture capable of being crossed by the mobile robot, controlling the obstacle positioning triggering module not to trigger a collision warning signal;
the first safe depth threshold is larger than the second safe depth threshold, and the second safe depth threshold is larger than the third safe depth threshold.
3. The obstacle avoidance control system according to claim 2, wherein in the obstacle avoidance module, the method for adjusting the current pose of the mobile robot according to the relative position relationship of the obstacles triggering the collision warning signals transmitted from the obstacle positioning and marking module and the size of the same obstacle at least comprises:
when the type of the target obstacle is a winding object and the obstacle positioning and marking module triggers the collision warning signal, controlling the mobile robot to reduce the real-time speed and change the current advancing direction according to the contour width of the target obstacle so that a straight path in the changed advancing direction avoids the target obstacle;
when the type of the target obstacle is a threshold capable of being crossed or furniture capable of being crossed by the mobile robot, controlling the mobile robot to keep the current advancing direction;
when the type of the target obstacle is an island obstacle and the obstacle positioning and marking module triggers the collision warning signal, controlling the mobile robot to change the current advancing direction so that the mobile robot starts to walk around the outline of the island obstacle;
and when the type of the target barrier is a wall body and the barrier positioning and marking module triggers the collision warning signal, controlling the mobile robot to adjust the optimal edgewise direction so that the mobile robot starts to enter an edgewise mode.
4. The obstacle avoidance control system according to claim 3, wherein in the obstacle avoidance module, the method for planning the obstacle avoidance path of the mobile robot according to the relative position relationship between the currently recognized type of obstacle and the size of the same obstacle, which are transmitted from the obstacle positioning and marking module, at least comprises:
firstly, planning a passable area between the mobile robot and the obstacles according to the relative position relation of the obstacles of the currently identified type and the size of the same obstacle, and planning an obstacle avoidance path of the mobile robot in the passable area; the currently identified type of obstacles transmitted by the obstacle positioning and marking module comprise target obstacles triggering collision warning signals and other target obstacles in a TOF camera detection visual angle range;
when the type of the target obstacle is a winding object and the obstacle positioning and marking module triggers the collision warning signal, planning an initial obstacle avoiding direction of the mobile robot according to the outline width of the target obstacle, and simultaneously adjusting the initial obstacle avoiding direction of the mobile robot in real time according to the position coordinates corresponding to each point on the identified outline of the winding object and the position coordinates of other target obstacles so as to avoid touching the identified winding object and keep adjusting the depth data corresponding to each point on the outline of the winding object acquired in real time to be larger than a first safety depth threshold;
when the type of the target obstacle is a threshold capable of being crossed or furniture capable of being crossed by the mobile robot, controlling the mobile robot to walk in a straight line along the current advancing direction so as to cross or cross the target obstacle;
when the type of the target obstacle is an island obstacle and the obstacle positioning and marking module triggers the collision warning signal, controlling the mobile robot to walk along the extension direction of the outline of the island obstacle, and keeping adjusting depth data corresponding to each point on the outline of the island obstacle collected and identified in real time to be larger than a second safety depth threshold so as to realize obstacle detouring walking of the mobile robot;
and when the type of the target barrier is a wall body and the barrier positioning and marking module triggers the collision warning signal, controlling the mobile robot to adjust the optimal edgewise direction and starting to walk along the wall along the optimal edgewise direction.
5. The obstacle classification and obstacle avoidance control system according to claim 4, wherein the TOF camera is installed in front of the mobile robot, the installation height of the TOF camera relative to a traveling plane is 6.7 centimeters, the optical axis of the TOF camera forms an inclination angle of 10 degrees relative to the top surface of the robot, and driving wheels at the bottom of the mobile robot are in contact with the traveling plane.
6. The obstacle classification and avoidance control system according to claim 5, wherein the obstacle classification module and the obstacle avoidance module are respectively connected to the obstacle positioning and triggering module through bidirectional communication interfaces to establish a communication relationship of the transceiving responses between the signal data.
7. The obstacle classification and obstacle avoidance control system according to any one of claims 2 to 6, wherein the size of the same obstacle includes the height of each position point of the contour line of the corresponding obstacle within a preset horizontal width;
the preset horizontal width is the length of a horizontal line segment in a pre-selected connectable domain corresponding to the surface of the obstacle, and horizontal projections of the contour line of the corresponding obstacle at each position point in the preset horizontal width are all on the horizontal line segment.
8. The obstacle classification obstacle avoidance control system according to claim 7, wherein the method for reselecting the brightness image data matched with the corresponding obstacle and output in real time by combining with the TOF camera at least comprises the following steps:
when the obstacle classification module judges that the height of each position point of the contour line of the corresponding obstacle in the preset horizontal width is in the threshold height range, the corresponding obstacle is identified as a threshold capable of being crossed; all heights in the threshold height range are smaller than the height of the body of the mobile robot;
when the obstacle classification module judges that the heights of the lowest position points of the contour lines of the corresponding obstacles in the preset horizontal width are higher than the height of the body of the mobile robot, the corresponding obstacles are identified as furniture for the mobile robot to pass through;
when the obstacle classification module judges that the contour lines of the corresponding obstacles accord with the rectangular characteristic condition and the depth values corresponding to each pixel point on the corresponding depth images are equal, and the heights of the position points of the contour lines in the preset horizontal width are higher than a preset passable height threshold value, identifying the corresponding obstacles as a wall body; the preset passable height threshold is higher than the height of the body of the mobile robot;
when the obstacle classification module judges that the variance of the height of each position point of the contour line of the corresponding obstacle in the preset horizontal width meets a first winding condition and/or judges that the variance of the product of the depth of the pixel point of the depth image corresponding to each position point of the contour line of the corresponding obstacle in the preset horizontal width and the gray level of the pixel point of the matched brightness image meets a second winding condition, the corresponding obstacle is identified as a winding;
when the obstacle classification module judges that the horizontal width of the space occupied by the corresponding obstacle is smaller than the preset island width and the vertical height of the space occupied by the corresponding obstacle is smaller than the preset island height, the corresponding obstacle is identified as an island obstacle;
the preset horizontal width is larger than the width of the body of the mobile robot; and the preset island height falls into the threshold height range, and the preset island width is smaller than or equal to the preset horizontal width.
9. The obstacle classification obstacle avoidance control system according to claim 8, wherein when the obstacle classification module determines that luminance image data matched with a communicable surface area corresponding to an obstacle is within a first preset medium gray level threshold range, and/or when the obstacle classification module determines that a product of depths of pixel points of a depth image corresponding to each position point of a communicable surface area corresponding to the obstacle and pixel gray levels of a matched luminance image is within a second preset medium gray level threshold range, the obstacle classification module determines that a surface medium corresponding to the obstacle is a planar medium allowing the mobile robot to move without an obstacle.
10. The obstacle classification obstacle avoidance control system according to claim 9, wherein the luminance image data matched with the corresponding obstacle is light luminance information reflected from a surface of the corresponding obstacle back to an imaging plane of the TOF camera, and is matched with depths of pixel points of a depth image of the corresponding obstacle acquired by the TOF camera in a one-to-one correspondence manner.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011159235.7A CN112415998B (en) | 2020-10-26 | 2020-10-26 | Obstacle classification obstacle avoidance control system based on TOF camera |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011159235.7A CN112415998B (en) | 2020-10-26 | 2020-10-26 | Obstacle classification obstacle avoidance control system based on TOF camera |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112415998A true CN112415998A (en) | 2021-02-26 |
| CN112415998B CN112415998B (en) | 2024-06-18 |
Family
ID=74840561
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011159235.7A Active CN112415998B (en) | 2020-10-26 | 2020-10-26 | Obstacle classification obstacle avoidance control system based on TOF camera |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112415998B (en) |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113232585A (en) * | 2021-05-07 | 2021-08-10 | 广州小鹏汽车科技有限公司 | Anti-collision method and device for vehicle door opening, vehicle and storage medium |
| CN113311837A (en) * | 2021-05-26 | 2021-08-27 | 上海十方生态园林股份有限公司 | Unmanned ship obstacle avoidance system based on overwater and underwater environment recognition |
| CN113341978A (en) * | 2021-06-10 | 2021-09-03 | 西北工业大学 | Intelligent trolley path planning method based on ladder-shaped barrier |
| CN113475978A (en) * | 2021-06-23 | 2021-10-08 | 深圳乐动机器人有限公司 | Robot recognition control method and device, robot and storage medium |
| CN113558536A (en) * | 2021-09-24 | 2021-10-29 | 莱克电气绿能科技(苏州)有限公司 | Intelligent calibration method, device and system of intelligent sweeping robot |
| CN113625704A (en) * | 2021-06-30 | 2021-11-09 | 北京旷视科技有限公司 | Obstacle avoidance method and device and automatic navigation device |
| CN114047753A (en) * | 2021-11-03 | 2022-02-15 | 哈尔滨鹏路智能科技有限公司 | Obstacle recognition and avoidance method of sweeping robot based on depth vision |
| CN114077252A (en) * | 2021-11-16 | 2022-02-22 | 中国人民解放军陆军工程大学 | Robot collision obstacle distinguishing device and method |
| CN114089756A (en) * | 2021-11-16 | 2022-02-25 | 合肥瓦力觉启机器人科技有限公司 | Robot obstacle avoidance method and system |
| CN114185356A (en) * | 2022-02-16 | 2022-03-15 | 莱陆科技(天津)有限公司 | Method and device for intelligently identifying obstacles and controlling robot to pass through obstacles |
| CN114831356A (en) * | 2022-07-05 | 2022-08-02 | 深圳市恒尔创科技有限公司 | Electronic cigarette control method based on mobile terminal and related equipment |
| CN114895688A (en) * | 2022-05-31 | 2022-08-12 | 合肥新思路智能科技有限公司 | Mobile robot line planning system and method based on artificial intelligence |
| WO2022213732A1 (en) * | 2021-04-09 | 2022-10-13 | 美智纵横科技有限责任公司 | Method for controlling movement of cleaning robot along obstacle, and cleaning robot |
| CN115525047A (en) * | 2022-03-21 | 2022-12-27 | 江苏集萃清联智控科技有限公司 | Vehicle local track planning method and system with multi-type obstacle avoidance mode |
| WO2023115658A1 (en) * | 2021-12-22 | 2023-06-29 | 广东栗子科技有限公司 | Intelligent obstacle avoidance method and apparatus |
| CN113741467B (en) * | 2021-09-07 | 2023-10-13 | 深圳大方智能科技有限公司 | Vertical wall surface construction method and construction robot |
| CN117249832A (en) * | 2023-10-25 | 2023-12-19 | 深圳市天鹤科技有限公司 | Method, device and storage medium for robot indoor navigation |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE102016011378A1 (en) * | 2016-09-21 | 2017-04-13 | Daimler Ag | Method for self-localization of a vehicle |
| CN107544501A (en) * | 2017-09-22 | 2018-01-05 | 广东科学技术职业学院 | A kind of intelligent robot wisdom traveling control system and its method |
| CN109284348A (en) * | 2018-10-30 | 2019-01-29 | 百度在线网络技术（北京）有限公司 | A kind of update method of electronic map, device, equipment and storage medium |
| CN110605713A (en) * | 2018-06-15 | 2019-12-24 | 科沃斯机器人股份有限公司 | Robot positioning method, robot, and storage medium |
| CN110622085A (en) * | 2019-08-14 | 2019-12-27 | 珊口（深圳）智能科技有限公司 | Mobile robot and control method and control system thereof |
| CN110727270A (en) * | 2019-10-24 | 2020-01-24 | 常州格力博有限公司 | Automatic working system and method for establishing working area control map thereof |
| CN111624997A (en) * | 2020-05-12 | 2020-09-04 | 珠海市一微半导体有限公司 | Robot control method and system based on TOF camera module and robot |
-
2020
- 2020-10-26 CN CN202011159235.7A patent/CN112415998B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE102016011378A1 (en) * | 2016-09-21 | 2017-04-13 | Daimler Ag | Method for self-localization of a vehicle |
| CN107544501A (en) * | 2017-09-22 | 2018-01-05 | 广东科学技术职业学院 | A kind of intelligent robot wisdom traveling control system and its method |
| CN110605713A (en) * | 2018-06-15 | 2019-12-24 | 科沃斯机器人股份有限公司 | Robot positioning method, robot, and storage medium |
| CN109284348A (en) * | 2018-10-30 | 2019-01-29 | 百度在线网络技术（北京）有限公司 | A kind of update method of electronic map, device, equipment and storage medium |
| CN110622085A (en) * | 2019-08-14 | 2019-12-27 | 珊口（深圳）智能科技有限公司 | Mobile robot and control method and control system thereof |
| CN110727270A (en) * | 2019-10-24 | 2020-01-24 | 常州格力博有限公司 | Automatic working system and method for establishing working area control map thereof |
| CN111624997A (en) * | 2020-05-12 | 2020-09-04 | 珠海市一微半导体有限公司 | Robot control method and system based on TOF camera module and robot |
Cited By (23)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022213732A1 (en) * | 2021-04-09 | 2022-10-13 | 美智纵横科技有限责任公司 | Method for controlling movement of cleaning robot along obstacle, and cleaning robot |
| CN113232585A (en) * | 2021-05-07 | 2021-08-10 | 广州小鹏汽车科技有限公司 | Anti-collision method and device for vehicle door opening, vehicle and storage medium |
| CN113232585B (en) * | 2021-05-07 | 2023-02-10 | 广州小鹏汽车科技有限公司 | Anti-collision method and device for vehicle door opening, vehicle and storage medium |
| CN113311837A (en) * | 2021-05-26 | 2021-08-27 | 上海十方生态园林股份有限公司 | Unmanned ship obstacle avoidance system based on overwater and underwater environment recognition |
| CN113341978A (en) * | 2021-06-10 | 2021-09-03 | 西北工业大学 | Intelligent trolley path planning method based on ladder-shaped barrier |
| CN113341978B (en) * | 2021-06-10 | 2023-03-07 | 西北工业大学 | Intelligent trolley path planning method based on ladder-shaped barrier |
| CN113475978A (en) * | 2021-06-23 | 2021-10-08 | 深圳乐动机器人有限公司 | Robot recognition control method and device, robot and storage medium |
| CN113625704A (en) * | 2021-06-30 | 2021-11-09 | 北京旷视科技有限公司 | Obstacle avoidance method and device and automatic navigation device |
| CN113741467B (en) * | 2021-09-07 | 2023-10-13 | 深圳大方智能科技有限公司 | Vertical wall surface construction method and construction robot |
| CN113558536A (en) * | 2021-09-24 | 2021-10-29 | 莱克电气绿能科技(苏州)有限公司 | Intelligent calibration method, device and system of intelligent sweeping robot |
| CN113558536B (en) * | 2021-09-24 | 2022-01-25 | 莱克电气绿能科技(苏州)有限公司 | Intelligent calibration method, device and system of intelligent sweeping robot |
| CN114047753A (en) * | 2021-11-03 | 2022-02-15 | 哈尔滨鹏路智能科技有限公司 | Obstacle recognition and avoidance method of sweeping robot based on depth vision |
| CN114089756A (en) * | 2021-11-16 | 2022-02-25 | 合肥瓦力觉启机器人科技有限公司 | Robot obstacle avoidance method and system |
| CN114089756B (en) * | 2021-11-16 | 2024-04-30 | 合肥瓦力觉启机器人科技有限公司 | Robot obstacle avoidance method and system |
| CN114077252A (en) * | 2021-11-16 | 2022-02-22 | 中国人民解放军陆军工程大学 | Robot collision obstacle distinguishing device and method |
| CN114077252B (en) * | 2021-11-16 | 2023-09-12 | 中国人民解放军陆军工程大学 | Robot collision obstacle distinguishing device and method |
| WO2023115658A1 (en) * | 2021-12-22 | 2023-06-29 | 广东栗子科技有限公司 | Intelligent obstacle avoidance method and apparatus |
| CN114185356A (en) * | 2022-02-16 | 2022-03-15 | 莱陆科技(天津)有限公司 | Method and device for intelligently identifying obstacles and controlling robot to pass through obstacles |
| CN115525047A (en) * | 2022-03-21 | 2022-12-27 | 江苏集萃清联智控科技有限公司 | Vehicle local track planning method and system with multi-type obstacle avoidance mode |
| CN114895688A (en) * | 2022-05-31 | 2022-08-12 | 合肥新思路智能科技有限公司 | Mobile robot line planning system and method based on artificial intelligence |
| CN114831356B (en) * | 2022-07-05 | 2022-11-01 | 深圳市恒尔创科技有限公司 | Electronic cigarette control method based on mobile terminal and related equipment |
| CN114831356A (en) * | 2022-07-05 | 2022-08-02 | 深圳市恒尔创科技有限公司 | Electronic cigarette control method based on mobile terminal and related equipment |
| CN117249832A (en) * | 2023-10-25 | 2023-12-19 | 深圳市天鹤科技有限公司 | Method, device and storage medium for robot indoor navigation |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112415998B (en) | 2024-06-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112415998B (en) | Obstacle classification obstacle avoidance control system based on TOF camera | |
| CN112327878B (en) | Obstacle classification and obstacle avoidance control method based on TOF camera | |
| CN111067439B (en) | Obstacle processing method and cleaning robot | |
| CN107041718B (en) | Cleaning robot and control method thereof | |
| US20200409382A1 (en) | Intelligent cleaning robot | |
| CN112004645A (en) | Intelligent cleaning robot | |
| CN112363513B (en) | Obstacle classification obstacle avoidance control method based on depth information | |
| CN111624997A (en) | Robot control method and system based on TOF camera module and robot | |
| CN110147106A (en) | Has the intelligent Mobile Service robot of laser and vision fusion obstacle avoidance system | |
| EP2903787B1 (en) | Robot management systems for determining docking station pose including mobile robots and methods using same | |
| US8209074B2 (en) | Robot and method for controlling the same | |
| CN108536149A (en) | A kind of automatic driving vehicle avoidance obstacle device and control method based on the paths Dubins | |
| KR100901311B1 (en) | Autonomous mobile platform | |
| CN110554696B (en) | Robot system, robot and robot navigation method based on laser radar | |
| CN112347876B (en) | Obstacle recognition method based on TOF camera and cleaning robot | |
| KR20190123673A (en) | Moving Robot and controlling method for the same | |
| CN112327879A (en) | Edge obstacle avoidance method based on depth information | |
| CN110946508A (en) | Control method and device of sweeping robot using laser radar and camera | |
| CN110088701B (en) | Operating method for a self-propelled cleaning device and such a cleaning device | |
| US11960296B2 (en) | Method and apparatus for autonomous mobile device | |
| CN111602028A (en) | Method for automatically guiding a vehicle along a virtual rail system | |
| CN114052561A (en) | Self-moving robot | |
| Andersen et al. | Traversable terrain classification for outdoor autonomous robots using single 2D laser scans | |
| CN113848944A (en) | Map construction method and device, robot and storage medium | |
| CN112308033B (en) | Obstacle collision warning method based on depth data and visual chip |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 519000 2706, No. 3000, Huandao East Road, Hengqin new area, Zhuhai, Guangdong Applicant after: Zhuhai Yiwei Semiconductor Co.,Ltd. Address before: Room 105-514, No.6 Baohua Road, Hengqin New District, Zhuhai City, Guangdong Province Applicant before: AMICRO SEMICONDUCTOR Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant |