WO2023115658A1

WO2023115658A1 - Intelligent obstacle avoidance method and apparatus

Info

Publication number: WO2023115658A1
Application number: PCT/CN2022/070836
Authority: WO
Inventors: 陈小平; 王云华; 杨旭; 罗韬
Original assignee: 广东栗子科技有限公司
Priority date: 2021-12-22
Filing date: 2022-01-07
Publication date: 2023-06-29
Also published as: CN116416518A

Abstract

An intelligent obstacle avoidance method and apparatus, and an intelligent sweeping device. The method comprises: during a movement process of an intelligent sweeping device, determining a target movement area of the intelligent sweeping device (101); collecting area information of the target movement area (102); determining whether there is a target obstacle in the target moving area according to the area information (103); when a determination result is yes, determining the position and other information of the target obstacle (104); and controlling, according to the position and the other information of the target obstacle, the intelligent sweeping device to perform obstacle avoidance (105), wherein the other information comprises size information and category information. By means of the method, area information on an advancing path can be collected during an advancing process of an intelligent sweeping device, and an obstacle on the path is intelligently identified, so as to realize accurate obstacle avoidance with respect to the obstacle, thereby improving the sweeping efficiency of the intelligent sweeping device; moreover, a corresponding obstacle avoidance strategy is matched by means of accurately identifying the category of the obstacle, thereby further improving the obstacle avoidance precision, and increasing the coverage of a sweeping surface to the greatest possible extent.

Description

Method and device for avoiding obstacles intelligently

technical field

The invention relates to the technical field of intelligent equipment, in particular to a method and device for intelligently avoiding obstacles, and intelligent cleaning equipment.

Background technique

With the continuous development of artificial intelligence technology, artificial intelligence products have gradually entered people's lives, providing many conveniences for people's lives, such as intelligent sweeping robots, which can automatically clean areas and reduce cleaning tasks for users.

In practical applications, the intelligent sweeping robot mainly cleans according to a preset path or a random path. However, it has been found in practice that the existing intelligent sweeping robots often bump into obstacles when they encounter obstacles during the cleaning process, resulting in reduced cleaning efficiency, and even being blocked by obstacles and unable to continue cleaning. It can be seen that how to provide a technical solution for intelligently identifying obstacles to improve the cleaning efficiency of intelligent cleaning equipment is particularly important.

Contents of the invention

The invention provides a method and device for intelligently avoiding obstacles, and intelligent cleaning equipment, which can collect area information on the traveling path during the traveling process of the intelligent cleaning equipment, intelligently identify obstacles on the path, and then realize accurate detection of obstacles. Obstacle avoidance improves the cleaning efficiency of intelligent cleaning equipment. At the same time, by accurately identifying the types of obstacles and matching corresponding obstacle avoidance strategies, the accuracy of obstacle avoidance is further improved and the coverage of the cleaning surface is maximized.

In order to solve the above technical problems, the first aspect of the present invention discloses a method for intelligently avoiding obstacles, the method comprising:

During the moving process of the intelligent cleaning equipment, determining the target moving area of the intelligent cleaning equipment;

collecting area information of the target moving area, where the area information includes image information and/or infrared information;

judging according to the area information whether there is a target obstacle in the target movement area that hinders the movement of the intelligent cleaning device;

When the judgment result is yes, determine the position of the target obstacle and other information, and control the intelligent cleaning device to avoid obstacles according to the position of the target obstacle and the other information, the other information includes size information and category information.

As an optional implementation manner, in the first aspect of the present invention, the determining the position of the target obstacle and other information includes:

determining the position of the target obstacle according to the area information;

determining the target image set of the target obstacle, and based on a preset image detection model, performing feature extraction on the target image set to obtain a feature detection result of the target obstacle, and using the feature detection result as the other information of the target obstacle, the feature detection result includes target category information of the target obstacle and target size information of the target obstacle;

Wherein, the determination of the target image set of the target obstacle includes:

According to the real-time position of the intelligent cleaning device and the position of the target obstacle, it is judged whether the distance between the intelligent cleaning device and the target obstacle is greater than or equal to a first preset distance, and when the judgment result is yes, acquiring a first image of the target obstacle, where the first image is used to identify category information of the target obstacle;

Judging whether the distance between the intelligent cleaning device and the target obstacle is less than the first preset distance and greater than or equal to a second preset distance, and when the judgment result is yes, obtain the second distance of the target obstacle An image, the second image includes depth of field information for detecting three-dimensional size information of the target obstacle;

The first image and the second image are determined as a target image set of the target obstacle.

As an optional implementation manner, in the first aspect of the present invention, the feature extraction of the target image set based on the preset image detection model to obtain the feature detection result of the target obstacle includes:

Inputting the first image into the image recognition branch of the preset image detection model, performing an image recognition operation on the first image to obtain a feature vector corresponding to the target obstacle in the first image; according to the target The feature vector corresponding to the obstacle is used to determine the category probability set corresponding to the target obstacle; the category with the highest probability value is determined from the category probability set as the target category information of the target obstacle;

determining an initial size model of the target obstacle according to the category information of the target obstacle;

Input the second image into the image detection branch of the image detection model to obtain the depth information of the target obstacle in the second image; reconstruct the initial size model according to the depth information to obtain the The target size information of the target obstacle;

The target category information and the target size information of the target obstacle are determined as the feature detection result of the target obstacle.

As an optional implementation manner, in the first aspect of the present invention, the reconstruction of the initial size model according to the depth information to obtain the target size information of the target obstacle includes:

According to the depth of field information, determine the depth of field features of all pixels corresponding to the target obstacle in the second image;

Transforming the depth features of all the pixels to determine the target depth features of all the pixels and the set of target pixels that need to be reconstructed in the initial size model; wherein, the Transforming the depth of field features, including: mapping the depth of field features of all the pixels into the initial size model;

According to the target depth features of all the pixels, the set of target pixels is reconstructed to obtain the target size information of the target obstacle.

As an optional implementation manner, in the first aspect of the present invention, the controlling the intelligent cleaning device to avoid obstacles according to the position of the target obstacle and the other information includes:

determining a target obstacle avoidance path according to characteristic information of the target obstacle and predetermined map information;

determining an obstacle avoidance control mode that matches the target category information according to the target category information of the target obstacle;

controlling the intelligent cleaning device to avoid obstacles according to the target obstacle avoidance path and the obstacle avoidance control mode;

Wherein, according to the target category information of the target obstacle, determining an obstacle avoidance control method that matches the target category information includes:

Judging whether the target category belongs to the first category, when the judgment result is yes, controlling the intelligent cleaning device to perform an automatic obstacle avoidance operation, the first category is used to represent obstacles that need to be avoided and cannot be cleaned;

Judging whether the target category information belongs to the second category, when the judgment result is yes, controlling the intelligent cleaning device to perform an automatic obstacle avoidance operation and output a warning message, the second category is used to indicate that the category needs to be avoided, cleaned and For obstacles that do not belong to the cleaning category of the intelligent cleaning device, the warning information is used to prompt the target personnel to clean the target obstacle;

Judging whether the target category information belongs to the third category, when the judgment result is yes, controlling the intelligent cleaning device not to perform avoidance operation, the third category is used to indicate that the category needs to be cleaned and belongs to the cleaning of the intelligent cleaning device Category-wide obstacles.

As an optional implementation manner, in the first aspect of the present invention, the determining the target obstacle avoidance path according to the characteristic information of the target obstacle and the predetermined map information includes:

Updating predetermined map information to generate target map information according to characteristic information of the target obstacle;

determining a target obstacle avoidance path of the intelligent cleaning device according to the position of the target obstacle and the target map information;

Wherein, the updating of the predetermined map information to generate the target map information according to the characteristic information of the target obstacle includes:

judging whether the target obstacle is a moving obstacle according to the characteristic information of the target obstacle;

When the judgment result is yes, determine the estimated moving position of the target obstacle according to the moving direction and moving speed of the target obstacle;

According to the estimated moving position of the target obstacle, the predetermined map information is dynamically updated to generate target map information.

As an optional implementation manner, in the first aspect of the present invention, the controlling the intelligent cleaning device to perform an automatic obstacle avoidance operation includes:

calculating the steering distance information, steering direction information, and steering angle information of the intelligent cleaning device according to the target size information of the target obstacle;

determining a target steering path according to the steering direction information and the steering angle information;

The intelligent cleaning device is controlled to perform an automatic obstacle avoidance operation according to the target steering path.

As an optional implementation manner, in the first aspect of the present invention, the calculation of the steering distance information, steering direction information, and steering angle information of the intelligent cleaning device according to the target size information of the target obstacle includes :

According to the target size information of the target obstacle, the top position of the intelligent cleaning device and the bottom position of the intelligent cleaning device, an edge pixel sequence of the target obstacle is extracted, the edge pixel sequence includes the target size A collection of all pixel points within the absolute height range between the apex of the intelligent cleaning device and the bottom point of the intelligent cleaning device in the information;

Calculating the horizontal distance between each edge pixel in the edge pixel sequence and the intelligent cleaning device to obtain a set of horizontal distances corresponding to all the edge pixels;

Selecting the edge pixel point with the smallest horizontal distance from the horizontal distance set as the target obstacle avoidance pixel point;

Calculate the steering distance information, steering direction information, and steering angle information of the intelligent cleaning device according to the target obstacle avoidance pixel.

The second aspect of the present invention discloses an intelligent obstacle avoidance device, which includes:

A first determination module, configured to determine a target moving area of the intelligent cleaning device during the movement of the intelligent cleaning device;

A collection module, configured to collect area information of the target moving area, where the area information includes image information and/or infrared information;

A judging module, configured to judge, according to the area information, whether there is a target obstacle in the target moving area that hinders the movement of the intelligent cleaning device;

A second determination module, configured to determine the position of the target obstacle and other information when the determination result of the determination module is yes;

A control module, configured to control the intelligent cleaning device to avoid obstacles according to the position of the target obstacle and the other information, the other information including size information and category information.

As an optional implementation manner, in the second aspect of the present invention, the second determination module includes:

A first determining submodule, configured to determine the position of the target obstacle according to the area information;

The second determining submodule is used to determine the target image set of the target obstacle, and based on a preset image detection model, perform feature extraction on the target image set to obtain a feature detection result of the target obstacle, and The feature detection result is used as other information of the target obstacle, and the feature detection result includes target category information of the target obstacle and target size information of the target obstacle;

Wherein, the specific manner in which the second determining submodule determines the target image set of the target obstacle is:

As an optional implementation, in the second aspect of the present invention, the second determining submodule extracts features from the target image set based on a preset image detection model to obtain the features of the target obstacle The specific methods of the test results are as follows:

As an optional implementation manner, in the second aspect of the present invention, the second determination submodule reconstructs the initial size model according to the depth information to obtain the target size information of the target obstacle. The specific way is:

As an optional implementation manner, in the second aspect of the present invention, the control module includes:

A third determining submodule, configured to determine an obstacle avoidance control method that matches the target category information according to the target category information of the target obstacle;

The control submodule controls the intelligent cleaning device to avoid obstacles according to the position of the target obstacle and the obstacle avoidance control mode;

The control submodule is specifically used for:

As an optional implementation manner, in the second aspect of the present invention, the specific manner in which the control submodule controls the intelligent cleaning device to perform an automatic obstacle avoidance operation is as follows:

As an optional implementation, in the second aspect of the present invention, the control submodule calculates the steering distance information, steering direction information, and steering angle of the intelligent cleaning device according to the target size information of the target obstacle The specific way of information is:

The third aspect of the present invention discloses an intelligent cleaning device, which includes:

a memory storing executable program code;

a processor coupled to the memory;

The processor invokes the executable program code stored in the memory to implement some or all of the steps in any intelligent obstacle avoidance method disclosed in the first aspect of the present invention.

The fourth aspect of the present invention discloses another intelligent obstacle avoidance device, which includes:

a memory storing executable program code;

a processor coupled to the memory;

The processor invokes the executable program code stored in the memory to execute some or all of the steps in any intelligent obstacle avoidance method disclosed in the first aspect of the present invention.

The fifth aspect of the present invention discloses a computer storage medium, the computer storage medium stores computer instructions, and when the computer instructions are invoked, it is used to execute any intelligent obstacle avoidance method disclosed in the first aspect of the present invention Some or all of the steps in .

Compared with the prior art, the present invention has the following beneficial effects:

The invention discloses a method and device for intelligently avoiding obstacles, and intelligent cleaning equipment. The method includes: determining the target moving area of the intelligent cleaning equipment during the moving process of the intelligent cleaning equipment; collecting area information of the target moving area; information to judge whether there is a target obstacle in the target moving area; when the judgment result is yes, determine the position of the target obstacle and other information, and control the intelligent cleaning device to avoid obstacles according to the position of the target obstacle and other information, other information includes size information and category information. It can be seen that the present invention can collect area information on the traveling path during the traveling process of the intelligent cleaning equipment, intelligently identify obstacles on the path, and then realize accurate obstacle avoidance of obstacles, improve the cleaning efficiency of the intelligent cleaning equipment, and at the same time pass accurate Identify the types of obstacles and match the corresponding obstacle avoidance strategies to further improve the accuracy of obstacle avoidance and maximize the coverage of the cleaning surface.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings that need to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those skilled in the art, other drawings can also be obtained based on these drawings without creative effort.

FIG. 1 is a schematic flowchart of a method for intelligently avoiding obstacles disclosed in an embodiment of the present invention;

Fig. 2 is a schematic structural diagram of an intelligent obstacle avoidance device disclosed in an embodiment of the present invention;

Fig. 3 is a schematic structural diagram of another intelligent obstacle avoidance device disclosed in an embodiment of the present invention;

Fig. 4 is a schematic structural diagram of another intelligent obstacle avoidance device disclosed in an embodiment of the present invention.

Detailed ways

In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only It is a part of embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

The terms "first", "second" and the like in the description and claims of the present invention and the above drawings are used to distinguish different objects, rather than to describe a specific order. Furthermore, the terms "include" and "have", as well as any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, device, product, or terminal comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally further includes For other steps or units inherent in these processes, methods, products or ends.

Reference herein to an "embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present invention. The occurrences of this phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is understood explicitly and implicitly by those skilled in the art that the embodiments described herein can be combined with other embodiments.

The invention discloses a method and device for intelligently avoiding obstacles, and intelligent cleaning equipment, which can collect area information on the traveling path during the traveling process of the intelligent cleaning equipment, such as image information and/or infrared information, and intelligently identify obstacles on the path Objects, thereby realizing accurate obstacle avoidance and improving the cleaning efficiency of intelligent cleaning equipment. At the same time, by accurately identifying the types of obstacles and matching corresponding obstacle avoidance strategies to meet different cleaning needs, the accuracy of obstacle avoidance is further improved, and the maximum Maximize the coverage of the cleaning surface. Each will be described in detail below.

Embodiment one

Please refer to FIG. 1 . FIG. 1 is a schematic flowchart of a method for intelligently avoiding obstacles disclosed in an embodiment of the present invention. Wherein, the method described in FIG. 1 can be applied to an automatic control device for intelligent obstacle avoidance, and the automatic control device for intelligent obstacle avoidance can include intelligent cleaning equipment or a server corresponding to the intelligent cleaning equipment, wherein the intelligent cleaning equipment can communicate with the user Terminals and/or other smart devices in the current area (such as: smart display screens, smart image acquisition devices, etc.) (Android mobile phone, iOS mobile phone, etc.), the embodiments of the present invention are not limited. As shown in Figure 1, the method for avoiding obstacles intelligently may include the following operations:

101. During the moving process of the smart cleaning device, determine the target moving area of the smart cleaning device.

In the embodiment of the present invention, the target movement area of the intelligent cleaning device may be the travel area determined according to the preset movement rules or cruising rules during the cleaning process or the cruising process, for example, the intelligent cleaning device follows the preset paper clip The route scans the map of the room, and the area corresponding to its forward direction is the above-mentioned target moving area; it can also be the target moving area analyzed based on the range of the cleaning area obtained from the mobile terminal. After the area to be cleaned is delineated on the virtual map in the APP, the area range is sent to the smart cleaning device, and the smart cleaning device can determine the target moving area corresponding to the area range according to the received area range; it can also be The target moving area analyzed by images or voice commands, for example, if the user sends a voice command of "sweeping robot, please clean around the dining table", the intelligent cleaning device can determine the range within 0.5m around the dining table according to the preset cleaning range area as its target mobile area.

102. Collect area information of the target moving area.

In the embodiment of the present invention, after the target moving area of the intelligent cleaning device is determined, the area information of the target moving area needs to be collected, including visual recognition information, distance measurement information, such as image information, infrared information, ultrasonic feedback One or more combinations of information, etc. The present invention can be acquired through the acquisition device installed by the intelligent cleaning equipment itself, such as a monocular camera, a binocular camera, an infrared sensor, an ultrasonic sensor, etc., or can be sent by receiving other intelligent acquisition equipment that can communicate with the intelligent cleaning equipment For example, the smart camera in the room sends the captured image to the smart cleaning device, etc., which is not limited in this embodiment of the present invention.

103. According to the area information, it is judged whether there is a target obstacle in the target moving area that hinders the movement of the intelligent cleaning device.

In the embodiment of the present invention, through the area information collected in step 102, a preliminary judgment can be made on whether there are obstacles that hinder the movement of the intelligent cleaning device in the target moving area. For example, by performing object recognition on the image information, the identified The object is matched with the obstacles in the database. If the match is successful, it can be determined that there is a target obstacle in the target moving area that hinders the movement of the intelligent cleaning device.

104. When the judgment result is yes, determine the position of the target obstacle and other information.

In the embodiment of the present invention, after it is determined that there is a target obstacle in the target moving area that hinders the movement of the intelligent cleaning device, the position of the target obstacle and other information are determined to perform obstacle avoidance control on the intelligent cleaning device. Among them, other information includes category information (bed, dining table, desk, water cup, garbage bag, orange peel, etc.) and size information (two-dimensional size information, three-dimensional size information, etc.) of the target obstacle.

105. Control the intelligent cleaning equipment to avoid obstacles according to the position of the target obstacle and other information.

In the embodiment of the present invention, during the traveling process of the intelligent cleaning device, according to its real-time position, the position information of the target obstacle and other information, the obstacle avoidance points during the traveling process are calculated, dynamic detection and route planning are maintained, and then the steering motor is controlled. Adjust with the traveling motor to control the intelligent cleaning equipment for obstacle avoidance.

It can be seen that the method described in the embodiment of the present invention can collect area information on the travel path of the intelligent cleaning device, such as image information and/or infrared information, and intelligently identify obstacles on the path, thereby realizing accurate detection of obstacles. Obstacle avoidance improves the cleaning efficiency of intelligent cleaning equipment. At the same time, by accurately identifying the type of obstacle and matching the corresponding obstacle avoidance strategy, it can meet different cleaning needs, further improve the accuracy of obstacle avoidance, and maximize the coverage of the cleaning surface.

In an optional embodiment, determining the position of the target obstacle and other information includes:

Determine the position of the target obstacle according to the area information;

Determine the target image set of the target obstacle, and based on the preset image detection model, perform feature extraction on the target image set to obtain the feature detection result of the target obstacle, and use the feature detection result as other information of the target obstacle, and the feature detection result Including the target category information of the target obstacle and the target size information of the target obstacle;

Among them, the target image set of the target obstacle is determined, including:

According to the real-time position of the intelligent cleaning device and the position of the target obstacle, determine whether the distance between the intelligent cleaning device and the target obstacle is greater than or equal to the first preset distance, and when the judgment result is yes, obtain the first image of the target obstacle , the first image is used to identify the category information of the target obstacle;

Judging whether the distance between the intelligent cleaning device and the target obstacle is less than the first preset distance and greater than or equal to the second preset distance, when the judgment result is yes, acquiring a second image of the target obstacle, the second image includes depth of field information , used to detect the three-dimensional size information of the target obstacle;

In the embodiment of the present invention, the position of the target obstacle can be determined according to the collected area information of the target moving area. For example, the intelligent cleaning device continuously emits infrared or ultrasonic signals during its movement, and by receiving the infrared or ultrasonic signals fed back by the target obstacle in real time, the distance between it and the target obstacle can be calculated, and then obtained The location of the target obstacle. At the same time, the intelligent cleaning device can collect the target image set of the target obstacle, and perform feature extraction on the images in the target image set through the pre-trained image detection model to obtain the feature detection result of the target obstacle. Wherein, the target image set may include one image to be detected, or may include multiple images to be detected, which is not limited in this embodiment of the present invention. It should be noted that the feature detection result is the above-mentioned other information, and the feature detection result may include target category information and target size information of the target obstacle.

In this optional embodiment, during the travel of the smart cleaning device, two images of the target obstacle are collected as the target image set according to the real-time position of the smart cleaning device and the position of the target obstacle. Among them, considering that when the intelligent cleaning device is far away from the target obstacle, the field of view of the collected image is larger, and the overall recognition of the target obstacle is better, so when judging that the distance between the intelligent cleaning device and the target obstacle is greater than or equal to At the first preset distance (for example, at 0.5m), an image for identifying the category of the target obstacle is collected. The resolution of the images collected at this time can be low-resolution, which reduces the proportion of computing power of the intelligent cleaning equipment and facilitates quick identification of category information. It should be noted that the present invention does not limit the first preset distance to be fixed, and the first preset distance can be adjusted according to the size of the target obstacle in the image field of view, such as the proportion of the target obstacle in the entire image When it is close to 70%, set the first preset distance to 0.7m; when the proportion of the target obstacle in the entire image is close to 50%, set the first preset distance to 0.5m. Correspondingly, considering that when the intelligent cleaning device is closer to the target obstacle, the details of the collected images are clearer and more accurate, so when judging that the distance between the intelligent cleaning device and the target obstacle is less than the first preset distance and greater than When it is equal to the second preset distance, an image for detecting the three-dimensional size information of the target obstacle is collected, and the image should include field depth information. For example, when the distance between the smart cleaning device and the target obstacle is 0.2m, the 3D ToF camera can be used to capture an image including depth of field information. At the same time, the smart cleaning device can also be equipped with a pan/tilt to control the 3D ToF camera to scan up and down , to obtain a more accurate image.

It can be seen that the method described in the embodiment of the present invention can quickly identify the type of the target obstacle through the first image collected at a relatively long distance, and measure the size information of the target obstacle through the second image collected at a relatively short distance. It is beneficial to improve the identification and detection efficiency of target obstacles, and improve the cleaning efficiency of intelligent cleaning equipment.

In another optional embodiment, based on a preset image detection model, feature extraction is performed on the target image set to obtain a feature detection result of the target obstacle, including:

Input the first image into the image recognition branch of the preset image detection model, perform image recognition operation on the first image, and obtain the feature vector corresponding to the target obstacle in the first image; determine the target obstacle according to the feature vector corresponding to the target obstacle The category probability set corresponding to the object; the category with the highest probability value is determined from the category probability set as the target category information of the target obstacle;

Determine the initial size model of the target obstacle according to the category information of the target obstacle;

Input the second image into the image detection branch of the image detection model to obtain the depth information of the target obstacle in the second image; reconstruct the initial size model according to the depth information to obtain the target size information of the target obstacle;

The target category information and target size information of the target obstacle are determined as the feature detection result of the target obstacle.

In the embodiment of the present invention, the image detection model is a deep neural network model obtained based on forward sample training, and the first image is input into the image recognition branch of the image detection model, and processed through convolution, pooling, activation layer and fully connected layer , the feature vector corresponding to the first image can be obtained. The feature vector is matched with the reference feature vector of each category reference obstacle image in the preset database to obtain the category probability set corresponding to each category reference obstacle of the target obstacle, and the probability value is selected from the category probability set The highest category is used as the target category information of the target obstacle. Furthermore, the initial size model of the target obstacle can be determined from the preset size model database according to the category information of the target obstacle. For example, when it is detected that the target obstacle is a basketball, it can be found from the preset size model database that the model matching the basketball is spherical.

In this optional embodiment, the second image is input into the image detection branch of the image detection model, through the analysis of the image detection branch, the field depth information of each pixel in the second image can be obtained, and then the field depth information reflected according to the field depth information The distance information between the intelligent cleaning device and the target obstacle is reconstructed from the initial size model determined above, and the target size information of the target obstacle can be obtained. For example, if the detected target obstacle is a basketball, it can be calculated from the depth of field information that the circumference of the basketball is 75 cm and the diameter is 24.6 cm.

The method described in the embodiment of the present invention can quickly identify the category of the target obstacle through the first image collected at a relatively long distance, accurately identify the category while reducing the consumption of computing power, and calculate it through the second image collected at a relatively short distance The size information of target obstacles and accurate identification of three-dimensional size details are conducive to improving the identification and detection efficiency of target obstacles, further improving the accuracy of obstacle avoidance of intelligent cleaning equipment, and improving the cleaning efficiency of intelligent cleaning equipment.

In another optional embodiment, the initial size model is reconstructed according to the depth information to obtain the target size information of the target obstacle, including:

Determining the depth of field features of all pixels corresponding to the target obstacle in the second image according to the depth of field information;

Transforming the depth of field features of all the pixels to determine the target depth of field features of all pixels and the set of target pixels to be reconstructed in the initial size model; wherein, the depth of field features of all the pixels are Performing a conversion operation, including: mapping the depth features of all the pixels into the initial size model;

According to the target depth features of all pixels, the target pixel set is reconstructed to obtain target size information of the target obstacle.

In the embodiment of the present invention, due to the limitation of the shooting distance or field of view of the intelligent cleaning device, it may not be possible to capture a complete image of the target obstacle, so it is necessary to reconstruct the initial size model according to the depth of field information to obtain the target obstacle more accurate size information. Through the field depth information of the second image, the field depth features of all pixels in the image area corresponding to the target obstacle in the second image can be obtained. If the recognized target obstacle is a basketball, only the image corresponding to the basketball in the second image is extracted. Depth of field characteristics of image regions. According to the size information reflected by these depth features, the set of target pixel points to be reconstructed in the initial size model is determined. For example, the depth of field information scanned by the intelligent cleaning device only includes 1/4 spherical surface of the entire basketball. Correspondingly, the pixel at the corresponding 1/4 spherical surface position in the initial size model is the target pixel point to be reconstructed. These depth features can be mapped to the determined initial size model, and the size of the image model composed of the target pixels to be reconstructed is reconstructed to obtain the target size information of the target obstacle.

It can be seen that the method described in the embodiment of the present invention can reconstruct the size model of the target obstacle through the depth of field information, so as to obtain the accurate visual size information of the target obstacle, and improve the precision and accuracy of the intelligent cleaning equipment for the obstacle measurement , which is conducive to improving the obstacle avoidance precision and accuracy of intelligent cleaning equipment, and further improving the cleaning efficiency of intelligent cleaning equipment.

In yet another optional embodiment, controlling the intelligent cleaning device to avoid obstacles according to the position of the target obstacle and other information includes:

Determine the target obstacle avoidance path according to the characteristic information of the target obstacle and the predetermined map information;

According to the target category information of the target obstacle, determine the obstacle avoidance control mode that matches the target category information;

According to the target obstacle avoidance path and obstacle avoidance control method, control the intelligent cleaning equipment to avoid obstacles;

Among them, according to the target category information of the target obstacle, the obstacle avoidance control method matching the target category information is determined, including:

Determine whether the target category information belongs to the second category. When the judgment result is yes, control the intelligent cleaning device to perform automatic obstacle avoidance operation and output a warning message. The second category is used to indicate that the category needs to avoid, clean and does not belong to intelligent cleaning equipment cleaning. For obstacles in the category range, the warning message is used to prompt the target personnel to clean the target obstacle;

It is judged whether the target category information belongs to the third category, and when the judgment result is yes, the intelligent cleaning device is controlled not to perform an avoidance operation, and the third category is used to represent obstacles that need to be cleaned and belong to the cleaning category range of the intelligent cleaning device.

In the embodiment of the present invention, the feature information (such as location information, category information, size information, etc.) of the target obstacle acquired by the intelligent cleaning device is firstly fused with the acquired map data information to obtain the fused map data Information, and then based on the fused map data information, the target obstacle avoidance path of the intelligent cleaning equipment can be generated. Through the target category information of the target obstacle, the control mode matching the target category information can be determined from the preset obstacle avoidance control modes, and different obstacle avoidance controls can be implemented for different target obstacles. Among them, the present invention provides three solutions. The obstacles that the intelligent cleaning equipment needs to avoid and cannot be cleaned are taken as the first category, such as shoe cabinets, beds and other items; the intelligent cleaning equipment needs to be avoided and cleaned and does not belong to intelligent cleaning equipment. Obstacles within the scope of cleaning categories are taken as the second category, such as slippers, children’s toys, thread balls, animal excrement, etc.; obstacles that need to be cleaned by smart cleaning equipment and belong to the scope of cleaning categories of intelligent cleaning equipment are taken as the third category, such as paper crumbs, hair, etc. After classifying the target obstacle, if it is judged as an item of the first category, control the intelligent cleaning device to perform automatic obstacle avoidance operation; if it is judged as an item of the second category, control the intelligent cleaning device to perform automatic obstacle avoidance operation and output The warning message is used to remind the user that the detected items need to be cleaned manually. It should be noted that the way to output the warning information can be to generate a warning voice prompt, or to send the information of the item (category information, location information, etc.) to other smart devices (smart phones, smart display screens, etc.) to remind The user is not limited in this embodiment of the present invention; if it is judged to be an item of the third category, the avoidance operation is not performed, and the item is directly cleaned across the item.

It can be seen that the method described in the embodiment of the present invention matches different obstacle avoidance strategies through different types of detected target obstacles, greatly enhances the intelligence level of intelligent cleaning equipment, and improves the refined control degree of intelligent cleaning equipment. It solves the problem of insufficient cleaning surface coverage caused by lack of obstacle avoidance strategies, and further improves the cleaning quality and cleaning efficiency of intelligent cleaning equipment.

In another optional embodiment, the target obstacle avoidance path is determined according to the characteristic information of the target obstacle and the predetermined map information, including:

According to the feature information of the target obstacle, update the predetermined map information to generate the target map information;

According to the position of the target obstacle and the target map information, determine the target obstacle avoidance path of the intelligent cleaning equipment;

Wherein, according to the feature information of the target obstacle, the predetermined map information is updated to generate the target map information, including:

According to the feature information of the target obstacle, it is judged whether the target obstacle is a moving obstacle;

According to the estimated moving position of the target obstacle, the predetermined map information is dynamically updated to generate the target map information.

In the embodiment of the present invention, it should be noted that the target obstacle avoidance path may be a global obstacle avoidance path (for example, if the target area selected by the user is the entire room, then the obstacle avoidance path generated for the entire room is the global area obstacle avoidance path) , may also be a local obstacle avoidance path (for example, if the target area specified by the user is the area around the dining table, then the generated obstacle avoidance path within the area around the dining table is the local obstacle avoidance path), which is not limited in this embodiment of the present invention. According to the characteristic information of the target obstacle, such as size information, the coordinate and scale conversion of the size information is carried out to obtain the corresponding coordinates and size of the target obstacle on the predetermined map, and then the target obstacle can be compared with the map The information is fused to obtain the fused map information, that is, the above-mentioned target map information. Further, it may also be judged whether it is necessary to update the target map information in real time based on the category information of the target obstacle or the like. For example, when it is judged that the target obstacle is a puppy, it is a moving obstacle, and the estimated moving position of the puppy can be calculated according to the monitored moving speed and direction of the puppy, and at the same time according to the estimated moving The location and the pre-determined map information are dynamically updated to dynamically generate real-time target map information. Furthermore, the target obstacle avoidance route of the intelligent cleaning device can be planned according to the position of existing obstacles and the position of the target obstacle in the target map information.

It can be seen that the method described in the embodiment of the present invention can combine the characteristic information of the target obstacle with the acquired map data information while matching different control methods through the characteristic information of the target obstacle for global path planning Or local path planning, which improves the efficiency of path planning and tracking, and further improves the intelligent obstacle avoidance level and obstacle avoidance efficiency of intelligent cleaning equipment.

In yet another optional embodiment, controlling the intelligent cleaning equipment to perform automatic obstacle avoidance operations includes:

According to the target size information of the target obstacle, calculate the steering distance information, steering direction information and steering angle information of the intelligent cleaning equipment;

Determine the target steering path according to the steering distance information, steering direction information and steering angle information;

Control the intelligent cleaning equipment to perform automatic obstacle avoidance operations according to the target steering path.

In the embodiment of the present invention, according to the target size information of the target obstacle and the position of the smart cleaning device, the shortest distance of the target obstacle relative to the front of the smart cleaning device is determined as the turning distance information, and at the same time, according to the target size information of the target obstacle , to determine whether the left and right sides of the target obstacle can be turned by the intelligent cleaning device to avoid obstacles, so as to determine the steering direction information and steering angle information. After determining the steering distance information, steering direction information, and steering angle information, the target steering path is planned according to these information, so as to control the intelligent cleaning equipment to perform automatic obstacle avoidance operations with the planned target steering path.

Further, the steering distance information can also be corrected according to the detected category information of the target obstacle. For the above-mentioned first category of items, the steering and obstacle avoidance should be performed as close as possible to the target obstacle. For the second category of items above, turn and avoid obstacles as far as possible from the target obstacle. For example, if the detected target obstacle is a bed, on the premise of avoiding collisions, in order to achieve a better cleaning effect and a larger cleaning area, obstacle avoidance can be performed at a distance of 3 cm from the edge of the bed; if the detected target obstacle Objects are wire balls. In order to avoid the wire balls from being entangled with the smart cleaning equipment, obstacle avoidance can be performed at a distance of 10cm from the edge of the wire balls.

It can be seen that the method described in the embodiment of the present invention can determine accurate steering distance information, steering direction information, and steering angle information through the target size information of the target obstacle, improve the accuracy of determining the target steering path, and further improve the intelligent cleaning equipment. Obstacle avoidance efficiency and accuracy are further improved to improve the cleaning quality and cleaning efficiency of intelligent cleaning equipment.

In yet another optional embodiment, the steering distance information, steering direction information, and steering angle information of the intelligent cleaning device are calculated according to the target size information of the target obstacle, including:

According to the target size information of the target obstacle, the top position of the intelligent cleaning device, and the bottom position of the intelligent cleaning device, the edge pixel sequence of the target obstacle is extracted, and the edge pixel sequence includes the vertices of the intelligent cleaning device in the target size information and the intelligent cleaning device The collection of all pixel points within the absolute height range between the bottom points of ;

Calculate the horizontal distance between each edge pixel in the edge pixel sequence and the intelligent cleaning device, and obtain the set of horizontal distances corresponding to all edge pixels;

Select the edge pixel with the smallest horizontal distance from the horizontal distance set as the target obstacle avoidance pixel;

Calculate the steering distance information, steering direction information and steering angle information of the intelligent cleaning device according to the target obstacle avoidance pixel.

In the embodiment of the present invention, the edge pixel sequence of the target obstacle is extracted according to the target size information of the target obstacle, the top position of the intelligent cleaning device, and the bottom position of the intelligent cleaning device, wherein the extracted edge pixel sequence includes the target size information located at A collection of all pixels within the absolute height range between the vertex of the smart cleaning device and the bottom point of the smart cleaning device. For example, if the absolute height range between the apex of the smart cleaning device and the bottom point of the smart cleaning device is 10cm, then the bottom point of the smart cleaning device is taken as the lowest point of the horizontal position, and the height range in the target size information intercepted from this position is All pixels within 10cm are used as the edge pixel sequence of the target obstacle. From the horizontal distance set composed of the horizontal distance between each edge pixel point and the intelligent cleaning device, the edge pixel point with the smallest horizontal distance is selected as the target obstacle avoidance pixel point. Furthermore, steering distance information, steering direction information, and steering angle information are calculated according to the target pixel.

Further, when the intelligent cleaning device travels along the edge of the target obstacle to avoid obstacles, the target steering path can be divided into multiple steering path nodes according to the steering direction and steering angle, and each time the intelligent cleaning device travels to a steering path node , the operation of calculating the steering distance information, steering direction information, and steering angle information of the intelligent cleaning device according to the target size information of the target obstacle is retriggered, so as to maintain dynamic survey and path planning during the traveling process to achieve Dynamically corrects the purpose of the target steering path.

It can be seen that the method described in the embodiment of the present invention can save the computing power of the intelligent cleaning equipment to calculate the target obstacle avoidance point by extracting the edge pixel sequence, improve the precision and accuracy of measuring the target obstacle avoidance point, and improve the rapid calculation of the intelligent cleaning equipment Low space and other gaps can avoid collision damage at the bottom of objects, thereby improving the obstacle avoidance efficiency and accuracy of intelligent cleaning equipment, and further improving the cleaning quality and cleaning efficiency of intelligent cleaning equipment.

Embodiment two

Please refer to FIG. 2 . FIG. 2 is a schematic structural diagram of an intelligent obstacle avoidance device disclosed in an embodiment of the present invention. Wherein, the device for intelligently avoiding obstacles may include an intelligent cleaning device or a server corresponding to the intelligent cleaning device, wherein the intelligent cleaning device may communicate with the user terminal and/or other intelligent devices in the current area (such as: a smart display screen, a smart image acquisition device, etc.) etc.), where the user terminal includes but is not limited to smart wearable devices (such as smart bracelets, etc.) and/or smart phones (Android phones, iOS phones, etc.), which are not limited in this embodiment of the present invention. It should be noted that the intelligent obstacle avoidance device refers to the steps in the intelligent obstacle avoidance method described in Embodiment 1, and the detailed description will not be repeated in this embodiment, as shown in Figure 2 As shown, the device for intelligently avoiding obstacles may include:

The first determination module 201 is used to determine the target movement area of the intelligent cleaning equipment during the movement of the intelligent cleaning equipment;

A collection module 202, configured to collect area information of the target moving area, where the area information includes image information and/or infrared information;

A judging module 203, configured to judge according to the area information whether there is a target obstacle in the target moving area that hinders the movement of the intelligent cleaning device;

The second determination module 204 is used to determine the position of the target obstacle and other information when the determination result of the determination module 203 is yes;

The control module 205 is configured to control the intelligent cleaning device to avoid obstacles according to the position of the target obstacle and other information, and the other information includes size information and category information.

It can be seen that the device described in the embodiment of the present invention can collect area information on the travel path, such as image information and/or infrared information, intelligently identify obstacles on the path during the travel of the intelligent cleaning device, and then realize accurate detection of obstacles. Obstacle avoidance improves the cleaning efficiency of intelligent cleaning equipment. At the same time, by accurately identifying the type of obstacle and matching the corresponding obstacle avoidance strategy, it can meet different cleaning needs, further improve the accuracy of obstacle avoidance, and maximize the coverage of the cleaning surface.

In an optional embodiment, as shown in FIG. 3, the second determination module 204 may include:

The first determining submodule 2041 is configured to determine the position of the target obstacle according to the area information;

The second determination sub-module 2042 is configured to determine a target image set of the target obstacle, and perform feature extraction on the target image set based on a preset image detection model to obtain a feature detection result of the target obstacle, Using the feature detection result as other information of the target obstacle, the feature detection result includes target category information of the target obstacle and target size information of the target obstacle;

Wherein, the second determination sub-module 2042 determines the target image set of the target obstacle in a specific manner as follows:

It can be seen that the device described in the embodiment of the present invention can quickly identify the type of the target obstacle through the first image collected at a relatively long distance, and measure the size information of the target obstacle through the second image collected at a relatively short distance. It is beneficial to improve the identification and detection efficiency of target obstacles, and improve the cleaning efficiency of intelligent cleaning equipment.

In another optional embodiment, as shown in FIG. 3 , the second determining submodule 2042 performs feature extraction on the target image set based on a preset image detection model to obtain a feature detection result of the target obstacle. The specific way is:

It can be seen that the device described in the embodiment of the present invention can quickly identify the category of the target obstacle through the first image collected at a relatively long distance, accurately identify the category while reducing computing power consumption, and use the second image collected at a relatively short distance To measure the size information of the target obstacle and accurately identify the three-dimensional size details, it is beneficial to improve the identification and detection efficiency of the target obstacle, further improve the accuracy of the obstacle avoidance of the intelligent cleaning equipment, and improve the cleaning efficiency of the intelligent cleaning equipment.

In yet another optional embodiment, as shown in FIG. 3 , the second determination submodule 2042 reconstructs the initial size model according to the depth information to obtain the target size information of the target obstacle. for:

It can be seen that the device described in the embodiment of the present invention can reconstruct the size model of the target obstacle through the depth of field information, so as to obtain the accurate visual size information of the target obstacle, and improve the precision and accuracy of the intelligent cleaning equipment for the obstacle measurement , which is conducive to improving the obstacle avoidance precision and accuracy of intelligent cleaning equipment, and further improving the cleaning efficiency of intelligent cleaning equipment.

In yet another optional embodiment, as shown in Figure 3, the control module may include:

The third determination sub-module 2051 is configured to determine an obstacle avoidance control method that matches the target category information according to the target category information of the target obstacle;

The control sub-module 2052 controls the intelligent cleaning device to avoid obstacles according to the position of the target obstacle and the obstacle avoidance control mode;

The control submodule 2052 is specifically used for:

Judging whether the target category information belongs to the third category, when the judgment result is yes, controlling the intelligent cleaning device not to perform avoidance operation, the third category is used to indicate that the category needs to be cleaned and belongs to the cleaning of the intelligent cleaning device Category-wide obstacles. .

It can be seen that the device described in the embodiment of the present invention can match different obstacle avoidance strategies through different types of detected target obstacles, greatly enhance the intelligence level of intelligent cleaning equipment, and improve the refined control degree of intelligent cleaning equipment , solve the problem of insufficient coverage of the cleaning surface caused by lack of obstacle avoidance strategies, and further improve the cleaning quality and cleaning efficiency of intelligent cleaning equipment.

In yet another optional embodiment, as shown in FIG. 3 , the specific manner in which the control submodule 2052 controls the intelligent cleaning device to perform an automatic obstacle avoidance operation is as follows:

It can be seen that the device described in the embodiment of the present invention can determine accurate steering distance information, steering direction information, and steering angle information through the target size information of the target obstacle, improve the accuracy of determining the target steering path, and further improve the intelligent cleaning equipment. Obstacle avoidance efficiency and accuracy are further improved to improve the cleaning quality and cleaning efficiency of intelligent cleaning equipment.

In yet another optional embodiment, as shown in FIG. 3 , the control submodule 2052 calculates the specific method of steering distance information, steering direction information, and steering angle information of the intelligent cleaning device according to the target size information of the target obstacle. for:

Screen out the edge pixel point with the smallest horizontal distance as the target obstacle avoidance pixel point from the horizontal distance set;

It can be seen that the device described in the embodiment of the present invention can save the computing power of the intelligent cleaning equipment to calculate the target obstacle avoidance point by extracting the edge pixel sequence, improve the precision and accuracy of measuring the target obstacle avoidance point, and improve the rapid calculation of the intelligent cleaning equipment Low space and other gaps can avoid collision damage at the bottom of objects, thereby improving the obstacle avoidance efficiency and accuracy of intelligent cleaning equipment, and further improving the cleaning quality and cleaning efficiency of intelligent cleaning equipment.

Embodiment three

Please refer to FIG. 4 . FIG. 4 is a schematic structural diagram of another intelligent obstacle avoidance device disclosed in an embodiment of the present invention. Wherein, the device for intelligently avoiding obstacles may include an intelligent cleaning device or a server corresponding to the intelligent cleaning device, wherein the intelligent cleaning device may communicate with the user terminal and/or other intelligent devices in the current area (such as: a smart display screen, a smart image acquisition device, etc.) etc.), where the user terminal includes but is not limited to smart wearable devices (such as smart bracelets, etc.) and/or smart phones (Android phones, iOS phones, etc.), which are not limited in this embodiment of the present invention. As shown in Figure 5, the device for avoiding obstacles intelligently may include:

A memory 301 storing executable program codes;

a processor 302 coupled to the memory 301;

The processor 302 invokes the executable program code stored in the memory 302 to execute some or all of the steps in the intelligent obstacle avoidance method disclosed in Embodiment 1 of the present invention.

Embodiment four

The embodiment of the present invention discloses a computer storage medium. The computer storage medium stores computer instructions. When the computer instructions are invoked, they are used to execute the steps in the intelligent obstacle avoidance method disclosed in the first embodiment of the present invention.

Embodiment five

The embodiment of the present invention discloses an intelligent cleaning device, wherein the intelligent cleaning device may include an intelligent obstacle avoidance device, and is used to implement some or all of the steps in the intelligent obstacle avoidance method described in FIG. 1 . Optionally, the intelligent obstacle avoidance device may be the intelligent obstacle avoidance device described in any one of FIGS. 2-4 , which is not limited in this embodiment of the present invention.

The device embodiments described above are only illustrative, and the modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in One place, or it can be distributed to multiple network modules. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. It can be understood and implemented by those skilled in the art without any creative efforts.

Through the specific description of the above embodiments, those skilled in the art can clearly understand that each implementation manner can be implemented by means of software plus a necessary general-purpose hardware platform, and of course also by hardware. Based on this understanding, the above-mentioned technical solution essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), programmable read-only memory (Programmable Read-only Memory, PROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, EPROM ), One-time Programmable Read-Only Memory (OTPROM), Electronically Erasable Programmable Read-Only Memory (EEPROM), Compact Disc Read -Only Memory, CD-ROM) or other optical disk storage, magnetic disk storage, tape storage, or any other computer-readable medium that can be used to carry or store data.

It should be noted that the computer program codes required for the operation of each part of this manual can be written in any one or more programming languages, including object-oriented programming languages such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB .NET, Python, etc., conventional programming languages such as C language, Visual Basic, Fortran2003, Perl, COBOL 2002, PHP, ABAP, dynamic programming languages such as Python, Ruby and Groovy, or other programming languages. The program code can run entirely on the computer (PC, embedded smart device, etc.), or as an independent software package on the user's computer, or partly on the user's computer and partly on the remote computer, or completely on the remote computer or run on the server. In the latter case, the remote computer can be connected to the user computer through any form of network, such as a local area network (LAN) or wide area network (WAN), or to an external computer (such as through the Internet), or in a cloud computing environment, or as a service Use software as a service (SaaS).

Finally, it should be noted that: a method and device for intelligently avoiding obstacles disclosed in the embodiments of the present invention, and intelligent cleaning equipment disclosed are only preferred embodiments of the present invention, and are only used to illustrate the technical solutions of the present invention. It is not limited thereto; although the present invention has been described in detail with reference to the aforementioned embodiments, those of ordinary skill in the art should understand; it can still modify the technical solutions described in the aforementioned embodiments, or modify some of the technical features thereof. Equivalent replacements are carried out; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

A method for intelligently avoiding obstacles, characterized in that the method comprises:

During the moving process of the intelligent cleaning equipment, determining the target moving area of the intelligent cleaning equipment;

collecting area information of the target moving area, where the area information includes image information and/or infrared information;

judging according to the area information whether there is a target obstacle in the target movement area that hinders the movement of the intelligent cleaning device;

When the judgment result is yes, determine the position of the target obstacle and other information, and control the intelligent cleaning device to avoid obstacles according to the position of the target obstacle and the other information, the other information includes size information and category information.
The method for avoiding obstacles intelligently according to claim 1, wherein said determining the position of said target obstacle and other information includes:

determining the position of the target obstacle according to the area information;

determining the target image set of the target obstacle, and based on a preset image detection model, performing feature extraction on the target image set to obtain a feature detection result of the target obstacle, and using the feature detection result as the other information of the target obstacle, the feature detection result includes target category information of the target obstacle and target size information of the target obstacle;

Wherein, the determination of the target image set of the target obstacle includes:

According to the real-time position of the intelligent cleaning device and the position of the target obstacle, it is judged whether the distance between the intelligent cleaning device and the target obstacle is greater than or equal to a first preset distance, and when the judgment result is yes, acquiring a first image of the target obstacle, where the first image is used to identify category information of the target obstacle;

Judging whether the distance between the intelligent cleaning device and the target obstacle is less than the first preset distance and greater than or equal to a second preset distance, and when the judgment result is yes, obtain the second distance of the target obstacle An image, the second image includes depth of field information for detecting three-dimensional size information of the target obstacle;

The first image and the second image are determined as a target image set of the target obstacle.
The method for intelligently avoiding obstacles according to claim 2, wherein the feature extraction is performed on the target image set based on the preset image detection model, and the feature detection result of the target obstacle is obtained, including :

Inputting the first image into the image recognition branch of the preset image detection model, performing an image recognition operation on the first image to obtain a feature vector corresponding to the target obstacle in the first image; according to the target The feature vector corresponding to the obstacle is used to determine the category probability set corresponding to the target obstacle; the category with the highest probability value is determined from the category probability set as the target category information of the target obstacle;

determining an initial size model of the target obstacle according to the category information of the target obstacle;

Input the second image into the image detection branch of the image detection model to obtain the depth information of the target obstacle in the second image; reconstruct the initial size model according to the depth information to obtain the The target size information of the target obstacle;

The target category information and the target size information of the target obstacle are determined as the feature detection result of the target obstacle.
The method for intelligently avoiding obstacles according to claim 3, wherein the reconstruction of the initial size model according to the depth information to obtain the target size information of the target obstacle includes:

According to the depth of field information, determine the depth of field features of all pixels corresponding to the target obstacle in the second image;

Transforming the depth features of all the pixels to determine the target depth features of all the pixels and the set of target pixels that need to be reconstructed in the initial size model; wherein, the Transforming the depth of field features, including: mapping the depth of field features of all the pixels into the initial size model;

According to the target depth features of all the pixels, the set of target pixels is reconstructed to obtain the target size information of the target obstacle.
The method for intelligent obstacle avoidance according to any one of claims 1-4, wherein the controlling the intelligent cleaning device to avoid obstacles according to the position of the target obstacle and the other information includes:

determining a target obstacle avoidance path according to characteristic information of the target obstacle and predetermined map information;

According to the target category information of the target obstacle, determine an obstacle avoidance control mode that matches the target category information;

controlling the intelligent cleaning device to avoid obstacles according to the target obstacle avoidance path and the obstacle avoidance control mode;

Wherein, according to the target category information of the target obstacle, determining an obstacle avoidance control method that matches the target category information includes:

Judging whether the target category belongs to the first category, when the judgment result is yes, controlling the intelligent cleaning device to perform an automatic obstacle avoidance operation, the first category is used to represent obstacles that need to be avoided and cannot be cleaned;

Judging whether the target category information belongs to the second category, when the judgment result is yes, controlling the intelligent cleaning device to perform an automatic obstacle avoidance operation and output a warning message, the second category is used to indicate that the category needs to be avoided, cleaned and For obstacles that do not belong to the cleaning category of the intelligent cleaning device, the warning information is used to prompt the target personnel to clean the target obstacle;

Judging whether the target category information belongs to the third category, when the judgment result is yes, controlling the intelligent cleaning device not to perform avoidance operation, the third category is used to indicate that the category needs to be cleaned and belongs to the cleaning of the intelligent cleaning device Category-wide obstacles.
The method for intelligent obstacle avoidance according to claim 5, wherein said determining the target obstacle avoidance path according to the feature information of the target obstacle and the predetermined map information comprises:

Updating predetermined map information to generate target map information according to characteristic information of the target obstacle;

determining a target obstacle avoidance path of the intelligent cleaning device according to the position of the target obstacle and the target map information;

Wherein, the updating of the predetermined map information to generate the target map information according to the feature information of the target obstacle includes:

judging whether the target obstacle is a moving obstacle according to the characteristic information of the target obstacle;

When the judgment result is yes, determine the estimated moving position of the target obstacle according to the moving direction and moving speed of the target obstacle;

According to the estimated moving position of the target obstacle, the predetermined map information is dynamically updated to generate target map information.
The method for avoiding obstacles intelligently according to claim 5 or 6, wherein said controlling said intelligent cleaning equipment to perform an automatic obstacle avoidance operation comprises:

calculating the steering distance information, steering direction information, and steering angle information of the intelligent cleaning device according to the target size information of the target obstacle;

determining a target steering path according to the steering direction information and the steering angle information;

The intelligent cleaning device is controlled to perform an automatic obstacle avoidance operation according to the target steering path.
The method for intelligently avoiding obstacles according to claim 7, characterized in that, according to the target size information of the target obstacle, calculating the steering distance information, steering direction information and steering angle information of the intelligent cleaning device, include:

According to the target size information of the target obstacle, the top position of the intelligent cleaning device and the bottom position of the intelligent cleaning device, an edge pixel sequence of the target obstacle is extracted, the edge pixel sequence includes the target size A collection of all pixel points within the absolute height range between the apex of the intelligent cleaning device and the bottom point of the intelligent cleaning device in the information;

Calculating the horizontal distance between each edge pixel in the edge pixel sequence and the intelligent cleaning device to obtain a set of horizontal distances corresponding to all the edge pixels;

Selecting the edge pixel point with the smallest horizontal distance from the horizontal distance set as the target obstacle avoidance pixel point;

Calculate the steering distance information, steering direction information, and steering angle information of the intelligent cleaning device according to the target obstacle avoidance pixel.
A device for intelligently avoiding obstacles, characterized in that the device includes:

A first determination module, configured to determine a target moving area of the intelligent cleaning device during the movement of the intelligent cleaning device;

A collection module, configured to collect area information of the target moving area, where the area information includes image information and/or infrared information;

A judging module, configured to judge, according to the area information, whether there is a target obstacle in the target moving area that hinders the movement of the intelligent cleaning device;

A second determination module, configured to determine the position of the target obstacle and other information when the determination result of the determination module is yes;

A control module, configured to control the intelligent cleaning device to avoid obstacles according to the position of the target obstacle and the other information, the other information including size information and category information.
An intelligent cleaning device, characterized in that the intelligent cleaning device comprises:

a memory storing executable program code;

a processor coupled to the memory;

The processor invokes the executable program code stored in the memory, so as to realize the intelligent obstacle avoidance method according to any one of claims 1-8.