WO2021103987A1

WO2021103987A1 - Control method for sweeping robot, sweeping robot, and storage medium

Info

Publication number: WO2021103987A1
Application number: PCT/CN2020/127238
Authority: WO
Inventors: 杨勇; 吴泽晓
Original assignee: 深圳市杉川机器人有限公司
Priority date: 2019-11-29
Filing date: 2020-11-06
Publication date: 2021-06-03
Also published as: CN110974088B; CN110974088A

Abstract

An automatic control method for a sweeping robot. The method comprises the following steps: obtaining image information of an obstacle during the travel of a sweeping robot; obtaining the category, position and depth information of the obstacle according to the image information of the obstacle; and according to the obstacle category, position and depth information, controlling the sweeping robot to perform an automatic obstacle avoidance operation.

Description

Control method of sweeping robot, sweeping robot and storage medium

This application claims the priority of the Chinese patent application filed on November 29, 2019 with the application number 201911212870.4 and titled "Sweeping Robot Control Method, Sweeping Robot and Storage Medium", which is hereby incorporated by reference in its entirety.

Technical field

This application relates to the field of artificial intelligence technology, and in particular to an automatic control method of a sweeping robot, a sweeping robot, and a computer-readable storage medium.

Background technique

With the continuous development of artificial intelligence technology, a series of artificial intelligence products based on artificial intelligence technology have gradually entered people's lives and provided many conveniences to people's lives. For example, an intelligent sweeping robot can automatically recognize obstacles and perform cleaning tasks on objects to be cleaned by avoiding obstacles, thereby reducing cleaning tasks for users.

In the current sweeping robot solution, when the sweeping robot recognizes obstacles, it usually only recognizes whether there are obstacles, but cannot identify the type of obstacle, and cannot perform corresponding obstacle avoidance operations for different types of obstacles; and when the obstacle is recognized Behind the object, obstacle avoidance is usually carried out by means of distance measurement, that is, obstacle avoidance by measuring the distance between the sweeping robot and the obstacle. However, the method based on distance measurement has the problem that the obstacle cannot be avoided more accurately.

The above content is only used to assist the understanding of the technical solution of the application, and does not mean that the above content is recognized as prior art.

Technical solutions

The main purpose of this application is to provide an automatic control method for a sweeping robot, a sweeping robot, and a computer-readable storage medium, which aims to solve the problem of poor obstacle avoidance effects of the sweeping robot in the prior art.

In order to achieve the above objective, the present application provides an automatic control method for a sweeping robot. The method includes the following steps:

Obtain the image information of the obstacles in the sweeping robot's traveling process;

Acquiring the category, location and depth information of the obstacle according to the image information of the obstacle;

According to the obstacle category, position and depth information, the sweeping robot is controlled to perform an automatic obstacle avoidance operation.

In an embodiment, the step of controlling the sweeping robot to perform an automatic obstacle avoidance operation according to the obstacle category, position and depth information includes:

If the category of the obstacle is the first category, control the sweeping robot to perform an automatic obstacle avoidance operation based on the position and depth information;

If the category of the obstacle is the second category, control the sweeping robot to output alarm information after performing automatic obstacle avoidance operations based on the position and depth information;

If the category of the obstacle is the third category, the sweeping robot is controlled not to perform the obstacle avoidance operation based on the position and depth information.

In an embodiment, if the category of the obstacle is the first category, the step of controlling the sweeping robot to perform an automatic obstacle avoidance operation based on the position and depth information includes:

If the category of the obstacle is the first category, determine the steering distance information, steering direction information, and steering angle information of the sweeping robot according to the position and depth information of the obstacle;

Determining the obstacle avoidance route according to the turning distance information, turning direction information, and turning angle information of the sweeping robot;

Control the sweeping robot to perform an automatic obstacle avoidance operation according to the obstacle avoidance route.

In an embodiment, if the category of the obstacle is the second category, the step of controlling the sweeping robot to output alarm information after performing the automatic obstacle avoidance operation based on the position and depth information includes:

If the category of the obstacle is the second category, control the sweeping robot to perform an automatic obstacle avoidance operation based on the position and depth information;

After the sweeping robot performs the automatic obstacle avoidance operation, control the sweeping robot to output alarm information, and send the category and location of the second type of obstacle and the alarm information to the remote terminal;

Judging whether the processing information sent by the remote terminal is received within a preset time corresponding to the category and location information;

If not received, control the cleaning robot to send alarm information to the remote terminal again until the processing information is received.

In an embodiment, if the category of the obstacle is the third category, the step of controlling the sweeping robot not to perform the obstacle avoidance operation based on the position and depth information includes:

If the category of the obstacle is the third category, judge whether the sweeping robot needs to perform obstacle avoidance operations according to the power parameters and morphological parameters of the sweeping robot;

If it is not required, the sweeping robot is controlled not to perform obstacle avoidance operations based on the position and depth information.

In an embodiment, the step of obtaining the category, position and depth information of the obstacle according to the image information of the obstacle includes:

Using an obstacle detection network to identify the category information and coordinate information of the obstacle in the image information;

Based on the coordinate information, a preset depth camera is used to identify obstacle position information and depth information in the image information.

In an embodiment, the step of using an obstacle detection network to identify the category information and coordinate information of the obstacle in the image information includes:

Input the image information into an obstacle detection network, and determine whether the feature information of the obstacle extracted by the detection network matches the trained obstacle detection model;

If they match, the obstacle category identified by the obstacle detection network and the coordinate information of the obstacle detection frame are acquired.

In an embodiment, after the step of obtaining the obstacle category and the coordinate information of the obstacle detection frame recognized by the obstacle detection network, the method further includes:

Determine the position of the obstacle in the image information collected by the preset depth camera based on the coordinate information of the obstacle detection frame;

Obtain the position and depth information of the obstacle based on the position of the obstacle in the image information collected by the preset depth camera.

In addition, in order to achieve the above-mentioned object, the present application also provides a cleaning robot including a memory, a processor, and a cleaning robot control method program stored on the processor and running on the processor. When the controller executes the program of the cleaning robot control method, the steps of the cleaning robot control method described above are implemented.

In addition, in order to achieve the above object, the present application also provides a computer-readable storage medium, the computer-readable storage medium stores a cleaning robot control method program, when the cleaning robot control method program is executed by a processor, the implementation is as described above. The steps of the sweeping robot control method described.

In the embodiment of the present application, by acquiring the image information of the obstacle during the travel of the sweeping robot, and then acquiring the category, position, and depth information of the obstacle according to the image information of the obstacle, the obstacle category, position, and depth information are obtained according to the obstacle category, position, and depth information. The depth information controls the sweeping robot to perform automatic obstacle avoidance operations. It can not only perform automatic obstacle avoidance operations, but also perform obstacle avoidance operations based on different obstacle categories. It can also perform more effective automatic obstacle avoidance operations based on the position and depth information of the obstacles, thereby improving the automatic obstacle avoidance of the sweeping robot. Efficiency and effectiveness.

Description of the drawings

FIG. 1 is a schematic structural diagram of a sweeping robot in a hardware operating environment involved in a solution of an embodiment of the application;

2 is a schematic flowchart of a first embodiment of a method for controlling a cleaning robot according to the present application;

3 is a schematic flowchart of a second embodiment of a method for controlling a cleaning robot according to the present application;

The realization, functional characteristics, and advantages of the purpose of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings.

Embodiments of the present invention

It should be understood that the specific embodiments described here are only used to explain the present application, and are not used to limit the present application.

The main solution of this application is:

Current sweeping robots cannot recognize obstacle types to perform corresponding obstacle avoidance operations for different types of obstacles, and usually use laser ranging and other methods to determine the distance between the sweeping robot and the obstacle to perform obstacle avoidance operations, and there is no obstacle avoidance effect. Good question. Therefore, this application proposes a cleaning robot control method, a cleaning robot, and a computer-readable storage medium, which can obtain the category, position, and depth information of the obstacle by acquiring the image information of the obstacle during the travel of the cleaning robot. The category, position and depth information of the object controls the sweeping robot to perform automatic obstacle avoidance operations, which can not only identify the obstacle category, but also perform automatic obstacle avoidance based on the position and depth information of the obstacle, which improves the automatic obstacle avoidance effect of the sweeping robot.

Referring to FIG. 1, FIG. 1 is a schematic diagram of the structure of the sweeping robot in the hardware operating environment involved in the solution of the embodiment of the application.

The sweeping robot in the embodiment of the present application may be connected to terminal devices such as PCs, smart phones, and tablet computers, and the connection may be a wireless connection or a wired connection.

As shown in FIG. 1, the cleaning robot may include: a communication bus 1002, a processor 1001, such as a CPU, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to implement connection and communication between these components. The user interface 1003 may include a display screen (Display) and an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a magnetic disk memory. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

In an embodiment, the sweeping robot may further include a camera, an RF (Radio Frequency, radio frequency) circuit, a sensor, an audio circuit, a WiFi module, and so on. Among them, sensors such as light sensors, motion sensors and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display screen according to the brightness of the ambient light, and the proximity sensor can turn off the display screen and/or when the mobile terminal is moved to the ear. Backlight. As a kind of motion sensor, the gravity acceleration sensor can detect the magnitude of acceleration in various directions (usually three-axis), and can detect the magnitude and direction of gravity when it is stationary. It can be used to identify the application of the mobile terminal's posture (such as horizontal and vertical screen switching, Related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, percussion), etc.; of course, the mobile terminal can also be equipped with other sensors such as gyroscope, barometer, hygrometer, thermometer, infrared sensor, temperature sensor, etc. , I won’t repeat it here.

Those skilled in the art can understand that the terminal structure shown in FIG. 1 does not constitute a limitation on the sweeping robot, and may include more or less components than shown in the figure, or combine certain components, or arrange different components.

As shown in FIG. 1, a memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a cleaning robot control method program.

In the sweeping robot shown in FIG. 1, the network interface 1004 is mainly used to connect to the back-end server and communicate with the back-end server; the user interface 1003 is mainly used to connect to the client (user side) and communicate with the client; The device 1001 can be used to call the cleaning robot control method program stored in the memory 1005, and perform the following operations:

In an embodiment, the processor 1001 may call the cleaning robot control method program stored in the memory 1005, and also perform the following operations:

In an embodiment, the processor 1001 may call the cleaning robot control program stored in the memory 1005, and also perform the following operations:

In an embodiment, the processor 1001 calls the cleaning robot control program stored in the memory 1005, and performs the following operations:

In an embodiment, the processor 1001 may call the cleaning robot control program stored in the memory 1005, and also perform the following operations: use an obstacle detection network to identify the category information and coordinate information of the obstacle in the image information;

In an embodiment, the processor 1001 may call a control program of the sweeping robot stored in the memory 1005, and also perform the following operations: input the image information into the obstacle detection network, and determine whether the characteristic information of the obstacle extracted by the detection network is Match with the trained obstacle detection model;

In an embodiment, after the step of obtaining the obstacle category and the coordinate information of the obstacle detection frame recognized by the obstacle detection network, the method further includes that the processor 1001 may call the sweeping robot control program stored in the memory 1005 , And also do the following:

Referring to Fig. 2, Fig. 2 is a flowchart of a first embodiment of a cleaning robot control method according to the present application. In this embodiment, the cleaning robot control method includes the following steps:

Step S10: Obtain image information of obstacles during the traveling of the sweeping robot;

The sweeping robot includes an image acquisition module, the image acquisition module includes a camera, the camera is used to collect real-time environmental image information of the surrounding environment of the sweeping robot in the process of traveling, and the sweeping robot is obtained by judging whether there are obstacles in the environmental image information Image information of obstacles in the process of traveling. The method of judging whether there are obstacles in the environmental image information can use image recognition technology similar to face recognition. By training the obstacle detection network, the collected environmental image information is input into the trained obstacle detection network. When feature extraction is performed on the collected environmental image information, if the extracted feature information can match the trained obstacle detection model, then there are obstacles in the collected environmental image information. If the extracted feature information can match the trained obstacle detection model If the network matches, there are no obstacles in the collected environmental image information. After judging whether there are obstacles in the collected environmental image information, if so, obtain the image information of the obstacles as the image information of the obstacles in the process of the sweeping robot during the travel, and further identify the obstacles through the trained obstacle detection network The category and other related information.

Step S20: Obtain the category, position and depth information of the obstacle according to the image information of the obstacle;

After obtaining the image information of the obstacle, the obstacle detection network is used to identify the category information and coordinate information of the obstacle in the image information, and based on the coordinate information, the preset depth camera is used to identify the obstacle position information and the obstacle position information in the image information. In-depth information. Specifically, the image information of the obstacle is input into the obstacle detection network, and when it is determined that there is an obstacle in the image information collected by the camera, the obstacle category information and the obstacle detection frame recognized by the obstacle detection network are obtained. The coordinate information. Before judging whether there are obstacles in the image information collected by the camera, it is necessary to train the obstacle detection network. In one embodiment, the specific training process is as follows: Collect a large number of pictures of obstacles that need to be avoided (for example: 10,000), Use the segmentation tool to show the outline of the obstacle that needs to be labeled to obtain the labeled image, and save it as a corresponding json format file after labeling. Generate the coordinate information of the four vertices of the corresponding detection frame and the obstacle category information in the annotated image in the saved json format file for the annotated image. In the labeling process, the labeling is increased in order from the time when the object category is 0, until all the recognition categories are labeled. After the annotation is completed, the json format file is converted into an xml format file for training the obstacle detection network, and finally converted into a record file that the obstacle network model can recognize and train. After the format conversion is completed, the converted file is sent to the obstacle detection model for training and the parameters are configured, and the number of training steps is set according to the number of pictures. When the obstacle picture in the file after the training format conversion is set by the parameters until the loss drops to 0.1 and no longer attenuates, the training is stopped. Convert the trained name into a file that can be used for obstacle detection. The training process of the obstacle detection network is essentially a feature extraction process. After the features are extracted, the network is continuously predicted. The predicted value is compared with the real value to optimize the parameters, and finally the ability to accurately predict the object and location information is realized. After the obstacle detection network is trained, the real-time collected image data is input to the obstacle detection network. The obstacle detection network extracts obstacle features in the data image to output information such as object category, vertex coordinates of the detection frame, and confidence. When the confidence level exceeds the preset confidence level, it can be judged that there is an obstacle in the image information collected in real time. When there is an obstacle, the coordinate information and category information of the detection frame corresponding to the obstacle are output through the obstacle detection network. Determine the position of the obstacle in the image information collected by the preset depth camera based on the coordinate information of the obstacle detection frame, so as to obtain the obstacle corresponding to the position according to the position of the obstacle in the image information collected by the preset depth camera In-depth information. Among them, the preset depth camera can be TOF (Time of flight, flight time) camera, it can also be a binocular stereo camera, or a monocular structured light camera and other depth cameras that can obtain object depth information. ToF (Time of flight, flight time) camera as an example, by continuously sending light pulses to the target, and then using the sensor to receive the light returned from the object, by detecting the flight time difference or phase difference between these transmitted and received light pulses to obtain the target distance, and according to the target The object distance generates corresponding depth information. The way to send light pulses to scan to obtain depth information can be point scanning, point-by-point scanning to obtain the three-dimensional geometric structure of the target, or surface scanning, which can be real-time by taking a picture of the scene Obtain the surface geometry information of the entire scene. In this embodiment, the method of surface scanning may be adopted. First, a scene image is collected, and the scene image is sent to the detection network for detection. When there are obstacles in the scene image, the detection information output by the detection network is used. TOF (Time of flight, flight time) The camera further obtains the depth information of the obstacle.

Step S30: Control the sweeping robot to perform an automatic obstacle avoidance operation according to the obstacle category, position and depth information;

According to the obstacle category, position and depth information output by the detection network, the sweeping robot is controlled to automatically perform corresponding obstacle avoidance operations. Before performing the corresponding obstacle avoidance operation, you can first judge whether the obstacle is an obstacle that needs to be avoided based on the obstacle type, position and depth information. If it is an obstacle that does not need to be avoided (such as a low step, etc.), then Continue to perform the original cleaning task along the direction of the current obstacle. If it is an obstacle that needs to be avoided, judge whether the obstacle is an obstacle avoidable type, if it is an obstacle avoidable type, control the sweeping robot Perform automatic obstacle avoidance operation. If the obstacle is an unavoidable obstacle, re-plan the cleaning route and do not perform automatic obstacle avoidance operation; when the obstacle is an obstacle avoidable type, it can be further judged whether the obstacle is The obstacle that needs to be cleaned, if it is an obstacle that needs to be cleaned, it will directly perform the cleaning operation without automatic obstacle avoidance operation, if it is an obstacle that needs to be cleaned, it will judge whether the sweeping robot can clean autonomously, if it can clean autonomously, then Control the sweeping robot to clean autonomously. If it cannot clean, it will control the sweeping robot to receive the cleaning instruction sent by the terminal device to clean. When the cleaning instruction of the terminal device is not received or the cleaning instruction of the terminal device cannot be cleaned or cannot be cleaned according to the cleaning instruction of the terminal device When cleaning, prompt information is sent to the terminal device for timely processing by the terminal device side user. When it is determined that the sweeping robot needs to avoid obstacles and can avoid obstacles, the corresponding obstacle avoidance route can be planned for the sweeping robot according to the position and depth information corresponding to the obstacle category. The planning of obstacle avoidance routes can be based on obstacle avoidance safety, obstacle avoidance time, obstacle avoidance routes, etc., and one of the planned obstacle avoidance routes can be selected to ensure the safety factor of obstacle avoidance and ensure the avoidance of obstacles. After the obstacle is cleared, follow the original cleaning trajectory and the obstacle avoidance route with the shortest turning time when avoiding the obstacle. Of course, multiple planned obstacle avoidance routes can also be sent to the terminal device, and after the terminal device specifies the corresponding obstacle avoidance route, the sweeping robot is controlled to perform the obstacle avoidance operation according to the obstacle avoidance route specified by the user on the terminal device side.

Obtain the image information of the obstacle during the travel of the sweeping robot, obtain the category, position and depth information of the obstacle according to the image information of the obstacle, and then control the sweeping robot to perform automatic execution according to the obstacle category, position and depth information. Obstacle avoidance operations can perform corresponding automatic obstacle avoidance operations for different types of obstacles, and perform automatic obstacle avoidance operations for obstacles with different position and depth information, so as to improve the accuracy of automatic obstacle avoidance and improve automatic avoidance. The efficiency and effectiveness of barriers.

Referring to Fig. 3, Fig. 3 is a flowchart of a second embodiment of a cleaning robot control method according to the present application. In this embodiment, the cleaning robot control method includes the following steps:

Step S11: Obtain image information of obstacles during the traveling of the sweeping robot;

Step S12: Obtain the category, position and depth information of the obstacle according to the image information of the obstacle;

Step S13: If the category of the obstacle is the first category, control the sweeping robot to perform an automatic obstacle avoidance operation based on the position and depth information;

Step S14: if the category of the obstacle is the second category, based on the position and depth information, control the sweeping robot to output alarm information after performing the automatic obstacle avoidance operation;

Step S15: If the category of the obstacle is the third category, control the sweeping robot not to perform obstacle avoidance operations based on the position and depth information.

This embodiment provides a better implementation solution. After acquiring the image information of the obstacle during the traveling of the sweeping robot, the category, position, and depth information of the obstacle are acquired according to the image information of the obstacle. According to the types of obstacles, when the obstacles are of different types, based on different types of obstacles, different obstacle avoidance operations are performed based on the location and depth information of the obstacles. Therefore, before performing different obstacle avoidance operations based on different types of obstacles, it is necessary to classify the obstacles. The classification method can be based on whether the obstacle is an obstacle that needs to be avoided, whether it is an obstacle that can be avoided, whether it is an obstacle that needs to be cleaned, whether it is an obstacle that can be cleaned by a sweeping robot, etc., as classification conditions, Classify obstacles. In this embodiment, a better classification method takes the obstacles that the robot sweeping needs to avoid and can avoid obstacles as the first type of obstacles, and the obstacles that the sweeping robot needs to avoid obstacles and need to be cleaned but cannot be cleaned by the sweeping robot are regarded as the first type of obstacles. Obstacles of the second type, the obstacles that the sweeping robot does not need to avoid obstacles but need to be cleaned and can be cleaned are regarded as the third type of obstacles. After the obstacles are classified, if the category of the obstacle is the first type, the steering distance information, steering direction information, and steering angle information of the sweeping robot are determined according to the position and depth information of the obstacle; according to the sweeping robot The robot steering distance information, steering direction information, and steering angle information determine the obstacle avoidance route; the sweeping robot is controlled to perform automatic obstacle avoidance operations according to the obstacle avoidance route. Among them, according to the position and depth information of the obstacle, the shortest distance of the obstacle relative to the front of the sweeping robot is determined to determine the steering distance information. According to the position and depth information of the obstacle, it is judged whether the area on the left and right sides of the obstacle can be used for the current robot to avoid obstacles. And whether the robot has completed the cleaning of the left and right areas to determine the steering direction information. According to the position and depth information of the obstacle, it will be able to effectively avoid the obstacle and continue to clean along the cleaning route specified by the original cleaning task to determine the best obstacle avoidance angle to determine the steering Angle information. After determining the steering distance information, steering direction information, and steering angle information, the obstacle avoidance route can be planned based on these information, so as to control the sweeping robot to perform automatic obstacle avoidance operations on the planned obstacle avoidance route; if the obstacle category is the first In the second category, the sweeping robot is controlled to perform automatic obstacle avoidance operations based on the position and depth information; after the sweeping robot performs the obstacle avoidance operation, because the second category of obstacles cannot be cleaned, the user on the terminal device side needs to be notified to come To process. At this time, after the sweeping robot performs the automatic obstacle avoidance operation, the sweeping robot is controlled to output alarm information. If the alarm information is output, there are two ways: the first is the way that the sweeping robot outputs in the form of audio through its own alarm , The second way is to output the alarm information to the terminal device. Of course, it can also be a combination of the two methods. When the second method is used, the category and location of the second type of obstacle and the alarm information are sent to the terminal; by recording the time when the sweeping robot sends the alarm information, Determine whether the processing information sent by the terminal is received within the preset time corresponding to the category and location information; if not, control the sweeping robot to send alarm information to the terminal again, and stop sending the alarm until the processing information is received information. If the category of the obstacle is the third category, the sweeping robot is controlled not to perform the obstacle avoidance operation based on the position and depth information. Before controlling the sweeping robot not to perform obstacle avoidance operations based on the position and depth information, it is determined whether the sweeping robot needs to perform obstacle avoidance operations according to the power parameters and morphological parameters of the sweeping robot combined with the position and depth information of the obstacle. Specifically, according to the position and depth information of the obstacle, if the sweeping robot can step up to the current third type of obstacle, the sweeping robot can be judged according to the morphological parameters of the sweeping robot (such as height, whether it has supporting feet for lifting, etc.) Whether the robot can step over the current obstacle and continue to perform the cleaning route of the original cleaning task. After the sweeping robot has worked for a period of time, judge whether the sweeping robot can step over the current obstacle under the current power parameters according to the power parameters of the sweeping robot (such as remaining power, output power, etc.), and continue to perform the cleaning route performed by the original cleaning task Clean up. When the two parameters meet the conditions, that is, it can provide sufficient power and has a form capable of stepping over the third type of obstacle, the sweeping robot can directly step over the third type of obstacle without performing automatic obstacle avoidance operations.

In this embodiment, by acquiring the image information of the obstacle during the travel of the sweeping robot, the category, position, and depth information of the obstacle are acquired according to the image information of the obstacle. If the category of the obstacle is the first type, it is based on The position and depth information controls the sweeping robot to perform automatic obstacle avoidance operations. If the category of the obstacle is the second type, the sweeping robot is controlled based on the position and depth information to output alarm information after performing the automatic obstacle avoidance operation, if If the category of the obstacle is the third category, based on the position and depth information, the sweeping robot is controlled not to perform obstacle avoidance operations. For different types of obstacles, perform corresponding automatic obstacle avoidance operations according to their position and depth information. , It can not only prevent missed scanning, but also alarm and notify the terminal user to deal with it, which can effectively improve the efficiency of obstacle avoidance.

In addition, an embodiment of the present application also provides a cleaning robot. The cleaning robot includes a memory, a processor, and a cleaning robot control program stored on the processor and running on the processor, and the processor executes the The cleaning robot control program implements the steps of the cleaning robot control method as described above.

In addition, an embodiment of the present application also provides a computer-readable storage medium, the computer-readable storage medium stores a cleaning robot control program, and the cleaning robot control program is executed by a processor to realize the above-mentioned cleaning robot control Method steps.

It should be noted that in this article, the term "include" or "include" or any other variants thereof is intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements not only includes those elements, It also includes other elements that are not explicitly listed, or elements inherent to the process, method, article, or system. If there are no more restrictions, the element defined by the sentence "including a..." does not exclude the existence of other identical elements in the process, method, article, or system that includes the element.

The serial numbers of the foregoing embodiments of the present application are for description only, and do not represent the superiority or inferiority of the embodiments.

Through the description of the above embodiments, those skilled in the art can clearly understand that the method of the above embodiments can be implemented by means of software plus the necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is better.的实施方式。 Based on this understanding, the technical solution of this application essentially or the part that contributes to the existing technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium (such as ROM/RAM) as described above. , Magnetic disk, optical disk), including several instructions to make a terminal device (can be a mobile phone, a computer, a server, a TV, or a network device, etc.) execute the method described in each embodiment of the present application.

The above are only the preferred embodiments of this application, and do not limit the scope of this application. Any equivalent structure or equivalent process transformation made using the content of the description and drawings of this application, or directly or indirectly used in other related technical fields , The same reason is included in the scope of patent protection of this application.

Claims

A control method of a sweeping robot, wherein the automatic control method of the sweeping machine includes the following steps:

Obtain the image information of the obstacles in the sweeping robot's traveling process;

Obtain the category, position and depth information of the obstacle according to the image information of the obstacle; and

According to the obstacle category, position and depth information, the sweeping robot is controlled to perform an automatic obstacle avoidance operation.
The control method of a sweeping robot according to claim 1, wherein the step of controlling the sweeping robot to perform an automatic obstacle avoidance operation according to the obstacle category, position and depth information comprises:

If the category of the obstacle is the first category, control the sweeping robot to perform an automatic obstacle avoidance operation based on the position and depth information;

If the category of the obstacle is the second category, control the sweeping robot to output alarm information after performing automatic obstacle avoidance operations based on the position and depth information; and

If the category of the obstacle is the third category, the sweeping robot is controlled not to perform the obstacle avoidance operation based on the position and depth information.
3. The control method of a sweeping robot according to claim 2, wherein if the category of the obstacle is the first category, the step of controlling the sweeping robot to perform an automatic obstacle avoidance operation based on the position and depth information comprises:

If the category of the obstacle is the first category, determine the steering distance information, steering direction information, and steering angle information of the sweeping robot according to the position and depth information of the obstacle;

Determine the obstacle avoidance route according to the turning distance information, turning direction information, and turning angle information of the sweeping robot; and

Control the sweeping robot to perform an automatic obstacle avoidance operation according to the obstacle avoidance route.
The control method of a cleaning robot according to claim 3, wherein if the category of the obstacle is the second type, based on the position and depth information, the cleaning robot is controlled to output alarm information after the automatic obstacle avoidance operation is executed. The steps include:

If the category of the obstacle is the second category, control the sweeping robot to perform an automatic obstacle avoidance operation based on the position and depth information;

After the sweeping robot performs the automatic obstacle avoidance operation, control the sweeping robot to output alarm information, and send the category and location of the second type of obstacle and the alarm information to the remote terminal;

Determine whether the processing information sent by the remote terminal is received within the preset time corresponding to the category and location information; and

If not received, control the cleaning robot to send alarm information to the remote terminal again until the processing information is received.
3. The control method of a sweeping robot according to claim 2, wherein if the category of the obstacle is the third category, the step of controlling the sweeping robot not to perform obstacle avoidance operations based on the position and depth information comprises:

If the category of the obstacle is the third category, judging whether the sweeping robot needs to perform obstacle avoidance operations according to the power parameters and morphological parameters of the sweeping robot; and

If it is not required, the sweeping robot is controlled not to perform obstacle avoidance operations based on the position and depth information.
The control method of a sweeping robot according to claim 1, wherein the step of obtaining the category, position and depth information of the obstacle according to the image information of the obstacle comprises:

Using an obstacle detection network to identify the category information and coordinate information of the obstacle in the image information; and

Based on the coordinate information, a preset depth camera is used to identify obstacle position information and depth information in the image information.
7. The control method of a sweeping robot according to claim 6, wherein the step of using an obstacle detection network to identify the category information and coordinate information of the obstacle in the image information comprises:

Input the image information into an obstacle detection network, and determine whether the feature information of the obstacle extracted by the detection network matches the trained obstacle detection model; and

If they match, the obstacle category identified by the obstacle detection network and the coordinate information of the obstacle detection frame are acquired.
7. The control method of a sweeping robot according to claim 7, wherein after the step of acquiring the obstacle category and the coordinate information of the obstacle detection frame recognized by the obstacle detection network, the method further comprises:

Determine the position of the obstacle in the image information collected by the preset depth camera based on the coordinate information of the obstacle detection frame; and

Obtain the position and depth information of the obstacle based on the position of the obstacle in the image information collected by the preset depth camera.
A sweeping robot, wherein the sweeping robot includes a memory, a processor, and an automatic control program for the sweeping robot stored in the memory and running on the processor, and the processor implements the claims when the automatic control program for the sweeping robot is executed Steps of the automatic control method for a cleaning robot described in any one of 1-8.
A computer-readable storage medium, wherein an automatic control program of a cleaning robot is stored on the computer-readable storage medium, and the automatic control program of the cleaning robot is executed by a processor as described in any one of claims 1-8. The steps of the automatic control method of the sweeping robot described.