CN113534820A

CN113534820A - Method and device for motion obstacle avoidance and route control of sweeping robot and sweeping robot

Info

Publication number: CN113534820A
Application number: CN202111077265.8A
Authority: CN
Inventors: 汪洋
Original assignee: Shenzhen Yuanding Intelligent Innovation Co ltd
Current assignee: Shenzhen Yuanding Intelligent Innovation Co ltd
Priority date: 2021-09-14
Filing date: 2021-09-14
Publication date: 2021-10-22
Anticipated expiration: 2041-09-14
Also published as: CN113534820B

Abstract

The embodiment of the invention discloses a method and a device for motion obstacle avoidance and route control of a sweeping robot and the sweeping robot, wherein the method comprises the following steps: controlling the sweeping robot to move based on the first planned route; detecting whether an obstacle exists or not, and if so, controlling one or more sensors to acquire sensor data of the obstacle; determining obstacle parameters corresponding to the obstacles according to the sensor data, wherein the obstacle parameters comprise the size, the area, the type, the position information and the like of the obstacles; determining the influence of the obstacles on the first planned route, wherein the influence comprises the relative position of the obstacles in the map and a closed route in the first planned route based on the obstacles, carrying out route optimization on the part of the route which does not move again to obtain a second planned route, and controlling the sweeping robot to move according to the second planned route to avoid the obstacles. By adopting the invention, the obstacle avoidance accuracy of the sweeping robot can be improved.

Description

Method and device for motion obstacle avoidance and route control of sweeping robot and sweeping robot

Technical Field

The invention relates to the technical field of sweeping robots, in particular to a method and a device for motion obstacle avoidance and route control of a sweeping robot and the sweeping robot.

Background

The floor sweeping robot is also called an automatic cleaner, intelligent dust collection, a robot dust collector and the like, is one of intelligent household appliances, and can automatically complete floor cleaning work in a room by means of certain artificial intelligence. In the existing sweeping robot scheme, when a sweeping robot identifies an obstacle, the robot usually only identifies whether the obstacle exists, but cannot further identify the size and the type of the obstacle, and cannot perform corresponding obstacle avoidance operation for different obstacles, so that the obstacle cannot be completely avoided or an area around the obstacle cannot be swept.

Disclosure of Invention

In view of the above, it is necessary to provide a method and a device for obstacle avoidance and route control of a sweeping robot, and a sweeping robot.

In a first aspect of the present invention, there is provided a method for obstacle avoidance and route control of a sweeping robot, including:

determining a map of the work of the sweeping robot and a first planned route, and controlling the sweeping robot to move based on the first planned route;

detecting whether an obstacle exists or not through one or more sensors arranged on the sweeping robot, and controlling the sweeping robot to move around the detected obstacle and controlling the one or more sensors to acquire sensor data of the obstacle under the condition that the obstacle is detected;

determining obstacle parameters corresponding to the obstacles according to the sensor data, wherein the obstacle parameters comprise the size, the area, the type, the position information and the like of the obstacles;

determining the influence of the obstacle on the first planned route based on a map of the work of the sweeping robot, the first planned route and obstacle parameters corresponding to the obstacle, wherein the influence comprises the relative position of the obstacle in the map and a closed route in the first planned route based on the obstacle;

and based on the relative position of the obstacle in the map, the first planned route and the closed route, carrying out route optimization on the part of the route which does not move again to obtain a second planned route, and controlling the sweeping robot to move according to the second planned route, so that the sweeping robot can avoid the area of the obstacle and can finish sweeping the area corresponding to the map in the moving process.

Optionally, in the case that an obstacle is detected, the step of controlling the sweeping robot to move around the detected obstacle and controlling the one or more sensors to collect sensor data of the obstacle further includes:

determining a distance value between the sweeping robot and the detected obstacle;

determining a detection route according to the distance value, wherein the detection route is used for indicating a movement route of the sweeping robot in the process of acquiring sensor data of the obstacle, and the detection route at least comprises at least two directions of the obstacle;

and controlling the sweeping robot to move based on the detection route so that the sweeping robot collects sensor data at least in two directions around the obstacle.

Optionally, the method further includes:

determining a historical obstacle area according to historical data, and expanding the historical obstacle area according to a preset expansion radius to determine an expansion area;

and calculating the overlap between the first planned route and the expanded area as an overlap area, and executing the steps of controlling the sweeping robot to move around the detected obstacle and controlling the one or more sensors to acquire sensor data of the obstacle when the sweeping robot moves to the overlap area.

Wherein the step of determining a historical obstacle area from the historical data further comprises:

and taking the obstacle area with the obstacle occurrence rate larger than a preset value as the historical obstacle area according to historical data.

Optionally, the step of determining an influence of the obstacle on the first planned route based on the map of the work of the sweeping robot, the first planned route, and the obstacle parameter corresponding to the obstacle further includes:

determining an overlapping area between the obstacle and the first planned route according to the size, the area and the position information of the obstacle, and taking the first planned route in the overlapping area as a closed route;

wherein the closed route comprises one or more motion sub-routes in the first planned route.

Optionally, the step of performing route optimization again on the unmoved route portion based on the relative position of the obstacle in the map, the first planned route, and the closed route to obtain a second planned route further includes:

carrying out route planning again on the area where the closed route is located to obtain a first sub-route;

performing route fusion on the first sub-route and other routes except the closed route in the first planned route to obtain a second planned route;

the area where the closed route is located is the area where the closed route is expanded according to the preset expansion radius.

Optionally, the step of re-planning the route in the area where the closed route is located to obtain the first sub-route further includes:

carrying out route planning on an area where the closed route is located to obtain a plurality of sub-routes;

respectively calculating a first consumption value corresponding to each sub-route, wherein the first consumption value is used for representing the route length and the running time

Selecting one or more sub-routes from the plurality of sub-routes as a first sub-route according to a first consumption value;

the step of performing route fusion on the first sub-route and other routes in the first planned route except the closed route to obtain a second planned route further includes:

combining the plurality of first sub-routes and the first planned route respectively, and calculating a second consumption value of the combined route obtained by combining;

and taking the combined route with the minimum second consumption value as a second planned route.

Optionally, the step of performing route fusion on the first sub-route and the other routes in the first planned route except the closed route to obtain the second planned route further includes:

at least one first alternative planned route stored when planning the first planned route is obtained,

comparing the closed route with each first alternative planning route, and calculating a fit value between the closed route and each first alternative planning route, wherein the fit value is used for representing the concentration of routes influenced by the closed route in the first alternative planning routes;

determining at least one second alternative planned route in the at least one alternative planned route according to the fit value;

and respectively combining the plurality of first sub-routes and at least one second alternative planning route, calculating a second consumption value corresponding to the combined route after combination, and taking the combined route with the minimum second consumption value as a second planning route.

determining whether the obstacle belongs to a preset low obstacle type or not according to obstacle parameters corresponding to the obstacle, wherein the height of the obstacle in the low obstacle type is smaller than the preset height;

and under the condition that the obstacle belongs to a preset low obstacle type, ignoring the detected obstacle, and controlling the sweeping robot to move according to a first planned route, wherein when passing through the area corresponding to the detected obstacle, the sweeping robot is controlled to sweep the area where the obstacle is located according to a preset sweeping mode.

In a second aspect of the present invention, there is provided a motion obstacle avoidance device for a sweeping robot based on multi-sensor fusion detection, including:

the first route planning module is used for determining a map of the work of the sweeping robot and a first planned route and controlling the sweeping robot to move based on the first planned route;

the sensor data acquisition module is used for detecting whether an obstacle exists through one or more sensors arranged on the sweeping robot, controlling the sweeping robot to move around the detected obstacle and controlling the one or more sensors to acquire sensor data of the obstacle under the condition that the obstacle is detected;

the obstacle parameter calculation module is used for determining obstacle parameters corresponding to the obstacles according to the sensor data, and the obstacle parameters comprise the size, the area, the type, the position information and the like of the obstacles;

the closed route calculation module is used for determining the influence of the obstacle on the first planned route based on a map of the work of the sweeping robot, the first planned route and obstacle parameters corresponding to the obstacle, wherein the influence comprises the relative position of the obstacle in the map and the closed route in the first planned route based on the obstacle;

and the second route planning module is used for optimizing the route of the part of the route which does not move again based on the relative position of the obstacle in the map, the first planned route and the closed route so as to obtain a second planned route, and controlling the sweeping robot to move according to the second planned route, so that the sweeping robot can avoid the area of the obstacle and can sweep the area corresponding to the map in the moving process.

In a third aspect of the present invention, a sweeping robot is provided, where the sweeping robot includes a memory and a processor, and the memory has executable codes, and when the executable codes are run on the processor, the method for avoiding obstacle during movement of the sweeping robot based on multi-sensor fusion detection according to the first aspect of the present invention is implemented.

The embodiment of the invention has the following beneficial effects:

after the method and the device for movement obstacle avoidance and route control of the sweeping robot and the sweeping robot are adopted, firstly, a map and a first planned route of the sweeping robot are determined, and the sweeping robot is controlled to move based on the first planned route; detecting whether an obstacle exists or not through one or more sensors arranged on the sweeping robot, and controlling the sweeping robot to move around the detected obstacle and controlling the one or more sensors to acquire sensor data of the obstacle under the condition that the obstacle is detected; then determining barrier parameters corresponding to the barriers according to the sensor data, wherein the barrier parameters comprise the size, the area, the type, the position information and the like of the barriers; determining the influence of the obstacle on the first planned route based on a map of the work of the sweeping robot, the first planned route and obstacle parameters corresponding to the obstacle, wherein the influence comprises the relative position of the obstacle in the map and a closed route in the first planned route based on the obstacle; and finally, based on the relative position of the obstacle in the map, the first planned route and the closed route, carrying out route optimization on the part of the route which does not move again to obtain a second planned route, and controlling the sweeping robot to move according to the second planned route, so that the sweeping robot can avoid the area of the obstacle in the moving process and can finish sweeping the area corresponding to the map. That is to say, in the present embodiment, the sensor data of the obstacle is accurately detected by the plurality of sensors, then the specific parameters of the obstacle are calculated according to the acquired sensor data, and the route which cannot continue to move due to the influence of the existence of the obstacle on the original planned route is calculated according to the specific parameters, then the route planning is performed again on the part corresponding to the route which cannot continue to move, and the sweeping robot is controlled to move again based on the route after the route planning is performed again, so as to improve the accuracy of the obstacle detection and the accuracy of the obstacle avoidance movement.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

Wherein:

fig. 1 is a schematic flow chart illustrating a method for obstacle avoidance and route control of a sweeping robot in an embodiment;

fig. 2 is a schematic diagram illustrating an example of a device for obstacle avoidance and route control of a sweeping robot;

fig. 3 is a schematic structural diagram of a computer device for operating the above-described method for obstacle avoidance and route control of the sweeping robot in one embodiment.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

In this embodiment, a method for motion obstacle avoidance and route control of a sweeping robot is provided, and the method may be implemented based on a sweeping robot.

In this embodiment, in order to avoid the robot from being damaged due to collision between the sweeping robot and the obstacle, a sensor for detecting the obstacle needs to be provided on the sweeping robot. The sweeping robot is provided with a plurality of sensors, and the sensors are used for detecting obstacles. Further, in the present embodiment, in order to improve the accuracy and comprehensiveness of obstacle detection, the plurality of sensors disposed on the sweeping robot at least include 2 different sensor types, that is, in the present embodiment, a plurality of sensors of different types are required to detect obstacles together, so as to enhance the detection of obstacles.

Further, the sensors are arranged in various directions of the sweeping robot, including the traveling direction of the sweeping robot and other directions except the traveling direction, so as to ensure that the sweeping robot detects the obstacle which may collide with the traveling direction and other directions (for example, two sides of the traveling direction) of the traveling direction in the movement process, so that the sweeping robot can effectively avoid colliding with the obstacle in the subsequent movement process.

In a specific embodiment, the plurality of sensors may be disposed at various orientations of the sweeping robot, and the specific arrangement orientation is not limited in this embodiment. The types of sensors are not limited to two, and may be various for acquiring data of an obstacle from respective different sensors. For example, different types of sensors are arranged in each direction of the sweeping robot in a crossed manner, so that data of obstacles can be acquired as comprehensively as possible, the comprehensiveness of obstacle detection is improved, and the accuracy of subsequent obstacle determination is improved.

Referring to fig. 1, a schematic flow chart of an implementation of a method for obstacle avoidance and route control of a sweeping robot is shown. The method for obstacle avoidance and route control of the sweeping robot includes steps S101 to S106 shown in fig. 1:

step S101: determining a map of the work of the sweeping robot and a first planned route, and controlling the sweeping robot to move based on the first planned route.

In this embodiment, a map corresponding to the sweeping robot is pre-constructed, where the map is an area where the sweeping robot moves during sweeping work, and the map includes an area to be swept and obstacle areas in the middle and outside of the area; the map is constructed by collecting surrounding environment data through a sensor arranged on the sweeping robot when the sweeping robot starts sweeping, for example, the map constructed based on the SLAM technology.

The method comprises the steps that a built map of the sweeping robot in work needs to be further planned to be a route of the sweeping robot in the sweeping process, the route is a first planned route, all areas of the map can be covered under the indication of the first planned route of the sweeping robot, the repeated driving route and the whole duration of the sweeping robot are reduced as far as possible, and the optimal driving route is obtained to serve as the first planned route. In the process of working of the sweeping robot, the robot moves based on the first planned route so as to sweep the area corresponding to the map.

Step S102: whether an obstacle exists is detected through one or more sensors arranged on the sweeping robot, and under the condition that the obstacle is detected, the sweeping robot is controlled to move around the detected obstacle and the one or more sensors are controlled to acquire sensor data of the obstacle.

In this embodiment, in the process that the sweeping robot moves and sweeps based on the first planned route, it is necessary to detect whether there is an obstacle around the sweeping robot and in the driving direction by using a sensor, so as to avoid colliding with the obstacle. When the obstacle is not detected, all sensors on the sweeping robot are not required to work, and only part of the sensors in all the sensors can be controlled to work to detect whether the obstacle exists. The detection of the existence of the obstacle can be realized only by the operation of part of the sensors; however, in order to further acquire comprehensive information of the obstacle, more sensors (for example, all sensors) may be further controlled to operate.

In a specific implementation, in a process of a movement of the sweeping robot, for example, in a process of performing a movement according to a pre-planned route and cleaning an area corresponding to the movement route, it is required to detect whether there is an obstacle by using a sensor provided on the sweeping robot.

In this step, whether an obstacle exists is detected by one or more sensors of a plurality of sensors on the sweeping robot, and the one or more sensors are part of all the sensors arranged on the sweeping robot, but not all the sensors.

For example, a distance between an object closest to the sweeping robot in the detection direction of the sensor and the sweeping robot is detected by a distance sensor (e.g., a laser sensor or a radar sensor) provided on the sweeping robot, and if the detected distance is smaller than a preset value, it is determined that an obstacle is present.

When an obstacle is detected, in order to acquire more detailed information about the obstacle, it is necessary to further perform detailed detection on the obstacle to acquire more comprehensive information about the obstacle. In specific implementation, when an obstacle is detected, more sensors are used for acquiring information of the obstacle, specifically, more sensors or all sensors in a plurality of sensors on the sweeping robot are used for acquiring information of the obstacle, that is, each sensor acquires sensor data corresponding to the obstacle, and the acquired sensor data represents information of the obstacle in a certain dimension or multiple dimensions.

It should be noted that, in this embodiment, each sensor corresponds to one set of sensor data, and the sensor data of the same type of sensor can be classified into one large group, and the data under the same type of sensor can be merged.

In this embodiment, in the process of acquiring data of an obstacle through the sensor, in order to acquire more obstacle information as much as possible, the sweeping robot needs to be further controlled to move, so that more data of the sweeping robot can be acquired from different angles in the moving process.

Specifically, when an obstacle is detected, the sweeping robot needs to be controlled to further move, and a plurality of sensors on the sweeping robot are controlled to collect more comprehensive obstacle data during the movement. During the acquisition of the sensor data, the sweeping robot needs to move at least partially around the obstacle and acquire the obstacle data.

In a specific embodiment, in the process of detecting the obstacle, firstly, a sensor is needed to acquire a distance value between the sweeping robot and the detected obstacle; according to the distance value between the sweeping robot and the obstacle, at least part of information of the obstacle can be acquired only by determining the distance which the sweeping robot needs to move, for example, when the distance between the sweeping robot and the obstacle is short, the more comprehensive information of the obstacle can be acquired only by moving a small distance, and when the distance between the sweeping robot and the obstacle is long, the more comprehensive information of the obstacle can be acquired only by moving a long distance. In the embodiment, the length of the path to be moved is inversely related to the distance between the sweeping robot and the obstacle. Further, after the distance value between the sweeping robot and the obstacle is determined, a movement route of the sweeping robot in the process of acquiring the data of the obstacle (namely, a detection route) needs to be further determined, and an inverse correlation relationship exists between the route length of the detection route and the distance value between the sweeping robot and the obstacle. Further, in order to ensure that the data of the obstacle is collected as comprehensively as possible, in the orientation relationship between the detection route and the obstacle, the detection route should at least include at least two orientations of the obstacle, where the orientations may correspond to one orientation according to 90 degrees, that is, the angle between the start point and the end point of the detection route and the obstacle is at least 90 degrees, and preferably 120-180 degrees.

Further, in order to reduce unnecessary route movement of the sweeping robot in the process of acquiring data of the obstacle, so as to improve efficiency, in this embodiment, the detection route may be a partial route directly in the first planned route, and the partial route may be continuous or discontinuous, that is, the acquisition of the data of the obstacle may be the acquisition of a continuous period of time, or may not be the acquisition of data of a plurality of discontinuous periods of time.

Generally, if an obstacle frequently occurs in a certain area in the history data, for example, the position of a chair in a restaurant area is frequently changed, and an object such as a chair is recognized as the obstacle during the cleaning process, it is necessary to bypass the area for cleaning. In this step, it may be determined that the record of the occurrence of the obstacle in the map is greater than the preset value according to the historical data, the corresponding area is used as a historical obstacle area, and then when the sweeping robot moves in the historical obstacle area, the acquisition of the data of the obstacle is triggered, that is, the execution of step S102 is triggered, so as to obtain whether the historical obstacle area has the obstacle and the existing obstacle condition. Considering that an obstacle moves in an actual situation, when a historical obstacle area is considered, the area needs to be expanded, for example, the historical obstacle area is expanded or expanded according to a preset expansion radius to determine an expanded area, and then, when the sweeping robot moves to the historical obstacle area or the expanded area according to the first planned route, execution of step S102 is triggered to acquire sensor data corresponding to the obstacle.

In specific execution, determining a historical obstacle area according to historical data, and expanding the historical obstacle area according to a preset expansion radius to determine an expansion area; calculating the overlap between the first planned route and the expanded area as an overlap area, and when the sweeping robot moves to the overlap area, executing the steps of controlling the sweeping robot to move around the detected obstacle and controlling the one or more sensors to acquire sensor data of the obstacle; wherein the step of determining a historical obstacle area from the historical data further comprises: and taking the obstacle area with the obstacle occurrence rate larger than a preset value as the historical obstacle area according to historical data.

Step S103: obstacle parameters corresponding to the obstacles are determined from the sensor data, including the size, area, type, location information, etc. of the obstacles.

After the obstacle is detected, sensor data corresponding to the obstacle is acquired through the sensor, and then specific information of the obstacle is determined according to the sensor data.

In a specific implementation, the sensor data may be further processed, so that the subsequently acquired obstacle parameters are more accurate.

And performing cross fusion processing on the sensor data corresponding to the plurality of sensors based on a preset feature fusion algorithm to acquire fused target sensor data.

In step S102, sensor data related to obstacles are collected by a plurality of sensors, the sensor data are from different types of sensors and from data collected from different positions and different orientations of the sweeping robot, the sensor data can represent the situation of the obstacles from different angles, and the sensor data also have data repetition, redundancy and mutual evidence enhancement.

In order to further emphasize the features of obstacles in sensor data acquired by different sensors and reduce the influence of noise, in the present embodiment, it is necessary to further process the sensor data of multiple sensors, in the present embodiment, feature fusion processing is performed on the sensor data, for example, cross fusion processing is performed on features corresponding to the sensor data to acquire target sensor data after fusion, the features of information of obstacles included in the target sensor data after fusion processing are further enhanced, noise is further reduced, and the accuracy of extracting obstacle information based on the target sensor data subsequently is higher.

In a specific embodiment, the screening and filtering process is firstly performed on the sensor data corresponding to each sensor respectively, so as to filter part of noise data in the sensor data; in another embodiment, the sensor data may also be pre-processed, e.g., to remove duplicate values, to remove data from missing portions of the data, etc. In order to improve the accuracy of the screening, filtering and preprocessing of the sensor data, when the sensor data corresponding to each sensor is processed correspondingly, how to process the sensor data may be set for each sensor type, and may also be set based on the position of each sensor on the sweeping robot.

For the sensor after the screening, filtering and preprocessing, the feature data corresponding to the sensor data needs to be further extracted. Specifically, a feature extraction model corresponding to each sensor type is determined, and the feature extraction model may be a convolutional neural network. The characteristics of the sensor data detected by different sensor types are different, so that corresponding feature extraction models or convolutional neural networks for feature extraction are different, and the feature data of each group of sensor data, which can be used for representing the relevant conditions of the obstacles, can be extracted more accurately.

In the specific characteristic extraction process, the characteristic extraction can be carried out in multiple steps so as to improve the comprehensiveness and accuracy of the characteristic extraction.

Specifically, each group of sensor data is respectively input into a first convolutional neural network corresponding to each sensor type so as to extract first characteristic data corresponding to each group of sensor data; then inputting the sensor data and the first characteristic data corresponding to each sensor into a second convolutional neural network corresponding to each sensor type to extract second characteristic data corresponding to each group of sensor data; and finally, performing cross fusion processing on the first characteristic data and the second characteristic data corresponding to the plurality of sensors based on a preset characteristic fusion function to obtain target sensor data after fusion processing.

That is, the first feature extraction is performed on the sensor data through the first convolutional neural network, and first feature data corresponding to the sensor data is acquired.

In the second extraction of the feature data, not only the feature extraction needs to be performed on the sensor data, but also the feature extraction needs to be performed on the sensor data and the first feature data obtained by performing the first feature extraction before the sensor data.

After the second feature extraction, the third and fourth feature extractions may be further performed on the features, where each feature extraction needs to take the sensor data and all feature data obtained by the previous feature extraction as inputs, and extract corresponding feature data through a convolutional neural network.

After performing feature data extraction twice or more, further processing needs to be performed on the feature data to perform feature fusion processing on the feature data obtained by performing feature extraction multiple times. Here, the feature data extracted from the sensor data is subjected to fusion processing. Specifically, based on a preset feature fusion function, cross fusion processing is performed on first feature data and second feature data corresponding to the plurality of sensors to obtain target sensor data after the fusion processing.

In a particular embodiment, the sensor data obtained for each sensor

Wherein, in the step (A),

wherein n is the number of sensors. In this step, for n sensor data sets

Data of each sensor

Respectively input to corresponding first convolution neural networks

In which

To obtain data of each sensor

First characteristic data of

Are respectively marked as

Thus, the feature set based on the first feature data is

。

Further, for n sets of sensor data, is

Data of each sensor

And the first characteristicThe characteristics of the data are combined into

Respectively input to corresponding second convolutional neural networks

To obtain data of each sensor

Second characteristic data of

Are respectively marked as

Thus, the feature set based on the second feature data is

。

For each first/second feature data, performing cross fusion processing on the first/second feature data and other n feature data (the feature data are subjected to fusion processing on a preset dimension, wherein the fusion processing comprises one or more of dimension splicing, matrix addition, matrix element-by-element multiplication, normalization and the like), and obtaining the first feature data and the second feature data after fusion; and then carrying out further fusion processing on the first characteristic data and the second characteristic data to obtain the target sensor data.

In the specific operation, the calculation is carried out by the following formula:

to obtain the first characteristic data after fusion

And second characteristic data

The corresponding first characteristic data set is

The second feature data set is

. Wherein the content of the first and second substances,

the method is a function for processing new characteristics obtained after the same characteristics pass through different networks, the function operation can be concat or addition operation, and the concat operation is splicing the characteristics according to a certain dimension;

is to the same network

The function operation may be concat or an addition operation.

Then the collection of the feature data after the fusion

And a set of second characteristic data

Further fusion processing is carried out, wherein the fusion processing operation can be concat operation and can also be addition operation. In a particular embodiment, the first characteristic data may be aggregated

And a set of second characteristic data

Merge into target sensor data (FEAT):

in other embodiments, the feature fusion processing may be performed in other fusion processing manners, such as addition fusion, and the manner of the fusion processing is not limited in any way, and may be performed by performing cross fusion processing according to any preset fusion algorithm to obtain the feature after fusion.

In another embodiment, the feature fusion may be performed by taking into account the characteristics of the obstacles described by the different feature data, and performing feature fusion from the viewpoint of the obstacles. Specifically, a plurality of obstacle description dimensions are preset, and each feature data includes data in one or more obstacle description dimensions. For example, when a obstacle description dimensions are included, for any feature data, the feature data included in the feature data may be feature data covering all the a obstacle description dimensions, may only include feature data in a partial dimension under the a obstacle description dimensions, or may be feature data in a dimension not including any obstacle description dimension.

In this step, for the first feature data and the second feature data, it is necessary to divide them into feature data in each obstacle description dimension, that is, feature data of the first feature data and the second feature data in at least one preset obstacle description dimension is extracted so that each obstacle description dimension includes at least one feature data. The feature data in one obstacle description dimension should be similar to each other because it describes the features of the same obstacle, and therefore, in this step, the similarity between one or more feature data included in each obstacle description dimension is calculated separately in the first place. And then, performing fusion processing on one or more feature data contained in the obstacle description dimension according to the similarity. The fusion process includes a filtering process, a deduplication process, and a data fusion process.

Specifically, for two feature data, according to the similarity between the two feature data, the corresponding fusion processing operation is determined, and the two feature data are subjected to fusion processing to obtain feature data after the fusion processing. For example, when the similarity is greater than or equal to a preset similarity threshold, the feature data in the obstacle description dimension is subjected to deduplication processing or fusion processing.

For another example, in one embodiment, how to construct a similarity matrix in the obstacle description dimension according to at least one feature data when the similarity between the plurality of feature data included in one obstacle description dimension is smaller than a preset similarity threshold, where the similarity matrix includes the similarity between the plurality of feature data in the obstacle description dimension. And then, based on the similarity matrix, carrying out fusion processing on the feature data under the description dimension of the obstacle, wherein the weight coefficient for carrying out the fusion processing is determined according to the similarity matrix. Specifically, for example, according to all coefficients corresponding to a certain feature data in the similarity matrix, a weight coefficient corresponding to the feature data is calculated according to the coefficient, for example, all coefficients are summed to obtain a weight coefficient. And then, performing weighting processing on all the characteristic data according to the corresponding weight coefficients to obtain fused target sensor data under the obstacle description dimensionality.

After the sensor data is processed to obtain the corresponding target sensor data, the specific data corresponding to the obstacle needs to be specifically determined based on the obtained target sensor data, or the specific data corresponding to the obstacle may be directly obtained from the sensor data. Specifically, the obstacle parameter corresponding to the obstacle is used to describe a specific feature description characterizing the obstacle, such as the size, area, type, location information, and the like of the obstacle. In a specific implementation, the sensor data may be input into a preset convolutional neural network to obtain an output of the convolutional neural network as an obstacle parameter of the obstacle.

In a specific embodiment, in order to improve the accuracy of calculating the obstacle parameter, not only the finally obtained target sensor data but also intermediate data in the process of calculating the target sensor data need to be considered to reinforce the target sensor data, and specifically, the target sensor data, the first characteristic data and the second characteristic data are input into a third convolutional neural network, and the output of the third convolutional neural network is obtained as the obstacle parameter.

Step S104: based on a map of the sweeping robot working and the first planned route and barrier parameters corresponding to the barriers, determining the influence of the barriers on the first planned route, wherein the influence comprises the relative positions of the barriers in the map and the closed route in the first planned route based on the barriers.

After determining the obstacle parameters of the obstacle, the influence on the already planned first planned route needs to be further confirmed, for example, because the existence of the obstacle may result in which routes of the originally planned first planned route cannot be driven further. In particular, the influence of the obstacle on the first planned route is further determined according to the obstacle parameter of the obstacle, wherein the relative position of the obstacle in the map and the closed route in the first planned route influenced by the obstacle are determined. The closed route is one or more sub-routes which cannot move according to the original route in the first planned route due to the influence of the obstacles.

In a specific embodiment, according to the size, the area and the position information of the obstacle, determining an overlapping area between the obstacle and the first planned route, and then taking the first planned route in the overlapping area as a closed route; wherein the closed route comprises one or more motion sub-routes in the first planned route.

Step S105: and based on the relative position of the obstacle in the map, the first planned route and the closed route, carrying out route optimization on the part of the route which does not move again to obtain a second planned route, and controlling the sweeping robot to move according to the second planned route, so that the sweeping robot can avoid the area of the obstacle and can finish sweeping the area corresponding to the map in the moving process.

Because of the influence of the obstacle, a part of the closed route in the first planned route cannot move, and the closed route can influence the fact that a part of the original first planned route cannot run, therefore, the route planning needs to be performed again on the area which is not cleaned to obtain a second planned route, and then the sweeping robot is controlled to move based on the second planned route, wherein when the sweeping robot moves according to the second planned route, the sweeping robot can avoid the area where the obstacle is located, and can complete the cleaning of all other areas which are not cleaned.

In a specific embodiment, the route planning is performed on the area where the closed route is located to obtain a plurality of sub-routes, and then one of the sub-routes is taken as the first sub-route. In another embodiment, a plurality of sub-routes may be determined as the first sub-route in sub-routes that are greater than one of the plurality of sub-routes. That is, the number of the first sub-routes may be one or more. In a specific embodiment, in order to perform the new route planning, it is necessary to perform not only the route planning on the area where the obstacle is located, but also a certain expansion on the area, that is, the area after the closed route is expanded according to a preset expansion radius.

In order to obtain a plurality of selectable routes as sub-routes when planning a route in an area where a closed route is located, one or more sub-routes need to be determined as a first sub-route from among the plurality of sub-routes. However, how to determine the first sub-route in the plurality of sub-routes requires calculating a first consumption value corresponding to each sub-route, the first consumption value being used to represent the route length, the exercise time, etc., and the calculation of the first consumption value may be a weighted sum of the length and the exercise time. In the specific calculation, a first value of the route length and a second value of the movement time corresponding to the route may be calculated, then a weighted sum of the first value and the second value may be calculated according to a preset weighting coefficient, and the calculated result may be used as the first consumption value. And then selecting a sub-route with the smallest first consumption value from the plurality of sub-routes as the first sub-route according to the first consumption value, or selecting the plurality of sub-routes with the first consumption values smaller than a preset value, or selecting the N sub-routes which are sequenced at the top as the first sub-route.

In the case that the first sub-route is one, the first sub-route and the original second planned route may be directly combined to obtain a combined route after combination.

Combining the plurality of first sub-routes and the first planned route respectively under the condition that the first sub-route is a plurality of routes, and calculating a second consumption value of the combined route obtained by combining; the combined route with the smallest second consumption value is then used as the second planned route.

In another embodiment, not only the first planned route but also other planned routes to the map need to be considered when re-routing. Specifically, when the map is subjected to route planning, routes other than the first planned route are stored (a plurality of first alternative planned routes), wherein which routes are stored in advance, and the stored route or routes can be determined by calculating consumption values of the routes (in the same manner as the first consumption value and the second consumption value). When the route planning is performed again, not all the first alternative planned routes are considered, but whether the first alternative planned routes are applicable to the current route planning scenario needs to be considered. Specifically, the closed route is compared with each of the first alternative planned routes, and a matching value (a first matching value) between the closed route and each of the first alternative planned routes is calculated, where the first matching value is used to characterize the concentration of routes influenced by the closed route in the first alternative planned routes.

In another embodiment, it is further required to consider whether an area which is cleaned by the cleaning robot currently and an area which is not cleaned are matched with the first alternative planned route, and specifically, according to a route which is already driven and a route which is not driven in the first planned route, a second matching value between the first candidate planned route and each first alternative planned route is calculated, where the second matching value is used to represent whether the first alternative planned route is applicable to the current route planning scenario.

Determining at least one second alternative planning route in the at least one alternative planning route according to the magnitude of the first fit value and the second fit value; the first matching value and the second matching value are both required to meet a preset threshold value, so that the second alternative planning route serving as the alternative can be determined to be actually suitable for the current route planning scene, and a large calculation amount is not brought.

Specifically, the plurality of first sub-routes and the at least one second alternative planning route are respectively combined to obtain corresponding combined routes, then corresponding second consumption values are respectively calculated for each combined route, and the combined route with the minimum second consumption value is used as the second planning route, so that the optimal second planning route is obtained.

It should be noted that, in this embodiment, it is not necessary to perform route planning again on all maps, and it is also not necessary to bear a large amount of calculation for route planning, and it is only necessary to perform route planning on a small area where an obstacle is located, and then combine the route planning result of this small area with one or multiple original alternative route planning results, so as to select the route planning result with the most recent route planning result with the most optimal global route planning result according to the combination of the local optimal route planning results, thereby obtaining the optimal route re-planning result under the condition of a small calculation amount.

In another embodiment, all obstacles are not required to be subjected to obstacle avoidance operation, and for some short obstacles, the sweeping robot can be directly controlled to directly cover the obstacle area for sweeping. The sweeping robot has certain obstacle crossing capability, for example, certain slope or height climbing capability, so that the sweeping robot can cope with more complex environments and can cope with sweeping of partial low obstacle areas.

In a specific operation, after obtaining the obstacle parameter corresponding to the obstacle, it is further required to determine whether the obstacle belongs to a preset low obstacle type according to the obstacle parameter corresponding to the obstacle, where the obstacle may be determined according to the height, shape, and type of the obstacle in the determined obstacle parameter, and the obstacle in the low obstacle type at least needs to have a height smaller than the preset height. Then, under the condition that the obstacle belongs to the preset low obstacle type, neglecting the detected obstacle, not needing to carry out obstacle avoidance processing, but controlling the sweeping robot to move according to the first planned route, triggering the climbing, crawling ladder or obstacle overcoming capacity of the sweeping robot, so as to sweep the area corresponding to the obstacle, and controlling the sweeping robot to sweep according to the preset sweeping mode when passing through the area corresponding to the detected obstacle. For example, under the condition that detects the carpet that has a take the altitude, the robot of sweeping the floor can continue to move the region that the carpet corresponds and clean to can open the mode of sweeping that corresponds with the carpet and clean, with the completion degree that improves whole sweeping, improve the intelligence of the robot of sweeping the floor, promote user experience.

Referring to fig. 2, in the present embodiment, a motion obstacle avoidance device of a sweeping robot based on multi-sensor fusion detection is provided, the device includes:

the first route planning module 101 is configured to determine a map of work of the sweeping robot and a first planned route, and control the sweeping robot to move based on the first planned route;

the sensor data acquisition module 102 is configured to detect whether an obstacle exists through one or more sensors arranged on the sweeping robot, and control the sweeping robot to move around the detected obstacle and control the one or more sensors to acquire sensor data of the obstacle when the obstacle is detected;

the obstacle parameter calculation module 103 is configured to determine an obstacle parameter corresponding to an obstacle according to the sensor data, where the obstacle parameter includes a size, an area, a type, and position information of the obstacle;

the closed route calculation module 104 is used for determining the influence of the obstacle on the first planned route based on the map of the work of the sweeping robot, the first planned route and the obstacle parameters corresponding to the obstacle, wherein the influence comprises the relative position of the obstacle in the map and the closed route in the first planned route based on the obstacle;

the second route planning module 105 is configured to perform route optimization again on a part of the route that does not move based on the relative position of the obstacle in the map, the first planned route, and the closed route to obtain a second planned route, and control the sweeping robot to move according to the second planned route, so that the sweeping robot can avoid an area of the obstacle and can complete sweeping of an area corresponding to the map in the moving process.

Fig. 3 shows an internal structure diagram of a computer device (sweeping robot) for implementing the above-described method for obstacle avoidance and route control of the sweeping robot in one embodiment. As shown in fig. 3, the computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program which, when executed by the processor, causes the processor to carry out the above-mentioned method. The internal memory may also have stored therein a computer program which, when executed by the processor, causes the processor to perform the method described above. Those skilled in the art will appreciate that the architecture shown in fig. 3 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a non-volatile computer-readable storage medium, and can include the processes of the embodiments of the methods described above when the program is executed. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims. Please enter the implementation content part.

Claims

1. A method for motion obstacle avoidance and route control of a sweeping robot is characterized by comprising the following steps:

2. The method of claim 1, wherein the step of controlling the sweeping robot to move around the detected obstacle and controlling the one or more sensors to collect sensor data of the obstacle when the obstacle is detected further comprises:

3. The method for obstacle avoidance and route control of a sweeping robot according to claim 1, further comprising:

calculating the overlap between the first planned route and the expanded area as an overlap area, and when the sweeping robot moves to the overlap area, executing the steps of controlling the sweeping robot to move around the detected obstacle and controlling the one or more sensors to acquire sensor data of the obstacle;

4. The method of claim 1, wherein the step of determining the influence of the obstacle on the first planned route based on a map of the work of the sweeping robot and the first planned route and obstacle parameters corresponding to the obstacle further comprises:

5. The method of claim 1, wherein the step of re-optimizing the route of the non-moving route portion to obtain the second planned route based on the relative position of the obstacle in the map, the first planned route, and the closed route to obtain the second planned route further comprises:

6. The method of claim 5, wherein the step of re-routing the area of the closed route to obtain the first sub-route further comprises:

7. The method of claim 6, wherein the step of merging the first sub-route with the other routes of the first planned route except the closed route to obtain the second planned route further comprises:

8. The method of claim 1, wherein the step of determining the influence of the obstacle on the first planned route based on a map of the work of the sweeping robot and the first planned route and obstacle parameters corresponding to the obstacle further comprises:

9. The utility model provides a barrier device is kept away in motion of robot of sweeping floor based on multisensor fuses detection which characterized in that, the device includes:

10. A sweeping robot, characterized in that the sweeping robot comprises a memory and a processor, wherein the memory has executable codes, and when the executable codes run on the processor, the method for avoiding the obstacle by the movement of the sweeping robot based on the multi-sensor fusion detection as claimed in any one of claims 1 to 8 is realized.