WO2023050637A1 - Garbage detection - Google Patents

Abstract

Disclosed in the present description are a garbage detection method and apparatus. An area to be subjected to detection is divided into at least two preset areas in advance. The method comprises: acquiring a target image which is captured for an area to be subjected to detection; determining positions of preset areas, which are obtained by dividing said area, in the target image; for the target image, determining a garbage detection result by using a pre-trained garbage detection model; and according to the determined garbage detection result, determining a preset area where garbage is present to be a garbage area.

Description

Spam detection technical field

The embodiments of this specification relate to the field of object detection, and in particular to methods and devices for garbage detection.

Background technique

For garbage detection, usually after taking an image of the area to be cleaned, the garbage detection technology is used to determine the garbage in the captured image, which can be marked with a garbage detection frame.

Although the current garbage detection technology can detect the garbage in the image, it is difficult to further determine the specific location of the detected garbage, and the accuracy is low.

Contents of the invention

In order to solve the above technical problems, this specification provides a method and device for garbage detection. The technical scheme is as follows.

A garbage detection method, which divides an area to be detected into at least two preset areas in advance; the method includes: acquiring a target image taken for the area to be detected; determining that the divided preset area is in the target image position; for the target image, using a pre-trained garbage detection model to determine a garbage detection result; according to the determined garbage detection result, determine a preset area where garbage exists as a garbage area.

A garbage detection device, which divides an area to be detected into at least two preset areas in advance; the device includes: an acquisition unit, used to acquire a target image taken for the area to be detected; a mapping unit, used to determine the divided The position of the preset area in the target image; the detection unit is used to determine the garbage detection result by using the garbage detection model trained in advance for the target image; the positioning unit is used to determine the garbage detection result according to the determined garbage detection result. A preset area where garbage exists is determined as a garbage area.

The above technical solution can determine the position of the preset area in the target image, so as to facilitate the determination of the preset area where garbage exists according to the garbage detection result of the target image, efficiently, quickly and accurately determine the position of the garbage, and reduce computing resources loss.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of this specification or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only These are some embodiments described in the embodiments of this specification, and those skilled in the art can also obtain other drawings based on these drawings.

Fig. 1 is a schematic flow chart of a garbage detection method provided by the embodiment of this specification;

Fig. 2 is a schematic diagram of the principle of a preset area mapping method provided by the embodiment of this specification;

Fig. 3 is a schematic diagram of the principle of a garbage area determination method provided by the embodiment of this specification;

Fig. 4 is a schematic diagram of the principle of cleaning route planning provided by the embodiment of this specification;

Fig. 5 is a schematic diagram of the principle of a model structure provided by the embodiment of this specification;

Fig. 6 is a schematic diagram of the principle of another cleaning route planning provided by the embodiment of this specification;

Fig. 7 is a schematic structural diagram of a garbage detection device provided by an embodiment of this specification;

Fig. 8 is a schematic structural diagram of a device for configuring the method of the embodiment of the present specification.

Detailed ways

In order for those skilled in the art to better understand the technical solutions in the embodiments of this specification, the technical solutions in the embodiments of this specification will be described in detail below in conjunction with the drawings in the embodiments of this specification. Obviously, the described implementation Examples are only some of the embodiments in this specification, not all of them. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments in this specification shall fall within the scope of the disclosure.

Object detection is an important perception technique. For example, in the field of unmanned driving, obstacles on the driving route can be detected for unmanned vehicles, specifically including other cars, pedestrians, bicycles, motorcycles, etc., and correct driving decisions can be made based on the detection results. Such as stopping, avoiding or going around.

Wherein, the target detected by the target detection may be an obstacle or other objects. For example, spam detection.

In a specific example, the unmanned sweeper can detect garbage in the area to be cleaned, and detect the garbage in the area to be cleaned, so that the driving route of the unmanned sweeper can be planned to clean up the detected garbage.

For garbage detection, usually after taking an image of the area to be cleaned, the garbage detection technology is used to determine the garbage in the captured image, which can be marked with a garbage detection frame.

Although the current garbage detection technology can detect the garbage in the image, it is difficult to further determine the specific location of the detected garbage, and the accuracy is low.

For example, for paper scraps of different sizes, a large piece of paper dust in the distance and a small piece of paper dust near it may have similar sizes in the image, so it is difficult to determine the position of the paper dust.

Moreover, the method for locating obstacles is not suitable for locating garbage. For example, because obstacles are more obvious, such as pedestrians, motorcycles, etc., they can be scanned by lidar, and obstacles can be determined through more reflection information. However, many garbage is relatively small, such as paper scraps, fallen leaves, etc. Even if the position in the image is detected and scanned by lidar, there is reflection information for the paper scraps, making it difficult to locate the scraps of paper.

In order to solve the above technical problems and facilitate positioning of garbage, the embodiment of this specification provides a garbage detection method.

In order to facilitate positioning, the area to be detected can be divided in advance, and can be divided into multiple preset areas. After the image taken for the area to be detected is acquired, the position of the preset area in the captured image can be determined, so that after the garbage is detected, the preset area with garbage can be determined as the garbage area.

Wherein, the garbage area is a specific location determined for the garbage. Of course, in the same garbage area, there may be multiple garbage. This method does not need to determine the location of each garbage, and can directly determine the preset area where garbage exists as the garbage area, and the determined garbage area is the location of each garbage contained therein.

In a specific example, after the garbage area is determined, the cleaning route can be planned quickly and efficiently, so that the unmanned sweeper can clean along the cleaning route. It should be noted that the unmanned sweeper needs to clean the garbage area as a whole.

The garbage detection method provided by the embodiment of this specification can divide the area to be detected into a plurality of preset areas, and determine the position of the preset area in the image, so that after detecting garbage, it is convenient to further determine the presence of garbage in the image The preset area, so that the garbage can be located. High positioning efficiency, fast speed and high accuracy.

A garbage detection method provided in the embodiment of this specification will be explained in detail below in conjunction with the accompanying drawings.

As shown in FIG. 1 , it is a schematic flowchart of a garbage detection method provided in the embodiment of this specification. Wherein the region to be detected can be divided into at least two preset regions in advance.

Wherein, the area to be detected may be an area where garbage detection is required, specifically, it may be a pre-set area. For example, for areas such as parking lots, airport halls, shopping malls, etc., since it is necessary to clean up the garbage on the ground, it is necessary to perform garbage detection first to facilitate cleaning later.

For the preset area, in an optional embodiment, it can be used to map to the image that needs to be detected for garbage, so as to determine the position of the detected garbage, that is, the preset area where the garbage is located, and facilitate subsequent determination of cleaning route.

The method flow does not limit the method of dividing the preset area. Optionally, the preset area may be divided manually or automatically by the device.

In the case that the device automatically divides the preset area, optionally, the division may be performed according to a certain size standard. For example, the area to be detected can be divided into multiple grids, and the size and shape of different grids are the same, and the size of each grid can be comprehensively determined according to factors such as image accuracy, garbage detection accuracy, and garbage evaluation size. Of course, it can also be directly set to 1 square meter.

In a specific example, the cleaning range of the unmanned sweeper without moving can be determined, and then the grid size can be determined according to the determined cleaning range, specifically, the grid can be made equal to or smaller than the determined cleaning range, so that When the unmanned sweeper is cleaning the preset area, it can be cleaned without moving.

Of course, the sizes of different preset areas may be the same or different; the shapes of different preset areas may be the same or different.

In the case of manually dividing the preset area, optionally, it can be manually divided according to a special area in the area to be detected. For example, when the area to be detected is a parking lot, each parking space contained in it can be manually determined as a preset area, which is convenient for subsequent direct cleaning of the entire parking space. The aisle can be divided into grids with the same size and shape; when the area to be detected is an area with a height difference, it can be specifically a staircase, and each staircase can be manually determined as a preset area.

Of course, optionally, when specifically dividing the preset area, the division may also be performed in a manner of combining manual division and automatic division. For example, for shopping malls, the ground in each shop can be manually determined as the preset area, and then for the hall and aisle of the shopping mall, the preset area can be determined by using the method of automatic division of equipment, specifically, it can be divided by using a grid of preset size , divide the hall and aisle of the shopping mall into multiple grids with a size of 1 square meter, and each grid is a preset area.

Optionally, since the preset areas are used to determine the location of the garbage, there may be no overlap between different preset areas.

The method may include the following steps.

S101: Acquire a target image taken for a region to be detected.

S102: Determine the position of the divided preset area in the target image.

S103: For the target image, determine a garbage detection result by using a pre-trained garbage detection model.

S104: Determine a preset area where garbage exists as a garbage area according to the determined garbage detection result.

Optionally, S102 and S103 may be executed in parallel or successively, and this embodiment does not limit the execution sequence between S102 and S103.

The above method flow can determine the position of the preset area in the target image, so as to facilitate the determination of the preset area where garbage exists according to the garbage detection result of the target image, efficiently, quickly and accurately determine the position of the garbage, and reduce computing resources loss. After determining the location of the garbage, staff can be arranged to clean it, or unmanned cleaning equipment can clean it after determining the cleaning route.

1. The following is a detailed explanation of S101.

In an optional embodiment, the target image in S101 may specifically be any image captured for the region to be detected.

For the shooting operation, optionally, the process of this method can be applied to any electronic device. Therefore, specifically, the electronic device can shoot target images for the area to be detected through its own camera device; The target image is captured in the area, and then transmitted to the electronic device to execute the above-mentioned method flow for garbage detection.

In a specific example, the method flow can be applied to unmanned cleaning equipment, specifically, a mobile unmanned cleaning vehicle. The unmanned cleaning equipment can be equipped with a camera device to facilitate garbage detection. Therefore, the unmanned cleaning equipment itself can take pictures of the area to be detected to obtain target images.

In another specific example, a high-altitude camera or a drone can take target images of the area to be detected, and then transmit the target images to unmanned cleaning equipment for garbage detection. Specifically, the UAV can shoot from a top-down perspective.

Therefore, optionally, acquiring a target image taken for the area to be detected may specifically include: acquiring a target image taken by other devices for the area to be detected; or acquiring a target image taken by itself for the area to be detected.

Regarding the shooting content, optionally, due to different shooting angles, the captured target image may include all regions to be detected, or some regions to be detected. In the case that the captured target image only includes a part of the area to be detected, the position of the garbage can only be determined based on the preset area contained in the target image.

Optionally, an image captured in advance for the region to be detected may be determined as the target image, or an image captured in real time for the region to be detected may be determined as the target image.

Therefore, optionally, acquiring a target image taken for the area to be detected may specifically include: acquiring a target image taken in real time for the area to be detected; or acquiring a target image taken in advance for the area to be detected.

For the shooting result, optionally, there may be one or more target images.

In the garbage detection scenario, the garbage itself is not necessarily fixed at the same position, and the location of the garbage may change at any time. For example, discarded cans were kicked away, paper scraps and fallen leaves were blown away by the wind, etc.

Therefore, in order to improve the accuracy of garbage detection and garbage location conveniently, multiple target images may be obtained, so as to perform garbage detection and garbage location based on the multiple target images.

Different target images can contain different parts of the area to be detected, so that multiple parts of the area to be detected can be covered, and garbage detection and garbage location can be performed from different angles to improve the accuracy of garbage detection and garbage location.

Specifically, multiple images taken continuously for the region to be detected may be acquired, or multiple images taken for the region to be detected within a preset time period may be acquired.

Therefore, optionally, acquiring a target image taken for the region to be detected may include: acquiring a plurality of target images taken for the region to be detected; or acquiring a plurality of target images taken for the region to be detected within a preset time period; Or acquire multiple target images shot continuously for the area to be detected.

In this embodiment, the process of the method can be executed by acquiring multiple target images, so as to improve the accuracy of garbage detection and garbage location.

2. A detailed explanation of S102 is given below.

The flow of the method does not limit the method for specifically determining the position of the preset area in the target image.

Optionally, the position of the area to be detected mapped in the target image can be determined according to the pose of the target image when shooting the area to be detected, and further can be determined according to the actual position of the preset area contained in the area to be detected The location of the preset area map in the target image.

Optionally, determining the position of the divided preset area in the target image may include: determining the position, height and shooting angle of the camera when shooting the target image; determining the position of each preset area in the area to be detected; According to the position, height and shooting angle of the camera device, and the position of each preset area, the position of each preset area in the target image is determined.

Wherein, optionally, according to the position, height and shooting angle of the camera device, the actual position corresponding to the boundary of the captured target image can be determined, so that the distance between the actual position corresponding to the boundary of the target image and the actual position of the preset area can be determined. The positional relationship further determines the position of the preset area in the target image.

Optionally, the position of the preset area may also be characterized by the positions of multiple points. For example, for a rectangular preset area, the position of the preset area may be represented by the positions of four vertices. Afterwards, the position of the camera device in three-dimensional space can be determined according to the position and height of the camera device, so that the points used to represent the position of the preset area can be connected to the camera device, and then the points used to characterize the position of the preset area can be determined according to the shooting angle. The position of the point in the captured target image, so that the position of the preset area in the target image can be determined.

For ease of understanding, as shown in FIG. 2 , it is a schematic diagram of a principle of a preset area mapping method provided by an embodiment of this specification. There are two preset area mapping methods provided.

The area to be detected may be pre-divided into 4 square preset areas. For ease of description, FIG. 2 only shows the situation of mapping for preset area 1 . For ease of understanding, the position of the preset area 1 is represented by coordinates. Specifically (0,0), (0,1), (1,0) and (1,1).

In FIG. 2 , a preset area mapping method may be to photograph the area to be detected through the top view angle of the camera device. Wherein, the position of the camera is (0,0), the height is 2, and the shooting angle is 90 degrees. Therefore, for the captured square target image 1, it can be determined that the actual positions corresponding to the boundary of the target image 1 are (2,2), (2,-2), (-2,2) and (-2,-2 ).

Therefore, the corresponding position of the preset area 1 in the target image 1 can be determined according to the positional relationship between the position of the preset area 1 and the actual position of the boundary of the target image 1 .

Another preset area mapping method may be to determine the position of the camera in three-dimensional space according to the position and height of the camera. Specifically, the position of the camera in the three-dimensional space may be determined according to the position (0, -2) and the height 1 of the camera. After that, the range mapped by the target image 2 can be determined according to the shooting angle of 90 degrees. Wherein, it can be determined that the actual position (0, -1) falls on one of the boundaries of the target image 2 . Then connect each point used to represent the position of the preset area 1 with the position of the camera in three-dimensional space, and the position in the target image 2 mapped to each point used to characterize the position of the preset area 1 can be determined, Therefore, the position of the preset area 1 in the target image 2 can be determined.

Optionally, determining the position of the preset area in the target image may also be done in other ways.

For example, the position of the preset area can be identified by means of features in the area to be detected. Specifically, for a rectangular parking space drawn with a white line, the position of the preset area in the target image can be determined by identifying the position of the white line.

In this embodiment, the positional relationship between the camera device and the area to be detected can be determined through the relevant information of the camera device, and then the position of the preset area in the target image can be accurately determined, which facilitates subsequent garbage positioning.

In an optional embodiment, the position of the imaging device in the UTM coordinate system can be determined, and the position of the preset area in the UTM coordinate system can be determined. According to the relationship between the positions, it can be determined that the preset area is at location in the target image.

3. A detailed explanation of S103 is given below.

In S103 , mainly for the target image, a garbage detection method is used to determine a garbage detection result. Among them, the pre-trained garbage detection model can be used for detection.

1. Pretreatment.

Before using the garbage detection model for detection, preprocessing operations need to be performed on the target image. In an optional embodiment, the preprocessing operations may include operations such as scaling, normalization, and cropping.

Therefore, optionally, preprocessing may be performed on the target image; the preprocessing result is input into the pre-trained garbage detection model, and the garbage detection result is determined according to the output of the garbage detection model.

Optionally, since the position of the preset area in the target image is determined, and in general, only the content of the preset area is required for garbage detection and garbage location, the target image can be cropped to retain the preset area in the target image. By setting an area, the amount of data input into the garbage detection model can be reduced, thereby reducing the loss of computing resources and improving the efficiency of garbage detection.

Therefore, optionally, the preprocessing may include: cropping the target image, and retaining a preset area in the target image.

Of course, optionally, the preprocessing may also include: scaling the target image. By scaling the target image, the resolution of the image can be reduced, and the amount of data input to the garbage detection model can be reduced, thereby reducing the consumption of computing resources and improving the efficiency of garbage detection.

Optionally, the preprocessing may also include: performing normalization processing on the RGB channel data in the target image, so that it is convenient to input the preprocessing result into the garbage detection model for subsequent detection by the garbage detection model.

It should be noted that in unmanned cleaning equipment, it is usually necessary to use the local computing resources of the unmanned cleaning equipment for garbage detection. Therefore, improving the efficiency of garbage detection can make unmanned cleaning equipment increase the speed of garbage detection and save unmanned Clean up the local computing resources of the device.

2. Garbage detection model.

In an optional embodiment, the garbage detection model may specifically be constructed using a deep convolutional neural network. These can include base feature networks for extracting image features and detection heads for spam detection.

Optionally, the framework of the garbage detection model may adopt frameworks such as SSD, RetinaNet, CenterNet or YoLo. Among them, the framework of YoLov5 can be used for construction.

It should be noted that for the specific scenario of garbage detection, there are many specific problems. Specifically, it can include: the number of garbage samples is small, and the garbage itself is small. Therefore, based on the characteristics of the specific scene of garbage detection, the adaptability can be adjusted according to the needs of the garbage detection scene, and the garbage detection model can be adjusted.

In an optional embodiment, the garbage detection model itself can perform feature extraction by scaling the input image data. For example, feature pyramid networks.

However, in the scene of garbage detection, due to the existence of smaller garbage, such as paper scraps, fallen leaves, stones, etc., these smaller garbage maps are also smaller on the target image. If feature extraction is further performed by scaling, then It is easy to make the image features corresponding to these smaller garbage blurred, making it difficult to detect, and the effect of the garbage detection model is poor.

Therefore, for the garbage detection model, the degree of zooming can be limited, so that when zooming the image data, the loss of the image features corresponding to the garbage can be avoided as far as possible, thereby improving the effect of the garbage detection model.

Therefore, optionally, the scaling ratio of the zoomed image data may be limited, and the resolution of the zoomed image data may also be limited.

Optionally, the garbage detection model is used to perform garbage detection on at least one or more scaled copies of the target image; wherein, any scaled copy may have an image resolution greater than a preset resolution, or a scale ratio may be greater than a preset scale ratio .

In a specific embodiment, the garbage detection model may include a feature pyramid network.

A typical feature pyramid network can include 2, 3, 4, and 5 layers for extracting features through scaling. Specifically, 2 layers can be used to scale the input image data by 4 times, 3 layers can be used to scale the input image data by 8 times, 4 layers can be used to scale the input image data by 16 times, and 5 layers can be used for Scales by a factor of 32 for the input image data.

In this embodiment, however, the feature pyramid network in the garbage detection model may only include 2, 3, and 4 layers, so as to limit the scaling and avoid loss of image features of garbage.

In this embodiment, the image features of the garbage in the target image can be avoided by limiting the zoom ratio or the zoomed image resolution, thereby improving the detection effect of the garbage detection model and improving the accuracy of garbage detection.

In an optional embodiment, in the collected image samples, the number of garbage detection frames containing garbage is usually far less than that of non-garbage detection frames. Therefore, when training the garbage detection model, in the training sample set, usually The number of positive samples (junk detection boxes) is much less than negative samples (non-junk detection boxes).

In other words, in image samples, more candidate locations can be evaluated, and only a few of them contain objects (garbage), and the other candidate locations are all background images, resulting in far more negative samples than positive samples.

In order to improve the training efficiency and accuracy of the garbage detection model and avoid being dragged down by negative samples, a focal loss can optionally be introduced when building the garbage detection model. Focus loss can determine the weights of positive and negative samples separately, increase the weight of a small number of positive samples, and reduce the weight of a large number of negative samples, so as to improve the training efficiency, training effect and garbage detection accuracy of the garbage detection model.

Optionally, the loss function in the garbage detection model can include focal loss. Among them, the focus loss can be used to increase the weight of image samples labeled with garbage detection boxes.

In this embodiment, aiming at the problem of unbalanced samples in the garbage detection scene, the weight of positive samples can be increased by introducing focal loss, so as to improve the training efficiency, training effect and garbage detection accuracy of the garbage detection model.

In an optional embodiment, usually the image samples marked with garbage detection frames are difficult to label, and the acquired number is small. Therefore, the number of training samples in the garbage detection scene is small.

Since the basic feature network can be included in the garbage detection model, it is used to extract image features. In order to improve the representation ability of the garbage detection model, other labels similar to garbage can be introduced to train the basic feature network. For example, obstacles.

Therefore, optionally, the training method of the garbage detection model may include: determining the image sample marked with the obstacle detection frame label as the image sample marked with the garbage detection frame label, and training the basic feature network included in the garbage detection model.

Compared with the garbage detection frame, the obstacle detection frame is easy to label, and has more application scenarios, and the number of samples that can be obtained is larger. It can be used to increase the number of samples for training the basic feature network and improve the representation ability of the basic feature network.

In a specific example, when the unmanned cleaning device detects garbage, it also needs to detect obstacles because it also needs to clean. Therefore, the obstacle detection frame label and the garbage detection frame label can be used together to train the basic feature network to improve the representation ability, and then the same basic feature network can be used for garbage detection and obstacle detection to reduce the loss of computing resources.

In this embodiment, in view of the small number of samples in the garbage detection scene, it is possible to introduce similar samples, such as obstacle samples, to help train the basic feature network of the garbage detection model and improve the representation ability, so that the garbage detection model can be improved. detection effect.

3. Garbage detection results.

In an optional embodiment, the output of the garbage detection model usually includes a garbage detection frame, which can be used to mark the detected garbage in the target image, and can have a confidence level, that is, the marked image part Contains the credibility of garbage.

Optionally, the garbage detection model can also be used to identify the type of the detected garbage, for example, can identify the type of garbage contained in the garbage detection frame, which can specifically include: paper scraps, leaves, plastic bags, boxes, etc. Specifically, the type of garbage may be further identified for the detected garbage detection frame.

Therefore, optionally, the garbage detection frame output by the garbage detection model may also have the recognized garbage type.

Optionally, when the garbage detection frame output by the garbage detection model is obtained, the output garbage detection frame may be directly determined as the garbage detection result.

In the case of obtaining the garbage detection frame output by the garbage detection model, unreliable or repeated detection results can be further filtered, thereby improving the garbage detection accuracy of the garbage detection model.

Optionally, garbage detection frames with a confidence level lower than a preset confidence level may be deleted, or similar garbage detection frames may be considered as the same garbage detection frame.

In an optional embodiment, the garbage detection model may output multiple similar garbage detection frames for the same garbage. In order to filter these repeated detection results, filtering may be performed according to the intersection ratio between different garbage detection frames.

Wherein, the intersection ratio between different garbage detection frames may specifically be the ratio of the overlap area between two garbage detection frames to the merged area.

Optionally, for the target image, using a pre-trained garbage detection model to determine the garbage detection result may include: the output of the garbage detection model is a garbage detection frame, and the output garbage detection frame has a degree of confidence; In order to traverse the garbage detection frame output by the garbage detection model; if the intersection ratio between the currently traversed garbage detection frame and any other garbage detection frame is greater than the preset intersection ratio, delete the other garbage detection frame ; After the traversal, determine the remaining garbage detection frames as garbage detection results.

Optionally, specifically, for the currently traversed garbage detection frame, in other garbage detection frames whose confidence is lower than the currently traversed garbage detection frame, calculate the intersection and union between the currently traversed garbage detection frame and other garbage detection frames ratio, delete other garbage detection frames whose calculated intersection ratio is greater than the preset intersection ratio.

Optionally, in order to improve the garbage detection accuracy of the garbage detection model, multiple target images taken at the same location for the area to be detected can be obtained, so that multiple target images can be used for multiple garbage detections, and the garbage detection results can be determined comprehensively .

Specifically, it may be a plurality of target images taken continuously at the same position for the region to be detected, or a plurality of target images taken periodically or irregularly at the same position for the region to be detected.

Since it is taken at the same location, the garbage detection frames detected in different target images can be compared.

If the same garbage detection frame is detected in multiple target images, the garbage detection frame can be determined as the garbage detection result; if the garbage detection frame is only detected in one target image, while other target images If none of them can be detected, the garbage detection frame may not be determined as a garbage detection result.

Therefore, optionally, if the same garbage detection frame is detected in target images greater than the first preset number, the garbage detection frame can be determined as the garbage detection result; If two preset numbers of target images are detected, the garbage detection frame may not be determined as a garbage detection result, and specifically, the garbage detection frame may be deleted; the second preset number may be smaller than the first preset number.

In this embodiment, by post-processing the output of the garbage detection model, unreliable or repeated garbage detection frames can be filtered, thereby improving the garbage detection accuracy of the garbage detection model.

4. A detailed explanation of S104 is given below.

In the case of determining the garbage detection result, since the garbage contained in the target image has been clarified, and the position of the preset area in the target image has also been determined, the preset area containing garbage can be directly determined according to the target image .

Specifically, if any detected garbage is within the range of any preset area corresponding to the location in the target image, it may be determined that the preset area contains garbage.

Of course, since the actual garbage detection result may be a garbage detection frame, it is necessary to determine the garbage area based on the garbage detection frame. The garbage detection frame may be entirely contained within the position range of any preset area, or may be located within the position range of different preset areas. Specifically, the garbage area may be determined according to the center point of the garbage detection frame or the degree of overlap with the preset area.

In an optional embodiment, the garbage detection result is a garbage detection frame, and according to the determined garbage detection result, determining a preset area where garbage exists as a garbage area may include: setting the center point of any garbage detection frame The preset area where it is located is determined as a garbage area; or the preset area whose coincidence degree with any garbage detection frame satisfies the preset coincidence condition is determined as a garbage area.

Wherein, optionally, the preset coincidence condition may include: the coincidence degree is greater than the preset coincidence degree, or the coincidence degree is the highest.

For ease of understanding, as shown in FIG. 3 , it is a schematic diagram of the principle of a method for determining a garbage area provided by an embodiment of this specification.

It includes 4 preset regions mapped to the position range in the target image and 2 garbage detection frames, which are preset regions 1-4 and garbage detection frames 1-2 respectively. The preset areas are marked with numbers 1-4.

Obviously, since the center point of the garbage detection frame 1 is located in the preset area 2, it can be determined that there is garbage in the preset area 2, and the preset area 2 is a garbage area. The center point of the garbage detection frame 2 is the preset area 3, and it can be determined that there is garbage in the preset area 3, and the preset area 3 is a garbage area.

In this embodiment, based on the mapped position of the preset area in the target image, combined with the detected garbage detection frame, the preset area where garbage exists can be determined, and the detected garbage can be located efficiently, quickly and accurately, which is convenient for follow-up Cleaning route planning saves computing resources and improves computing efficiency.

In an optional embodiment, in order to improve the accuracy of determining the garbage area, the garbage area may be comprehensively determined according to the garbage detection results of multiple target images.

Therefore, optionally, acquiring a target image taken for the region to be detected may include: acquiring a plurality of target images taken for the region to be detected; or acquiring a plurality of target images taken for the region to be detected within a preset time period; Or acquire multiple target images shot continuously for the area to be detected.

Of course, the acquired multiple target images are not necessarily taken at the same position, and may respectively contain different parts of the region to be detected.

And if any preset area only detects the presence of garbage in one target image, but does not detect the presence of garbage in other target images, then the detection result may be wrong, or the position of the garbage has changed. Therefore, you can The preset area is not determined as a garbage area.

If rubbish is detected in multiple target images in any preset area, these multiple target images may be obtained by shooting the area to be detected from different positions. Then, there is likely to be garbage in the preset area, and the preset area can be determined as the garbage area.

Correspondingly, according to the determined garbage detection result, determining the preset area with garbage as the garbage area may include: if any preset area is in the garbage detection result determined for the target image of the preset number of images, all If garbage exists, the preset area may be determined as the garbage area.

In this embodiment, the accuracy of the garbage area can be improved by using the garbage detection results of multiple target images.

In the unmanned cleaning equipment, the above-mentioned embodiment can be used to determine the position of the preset area in the target image, so as to facilitate the positioning of the detected garbage, and it is also convenient for the subsequent unmanned cleaning equipment to plan cleaning routes. Garbage is cleaned up.

5. In addition, the cleaning route can be further determined in the process of the above method.

In an optional embodiment, after the garbage is detected and the location of the garbage is determined, cleaning may be performed on the detected garbage.

For example, unmanned cleaning equipment can detect and locate garbage based on local computing resources, and move and clean after determining the cleaning route.

Therefore, optionally, the process of the method may further include S105: determining a garbage area to be cleaned, and determining a cleaning route according to the garbage area to be cleaned.

Optionally, the determined cleaning route may include a garbage area to be cleaned.

Wherein, S105 may be performed after determining the garbage area, specifically, it may be performed after S104.

After the garbage area is determined, it is usually necessary to clean it by cleaning equipment and plan the cleaning route. To plan the cleaning route, it is necessary to determine the garbage area to be cleaned.

Wherein, the garbage area to be cleaned may specifically be a garbage area that needs to be cleaned later.

Optionally, each determined garbage area may be determined as a garbage area to be cleaned.

Optionally, the garbage area that meets the requirements may also be determined as the garbage area to be cleaned. In other words, there may be garbage areas that do not meet the requirements and do not need to be cleaned temporarily, so it does not need to be considered when planning the cleaning route.

For example, when a car is detected on the parking space in the preset area, even if it is determined that the parking space contains garbage, it cannot be cleaned because of the obstacle of the car, so that parking in the preset area can not be determined as waiting Garbage area for sweeping.

Of course, optionally, the garbage area containing a large amount of garbage may be determined as the garbage area to be cleaned, or the garbage area containing a specific type of garbage may be determined as the garbage area to be cleaned.

In an optional embodiment, the determined garbage area needs to be cleaned as a whole.

According to the garbage areas to be cleaned, the cleaning route is determined. Optionally, a route passing through all the garbage areas to be cleaned may be determined.

Optionally, start from any garbage area to be cleaned, randomly select an unpassed garbage area from other nearest garbage areas to be cleaned, move to the selected garbage area, and perform overall cleaning.

For ease of understanding, as shown in FIG. 4 , it is a schematic diagram of the principle of cleaning route planning provided by the embodiment of this specification. It includes preset areas 1-9, and each preset area is marked by a number contained therein, and preset areas 1, 3, 4, and 9 are determined as garbage areas.

Starting from the preset area 1, the nearest preset area 4 can be determined and moved to the preset area 4; since the distance between the preset area 4 and the preset areas 3 and 9 is the same, the preset area can be randomly selected Area 3, move to preset area 3, and then move to preset area 9.

Thus, the cleaning route can be obtained.

In this embodiment, the cleaning route can be determined quickly and efficiently based on the garbage area where the garbage is located and determined based on the garbage location, so as to avoid the loss of computing resources. Especially in unmanned cleaning equipment, it can save local computing resources of unmanned cleaning equipment.

In addition to the cleaning route, when cleaning, in an optional embodiment, different cleaning methods may also be determined for different types of garbage.

It should be noted that for small garbage, such as paper scraps, leaves, etc., usually can be cleaned by wind suction, while for larger garbage, such as boxes, cans, etc., tools, such as brooms, etc., can usually be used to clean up. clean up.

Therefore, when cleaning, the cleaning method can be determined according to the type of garbage identified by the garbage detection model.

Therefore, optionally, the above-mentioned method flow may also include: obtaining the corresponding relationship between garbage types and cleaning methods; according to the determined garbage detection results, determining the types of garbage contained in the garbage area; the garbage detection model can also be used to detect garbage types ; Determine the corresponding cleaning method according to the type of garbage contained in any garbage area.

Wherein, since there may be one or more types of garbage contained in the garbage area, one or more corresponding cleaning methods may be determined.

Optionally, different cleaning methods may have priorities, so that the cleaning method with the highest priority may be selected; optionally, each determined cleaning method may be used to clean once to improve the cleaning effect.

In this embodiment, according to the type of garbage detected by the garbage detection model, a corresponding cleaning method can be determined, so that subsequent cleaning can be facilitated, and the cleaning of garbage can be more convenient and thorough.

For unmanned cleaning equipment, after garbage detection and garbage location, the cleaning method can be determined quickly and efficiently according to the type of garbage contained in each garbage area, so that when the unmanned cleaning equipment specifically cleans any garbage area, Use the corresponding cleaning method to clean to improve cleaning efficiency.

6. In the process of the above method, obstacles may be further determined.

In the case of garbage detection, if unmanned cleaning equipment is used to clean the detected garbage, it is usually necessary to detect obstacles, so as to avoid the detected obstacles while cleaning the detected garbage.

For example, for an unmanned sweeper, it is necessary to avoid obstacles while cleaning up garbage. Therefore, for the area that needs to be cleaned, the unmanned sweeper needs to perform both garbage detection and obstacle detection.

There are many methods for obstacle detection, and this method flow does not limit a specific obstacle detection method.

Optionally, target detection may be used for obstacle detection, and an obstacle detection model may be used to determine obstacles in the target image; laser radar or the like may also be used for scanning or the like.

In an optional embodiment, multiple obstacle detection methods can be used, and the detection results of multiple obstacle detection methods can be combined to determine a preset area where an obstacle exists, that is, an obstacle area, so that Use unmanned cleaning equipment to avoid obstacle areas.

Optionally, the detection results of multiple obstacle detection methods may be used as the obstacle detection results to determine the obstacle area.

Wherein, optionally, using laser radar to scan for obstacles may be based on the point cloud data acquired by laser radar to determine a preset area where obstacles exist.

Specifically, for the point cloud data obtained by the lidar, the ground plane points are obtained according to the ground plane fitting method, and the remaining points are used as obstacle points. Then the three-dimensional obstacle point cloud is projected into the target image through the projection formula, and the number of obstacle points is counted for the preset area. If it is greater than the threshold, it is considered as an obstacle area with obstacles.

Optionally, based on images, obstacles can be determined by using a target detection method.

Therefore, optionally, the obstacle detection results based on images and point cloud data can be fused to determine the combined obstacle detection results, so as to facilitate the determination of obstacle areas where obstacles exist.

The execution sequence of obstacle detection and garbage detection can be executed in parallel or sequentially. This embodiment is not limited.

In an optional embodiment, after the obstacle is detected, the detected obstacle may be positioned according to the obstacle detection result, specifically, a preset area where the obstacle exists may be determined as the obstacle area. Specifically, the obstacle area may be a preset area where obstacles exist.

Optionally, the determination of the obstacle area and the garbage area may be performed in parallel or sequentially, which is not limited in this embodiment.

Among them, since the determination of the garbage area is the garbage detected by the positioning, and the garbage detected by the positioning is usually used to plan the cleaning route, and the obstacle area usually needs to be avoided, so it can be determined according to the determined garbage area and the obstacle area. Cleaning route, the determined cleaning route can pass through the garbage area that needs to be cleaned, and will not pass through any obstacle area.

For example, for the 5 parking spaces 1-5 contained in the parking lot, it is determined that there are cars parked in the parking spaces 1 and 3, and garbage exists in the parking spaces 2, 4, and 5. Therefore, it can be determined that parking spaces 1 and 3 are obstacle areas, and parking spaces 2, 4, and 5 are garbage areas. The cleaning route can be planned from parking space 2 to parking space 4 and then to parking space 5. Each parking space needs to be cleaned as a whole.

In addition, optionally, there may be a preset area, which is determined as both a garbage area and an obstacle area.

From the perspective of garbage detection, the preset area can be determined as both a garbage area and an obstacle area.

However, from the perspective of cleaning route planning, there is a contradiction because the cleaning route needs to pass through the garbage area and avoid the obstacle area.

In order to solve this contradiction, obstacles, such as cars and pedestrians, are difficult to move. Therefore, when planning the cleaning route, the garbage area determined as the obstacle area may not be considered.

In other words, determining the garbage area that needs to be cleaned may specifically include: determining the garbage area in the non-obstacle area as the garbage area that needs to be cleaned, so that the cleaning route can be planned only for the garbage area in the non-obstacle area.

Of course, when determining the garbage area, a preset area that contains garbage and belongs to the non-obstacle area may also be determined as the garbage area according to the pre-determined obstacle area.

Therefore, optionally, the above method flow may further include S106: Detecting obstacles in the area to be detected, and determining a preset area where obstacles exist as an obstacle area.

Optionally, determining the preset area where garbage is detected as the garbage area according to the determined garbage detection result may include: determining the non-obstacle area where garbage is detected as the garbage area according to the determined garbage detection result .

Optionally, determining the garbage area to be cleaned may include: determining the garbage area in the non-obstacle area as the garbage area to be cleaned.

Optionally, when planning the cleaning route, it is also possible to determine the garbage area belonging to the obstacle area and the garbage area in the non-obstacle area, so that the cleaning route can be determined according to the garbage area in the non-obstacle area, so that the planned cleaning route includes non-obstacle areas. The garbage area of the area, does not include the garbage area that belongs to the obstacle area, and does not include any obstacle area.

In the above embodiments, obstacle detection can be further combined to help plan cleaning routes and improve garbage cleaning efficiency.

Of course, optionally, after the obstacle area is determined, the image content contained in the obstacle area in the target image can be directly deleted, and garbage detection is performed on the deleted target image, which can save computing resources.

In an optional embodiment, obstacle detection may be performed by a target detection method. Therefore, detecting an obstacle in the region to be detected may include: using a pre-trained obstacle detection model to determine an obstacle detection result for the target image.

Among them, optionally, in order to save computing resources, since both obstacle detection and garbage detection usually need to extract image features for the target image, when training the obstacle detection model and garbage detection model, optionally, the labels marked with The image samples of the obstacle detection frame label, and the image samples marked with the garbage detection frame label, jointly train the basic feature network of the obstacle detection model and the garbage detection model, and obtain the same basic feature network, which can improve the representation of the basic feature network ability, improve the training effect of obstacle detection model and garbage detection model, and save computing resources.

Of course, optionally, for the detection head of the obstacle detection model, only the image samples marked with the obstacle detection frame label need to be used for training, and the parameters of the basic feature network can be fixed.

Therefore, optionally, the basic feature networks included in the obstacle detection model and the garbage detection model are trained using the same training sample set, and the trained basic feature networks are the same. The training sample set may include: image samples marked with obstacle detection frame labels, and image samples marked with garbage detection frame labels.

It should be noted that the garbage detection model in the embodiment of this specification can be trained by collecting data of different garbage in different scenarios, and output the garbage detection frame and garbage category in the detection head.

In order to reduce the loss of model computing resources and improve computing speed, while reducing the requirements for training data, in an optional embodiment, the method of sharing the basic feature network with the obstacle detection model but independently training the detection head can be used .

For ease of understanding, as shown in FIG. 5 , it is a schematic schematic diagram of a model structure provided by an embodiment of this specification. Among them, the obstacle detection model and the garbage detection model share the basic feature network. When the target image is used for garbage detection and obstacle detection, the feature extraction of the basic feature network can only be performed once without performing separate executions, thereby saving computing resources and improving computing speed and efficiency.

On the one hand, the basic feature network occupies the most computing resources in the deep learning model, and sharing this part of the calculation can greatly reduce the consumption of computing resources.

On the other hand, both obstacle detection and garbage detection are performed in the same scene, but the detection targets are different, so many basic features for detection can be shared.

Moreover, since obstacle data is easier to obtain than garbage data, the amount of training data is much larger than garbage detection training data. Therefore, the shared basic features can also greatly reduce the need for separate training data for garbage detection, so as to achieve the target accuracy faster.

The specific way to train the detection model of the shared basic feature network is to train a detection model together on the obstacle and garbage detection data to obtain the common basic feature network parameters.

Specifically, both the obstacle detection frame label and the garbage detection frame label can be regarded as the target detection frame label, which is used to train a detection model and obtain the basic feature network parameters.

Afterwards, based on the obtained basic feature network parameters, a garbage detection model and an obstacle detection model can be constructed respectively. The basic feature network parameters can be fixed during the training process, and the detection head of the obstacle detection model can be trained using the obstacle data. part, use the garbage detection data to train the detection head of the garbage detection model.

In an optional embodiment, after the obstacle area to be avoided is determined, a more accurate and suitable cleaning route can be further determined.

Optionally, the process of the above method may further include: determining a garbage area that needs to be cleaned and an obstacle area that needs to be avoided, and determining a cleaning route based on the determined garbage area and obstacle area.

For ease of understanding, as shown in FIG. 6 , it is a schematic diagram of another cleaning route planning provided by the embodiment of this specification.

It contains preset areas 1-9, and each preset area is marked by a number contained therein. Preset areas 1, 3, 4, and 9 are determined as garbage areas, and preset areas 5 and 6 are determined as obstacle areas.

Starting from the preset area 1, the nearest preset area 4 can be determined and moved to the preset area 4; since the distance between the preset area 4 and the preset areas 3 and 9 is the same, the preset area can be randomly selected Area 3, moving to the preset area 3, since moving from the preset area 4 to the preset area 3 in a straight line will pass through the preset area 5 (obstacle area), therefore, it can go around through the preset areas 1 and 2.

Moving to the preset area 9, since the preset areas 5 and 6 are obstacle areas, it is possible to detour from outside the preset area 6, thereby obtaining a cleaning route.

For ease of understanding, the embodiment of this specification also provides an application embodiment. It explains for unmanned cleaning equipment.

The unmanned cleaning equipment can take pictures of the area to be detected through its own camera device to obtain the target image. The area to be detected may be a hall of a shopping mall. The area to be detected is pre-divided into multiple preset areas, and each preset area is a square floor tile in the hall of the shopping mall.

Unmanned cleaning equipment can detect garbage and obstacles at the same time. Specifically, the target image can be input into the basic feature network, and then the image features output by the basic feature network can be input into the detection head and obstacle in the garbage detection model. The detection head in the object detection model determines the garbage detection result and the obstacle detection result. Specifically, obstacles may be pedestrians, goods, and the like.

In addition, the unmanned cleaning device can also determine the position of the preset area in the area to be detected in the target image, specifically, it can determine the position of the square floor tiles in the hall of the shopping mall in the target image.

Combined with the position of the square floor tiles in the target image, as well as the detected garbage and obstacles, the unmanned cleaning equipment can determine the square floor tiles with garbage and the square floor tiles with obstacles, and then plan the cleaning route to avoid obstacles Clean the square floor tiles with rubbish.

In the process of the above method, based on the location mapping of the preset area, specifically, the mapping of map information, the preset area can be mapped to the target image, which is convenient for garbage positioning, so that it can serve unmanned cleaning, and the obtained garbage positioning The information can be used by unmanned sweepers for efficient garbage cleaning path planning.

In addition, the garbage detection model can also share the basic feature network with the obstacle detection model, thereby reducing computing resource consumption, reducing training data requirements, and improving representation capabilities.

It is also possible to classify garbage on the basis of garbage detection and determine the cleaning method to better serve unmanned cleaning planning.

Corresponding to the above method flow, the embodiment of this specification also provides an apparatus embodiment.

As shown in FIG. 7 , it is a schematic structural diagram of a garbage detection device provided in the embodiment of this specification.

Wherein, the area to be detected can be divided into at least two preset areas in advance. The rubbish detection device may include the following units.

The obtaining unit 201 is configured to obtain a target image taken for the region to be detected.

The mapping unit 202 is configured to determine the position of the divided preset area in the target image.

The detection unit 203 is configured to determine a garbage detection result by using a pre-trained garbage detection model for the target image.

The positioning unit 204 is configured to determine a preset area where garbage exists as a garbage area according to the determined garbage detection result.

Optionally, the mapping unit 202 can be used to: determine the position, height and shooting angle of the camera when shooting the target image; determine the position of each preset area in the area to be detected; according to the position, height and shooting angle of the camera , and the position of each preset area to determine the position of each preset area in the target image.

Optionally, the detection unit 203 may include: a preprocessing subunit 203a, configured to perform preprocessing on the target image.

The detection subunit 203b is configured to input the preprocessing result into the pre-trained garbage detection model, and determine the garbage detection result according to the output of the garbage detection model.

Optionally, the preprocessing subunit 203a may be configured to: crop the target image, and retain a preset area in the target image.

Optionally, the garbage detection model is at least used to perform garbage detection on one or more scaled copies of the target image; wherein, any scaled copy has an image resolution greater than a preset resolution, or a scale ratio greater than a preset scale ratio.

Optionally, the loss function in the garbage detection model includes a focus loss; the focus loss is used to increase the weight of image samples marked with garbage detection frame labels.

Optionally, the garbage detection model may include a basic feature network; the training method of the garbage detection model may include: determining an image sample marked with an obstacle detection frame label as an image sample marked with a garbage detection frame label, and training the garbage detection model Included base feature network.

Optionally, the output of the garbage detection model is a garbage detection frame, and the output garbage detection frame has a confidence level; the detection unit 203 may include: a traversal subunit 203c for traversing the garbage detection The garbage detection box output by the model.

The deletion subunit 203d is used to delete the other garbage detection frame when the intersection ratio between the currently traversed garbage detection frame and any other garbage detection frame is greater than the preset intersection ratio; after the traversal, the The remaining garbage detection boxes are determined as garbage detection results.

Optionally, the garbage detection result is a garbage detection frame, and the positioning unit 204 can be used to: determine the preset area where the center point of any garbage detection frame is located as the garbage area; or determine the coincidence degree with any garbage detection frame The preset area that satisfies the preset coincidence condition is determined as a garbage area.

Optionally, the acquiring unit 201 can be used to: acquire multiple target images taken for the area to be detected; or acquire multiple target images taken for the area to be detected within a preset time period; or acquire continuous shooting of the area to be detected multiple target images.

Correspondingly, the positioning unit 204 may be configured to: if any preset area contains garbage in the garbage detection results determined for the preset number of target images, determine the preset area as a garbage area.

Optionally, the garbage detection device may further include: a cleaning route determining unit 205, configured to determine a garbage area to be cleaned, and determine a cleaning route according to the garbage area to be cleaned.

Optionally, the garbage detection device may also include: a cleaning mode determination unit 206, configured to obtain the correspondence between garbage types and cleaning modes; according to the determined garbage detection results, determine the types of garbage contained in the garbage area; the garbage detection model It is also used to detect the type of garbage; according to the type of garbage contained in any garbage area, determine the corresponding cleaning method.

Optionally, the garbage detection device may further include: an obstacle detection unit 207, configured to detect obstacles in the area to be detected, and determine a preset area where obstacles exist as an obstacle area.

Correspondingly, the positioning unit 204 may be configured to: determine the non-obstacle area in which garbage is detected as the garbage area according to the determined garbage detection result.

Optionally, the obstacle detection unit 207 may be configured to: use a pre-trained obstacle detection model to determine an obstacle detection result for the target image.

Optionally, the basic feature network included in the obstacle detection model and the garbage detection model can be trained using the same training sample set, and the trained basic feature network is the same; the training sample set can include: an image marked with an obstacle detection frame label samples, and image samples annotated with spam detection box labels.

For detailed explanations of the above device embodiments, reference may be made to the explanations of the above method flow.

The embodiment of this specification also provides a computer device, which can be specifically an unmanned cleaning device, which at least includes a memory, a processor, and a computer program stored in the memory and operable on the processor, wherein the processor executes the program When implementing a garbage detection method in any one of the above method embodiments.

FIG. 8 shows a schematic diagram of a more specific hardware structure of a computer device provided by the embodiment of this specification. The device may include: a processor 1010 , a memory 1020 , an input/output interface 1030 , a communication interface 1040 and a bus 1050 . The processor 1010 , the memory 1020 , the input/output interface 1030 and the communication interface 1040 are connected to each other within the device through the bus 1050 .

The processor 1010 may be implemented by a general-purpose CPU (Central Processing Unit, central processing unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute related programs to realize the technical solutions provided by the embodiments of this specification.

The memory 1020 can be implemented in the form of ROM (Read Only Memory, read-only memory), RAM (Random Access Memory, random access memory), static storage device, dynamic storage device, etc. The memory 1020 can store operating systems and other application programs. When implementing the technical solutions provided by the embodiments of this specification through software or firmware, the relevant program codes are stored in the memory 1020 and invoked by the processor 1010 for execution.

The input/output interface 1030 is used to connect the input/output module to realize information input and output. The input/output/module can be configured in the device as a component (not shown in the figure), or can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, mouse, touch screen, microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, and the like.

The communication interface 1040 is used to connect a communication module (not shown in the figure), so as to realize the communication interaction between the device and other devices. The communication module can realize communication through wired means (such as USB, network cable, etc.), and can also realize communication through wireless means (such as mobile network, WIFI, Bluetooth, etc.).

Bus 1050 includes a path that carries information between the various components of the device (eg, processor 1010, memory 1020, input/output interface 1030, and communication interface 1040).

It should be noted that although the above device only shows the processor 1010, the memory 1020, the input/output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components. In addition, those skilled in the art can understand that the above-mentioned device may only include components necessary to implement the solutions of the embodiments of this specification, and does not necessarily include all the components shown in the figure.

The embodiment of this specification also provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, a garbage detection method in any of the above method embodiments is implemented.

Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can be implemented by any method or technology for storage of information. Information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Flash memory or other memory technology, Compact Disc Read-Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, A magnetic tape cartridge, disk storage or other magnetic storage device or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media excludes transitory computer-readable media, such as modulated data signals and carrier waves.

It can be known from the above description of the implementation manners that those skilled in the art can clearly understand that the embodiments of this specification can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the essence of the technical solutions of the embodiments of this specification or the part that contributes to the prior art can be embodied in the form of software products, and the computer software products can be stored in storage media, such as ROM/RAM, A magnetic disk, an optical disk, etc., include several instructions to enable a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this specification.

The systems, devices, modules, or units described in the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementing device is a computer, which may take the form of a personal computer, laptop computer, cellular phone, camera phone, smart phone, personal digital assistant, media player, navigation device, e-mail device, game control device, etc. desktops, tablets, wearables, or any combination of these.

Each embodiment in this specification is described in a progressive manner, the same and similar parts of each embodiment can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, as for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and for relevant parts, please refer to part of the description of the method embodiment. The device embodiments described above are only illustrative, and the modules described as separate components may or may not be physically separated, and the functions of each module may be integrated in the same or multiple software and/or hardware implementations. Part or all of the modules can also be selected according to actual needs to achieve the purpose of the solution of this embodiment. It can be understood and implemented by those skilled in the art without creative effort.

The above is only the specific implementation of the embodiment of this specification. It should be pointed out that for those of ordinary skill in the art, without departing from the principle of the embodiment of this specification, some improvements and modifications can also be made. These Improvements and modifications should also be regarded as protections for the embodiments of this specification.

Claims (18)

1. A garbage detection method, which divides the area to be detected into at least two preset areas in advance; the method includes:

   Acquiring a target image taken for the region to be detected;

   determining the position of the divided preset area in the target image;

   For the target image, using a pre-trained garbage detection model to determine a garbage detection result;

   According to the determined garbage detection result, a preset area where garbage exists is determined as a garbage area.
2. The method according to claim 1, the determining the position of the divided preset area in the target image comprises:

   When determining the shooting of the target image, the position, height and shooting angle of the camera;

   determining the position of each preset area in the area to be detected;

   The position of each preset area in the target image is determined according to the position, height and shooting angle of the camera device, and the position of each preset area.
3. The method according to claim 1, said using a pre-trained garbage detection model to determine a garbage detection result for said target image, comprising:

   Preprocessing the target image;

   The preprocessing result is input into the pre-trained garbage detection model, and the garbage detection result is determined according to the output of the garbage detection model.
4. The method according to claim 3, wherein the preprocessing comprises: cropping the target image, and retaining a preset area in the target image.
5. The method of claim 1, wherein the garbage detection model is used to perform garbage detection on at least one or more scaled copies of the target image; wherein any scaled copy has an image resolution greater than a preset resolution, or The zoom ratio is greater than the preset zoom ratio.
6. The method according to claim 1, wherein the loss function in the garbage detection model includes a focus loss; and the focus loss is used to increase the weight of image samples labeled with garbage detection frame labels.
7. The method according to claim 1, the garbage detection model comprising a basic feature network; the training method of the garbage detection model comprising:

   The image samples marked with the obstacle detection frame label are determined as the image samples marked with the garbage detection frame label, and the basic feature network included in the garbage detection model is trained.
8. The method according to claim 1, said using a pre-trained garbage detection model to determine a garbage detection result for said target image, comprising:

   The output of the garbage detection model is a garbage detection frame, and the output garbage detection frame has a degree of confidence;

   According to the order of confidence from large to small, traverse the garbage detection frame output by the garbage detection model;

   When the currently traversed garbage detection frame has an intersection ratio with any other garbage detection frame that is greater than a preset intersection ratio, delete any other garbage detection frame;

   After the traversal, the remaining garbage detection frames are determined as garbage detection results.
9. The method according to claim 1, wherein the garbage detection result is a garbage detection frame, and according to the determined garbage detection result, determining a preset area where garbage exists as a garbage area includes:

   Determining the preset area where the center point of any garbage detection frame is located as the garbage area; or

   A preset area whose coincidence degree with any garbage detection frame satisfies a preset coincidence condition is determined as a garbage area.
10. The method according to claim 1, said acquiring the target image taken for said region to be detected comprises:

    Acquiring a plurality of target images taken for the region to be detected; or acquiring a plurality of target images taken for the region to be detected within a preset period of time; or acquiring a plurality of target images continuously taken for the region to be detected ;

    According to the determined garbage detection result, determining the preset area where garbage exists as the garbage area includes:

    If any preset area contains garbage in the garbage detection results determined for the preset number of target images, then any preset area is determined as a garbage area.
11. The method according to claim 1, further comprising:

    A garbage area to be cleaned is determined, and a cleaning route is determined according to the garbage area to be cleaned; the cleaning route includes the garbage area to be cleaned.
12. The method according to claim 1, further comprising:

    Obtain the corresponding relationship between garbage types and cleaning methods;

    According to the determined garbage detection result, determine the type of garbage contained in the garbage area; the garbage detection model is also used to detect the type of garbage;

    According to the type of garbage contained in any garbage area, determine the corresponding cleaning method.
13. The method according to claim 1, further comprising:

    Detecting obstacles in the area to be detected, and determining a preset area where obstacles exist as an obstacle area;

    According to the determined garbage detection result, determining the preset area where garbage is detected as the garbage area includes:

    According to the determined garbage detection result, the non-obstacle area in which garbage is detected is determined as the garbage area.
14. The method according to claim 13, said detecting obstacles in said region to be detected, comprising:

    For the target image, a pre-trained obstacle detection model is used to determine an obstacle detection result.
15. The method according to claim 14, the basic feature network included in the obstacle detection model and the garbage detection model is trained using the same training sample set, and the basic feature network obtained through training is the same; the training sample set includes: Image samples labeled with obstacle detection boxes, and image samples labeled with garbage detection boxes.
16. A garbage detection device, which divides the area to be detected into at least two preset areas in advance; the device includes:

    an acquisition unit, configured to acquire a target image taken for the region to be detected;

    a mapping unit, configured to determine the position of the divided preset area in the target image;

    A detection unit, configured to use a pre-trained garbage detection model to determine a garbage detection result for the target image;

    The positioning unit is configured to determine a preset area where garbage exists as a garbage area according to the determined garbage detection result.
17. An unmanned cleaning device, comprising a memory, a processor, and a computer program stored on the memory and operable on the processor, wherein, when the processor executes the program, the process described in any one of claims 1 to 15 is realized. described method.
18. A computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the method according to any one of claims 1 to 15 is implemented.

Images