CN111567331A - A grass garbage automatic cleaning machine and method based on deep convolutional neural network - Google Patents

Abstract

The present invention relates to the technical field of garbage disposal equipment, in particular to an automatic lawn garbage cleaning machine and method based on a deep convolutional neural network, comprising a frame and a walking assembly arranged on the frame, and further comprising: a cleaning assembly, The cleaning component is used to clean the garbage on the grass; the image acquisition device is used to collect the environmental image of the grass; the central controller, the image acquisition device and the cleaning component are all electrically connected with the central controller; the central controller is provided with a garbage identification device, It is electrically connected with the image acquisition device, and is used to process the environmental image of the grassland collected by the image acquisition device, and identify whether there is an image of garbage in the environmental image of the grassland; the garbage identification device recognizes that there is an image of garbage in the environmental image of the grassland , the central controller controls the cleaning component to clean up garbage; the image acquisition device, the cleaning component and the central controller are all arranged on the rack. The present invention has the advantage of improving the efficiency of garbage cleaning.

Description

An automatic lawn garbage cleaning machine and method based on deep convolutional neural network

technical field

The invention relates to the technical field of garbage disposal equipment, in particular to an automatic lawn garbage cleaning machine and method based on a deep convolutional neural network.

Background technique

In order to better improve people's quality of life, the current urban green area is increasing year by year. Some public places such as schools and parks have lawns. Urban lawns can purify the air and absorb carbon dioxide, sulfur dioxide, hydrogen fluoride, ammonia, chlorine, etc. in the atmosphere. Toxic and harmful gases. Grass can absorb a lot of carbon dioxide every hour, while releasing a lot of oxygen. The lawn can adjust the atmospheric temperature and humidity, and each hectare of lawn evaporates a lot of water every day, increasing the relative humidity in the air. The lawn can be vacuumed and sterilized, and the vacuuming capacity of the grass is 70 times greater than that of the bare field. Lawns can reduce noise pollution, and large squares with lawns can reduce noise. Lawns can also retain water and drought, beautify the environment and regulate the climate, but with the increase of people's activities, a lot of garbage often appears on the lawn.

The Chinese patent with the authorization announcement number CN104996175B discloses a high-efficiency hand-push lawn garbage cleaning equipment, which includes a base, a wheel frame, a ground wheel, a pick rod, a main casing and a control box. The lower part of the base is provided with four wheel frames. A ground wheel is installed on the wheel frame, a guide rail is arranged between the push handles, and a garbage bin is arranged on the guide rail. A pick-up rod is set on the roller, a main casing is fixedly installed on the base through a screw, a discharge door is installed on the right side of the main casing through a loose leaf, the discharge door is opposite to the garbage box, and a control box is installed on the upper part of the main casing. The pick-up roller is driven to rotate by the pick-up motor, and the pick-up rod on the pick-up roller picks up the garbage and leaves on the grass, and finally reaches the conveyor belt through centrifugal action, and imports it into the trash can that comes with the equipment to complete the process of picking up garbage. Work.

The prior art has the following technical defects: the above cleaning equipment still needs to manually identify the garbage, and after identifying the garbage, the control box is manually controlled to pick up the garbage, and the garbage cleaning efficiency is low.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide an automatic lawn garbage cleaning machine and method based on a deep convolutional neural network, which has the advantage of improving the efficiency of garbage cleaning.

In order to achieve the above purpose, the technical solution adopted in the present invention is: an automatic lawn garbage cleaning machine based on a deep convolutional neural network, comprising a frame and a walking assembly arranged on the frame, and further comprising:

a cleaning component, which is used for cleaning garbage on the grass;

an image acquisition device for acquiring environmental images of the grass;

a central controller, the image acquisition device and the cleaning assembly are all electrically connected to the central controller;

The central controller is provided with a garbage identification device, which is electrically connected to the image acquisition device, and is used for processing the environmental image of the grassland collected by the image acquisition device, and to identify whether there is an image of garbage in the environmental image of the grassland; When the garbage identification device recognizes that there is an image of garbage in the environmental image of the grass, the central controller controls the cleaning component to clean up garbage;

The image acquisition device, the cleaning assembly and the central controller are all arranged on the frame.

Preferably, the cleaning assembly includes a cleaning broom, a collecting shovel and a garbage bin, the cleaning broom includes a rotating rod and a broom head, one end of the rotating rod is rotatably arranged on the frame, and the frame is provided with a rotating rod and a broom head. a first steering gear that drives the rotating rod to rotate, and the other end of the rotating rod is connected with a broom head;

The garbage box is arranged on the frame, and a rotating shaft is rotatably connected to the side of the garbage box close to the cleaning broom, and the length direction of the rotating shaft is perpendicular to the length direction of the rotating rod. In the collection shovel, the trash box is provided with a second steering gear that drives the rotating shaft to rotate around the axis of the rotating shaft.

Preferably, the garbage identification device includes

The training module is used to train the convolutional neural network using multiple grass environment images to obtain a deep convolutional neural network model;

A target detection model for identifying images of garbage in multiple grass environment images;

The image preprocessing module is used to perform histogram equalization processing and logarithmic transformation processing on the environmental image of the grass, obtain the processed environmental image of the grass, and send the processed environmental image of the grass to the training module and the target detection model. .

Preferably, the deep convolutional neural network model is Tiny-YOLOv3.

Through the above technical solutions, compared with the previous YOLO algorithm, YOLOv3 adopts DarkNet53 with higher accuracy as the feature extraction network, designs the target multi-scale detection structure, and uses the logistic function to replace the traditional softmax function. DarkNet53 draws on the idea of ResNet residual network, and sets up a shortcut path between some layers. Research shows that DarkNet53 is similar in accuracy but faster than ResNet-152.

Preferably, the Tiny-YOLOv3 model uses a binary cross-entropy loss function for category prediction,

Among them, N is the total number of training images; yi takes a value of 0 or 1, yi takes a value of 1, which means that the ith input image contains garbage images, and a yi value of 0 means that the ith input image does not contain Image of garbage; pi value is the predicted probability of whether the ith input image contains garbage image, pi value is between 0 and 1.

Preferably, the deep convolutional neural network model includes a DarkNet framework, and the DarkNet framework includes 53 convolutional layers and 22 Residual layers, and the 53 convolutional layers in the DarkNet framework are used to perform image processing on the environment image of the grass. Feature extraction, the 22 Residual layers in the DarkNet framework are used to solve gradient dispersion or gradient explosion in deep convolutional neural network models. On the one hand, the Darknet-53 network adopts a fully convolutional structure. During the forward propagation of Yolo v3, the size transformation of the tensor is achieved by changing the stride of the convolution kernel. The stride of the convolution is 2. After each convolution, the side length of the image is reduced by half. On the other hand, the Darknet-53 network introduces the Residual structure. Yolo v2 is still a straight-tube network structure like VGG. If there are too many layers, there will be gradient problems in training, so Darknet-19 also has 19 layers. Thanks to the residual structure of ResNet, the difficulty of training deep networks is greatly reduced. Therefore, the Darknet-53 network achieves 53 layers, and the accuracy improvement is relatively obvious.

Preferably, the training module uses stochastic gradient descent to optimize the Tiny-YOLOv3 model. Stochastic Gradient Descent (SGD) is an extension of the gradient descent algorithm. The core of stochastic gradient descent is this: the gradient is the expectation. It is expected that a small sample size can be used to approximate the estimate. Batch Gradient Descent (BGD): It is the most primitive form of gradient descent. All sample data in the training set must be used for each iteration step or each parameter update. When the number of samples is large, the training process will very slow. Stochastic Gradient Descent (SGD): Since batch gradient descent requires all training samples when updating each parameter, the training process becomes abnormally slow as the number of samples increases. The stochastic gradient descent method is proposed to solve the drawback of the batch gradient descent method. Stochastic gradient descent is iteratively updated once per sample. A problem with SGD is that there is more noise than BGD, so that SGD does not proceed towards the optimization direction every iteration.

An automatic cleaning method for grass garbage based on a deep convolutional neural network, comprising the following steps:

S1: Use multiple grass environment images to train the convolutional neural network, obtain the Tiny-YOLOv3 model, and execute S2;

S2: Build an environmental grid map with the initial position as the origin, and execute S3;

S3: Acquire multiple real-time grass environment images, and execute S4;

S4: Identify whether there are garbage images in the multiple real-time grass environment images, if so, execute S5, if not, execute S6;

S5: Obtain the specific position and frame of the image of the garbage in the grass environment image, perform garbage cleaning, and execute S6;

S6: Move to the next grid, and execute S3.

Preferably, the S4 further comprises the following steps,

S41: Set the IOU threshold and the confidence threshold, and execute S42;

S42: adjust the size of the input environment image, and execute S43;

S43: Input to the Tiny-YOLOv3 model for feature extraction, and execute S44;

S44: Perform multi-scale fusion prediction on smoke or flame through a similar FPN network, and divide the feature map into multiple grids; use the K-means clustering method to cluster the bounding boxes of the training set to obtain appropriate anchor boxes, and 3 anchor boxes are generated on each grid to generate the predicted object bounding box, and the class is predicted by a binary cross-entropy loss function.

Preferably, the step S5 uses a deep convolutional neural network-based automatic lawn garbage cleaning machine for garbage cleaning, which specifically includes the following steps:

S51: The initial state of the rotating rod is that the end close to the broom head is inclined in the direction away from the collecting shovel. The first steering gear drives the rotating rod to take the end of the rotating rod as the center of the circle and the rotating rod as the radius to sweep the garbage on the grass. Enter the collecting shovel, the first steering gear drives the rotating rod to return to the initial state, and executes S52;

S52: The second steering gear drives the collection shovel to rotate, dumps the garbage in the collection shovel into the garbage bin, and the second steering gear drives the collection shovel to rotate to the initial position.

To sum up, the beneficial effects of the present invention are:

1. The present invention can automatically identify whether there is garbage on the grass, and when the garbage is identified, it can automatically carry out garbage cleaning without operator control, and has the advantage of improving the efficiency of garbage cleaning;

2. The cleaning assembly of the present invention includes a cleaning broom, a collecting shovel and a garbage bin. The cleaning broom includes a rotating rod and a broom head. One end of the rotating rod is rotated and arranged on the frame, and the frame is provided with a first rudder that drives the rotating rod to rotate. The other end of the rotating rod is connected with a broom head, the garbage box is set on the frame, and the side of the garbage box close to the cleaning broom is connected with a rotating shaft. The length direction of the rotating shaft is perpendicular to the length direction of the rotating rod. The shovel and the garbage bin are provided with a second steering gear that drives the rotating shaft to rotate around the axis of the rotating shaft, which has the advantage of efficiently cleaning garbage.

Description of drawings

1 is a schematic structural diagram of an automatic lawn garbage cleaning machine based on a deep convolutional neural network according to Embodiment 1 of the present invention;

2 is a system block diagram of a deep convolutional neural network-based automatic lawn garbage cleaning machine according to Embodiment 1 of the present invention;

3 is a schematic flowchart of a method for automatic cleaning of grass garbage based on a deep convolutional neural network according to the present invention;

4 is a schematic diagram of an image used for displaying the Tiny-YOLOv3 model for identifying garbage in an environmental image of grass according to an embodiment of the present invention;

5 is a schematic diagram illustrating an image of the Tiny-YOLOv3 model for recognizing garbage in an environmental image of a grass field according to an embodiment of the present invention;

6 is a schematic structural diagram of an automatic lawn garbage cleaning machine based on a deep convolutional neural network according to Embodiment 2 of the present invention;

FIG. 7 is a system block diagram of an automatic lawn garbage cleaning machine based on a deep convolutional neural network according to Embodiment 2 of the present invention.

In the figure, 1, the frame; 2, the walking component; 3, the cleaning component; 31, the cleaning broom; 32, the collecting shovel; 33, the garbage bin; 4, the image acquisition device.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to Figures 1 to 7 of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

Example 1

1 and 2, an automatic lawn garbage cleaning machine based on a deep convolutional neural network includes a rack 1 and a walking assembly 2 arranged on the rack 1. It is worth noting that the walking assembly 2 includes a front axle and a rear The axle, the front axle and the rear axle are respectively rotatably arranged at both ends of the frame 1, and both ends of the front axle and the rear axle are coaxially connected with wheels. In this embodiment, the cleaning machine is driven by the operator.

Referring to Figures 1 and 2, the cleaning machine also includes:

Cleaning component 3, cleaning component 3 is used to clean the garbage on the grass;

The image collection device 4 is used to collect the environmental image of the grass. It is worth noting that, in this embodiment, the image collection device 4 is a high-definition camera;

The central controller, the image acquisition device 4 and the cleaning component 3 are all electrically connected to the central controller. In this embodiment, the central controller adopts a Qualcomm single-core CPU;

The central controller is provided with a garbage identification device, which is electrically connected to the image acquisition device 4 for processing the environmental image of the grassland collected by the image acquisition device 4, and to identify whether there is an image of garbage in the environmental image of the grassland; garbage identification When the device recognizes that there is an image of garbage in the environmental image of the grass, the central controller controls the cleaning component 3 to clean up the garbage;

The image acquisition device 4 , the cleaning assembly 3 and the central controller are all arranged on the rack 1 .

The garbage identification device includes a training module, a target detection model and an image preprocessing module.

The training module is used to train the convolutional neural network using multiple grass environment images to obtain a deep convolutional neural network model. The deep convolutional neural network model is Tiny-YOLOv3. The training module uses the stochastic gradient descent method to optimize Tiny-YOLOv3 Model.

Object detection models, including Tiny-YOLOv3, are used to identify images of garbage in environmental images of multiple grasses; the Tiny-YOLOv3 model uses a binary cross-entropy loss function for class prediction,

Among them, N is the total number of training images; yi takes a value of 0 or 1, yi takes a value of 1, which means that the ith input image contains garbage images, and a yi value of 0 means that the ith input image does not contain Image of garbage; pi value is the predicted probability of whether the ith input image contains garbage image, pi value is between 0 and 1. The Tiny-YOLOv3 model includes the Darknet framework. The Darknet framework includes 53 convolutional layers and 22 Residual layers. The 53 convolutional layers in the Darknet framework are used to extract features from the environmental image of the grass. The Darknet framework includes 22 Residual layers. For solving gradient dispersion or gradient explosion in deep convolutional neural network models.

The image preprocessing module is used to perform histogram equalization processing and logarithmic transformation processing on the environmental image of the grass, obtain the processed environmental image of the grass, and send the processed environmental image of the grass to the training module and the target detection model. .

Referring to Figure 3, a method for automatic cleaning of grass garbage based on a deep convolutional neural network includes the following steps:

S1: Use multiple grass environment images to train the convolutional neural network, obtain the Tiny-YOLOv3 model, and execute S2;

S2: Build an environmental grid map with the initial position as the origin, and execute S3;

S3: Acquire multiple real-time grass environment images, and execute S4;

S4: Identify whether there are garbage images in the multiple real-time grass environment images, if so, execute S5, if not, execute S6;

S5: Obtain the specific position and frame of the image of the garbage in the grass environment image, perform garbage cleaning, and execute S6;

S6: Move to the next grid, and execute S3.

S4 also includes the following steps,

S41: Set the IOU threshold and the confidence threshold, and execute S42;

S42: adjust the size of the input environment image, and execute S43;

S43: Input to the Tiny-YOLOv3 model for feature extraction, and execute S44;

S44: Perform multi-scale fusion prediction on smoke or flame through a similar FPN network, and divide the feature map into multiple grids; use the K-means clustering method to cluster the bounding boxes of the training set to obtain appropriate anchor boxes, and 3 anchor boxes are generated on each grid to generate the predicted object bounding box, and the class is predicted by a binary cross-entropy loss function.

S5 uses a deep convolutional neural network-based automatic lawn garbage cleaning machine for garbage cleaning, which specifically includes the following steps:

S51: The initial state of the rotating rod is that the end close to the broom head is inclined in the direction away from the collecting shovel 32. The first steering gear drives the rotating rod to take the end point of the rotating rod as the center of the circle and the rotating rod as the radius to rotate the garbage on the grass. Sweep into the collecting shovel 32, the first steering gear drives the rotating rod to return to the initial state, and executes S52;

S52: The second steering gear drives the collection shovel 32 to rotate, dumps the garbage in the collection shovel 32 into the garbage bin 33, and the second steering gear drives the collection shovel 32 to rotate to the initial position.

Referring to Figures 4 and 5, it is worth noting that, before training the Tiny-YOLOv3 model in this embodiment, the daily garbage that often appears on the grass is collected, and then distributed to different types of grass to enhance the performance of the Tiny-YOLOv3 model. Generalization ability, obtain 20,000 grass garbage samples, perform data enhancement on the garbage samples, and obtain 8,000 grass garbage image samples with different angles, different lighting and different noises, and crop the collected grass garbage image samples to 416* 416 size, use the labelImage tool to label the garbage image of the grass. 65,000 grass garbage samples were randomly selected for training, and the remaining samples were used as the test set. There are 100,000 training times in total, and the weights are automatically saved every 5,000 times. The basic learning rate is 0.001, the batch size is 32, the momentum is 0.9, and the weight decay coefficient is 0.0005. L2 regularization is used to reduce overfitting.

In this embodiment, the performance of the Tiny-YOLOv3 model is also studied, and the accuracy (Accuracy) and recall (Recall) of the Tiny-YOLOv3 model for garbage identification are obtained, as shown in Table 1:

type Accuracy recall Rubbish 94.12 92.38

The Tiny-yolo v3 target detection model has an accuracy rate of 94.12% for grass litter detection and a recall rate of 92.38%, which has high accuracy and recall rate for grass litter detection.

The implementation principle of this embodiment is as follows: the operator pushes the frame 1 to walk to the area with garbage, the high-definition camera captures multiple environmental images of the current grass, and transmits the multiple environmental images of the current grass to the central controller. The garbage identification device in the central controller recognizes whether there are garbage images in the environmental images of the current grass. If there is, the initial state of the rotating lever is that the end close to the broom head is inclined in the direction away from the collecting shovel 32, and the first rudder The motor drives the rotating rod with the end point of the rotating rod as the center of the circle and the rotating rod as the radius, and sweeps the garbage on the grass into the collection shovel 32. The first steering gear drives the rotating rod to return to the initial state, and the second steering gear drives the collection shovel. 32 is rotated to dump the garbage in the collection shovel 32 into the garbage bin 33, and the second steering gear drives the collection shovel 32 to rotate to the initial position to complete the cleaning work.

Example 2

6 and 7, an automatic lawn garbage cleaning machine based on a deep convolutional neural network includes a rack 1 and a walking assembly 2 arranged on the rack 1. It is worth noting that the walking assembly 2 includes a planetary gear reduction mechanism , hub mechanism and rubber track drive. The planetary gear reduction mechanism includes a front planetary carrier, a rear planetary carrier, a drive shaft, a sun gear, a planetary shaft, a planetary gear and a clutch; the sun gear is fixedly installed on the drive shaft, the drive shaft passes through the rear planetary carrier, and the planetary gear is connected by bearings. On the planetary shaft, three planetary gears mesh with the sun gear, the planetary shafts are installed and connected between the front planetary carrier and the rear planetary carrier, and the clutch is installed between the drive shaft and the rear planetary carrier. The planetary gear reduction mechanism also includes a brake, which is installed between the rear planet carrier and the body 2 . The hub mechanism includes a hub plate, a reinforcing plate and a load-bearing shaft. Each side of the hub plate is connected to the reinforcing plate correspondingly through the load-bearing shaft. One end of the support rod is hinged with the hub plate, and the other end is hinged with the front planet carrier or the rear planet carrier. The rubber track transmission mechanism includes a rubber track, a driving wheel and a load-bearing wheel; the driving wheel is installed on the planetary shaft and is fixedly connected to the planetary wheel. There is a driving wheel at the left and right ends of the planetary wheel, and the load-bearing wheel is installed between the hub plate and the reinforcing plate. On the bearing shaft, the rubber track wraps the driving wheel and the bearing wheel, and the inner side of the rubber track meshes with the driving wheel.

The cleaning machine also includes an obstacle avoidance device, the obstacle avoidance device is electrically connected with the central controller, and the obstacle avoidance device is used to detect whether there are obstacles around the body. The obstacle avoidance device includes a front infrared ranging sensor, a left infrared ranging sensor and a right infrared ranging sensor, and the front infrared ranging sensor and the left infrared ranging sensor are all electrically connected to the central controller.

6, 7, this cleaning machine also includes:

Cleaning component 3, cleaning component 3 is used to clean the garbage on the grass;

The image collection device 4 is used to collect the environmental image of the grass. It is worth noting that, in this embodiment, the image collection device 4 is a high-definition camera;

The central controller, the image acquisition device 4 and the cleaning component 3 are all electrically connected to the central controller. In this embodiment, the central controller adopts a Qualcomm single-core CPU;

The central controller is provided with a garbage identification device, which is electrically connected to the image acquisition device 4 for processing the environmental image of the grassland collected by the image acquisition device 4, and to identify whether there is an image of garbage in the environmental image of the grassland; garbage identification When the device recognizes that there is an image of garbage in the environmental image of the grass, the central controller controls the cleaning component 3 to clean up the garbage;

The image acquisition device 4 , the cleaning assembly 3 and the central controller are all arranged on the rack 1 .

The garbage identification device includes a training module, a target detection model and an image preprocessing module.

The training module is used to train the convolutional neural network using multiple grass environment images to obtain a deep convolutional neural network model. The deep convolutional neural network model is Tiny-YOLOv3. The training module uses the stochastic gradient descent method to optimize Tiny-YOLOv3 Model.

Object detection models, including Tiny-YOLOv3, are used to identify images of garbage in environmental images of multiple grasses; the Tiny-YOLOv3 model uses a binary cross-entropy loss function for class prediction,

Among them, N is the total number of training images; yi takes a value of 0 or 1, yi takes a value of 1, which means that the ith input image contains garbage images, and a yi value of 0 means that the ith input image does not contain Image of garbage; pi value is the predicted probability of whether the ith input image contains garbage image, pi value is between 0 and 1. The Tiny-YOLOv3 model includes the Darknet framework. The Darknet framework includes 53 convolutional layers and 22 Residual layers. The 53 convolutional layers in the Darknet framework are used to extract features from the environmental image of the grass. The Darknet framework includes 22 Residual layers. For solving gradient dispersion or gradient explosion in deep convolutional neural network models.

The image preprocessing module is used to perform histogram equalization processing and logarithmic transformation processing on the environmental image of the grass, obtain the processed environmental image of the grass, and send the processed environmental image of the grass to the training module and the target detection model. .

Referring to Figure 3, a method for automatic cleaning of grass garbage based on a deep convolutional neural network includes the following steps:

S1: Use multiple grass environment images to train the convolutional neural network, obtain the Tiny-YOLOv3 model, and execute S2;

S2: Build the environmental grid map with the initial position as the origin. It is worth noting that when establishing the environmental grid map, the central controller takes the initial position of the current rack 1 as the origin O, and is a boundary point according to the setting The length and width of the map to create a grid map, the operator can also change the length or width of the grid map through the host computer or mobile terminal; the grid in the grid map is represented by a two-dimensional array map[][], map[] x][y]=0, indicating that the grid has not been visited, map[x][y]=1, indicating that there are obstacles in the grid, map[x][y]=2, indicating that the cleaning machine has Walk through, execute S3;

S3: Acquire multiple real-time grass environment images, and execute S4;

S4: Identify whether there are garbage images in the multiple real-time grass environment images, if so, execute S5, if not, execute S6;

S5: Obtain the specific position and frame of the image of the garbage in the grass environment image, perform garbage cleaning, and execute S6;

S6: Detect whether there are obstacles around the frame 1, and control the walking device to drive the cleaning machine to move to the next grid according to the walking rules and the inner spiral algorithm, and execute S3; the inner spiral algorithm means that the system aligns the grid in a certain direction The grid map is covered. In this embodiment, the system covers the grid map in a clockwise direction. The grids that the system does not pass through are marked with map[x][y]-=0, and the grids that the system passes through are marked with map[x][y]=2. The system detects the grids with obstacles and uses map[x][y]=1 to identify, each time the cleaning machine passes through a grid, the grid identification of the grid map is updated. The walking rules specifically include the following steps:

S61: Detect whether there is an obstacle in front of the rack 1, if not, go to S62, if there is, go to S63;

S62: Detect whether there is an obstacle on the left side of the rack 1, if not, turn left, if there is, go straight;

S63: Detect whether there is an obstacle on the left side of the rack 1, if not, turn left, if there is, turn right; When an obstacle is detected on the right side, the central controller selects the next grid with the shortest distance from the grid and has not been visited, and plans a route to control the body to move to the next grid;

S4 also includes the following steps,

S41: Set the IOU threshold and the confidence threshold, and execute S42;

S42: adjust the size of the input environment image, and execute S43;

S43: Input to the Tiny-YOLOv3 model for feature extraction, and execute S44;

S44: Perform multi-scale fusion prediction on smoke or flame through a similar FPN network, and divide the feature map into multiple grids; use the K-means clustering method to cluster the bounding boxes of the training set to obtain appropriate anchor boxes, and 3 anchor boxes are generated on each grid to generate the predicted object bounding box, and the class is predicted by a binary cross-entropy loss function.

S5 uses a deep convolutional neural network-based automatic lawn garbage cleaning machine for garbage cleaning, which specifically includes the following steps:

S51: The initial state of the rotating rod is that the end close to the broom head is inclined in the direction away from the collecting shovel 32. The first steering gear drives the rotating rod to take the end point of the rotating rod as the center of the circle and the rotating rod as the radius to rotate the garbage on the grass. Sweep into the collecting shovel 32, the first steering gear drives the rotating rod to return to the initial state, and executes S52;

S52: The second steering gear drives the collection shovel 32 to rotate, dumps the garbage in the collection shovel 32 into the garbage bin 33, and the second steering gear drives the collection shovel 32 to rotate to the initial position.

Referring to Figures 4 and 5, it is worth noting that, before training the Tiny-YOLOv3 model in this embodiment, the daily garbage that often appears on the grass is collected, and then distributed to different types of grass to enhance the performance of the Tiny-YOLOv3 model. Generalization ability, obtain 20,000 grass garbage samples, perform data enhancement on the garbage samples, and obtain 8,000 grass garbage image samples with different angles, different lighting and different noises, and crop the collected grass garbage image samples to 416* 416 size, use the labelImage tool to label the garbage image of the grass. 65,000 grass garbage samples were randomly selected for training, and the remaining samples were used as the test set. There are 100,000 training times in total, and the weights are automatically saved every 5,000 times. The basic learning rate is 0.001, the batch size is 32, the momentum is 0.9, and the weight decay coefficient is 0.0005. L2 regularization is used to reduce overfitting.

In this embodiment, the performance of the Tiny-YOLOv3 model is also studied, and the accuracy (Accuracy) and recall (Recall) of the Tiny-YOLOv3 model for garbage identification are obtained, as shown in Table 1:

type Accuracy recall Rubbish 94.12 92.38

The Tiny-yolo v3 target detection model has an accuracy rate of 94.12% for grass litter detection and a recall rate of 92.38%, which has high accuracy and recall rate for grass litter detection.

The implementation principle of this embodiment is as follows: the walking component 2 drives the frame 1 to walk to the area with garbage, the high-definition camera captures multiple environmental images of the current grass, and transmits the multiple environmental images of the current grass to the central controller. The garbage identification device in the central controller recognizes whether there are garbage images in the environmental images of the current grass. If there is, the initial state of the rotating lever is that the end close to the broom head is inclined to the direction away from the collection shovel 32, and the first rudder The motor drives the rotating rod with the end point of the rotating rod as the center of the circle and the rotating rod as the radius to sweep the garbage on the grass into the collecting shovel 32. The first steering gear drives the rotating rod to return to the initial state, and the second steering gear drives the collecting shovel. 32 is rotated to dump the garbage in the collection shovel 32 into the garbage bin 33, and the second steering gear drives the collection shovel 32 to rotate to the initial position to complete the cleaning work. Detect whether there are obstacles around the frame 1, and control the walking component 2 to drive the cleaning machine to move to the next grid according to the walking rules and the inner spiral algorithm for the next garbage identification and cleaning.

In the description of the present invention, it should be understood that the terms "counterclockwise", "clockwise", "longitudinal", "horizontal", "upper", "lower", "front", "rear", "left", The orientation or positional relationship indicated by "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc. is based on the orientation or positional relationship shown in the drawings, and is only for the purpose of It is convenient to describe the present invention, not to indicate or imply that the device or element referred to must have a particular orientation, be constructed and operate in a particular orientation, and therefore should not be construed as a limitation of the present invention.

Claims (10)

1. a grass garbage automatic cleaning machine based on a deep convolutional neural network, comprising a rack (1) and a walking assembly (2) arranged on the rack (1), it is characterized in that, also comprises: a cleaning assembly (3), the cleaning assembly (3) is used for cleaning garbage on the grass; an image acquisition device (4) for acquiring an environmental image of the grass; a central controller, the image acquisition device (4) and the cleaning assembly (3) are all electrically connected to the central controller; The central controller is provided with a garbage identification device, which is electrically connected to the image acquisition device (4), and is used for processing the environmental image of the grass field collected by the image acquisition device (4), and to identify the environmental image of the grass field. Whether there is an image of garbage; when the garbage identification device recognizes that there is an image of garbage in the environmental image of the grass, the central controller controls the cleaning component (3) to clean up garbage; The image acquisition device (4), the cleaning assembly (3) and the central controller are all arranged on the frame (1). 2. The automatic lawn garbage cleaning machine based on a deep convolutional neural network according to claim 1, wherein the cleaning component (3) comprises a cleaning broom, a collecting shovel (32) and a garbage bin (33) , the cleaning broom includes a rotating rod and a broom head, one end of the rotating rod is rotatably arranged on the frame (1), and the frame (1) is provided with a first rudder that drives the rotating rod to rotate machine, the other end of the rotating rod is connected with a broom head; The garbage box (33) is arranged on the frame (1), and a side of the garbage box (33) close to the cleaning broom is rotatably connected with a rotating shaft, and the length direction of the rotating shaft is the same as the length direction of the rotating rod. Vertically, the collecting shovel (32) is connected to the rotating shaft, and the garbage box (33) is provided with a second steering gear that drives the rotating shaft to rotate around the axis of the rotating shaft. 3. The automatic lawn garbage cleaning machine based on a deep convolutional neural network according to any one of claims 1-2, wherein the garbage identification device comprises: The training module is used to train the convolutional neural network using multiple grass environment images to obtain a deep convolutional neural network model; A target detection model for identifying images of garbage in multiple grass environment images; The image preprocessing module is used to perform histogram equalization processing and logarithmic transformation processing on the environmental image of the grass, obtain the processed environmental image of the grass, and send the processed environmental image of the grass to the training module and the target detection model. . 4 . The automatic lawn garbage cleaning machine based on a deep convolutional neural network according to claim 3 , wherein the deep convolutional neural network model is Tiny-YOLOv3. 5 . 5. A kind of automatic lawn garbage cleaning machine based on deep convolutional neural network according to claim 4, is characterized in that: described Tiny-YOLOv3 model uses binary cross entropy loss function to carry out category prediction,

Among them, N is the total number of training images; yi takes a value of 0 or 1, yi takes a value of 1, which means that the ith input image contains garbage images, and a yi value of 0 means that the ith input image does not contain Image of garbage; pi value is the predicted probability of whether the ith input image contains garbage image, pi value is between 0 and 1. 6. The automatic lawn garbage cleaning machine based on a deep convolutional neural network according to any one of claims 4-5, wherein the deep convolutional neural network model comprises a DarkNet framework, and the DarkNet framework comprises 53 convolutional layers and 22 Residual layers, the 53 convolutional layers in the DarkNet framework are used for feature extraction on the environmental image of the grass, and the 22 Residual layers in the DarkNet framework are used to solve the deep convolutional neural network. Gradient dispersion or gradient explosion in network models. 7 . The automatic lawn garbage cleaning machine based on a deep convolutional neural network according to claim 4 , wherein the training module adopts a stochastic gradient descent method to optimize the Tiny-YOLOv3 model. 8 . 8. an automatic cleaning method for grass garbage based on deep convolutional neural network, is characterized in that, comprises the following steps, S1: Use multiple grass environment images to train the convolutional neural network, obtain the Tiny-YOLOv3 model, and execute S2; S2: Build an environmental grid map with the initial position as the origin, and execute S3; S3: Acquire multiple real-time grass environment images, and execute S4; S4: Identify whether there are garbage images in the multiple real-time grass environment images, if so, execute S5, if not, execute S6; S5: Obtain the specific position and frame of the image of the garbage in the grass environment image, perform garbage cleaning, and execute S6; S6: Move to the next grid, and execute S3. 9. a kind of grass garbage automatic cleaning method based on deep convolutional neural network according to claim 8, is characterized in that, described S4 also comprises the following steps, S41: Set the IOU threshold and the confidence threshold, and execute S42; S42: adjust the size of the input environment image, and execute S43; S43: Input to the Tiny-YOLOv3 model for feature extraction, and execute S44; S44: Perform multi-scale fusion prediction on smoke or flame through a similar FPN network, and divide the feature map into multiple grids; use the K-means clustering method to cluster the bounding boxes of the training set to obtain appropriate anchor boxes, and 3 anchor boxes are generated on each grid to generate the predicted object bounding box, and the class is predicted by a binary cross-entropy loss function. 10. The method for automatic cleaning of grass garbage based on a deep convolutional neural network according to any one of claims 8-9, wherein the S5 uses the method described in any one of claims 1-7. The lawn garbage automatic cleaning machine based on the deep convolutional neural network performs garbage cleaning, which includes the following steps: S51: The initial state of the rotating rod is that the end close to the broom head is inclined to the direction away from the collecting shovel (32). The first steering gear drives the rotating rod to take the end point of the rotating rod as the center of the circle and the rotating rod as the radius to rotate the grass on the grass. The collected garbage is swept into the collecting shovel (32), the first steering gear drives the rotating rod to return to the initial state, and executes S52; S52: The second steering gear drives the collection shovel (32) to rotate, dumps the garbage in the collection shovel (32) into the garbage bin (33), and the second steering gear drives the collection shovel (32) to rotate to the initial position.

Images