CN111567331A - Automatic lawn garbage cleaning machine and method based on deep convolutional neural network - Google Patents
Automatic lawn garbage cleaning machine and method based on deep convolutional neural network Download PDFInfo
- Publication number
- CN111567331A CN111567331A CN202010391121.9A CN202010391121A CN111567331A CN 111567331 A CN111567331 A CN 111567331A CN 202010391121 A CN202010391121 A CN 202010391121A CN 111567331 A CN111567331 A CN 111567331A
- Authority
- CN
- China
- Prior art keywords
- garbage
- grassland
- image
- neural network
- convolutional neural
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
- A01G—HORTICULTURE; CULTIVATION OF VEGETABLES, FLOWERS, RICE, FRUIT, VINES, HOPS OR SEAWEED; FORESTRY; WATERING
- A01G20/00—Cultivation of turf, lawn or the like; Apparatus or methods therefor
- A01G20/40—Apparatus for cleaning the lawn or grass surface
- A01G20/43—Apparatus for cleaning the lawn or grass surface for sweeping, collecting or disintegrating lawn debris
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Physics & Mathematics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Multimedia (AREA)
- Environmental Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to the technical field of garbage disposal equipment, in particular to an automatic lawn garbage cleaning machine and a method based on a deep convolutional neural network, wherein the automatic lawn garbage cleaning machine comprises a rack and a walking assembly arranged on the rack, and further comprises the following steps: the cleaning assembly is used for cleaning garbage on the grassland; the image acquisition device is used for acquiring an environmental image of the grassland; the central controller, the image acquisition device and the cleaning assembly are electrically connected with the central controller; the central controller is internally provided with a garbage recognition device which is electrically connected with the image acquisition device and is used for processing the grassland environment image acquired by the image acquisition device and recognizing whether a garbage image exists in the grassland environment image; when the garbage recognition device recognizes that the images of the garbage exist in the environmental images of the grassland, the central controller controls the cleaning assembly to clean the garbage; the image acquisition device, the cleaning assembly and the central controller are all arranged on the frame. The garbage cleaning device has the advantage of improving the garbage cleaning efficiency.
Description
Technical Field
The invention relates to the technical field of garbage disposal equipment, in particular to an automatic lawn garbage cleaning machine and method based on a deep convolutional neural network.
Background
In order to better improve the life quality of people, the urban greening area is increasing year by year at present, public places such as schools, parks and the like all have lawns, and the urban lawns can purify air and absorb toxic and harmful gases such as carbon dioxide, sulfur dioxide, hydrogen fluoride, ammonia, chlorine and the like in the atmosphere. Grasslands can absorb a large amount of carbon dioxide per hour while releasing a large amount of oxygen. The lawn can regulate the atmospheric temperature and humidity, and evaporate a large amount of water every hectare lawn every day to increase the relative humidity in the air. The lawn can absorb dust and sterilize, and the dust absorption capacity of the lawn is 70 times larger than that of a bare ground. The lawn can reduce noise pollution, and the lawn can reduce noise in a wider field. The lawn can also preserve water, resist drought, beautify the environment and adjust the climate, but along with the increase of the activities of people, a large amount of garbage often appears on the lawn.
Chinese patent with grant bulletin number CN104996175B discloses a high-efficient hand propelled meadow rubbish clean-up equipment, the on-line screen storage device comprises a base, the wheel carrier, the land wheel, choose the material pole, main casing body and control box, the base lower part is provided with 4 wheel carriers, install the land wheel on the wheel carrier, it is provided with the guide rail to push away between the handle, be provided with the dustbin on the guide rail, the welding of dustbin left side has first conveyer belt roller support, the welding of first conveyer belt left side has second conveyer belt roller support, it chooses the material pole to be provided with on the material roller to pick up, there is main casing body through screw rod fixed mounting on the base, main casing body right side is installed through the loose-leaf and is unloaded the door. The picking roller is driven to rotate by the picking motor, the picking rod on the picking roller picks up the garbage and leaves on the grassland, the garbage and leaves finally arrive on the conveying belt through the centrifugal effect, and the garbage can on the leading-in equipment is taken to complete the garbage picking work.
The prior art has the following technical defects: above-mentioned cleaning equipment still needs artifical discernment rubbish, discerns rubbish back, and the work of picking up rubbish is carried out to manual control box, and rubbish clearance efficiency is lower.
Disclosure of Invention
The invention aims to provide an automatic lawn garbage cleaning machine and method based on a deep convolutional neural network, and the automatic lawn garbage cleaning machine and method have the advantage of improving garbage cleaning efficiency.
In order to achieve the purpose, the technical scheme adopted by the invention is as follows: the utility model provides an automatic descaling machine of meadow rubbish based on degree of depth convolution neural network, includes the frame and sets up walking subassembly in the frame still includes:
the cleaning assembly is used for cleaning garbage on the grassland;
the image acquisition device is used for acquiring an environmental image of the grassland;
the image acquisition device and the cleaning assembly are electrically connected with the central controller;
the central controller is internally provided with a garbage recognition device which is electrically connected with the image acquisition device and is used for processing the grassland environment image acquired by the image acquisition device and recognizing whether a garbage image exists in the grassland environment image; when the garbage recognition device recognizes images of garbage in the environmental images of the grassland, the central controller controls the cleaning assembly to clean the garbage;
the image acquisition device, the cleaning assembly and the central controller are all arranged on the rack.
Preferably, the cleaning assembly comprises a cleaning broom, a collecting shovel and a garbage can, the cleaning broom comprises a rotating rod and a broom head, one end of the rotating rod is rotatably arranged on the rack, a first steering engine for driving the rotating rod to rotate is arranged on the rack, and the other end of the rotating rod is connected with the broom head;
the garbage bin sets up in the frame, one side that the garbage bin is close to and cleans the broom is rotated and is connected with the pivot, the length direction of pivot with the length direction of pivot is perpendicular, be connected with in the pivot collect the shovel, the garbage bin is provided with the drive the pivot is around the axis pivoted second steering wheel of pivot.
Preferably, the garbage recognition device comprises
The training module is used for training the convolutional neural network by using the environmental images of a plurality of grasslands to obtain a deep convolutional neural network model;
the target detection model is used for identifying images of garbage in a plurality of grassland environment images;
and the image preprocessing module is used for carrying out histogram equalization processing and logarithmic transformation processing on the environmental image of the grassland, obtaining the processed environmental image of the grassland and sending the processed environmental image of the grassland to the training module and the target detection model.
Preferably, the deep convolutional neural network model is Tiny-Yolov 3.
Through the technical scheme, compared with the previous YOLO algorithm, the YOLOv3 adopts DarkNet53 with higher precision as a feature extraction network, designs a target multi-scale detection structure, and uses a logistic function to replace the traditional softmax function. The DarkNet53 consults the thought of ResNet residual network, and sets a shortcut path between some layers, and the research shows that: DarkNet53 is close in accuracy but faster than ResNet-152.
Preferably, the Tiny-YOLOv3 model uses a binary cross entropy loss function for class prediction,
wherein N is the total number of training pictures; the yi value is 0 or 1, the yi value is 1, the ith input picture contains the image of the garbage, and the yi value is 0, the ith input picture does not contain the image of the garbage; the pi value is a probability of prediction of whether the ith input picture contains a spam image, and is between 0 and 1.
Preferably, the deep convolutional neural network model comprises a DarkNet framework, the DarkNet framework comprises 53 convolutional layers and 22 Residual layers, 53 convolutional layers in the DarkNet framework are used for performing feature extraction on the environmental image of the grassland, and 22 Residual layers in the DarkNet framework are used for solving gradient diffusion or gradient explosion in the deep convolutional neural network model. In one aspect, the Darknet-53 network adopts a full convolution structure, and the tensor size transformation is realized by changing the step size of a convolution kernel in the forward propagation process of the Yolo v 3. The convolution step size is 2, and after each convolution, the image side length is reduced by half. On the other hand, the Darknet-53 network introduces a Residual structure. Also in the Yolo v2, the straight-tube type network structure like VGG has gradient problem when the number of layers is too large, so that Darknet-19 has 19 layers. Due to the residual structure of ResNet, the difficulty of training a deep network is greatly reduced. Therefore, the Darknet-53 network achieves 53 layers, and the precision is obviously improved.
Preferably, the training module optimizes the Tiny-Yolov3 model by using a random gradient descent method. Random Gradient Descent (SGD) is an extension of the Gradient Descent algorithm. The core of the stochastic gradient descent is: a gradient is desired. It is desirable to approximate the estimate using a small sample. Batch gradient descent method (Batch gradient descent, BGD): the method is the most primitive form of the gradient descent method, all sample data in a training set are used when each step is iterated or each parameter is updated, and when the number of samples is large, the training process is slow. Random gradient Descent method (StochasticGradient Descent, SGD): since the batch gradient descent method requires all training samples when updating each parameter, the training process becomes abnormally slow as the number of samples increases. The random gradient descent method is proposed to solve the disadvantage of the batch gradient descent method. The random gradient descent is iteratively updated once per sample. One problem with SGD is that it is more noisy than BGD, so that SGD does not go in the optimization direction for every iteration.
A lawn garbage automatic cleaning method based on a deep convolutional neural network comprises the following steps,
s1: training the convolutional neural network by using the environmental images of a plurality of grasslands to obtain a Tiny-Yolov3 model, and executing S2;
s2: establishing an environment grid map by taking the initial position as an origin, and executing S3;
s3: acquiring a plurality of real-time grassland environment images, and executing S4;
s4: identifying whether garbage images exist in a plurality of real-time grassland environment images, if so, executing S5, otherwise, executing S6;
s5: acquiring the specific position and the frame of the image of the rubbish in the grassland environment image, cleaning the rubbish, and executing S6;
s6: moving into the next grid, S3 is performed.
Preferably, the S4 further comprises the step of,
s41: setting an IOU threshold and a confidence threshold, and executing S42;
s42: resizing the input environment image, and performing S43;
s43: inputting the data into a Tiny-YOLOv3 model for feature extraction, and executing S44;
s44: carrying out multi-scale fusion prediction on smoke or flame through a similar FPN network, and dividing the characteristic diagram into a plurality of grids; and clustering the boundary frames of the training set by using a K-means clustering method to obtain a proper anchor box, generating 3 anchor box numbers on each grid to generate a predicted target boundary frame, and predicting the category by using a binary cross entropy loss function.
Preferably, the S5 is used for performing the garbage cleaning work by using an automatic lawn garbage cleaning machine based on a deep convolutional neural network, and specifically includes the following steps,
s51: the initial state of the rotating rod is that one end close to the broom head is arranged in an inclined mode towards the direction far away from the collecting shovel, the first steering engine drives the rotating rod to rotate by taking the end point of the rotating rod as the center of a circle and taking the rotating rod as the radius, the garbage on the grassland is swept into the collecting shovel, the first steering engine drives the rotating rod to return to the initial state, and S52 is executed;
s52: the second steering wheel drives the collection shovel to rotate, dumps garbage in the collection shovel to the garbage can, and the second steering wheel drives the collection shovel to rotate to the initial position.
In conclusion, the beneficial effects of the invention are as follows:
1. the automatic garbage cleaning device can automatically identify whether garbage exists on the grassland or not, automatically perform garbage cleaning work when the garbage is identified, does not need to be controlled by operators, and has the advantage of improving the garbage cleaning efficiency;
2. the cleaning assembly comprises a cleaning broom, a collecting shovel and a garbage can, the cleaning broom comprises a rotating rod and a broom head, one end of the rotating rod is rotatably arranged on a rack, a first steering engine for driving the rotating rod to rotate is arranged on the rack, the other end of the rotating rod is connected with the broom head, the garbage can is arranged on the rack, one side, close to the cleaning broom, of the garbage can is rotatably connected with a rotating shaft, the length direction of the rotating shaft is perpendicular to the length direction of the rotating shaft, the collecting shovel is connected onto the rotating shaft, and the garbage can is provided with a second steering engine for driving the rotating shaft to rotate around the axis of the rotating shaft.
Drawings
Fig. 1 is a schematic structural diagram of an automatic lawn garbage sweeper based on a deep convolutional neural network according to embodiment 1 of the present invention;
FIG. 2 is a system block diagram of an automatic lawn garbage sweeper based on a deep convolutional neural network according to embodiment 1 of the present invention;
FIG. 3 is a schematic flow chart of a method for automatically cleaning grassland garbage based on a deep convolutional neural network according to the present invention;
FIG. 4 is a schematic diagram of an image used to demonstrate the Tiny-YOLOv3 model identifying spam in an environmental image of grass;
FIG. 5 is a schematic diagram of an image used to demonstrate the Tiny-YOLOv3 model identifying spam in an environmental image of grass;
fig. 6 is a schematic structural diagram of an automatic lawn garbage sweeper based on a deep convolutional neural network according to embodiment 2 of the present invention;
fig. 7 is a system block diagram of an automatic lawn garbage sweeper based on a deep convolutional neural network according to embodiment 2 of the present invention.
In the figure, 1, a frame; 2. a walking assembly; 3. cleaning the assembly; 31. cleaning the broom; 32. collecting shovels; 33. a dustbin; 4. an image acquisition device.
Detailed Description
The technical solutions in the embodiments of the present invention are clearly and completely described below with reference to fig. 1 to 7 of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Example 1
Referring to fig. 1 and 2, an automatic lawn garbage cleaning machine based on a deep convolutional neural network comprises a frame 1 and a traveling assembly 2 arranged on the frame 1, and it is worth explaining that the traveling assembly 2 comprises a front shaft and a rear shaft, the front shaft and the rear shaft are respectively rotatably arranged at two ends of the frame 1, and two ends of the front shaft and the rear shaft are coaxially connected with wheels. In this embodiment, this descaling machine relies on operating personnel to promote the walking.
Referring to fig. 1 and 2, the cleaning machine further includes:
the cleaning assembly 3 is used for cleaning garbage on the grassland;
the image acquisition device 4 is used for acquiring an environmental image of a grassland, and it is worth to be noted that in the embodiment, the image acquisition device 4 is a high-definition camera;
the central controller, the image acquisition device 4 and the cleaning assembly 3 are electrically connected with the central controller, and in the embodiment, the central controller adopts a high-pass single-core CPU;
the central controller is internally provided with a garbage recognition device which is electrically connected with the image acquisition device 4 and is used for processing the grassland environment image acquired by the image acquisition device 4 and recognizing whether a garbage image exists in the grassland environment image; when the garbage recognition device recognizes that the images of the garbage exist in the environmental images of the grassland, the central controller controls the cleaning assembly 3 to clean the garbage;
the image acquisition device 4, the cleaning assembly 3 and the central controller are all arranged on the frame 1.
The garbage recognition device comprises a training module, a target detection model and an image preprocessing module.
The training module is used for training the convolutional neural network by using the environmental images of a plurality of grasslands to obtain a deep convolutional neural network model, the deep convolutional neural network model is Tiny-Yolov3, and the training module optimizes the Tiny-Yolov3 model by adopting a random gradient descent method.
A target detection model comprising Tiny-YOLOv3 for identifying images of litter in a plurality of images of an environment of grass; the Tiny-YOLOv3 model uses a binary cross entropy loss function for class prediction,
wherein N is the total number of training pictures; the yi value is 0 or 1, the yi value is 1, the ith input picture contains the image of the garbage, and the yi value is 0, the ith input picture does not contain the image of the garbage; the pi value is a probability of prediction of whether the ith input picture contains a spam image, and is between 0 and 1. The Tiny-YOLOv3 model includes a Darknet framework, the Darknet framework includes 53 convolutional layers and 22 Residual layers, 53 convolutional layers in the Darknet framework are used for carrying out feature extraction on the environmental image of the grassland, and 22 Residual layers in the Darknet framework are used for solving gradient diffusion or gradient explosion in the deep convolutional neural network model.
And the image preprocessing module is used for carrying out histogram equalization processing and logarithmic transformation processing on the environmental image of the grassland, obtaining the processed environmental image of the grassland and sending the processed environmental image of the grassland to the training module and the target detection model.
Referring to fig. 3, a method for automatically cleaning grassland garbage based on a deep convolutional neural network comprises the following steps,
s1: training the convolutional neural network by using the environmental images of a plurality of grasslands to obtain a Tiny-Yolov3 model, and executing S2;
s2: establishing an environment grid map by taking the initial position as an origin, and executing S3;
s3: acquiring a plurality of real-time grassland environment images, and executing S4;
s4: identifying whether garbage images exist in a plurality of real-time grassland environment images, if so, executing S5, otherwise, executing S6;
s5: acquiring the specific position and the frame of the image of the rubbish in the grassland environment image, cleaning the rubbish, and executing S6;
s6: moving into the next grid, S3 is performed.
S4 further includes the following steps,
s41: setting an IOU threshold and a confidence threshold, and executing S42;
s42: resizing the input environment image, and performing S43;
s43: inputting the data into a Tiny-YOLOv3 model for feature extraction, and executing S44;
s44: carrying out multi-scale fusion prediction on smoke or flame through a similar FPN network, and dividing the characteristic diagram into a plurality of grids; and clustering the boundary frames of the training set by using a K-means clustering method to obtain a proper anchor box, generating 3 anchor box numbers on each grid to generate a predicted target boundary frame, and predicting the category by using a binary cross entropy loss function.
S5, the automatic lawn garbage cleaning machine based on the deep convolutional neural network is used for cleaning garbage, and the method specifically comprises the following steps,
s51: the initial state of the rotating rod is that one end close to the broom head is arranged in an inclined way towards the direction far away from the collecting shovel 32, the first steering engine drives the rotating rod to rotate by taking the end point of the rotating rod as the center of a circle and taking the rotating rod as the radius, so that the garbage on the grassland is swept into the collecting shovel 32, the first steering engine drives the rotating rod to return to the initial state, and S52 is executed;
s52: the second steering wheel drives the collection shovel 32 to rotate, and dumps the rubbish in the collection shovel 32 to the dustbin 33, and the second steering wheel drives the collection shovel 32 to rotate to the initial position again.
Referring to fig. 4 and 5, it is worth explaining that, in the present embodiment, before training the Tiny-yollov 3 model, daily garbage that often appears on grass is collected and then distributed to different types of grass to enhance the generalization ability of the Tiny-yollov 3 model, 20000 grass garbage samples are obtained, data enhancement is performed on the garbage samples, 8000 grass garbage image samples with different angles, different lighting and different noises are obtained, all the collected grass garbage image samples are cut to 416 × 416, and a labelImage tool is selected to label the grass garbage images. 65000 grass litter samples were randomly selected for training, and the remaining samples were used as the test set. The training times are 100000 times in total, the weight is automatically saved every 5000 times of training, the basic learning rate is 0.001, the batch size is 32, the momentum is 0.9, the weight attenuation coefficient is 0.0005, and overfitting is reduced by adopting L2 regularization.
In this embodiment, the performance of the Tiny-yollov 3 model is also studied, and the Accuracy (Accuracy) and the Recall (Recall) of the Tiny-yollov 3 model for identifying garbage are obtained, as shown in table 1:
| type (B) | Rate of accuracy | Recall rate |
| Garbage collection | 94.12 | 92.38 |
The accuracy rate of the Tiny-yolo v3 target detection model on the detection of the grassland garbage is 94.12%, the recall rate is 92.38%, and the accuracy rate and the recall rate on the detection of the grassland garbage are higher.
The implementation principle of the embodiment is as follows: operating personnel promotes 1 walking of frame to the region that has rubbish, and the environmental image on many current grasslands is shot to high definition camera to transmit the environmental image on many current grasslands to central controller in. Whether there is the image of rubbish in the environmental image of many current grasslands of rubbish recognition device discernment in the central controller, if there is, the initial condition of bull stick sets up to the direction slope of keeping away from collection shovel 32 for the one end that is close to the broom head, first steering wheel drives the extreme point that the bull stick used the bull stick and is the centre of a circle, use the bull stick to rotate as the radius, sweep rubbish on the grassland into in collecting shovel 32, first steering wheel drives the bull stick and resumes to initial condition, the rotation of collection shovel 32 is driven to the second steering wheel, will collect the rubbish in the shovel 32 and empty to dustbin 33 in, the second steering wheel drives collection shovel 32 again and rotates to initial position, accomplish the work of clearing up.
Example 2
Referring to fig. 6 and 7, the automatic lawn garbage cleaning machine based on the deep convolutional neural network comprises a frame 1 and a traveling assembly 2 arranged on the frame 1, and it is worth explaining that the traveling assembly 2 comprises a planetary gear speed reducing mechanism, a hub mechanism and a rubber crawler drive. The planetary gear speed reducing mechanism comprises a front planetary frame, a rear planetary frame, a driving shaft, a sun gear, a planetary shaft, a planetary gear and a clutch; the sun gear is fixedly mounted on the driving shaft, the driving shaft penetrates through the rear planet carrier, the planet gears are connected and mounted on the planet shafts through bearings, the three planet gears are meshed with the sun gear, the planet shafts are connected between the front planet carrier and the rear planet carrier in an installing mode, and the clutch is mounted between the driving shaft and the rear planet carrier. The planetary gear reduction mechanism further comprises a brake, and the brake is mounted between the rear planet carrier and the machine body 2. The hub mechanism comprises hub plates, reinforcing plates and a bearing shaft, wherein the hub plate on each side is correspondingly connected with the reinforcing plates through the bearing shaft, one end of each supporting rod is hinged with the hub plate, and the other end of each supporting rod is hinged with the front planet carrier or the rear planet carrier. The rubber track transmission mechanism comprises a rubber track, a driving wheel and a bearing wheel; the driving wheel is arranged on the planet shaft and is fixedly connected to the planet wheel, the left end and the right end of the planet wheel are respectively provided with the driving wheel, the bearing wheels are arranged on the bearing shaft between the hub plate and the reinforcing plate, the rubber track wraps the driving wheel and the bearing wheels, and the inner side of the rubber track is meshed with the driving wheel.
The cleaning machine further comprises an obstacle avoidance device, the obstacle avoidance device is electrically connected with the central controller, and the obstacle avoidance device is used for detecting whether obstacles exist around the machine body. Keep away barrier device and include the infrared distance measuring sensor in place ahead, the infrared distance measuring sensor in left side and the infrared distance measuring sensor in right side, the infrared distance measuring sensor in place ahead, the infrared distance measuring sensor in left side all with central controller electric connection.
Referring to fig. 6 and 7, the cleaning machine further includes:
the cleaning assembly 3 is used for cleaning garbage on the grassland;
the image acquisition device 4 is used for acquiring an environmental image of a grassland, and it is worth to be noted that in the embodiment, the image acquisition device 4 is a high-definition camera;
the central controller, the image acquisition device 4 and the cleaning assembly 3 are electrically connected with the central controller, and in the embodiment, the central controller adopts a high-pass single-core CPU;
the central controller is internally provided with a garbage recognition device which is electrically connected with the image acquisition device 4 and is used for processing the grassland environment image acquired by the image acquisition device 4 and recognizing whether a garbage image exists in the grassland environment image; when the garbage recognition device recognizes that the images of the garbage exist in the environmental images of the grassland, the central controller controls the cleaning assembly 3 to clean the garbage;
the image acquisition device 4, the cleaning assembly 3 and the central controller are all arranged on the frame 1.
The garbage recognition device comprises a training module, a target detection model and an image preprocessing module.
The training module is used for training the convolutional neural network by using the environmental images of a plurality of grasslands to obtain a deep convolutional neural network model, the deep convolutional neural network model is Tiny-Yolov3, and the training module optimizes the Tiny-Yolov3 model by adopting a random gradient descent method.
A target detection model comprising Tiny-YOLOv3 for identifying images of litter in a plurality of images of an environment of grass; the Tiny-YOLOv3 model uses a binary cross entropy loss function for class prediction,
wherein N is the total number of training pictures; the yi value is 0 or 1, the yi value is 1, the ith input picture contains the image of the garbage, and the yi value is 0, the ith input picture does not contain the image of the garbage; the pi value is a probability of prediction of whether the ith input picture contains a spam image, and is between 0 and 1. The Tiny-YOLOv3 model includes a Darknet framework, the Darknet framework includes 53 convolutional layers and 22 Residual layers, 53 convolutional layers in the Darknet framework are used for carrying out feature extraction on the environmental image of the grassland, and 22 Residual layers in the Darknet framework are used for solving gradient diffusion or gradient explosion in the deep convolutional neural network model.
And the image preprocessing module is used for carrying out histogram equalization processing and logarithmic transformation processing on the environmental image of the grassland, obtaining the processed environmental image of the grassland and sending the processed environmental image of the grassland to the training module and the target detection model.
Referring to fig. 3, a method for automatically cleaning grassland garbage based on a deep convolutional neural network comprises the following steps,
s1: training the convolutional neural network by using the environmental images of a plurality of grasslands to obtain a Tiny-Yolov3 model, and executing S2;
s2: the method comprises the following steps that an environment grid map is established by taking an initial position as an original point, and when the environment grid map is established, a central controller establishes the grid map by taking the initial position of a current rack 1 as the original point O and a boundary point according to the set map length and width, and an operator can change the length or width of the grid map through an upper computer or a mobile terminal; a grid in the grid map is represented by a two-dimensional array map [ ] [ ], map [ x ] [ y ] ═ 0, which indicates that the grid has not been visited, map [ x ] [ y ] ═ 1, which indicates that an obstacle exists in the grid, map [ x ] [ y ] ═ 2, which indicates that the cleaner has run, and S3 is executed;
s3: acquiring a plurality of real-time grassland environment images, and executing S4;
s4: identifying whether garbage images exist in a plurality of real-time grassland environment images, if so, executing S5, otherwise, executing S6;
s5: acquiring the specific position and the frame of the image of the rubbish in the grassland environment image, cleaning the rubbish, and executing S6;
s6: detecting whether barriers exist around the rack 1 or not, controlling a walking device to drive the cleaning machine to move into the next grid according to a walking rule and an internal spiral algorithm, and executing S3; the internal spiral algorithm means that the system covers the grid map in a certain direction, and in this embodiment, the system covers the grid map in a clockwise direction. The grid which the system does not pass through is marked by map [ x ] [ y ] - (0), the grid which the system passes through is marked by map [ x ] [ y ] - (2), the system detects the grid with the obstacle, the grid is marked by map [ x ] [ y ] - (1), the cleaner updates the grid mark of the grid map every time the cleaner passes through one grid, and the walking rule concretely comprises the following steps,
s61: detecting whether an obstacle exists in front of the rack 1, if not, executing S62, and if so, executing S63;
s62: detecting whether an obstacle exists on the left side of the rack 1, if not, turning left, and if so, moving straight;
s63: detecting whether an obstacle exists on the left side of the rack 1, if not, turning left, and if so, turning right; it is worth to be noted that, when the locomotive is located in a certain grid and the front, left and right sides of the grid all detect the existence of obstacles, the central controller selects the next grid which has the shortest distance to the grid and is not accessed, and plans the route control locomotive to move to the next grid;
s4 further includes the following steps,
s41: setting an IOU threshold and a confidence threshold, and executing S42;
s42: resizing the input environment image, and performing S43;
s43: inputting the data into a Tiny-YOLOv3 model for feature extraction, and executing S44;
s44: carrying out multi-scale fusion prediction on smoke or flame through a similar FPN network, and dividing the characteristic diagram into a plurality of grids; and clustering the boundary frames of the training set by using a K-means clustering method to obtain a proper anchor box, generating 3 anchor box numbers on each grid to generate a predicted target boundary frame, and predicting the category by using a binary cross entropy loss function.
S5, the automatic lawn garbage cleaning machine based on the deep convolutional neural network is used for cleaning garbage, and the method specifically comprises the following steps,
s51: the initial state of the rotating rod is that one end close to the broom head is arranged in an inclined way towards the direction far away from the collecting shovel 32, the first steering engine drives the rotating rod to rotate by taking the end point of the rotating rod as the center of a circle and taking the rotating rod as the radius, so that the garbage on the grassland is swept into the collecting shovel 32, the first steering engine drives the rotating rod to return to the initial state, and S52 is executed;
s52: the second steering wheel drives the collection shovel 32 to rotate, and dumps the rubbish in the collection shovel 32 to the dustbin 33, and the second steering wheel drives the collection shovel 32 to rotate to the initial position again.
Referring to fig. 4 and 5, it is worth explaining that, in the present embodiment, before training the Tiny-yollov 3 model, daily garbage that often appears on grass is collected and then distributed to different types of grass to enhance the generalization ability of the Tiny-yollov 3 model, 20000 grass garbage samples are obtained, data enhancement is performed on the garbage samples, 8000 grass garbage image samples with different angles, different lighting and different noises are obtained, all the collected grass garbage image samples are cut to 416 × 416, and a labelImage tool is selected to label the grass garbage images. 65000 grass litter samples were randomly selected for training, and the remaining samples were used as the test set. The training times are 100000 times in total, the weight is automatically saved every 5000 times of training, the basic learning rate is 0.001, the batch size is 32, the momentum is 0.9, the weight attenuation coefficient is 0.0005, and overfitting is reduced by adopting L2 regularization.
In this embodiment, the performance of the Tiny-yollov 3 model is also studied, and the Accuracy (Accuracy) and the Recall (Recall) of the Tiny-yollov 3 model for identifying garbage are obtained, as shown in table 1:
| type (B) | Rate of accuracy | Recall rate |
| Garbage collection | 94.12 | 92.38 |
The accuracy rate of the Tiny-yolo v3 target detection model on the detection of the grassland garbage is 94.12%, the recall rate is 92.38%, and the accuracy rate and the recall rate on the detection of the grassland garbage are higher.
The implementation principle of the embodiment is as follows: the walking assembly 2 drives the frame 1 to walk to the region that has rubbish, and the high definition camera shoots the environmental image on many current meadows to transmit the environmental image on many current meadows to central controller in. Whether there is the image of rubbish in the environmental image of many current grasslands of rubbish recognition device discernment in the central controller, if there is, the initial condition of bull stick sets up to the direction slope of keeping away from collection shovel 32 for the one end that is close to the broom head, first steering wheel drives the extreme point that the bull stick used the bull stick and is the centre of a circle, use the bull stick to rotate as the radius, sweep rubbish on the grassland into in collecting shovel 32, first steering wheel drives the bull stick and resumes to initial condition, the rotation of collection shovel 32 is driven to the second steering wheel, will collect the rubbish in the shovel 32 and empty to dustbin 33 in, the second steering wheel drives collection shovel 32 again and rotates to initial position, accomplish the work of clearing up. Whether barriers exist around the rack 1 or not is detected, and the walking assembly 2 is controlled to drive the cleaning machine to move to the next grid according to the walking rule and the internal spiral algorithm, so that the next garbage recognition and cleaning work is carried out.
In the description of the present invention, it is to be understood that the terms "counterclockwise", "clockwise", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate orientations or positional relationships based on those shown in the drawings, and are used for convenience of description only, and do not indicate or imply that the referenced devices or elements must have a particular orientation, be constructed and operated in a particular orientation, and thus, are not to be considered as limiting.
Claims (10)
1. The utility model provides an automatic descaling machine of meadow rubbish based on degree of depth convolution neural network, includes frame (1) and sets up walking subassembly (2) on frame (1), its characterized in that still includes:
a cleaning assembly (3), wherein the cleaning assembly (3) is used for cleaning rubbish on the grassland;
the image acquisition device (4) is used for acquiring an environmental image of the grassland;
the image acquisition device (4) and the cleaning assembly (3) are electrically connected with the central controller;
the central controller is internally provided with a garbage recognition device which is electrically connected with the image acquisition device (4) and used for processing the grassland environment image acquired by the image acquisition device (4) and recognizing whether a garbage image exists in the grassland environment image; when the garbage recognition device recognizes that the images of the garbage exist in the environmental images of the grassland, the central controller controls the cleaning assembly (3) to clean the garbage;
the image acquisition device (4), the cleaning assembly (3) and the central controller are all arranged on the rack (1).
2. The automatic lawn garbage cleaning machine based on the deep convolutional neural network as claimed in claim 1, characterized in that: the cleaning assembly (3) comprises a cleaning broom, a collecting shovel (32) and a garbage can (33), the cleaning broom comprises a rotating rod and a broom head, one end of the rotating rod is rotatably arranged on the rack (1), a first steering engine for driving the rotating rod to rotate is arranged on the rack (1), and the other end of the rotating rod is connected with the broom head;
dustbin (33) set up in frame (1), dustbin (33) are close to the one side rotation that cleans the broom and are connected with the pivot, the length direction of pivot with the length direction of pivot is perpendicular, be connected with in the pivot collect shovel (32), dustbin (33) are provided with the drive the pivot revolutes the axis pivoted second steering wheel of axle.
3. The automatic lawn garbage cleaning machine based on the deep convolutional neural network as claimed in any one of claims 1-2, characterized in that: the garbage recognition device comprises
The training module is used for training the convolutional neural network by using the environmental images of a plurality of grasslands to obtain a deep convolutional neural network model;
the target detection model is used for identifying images of garbage in a plurality of grassland environment images;
and the image preprocessing module is used for carrying out histogram equalization processing and logarithmic transformation processing on the environmental image of the grassland, obtaining the processed environmental image of the grassland and sending the processed environmental image of the grassland to the training module and the target detection model.
4. The automatic lawn garbage cleaning machine based on the deep convolutional neural network as claimed in claim 3, characterized in that: the deep convolutional neural network model is Tiny-YOLOv 3.
5. The automatic lawn garbage cleaning machine based on the deep convolutional neural network as claimed in claim 4, wherein: the Tiny-YOLOv3 model uses a binary cross entropy loss function for class prediction,
wherein N is the total number of training pictures; the yi value is 0 or 1, the yi value is 1, the ith input picture contains the image of the garbage, and the yi value is 0, the ith input picture does not contain the image of the garbage; the pi value is a probability of prediction of whether the ith input picture contains a spam image, and is between 0 and 1.
6. The automatic lawn garbage cleaning machine based on the deep convolutional neural network as claimed in any one of claims 4-5, characterized in that: the deep convolutional neural network model comprises a DarkNet framework, the DarkNet framework comprises 53 convolutional layers and 22 Residual layers, 53 convolutional layers in the DarkNet framework are used for carrying out feature extraction on an environment image of a grassland, and 22 Residual layers in the DarkNet framework are used for solving gradient diffusion or gradient explosion in the deep convolutional neural network model.
7. The automatic lawn garbage cleaning machine based on the deep convolutional neural network as claimed in any one of claims 4-5, characterized in that: the training module adopts a random gradient descent method to optimize a Tiny-YOLOv3 model.
8. A grassland garbage automatic cleaning method based on a deep convolutional neural network is characterized by comprising the following steps,
s1: training the convolutional neural network by using the environmental images of a plurality of grasslands to obtain a Tiny-Yolov3 model, and executing S2;
s2: establishing an environment grid map by taking the initial position as an origin, and executing S3;
s3: acquiring a plurality of real-time grassland environment images, and executing S4;
s4: identifying whether garbage images exist in a plurality of real-time grassland environment images, if so, executing S5, otherwise, executing S6;
s5: acquiring the specific position and the frame of the image of the rubbish in the grassland environment image, cleaning the rubbish, and executing S6;
s6: moving into the next grid, S3 is performed.
9. The automatic lawn garbage cleaning method based on deep convolutional neural network as claimed in claim 8, wherein said S4 further comprises the following steps,
s41: setting an IOU threshold and a confidence threshold, and executing S42;
s42: resizing the input environment image, and performing S43;
s43: inputting the data into a Tiny-YOLOv3 model for feature extraction, and executing S44;
s44: carrying out multi-scale fusion prediction on smoke or flame through a similar FPN network, and dividing the characteristic diagram into a plurality of grids; and clustering the boundary frames of the training set by using a K-means clustering method to obtain a proper anchor box, generating 3 anchor box numbers on each grid to generate a predicted target boundary frame, and predicting the category by using a binary cross entropy loss function.
10. The automatic lawn garbage cleaning method based on the deep convolutional neural network as claimed in any one of claims 8-9, wherein the S5 performs garbage cleaning work by using the automatic lawn garbage cleaning machine based on the deep convolutional neural network as claimed in any one of claims 1-7, which comprises the following steps,
s51: the initial state of the rotating rod is that one end close to the broom head is arranged in an inclined mode towards the direction far away from the collecting shovel (32), the first steering engine drives the rotating rod to rotate by taking the end point of the rotating rod as the circle center and the rotating rod as the radius, the garbage on the grassland is swept into the collecting shovel (32), the first steering engine drives the rotating rod to return to the initial state, and S52 is executed;
s52: the second steering engine drives the collection shovel (32) to rotate, the garbage in the collection shovel (32) is dumped into the garbage can (33), and the second steering engine drives the collection shovel (32) to rotate to the initial position.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010391121.9A CN111567331A (en) | 2020-05-11 | 2020-05-11 | Automatic lawn garbage cleaning machine and method based on deep convolutional neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010391121.9A CN111567331A (en) | 2020-05-11 | 2020-05-11 | Automatic lawn garbage cleaning machine and method based on deep convolutional neural network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111567331A true CN111567331A (en) | 2020-08-25 |
Family
ID=72115352
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010391121.9A Pending CN111567331A (en) | 2020-05-11 | 2020-05-11 | Automatic lawn garbage cleaning machine and method based on deep convolutional neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111567331A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113420583A (en) * | 2021-01-29 | 2021-09-21 | 山东农业大学 | Color-changing wood monitoring system and method based on edge computing platform |
| CN113903006A (en) * | 2021-12-09 | 2022-01-07 | 北京云迹科技有限公司 | Robot monitoring sanitation method and device, electronic equipment and storage medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104563028A (en) * | 2015-01-08 | 2015-04-29 | 余朝松 | Road sweeper |
| CN104724426A (en) * | 2015-03-30 | 2015-06-24 | 济南大学 | Intelligent garbage collection vehicle |
| CN106553196A (en) * | 2016-11-09 | 2017-04-05 | 重庆向日葵机器人有限公司 | The sanitation robot and its control method of rubbish are unloaded automatically can |
| CN108086214A (en) * | 2018-01-26 | 2018-05-29 | 张伟 | A kind of outdoor sweeping robot |
| CN109934121A (en) * | 2019-02-21 | 2019-06-25 | 江苏大学 | A kind of orchard pedestrian detection method based on YOLOv3 algorithm |
| CN110222769A (en) * | 2019-06-06 | 2019-09-10 | 大连理工大学 | A kind of Further aim detection method based on YOLOV3-tiny |
| KR20190128500A (en) * | 2018-05-08 | 2019-11-18 | 한국전자통신연구원 | Method and apparatus for detecting garbage dumping action in real time on video surveillance system |
| CN110689118A (en) * | 2019-09-29 | 2020-01-14 | 华南理工大学 | Improved target detection method based on YOLO V3-tiny |
-
2020
- 2020-05-11 CN CN202010391121.9A patent/CN111567331A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104563028A (en) * | 2015-01-08 | 2015-04-29 | 余朝松 | Road sweeper |
| CN104724426A (en) * | 2015-03-30 | 2015-06-24 | 济南大学 | Intelligent garbage collection vehicle |
| CN106553196A (en) * | 2016-11-09 | 2017-04-05 | 重庆向日葵机器人有限公司 | The sanitation robot and its control method of rubbish are unloaded automatically can |
| CN108086214A (en) * | 2018-01-26 | 2018-05-29 | 张伟 | A kind of outdoor sweeping robot |
| KR20190128500A (en) * | 2018-05-08 | 2019-11-18 | 한국전자통신연구원 | Method and apparatus for detecting garbage dumping action in real time on video surveillance system |
| CN109934121A (en) * | 2019-02-21 | 2019-06-25 | 江苏大学 | A kind of orchard pedestrian detection method based on YOLOv3 algorithm |
| CN110222769A (en) * | 2019-06-06 | 2019-09-10 | 大连理工大学 | A kind of Further aim detection method based on YOLOV3-tiny |
| CN110689118A (en) * | 2019-09-29 | 2020-01-14 | 华南理工大学 | Improved target detection method based on YOLO V3-tiny |
Non-Patent Citations (2)
| Title |
|---|
| 刘军等: "基于增强Tiny YOLOV3算法的车辆实时检测与跟踪", 《农业工程学报》 * |
| 焦李成等: "《人工智能前沿技术丛书 模式识别》", 30 September 2019, 西安电子科技大学出版社 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113420583A (en) * | 2021-01-29 | 2021-09-21 | 山东农业大学 | Color-changing wood monitoring system and method based on edge computing platform |
| CN113903006A (en) * | 2021-12-09 | 2022-01-07 | 北京云迹科技有限公司 | Robot monitoring sanitation method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111567331A (en) | Automatic lawn garbage cleaning machine and method based on deep convolutional neural network | |
| CN111387886B (en) | Cleaning device and cleaning robot | |
| CN116508493B (en) | Gantry crawler type tea picking robot and picking method thereof | |
| CN109479628B (en) | Dead leaf collection device | |
| KR20210056033A (en) | Apparatus for cleaning panel | |
| CN204169783U (en) | Sucking mouth and the dust catcher possessing this sucking mouth | |
| CN111560896A (en) | Cleaning device for intelligence is made | |
| CN110593173A (en) | Novel independent multifunctional outdoor unmanned sweeper | |
| CN108967252A (en) | A kind of herding robot overall mechanism | |
| CN117456395A (en) | Sea and land two-domain garbage recycling planning method based on machine vision | |
| CN108677842A (en) | One kind being applied to the clean electrostatic generator absorption dust of urban road and transmits recovery system with crawler type rubbish | |
| CN116949986A (en) | Fallen leaf suction sweeper | |
| CN208762931U (en) | A kind of environment sweeping robot that open air uses | |
| CN111506074A (en) | Machine control method of crop tedding dust collection device | |
| CN115571337A (en) | Agricultural soil parameter information acquisition unmanned aerial vehicle | |
| CN109159106A (en) | Twin-stage round brush based on Mecanum wheel picks up shape fruit robot | |
| CN106564061B (en) | Lifting type garbage transmits sanitation robot and its control method | |
| CN114345853A (en) | Intelligent urban road pipeline cleaning system and cleaning process based on artificial intelligence | |
| CN209036519U (en) | Twin-stage round brush based on Mecanum wheel picks up spherical fruit robot | |
| CN107627314A (en) | A kind of pathfinding robot, Pathfinding system and method for searching based on genetic algorithm | |
| CN213476742U (en) | Garbage cleaning device for environmental protection | |
| CN216955737U (en) | Monitoring device for forest vegetation restoration in sample line investigation | |
| CN212533875U (en) | Afforestation refuse treatment device | |
| CN220402830U (en) | Mobile monitoring and early warning device for migratory rice pests | |
| CN2384985Y (en) | Light dust-proof road-spweeper |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200825 |
|
| RJ01 | Rejection of invention patent application after publication |