CN111914815A - Machine vision intelligent recognition system and method for garbage target - Google Patents
Machine vision intelligent recognition system and method for garbage target Download PDFInfo
- Publication number
- CN111914815A CN111914815A CN202010924563.5A CN202010924563A CN111914815A CN 111914815 A CN111914815 A CN 111914815A CN 202010924563 A CN202010924563 A CN 202010924563A CN 111914815 A CN111914815 A CN 111914815A
- Authority
- CN
- China
- Prior art keywords
- target
- data
- garbage
- layer
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 239000010813 municipal solid waste Substances 0.000 title claims abstract description 59
- 238000000034 method Methods 0.000 title claims abstract description 24
- 238000012549 training Methods 0.000 claims abstract description 21
- 238000012360 testing method Methods 0.000 claims abstract description 14
- 238000013507 mapping Methods 0.000 claims abstract description 7
- 239000002699 waste material Substances 0.000 claims description 10
- 230000006870 function Effects 0.000 claims description 9
- 238000002372 labelling Methods 0.000 claims description 9
- 238000012545 processing Methods 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 claims description 6
- 239000011159 matrix material Substances 0.000 claims description 6
- 230000009467 reduction Effects 0.000 claims description 6
- 230000001629 suppression Effects 0.000 claims description 6
- 238000010586 diagram Methods 0.000 claims description 5
- 230000001131 transforming effect Effects 0.000 claims description 5
- 230000008859 change Effects 0.000 claims description 4
- 238000005457 optimization Methods 0.000 claims description 4
- 230000008569 process Effects 0.000 claims description 4
- 230000003321 amplification Effects 0.000 claims description 3
- 230000009193 crawling Effects 0.000 claims description 3
- 238000005516 engineering process Methods 0.000 claims description 3
- 238000003199 nucleic acid amplification method Methods 0.000 claims description 3
- 238000007781 pre-processing Methods 0.000 claims description 3
- 230000001902 propagating effect Effects 0.000 claims description 3
- 238000007670 refining Methods 0.000 claims description 3
- 238000000638 solvent extraction Methods 0.000 claims description 3
- 230000001360 synchronised effect Effects 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 3
- 238000013519 translation Methods 0.000 claims description 3
- 238000012544 monitoring process Methods 0.000 abstract description 6
- 238000004140 cleaning Methods 0.000 abstract description 4
- 230000000694 effects Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 244000276331 Citrus maxima Species 0.000 description 1
- 235000001759 Citrus maxima Nutrition 0.000 description 1
- 241000255925 Diptera Species 0.000 description 1
- 241000282326 Felis catus Species 0.000 description 1
- 244000141359 Malus pumila Species 0.000 description 1
- 240000008790 Musa x paradisiaca Species 0.000 description 1
- 235000018290 Musa x paradisiaca Nutrition 0.000 description 1
- 235000021016 apples Nutrition 0.000 description 1
- 235000013405 beer Nutrition 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 235000013361 beverage Nutrition 0.000 description 1
- 235000019504 cigarettes Nutrition 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 239000010791 domestic waste Substances 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 238000012856 packing Methods 0.000 description 1
- 239000011087 paperboard Substances 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C22/00—Measuring distance traversed on the ground by vehicles, persons, animals or other moving solid bodies, e.g. using odometers, using pedometers
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/86—Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/26—Government or public services
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Business, Economics & Management (AREA)
- Data Mining & Analysis (AREA)
- Remote Sensing (AREA)
- Radar, Positioning & Navigation (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Tourism & Hospitality (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Multimedia (AREA)
- Computational Linguistics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Biomedical Technology (AREA)
- Development Economics (AREA)
- Educational Administration (AREA)
- Biophysics (AREA)
- Economics (AREA)
- Human Resources & Organizations (AREA)
- Marketing (AREA)
- Primary Health Care (AREA)
- Strategic Management (AREA)
- General Business, Economics & Management (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Computer Networks & Wireless Communication (AREA)
- Electromagnetism (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a machine vision intelligent recognition system of a garbage target, which comprises a milemeter module, a target recognition module and a mapping module, wherein the system of the invention establishes a dynamic map of garbage target distribution, reduces the monitoring pressure of managers and improves the working efficiency of cleaning personnel, and also discloses a machine vision intelligent recognition method of the garbage target, which comprises the following steps: the method comprises the steps of calculating pose data, identifying garbage targets, establishing a point cloud map, establishing a training set and a testing set, establishing an improved yolov3 model, training a network and testing the network, wherein the method uses a deep separable convolution to replace a part of traditional convolution in darknet-53, the calculated amount of the model is greatly reduced, the garbage targets can be identified more accurately based on an improved yolov3 target identification algorithm, the parameters of the model are fewer, the time for training and optimizing is reduced, the identification speed of the model is improved, and the requirement on hardware is reduced.
Description
[ technical field ]
The invention relates to the field of machine vision and machine learning, in particular to a machine vision intelligent recognition system and method for a garbage target.
[ background art ]
With the increase of the consumption level of residents, the quantity of household garbage generated by people in the community is continuously increased. The waste domestic garbage not only seriously affects the environmental sanitation of communities, but also is easy to breed mosquitoes and spread diseases so as to harm the health of people. At present, domestic garbage in the community is cleaned regularly by cleaning workers, the mode has good effect on treating the domestic garbage in the downstairs garbage can of residents, but the effect is poor and manpower is wasted for the domestic garbage which is nearly randomly generated in other areas in the community. Therefore, there is a need to develop a visual system capable of monitoring the changing conditions of the domestic waste in the community.
The existing garbage identification system is mainly provided with fixed cameras, and when the garbage change condition of the whole community needs to be monitored, more cameras need to be arranged; meanwhile, the identification system cannot integrate the identification results from all paths of monitoring videos into one map, so that huge challenges are brought to monitoring management personnel in the background.
In addition, the current garbage machine vision identification method mostly adopts two-step methods (R-CNN and false-RCNN) based on a deep learning target identification algorithm, but the method has the problems of more complicated training process, more difficult optimization result, large calculated amount, longer time for processing one frame of image, higher requirement on a hardware platform and the like. .
The present invention has been made in view of the above problems.
[ summary of the invention ]
The invention aims to provide a machine vision intelligent recognition system and a machine vision intelligent recognition method for garbage targets, which aim to overcome the defects of the prior art.
In order to solve the technical problems, the technical scheme adopted by the invention is as follows:
a machine vision intelligent recognition system for garbage targets comprises
The odometer module is used for calculating pose data of the robot;
the target recognition module is used for searching the position of the junk target in the image by using a target recognition algorithm and calculating the position data and the depth data of the junk target;
and the mapping module is used for receiving and processing the pose data of the odometer module, the garbage target position data and the depth data of the target identification module to obtain the coordinates of the garbage target in a world coordinate system, and splicing and constructing to obtain a point cloud map of the garbage target.
The machine vision intelligent recognition system for the garbage target comprises the wheel type odometer and the laser odometer.
The machine vision intelligent recognition system for the garbage target comprises a high-definition camera and a target recognition module.
A machine vision intelligent identification method of a rubbish target comprises the following steps:
step A, calculating pose data:
a1, wheel type odometer calibration: collecting a plurality of data segments, each data segment comprising angular velocities W of two wheelsLAnd WRCalculating the distance b between the two wheels and the radius of the two wheels by a model-based method according to the duration of the data and the matching value of the laser radar;
a2, removing the motion distortion of the laser radar data: the industrial personal computer reads laser data, the STM32 uploads odometer integral data, whether the position and attitude data in an odometer queue are synchronous with the laser radar data in time is checked, if so, the processing is not carried out, otherwise, secondary interpolation is carried out on the position and attitude data, and the laser radar data is transformed according to the position and attitude data to obtain the laser radar data with motion distortion removed;
a3, using a front-end registration algorithm on the acquired lidar data: firstly, searching a matching point in the point cloud, then carrying out pose change and error calculation on the point cloud according to a matching calculation rotation matrix R and a translation matrix T, and continuously iterating until the error is smaller than a tolerance value.
A4, optimizing the pose graph based on a graph optimization technology;
step B, identifying a garbage target:
the target recognition module finds the position of the junk target in the image by using a target recognition algorithm and then issues position data and depth data of the junk target;
step C, establishing a point cloud map:
the mapping module processes data from the odometer module and the target identification module, and performs coordinate transformation according to the following formula:
z＝d
and transforming the coordinates of the target under the camera coordinate system by using the optimized pose to obtain the coordinates of the target under the world coordinate system, and finally splicing and constructing a point cloud map.
The machine vision intelligent identification method of the rubbish target further comprises the following steps:
step D, establishing a training set and a testing set:
d1, refining the definition of the garbage target: selecting a plurality of waste products as definitions of garbage targets;
d2, collecting photographs of the waste: acquiring a corresponding sample image by adopting a field acquisition mode and a crawling mode;
d3, preprocessing of the image: cropping the collected image data and transforming to 416 x 416 standard sample data;
d4, amplification of sample data: sample data is expanded in the following way:
d4-1. magnification of width and height of sample image 1.5 times;
d4-2. width reduction 1/3 and height reduction 1/2 of the sample image, and ensuring that the image is a multiple of 32;
d4-3, brightness enhancement of the sample image;
d4-4, the brightness of the sample image is reduced;
d4-5, turning the sample image by 90 degrees or 180 degrees;
d4-6, adding noise to the sample image;
d5, manual labeling: according to the definition in the step D1, labeling the image data in the sample set by using an image labeling tool to obtain label data (x, y, w, h, c);
d6, partitioning of sample set: dividing a sample set into a training set and a test set according to a ratio of 98: 2;
step E, constructing an improved yolov3 model:
respectively creating a convolutional layer, a depth separable convolutional layer, an upsampling layer, a residual layer, a splicing layer and a prediction layer, wherein the upsampling layer uses a bilinear interpolation algorithm to expand a feature map, the residual layer adds the output of a lower layer and the output of an upper layer, the splicing layer superposes the output of the upper layer containing rich semantics and the output of a lower layer with higher resolution, the prediction layer comprises 3 feature maps, each grid of each feature map also comprises 3 anchor boxes with different length-width ratios, and each anchor box is a1 × 20 vector and comprises the following prediction information: t is tx,ty,tw,thThe bounding box contains the confidence of the object and the probability that the object belongs to the garbage target;
step F, training the network:
initializing the weight of the model by using a random initialization strategy, then, throwing training data into the model, predicting each grid of the layer feature diagram after forward propagation to obtain a prediction result of 3 x (4+1+15), and finally, calculating the error between the prediction result and the real result according to the following cost function formula:
calculating an error between a predicted value and a true value through the cost function, then reversely propagating the error, updating the weight and deviation of the network through a momentum gradient descent strategy, and stopping training when the loss value of the cost function is small and stable, namely, convergence;
step G, network testing:
test set data is lost into the model, a prediction result is obtained, and a confidence threshold of a boundary box is set to be ThscoresAnd (5) rejecting the prediction result with low confidence coefficient, setting a non-maximum suppression threshold value of 0.5, executing non-maximum suppression, preventing the same target from being identified in multiple ways, and finally outputting the corrected prediction result.
Compared with the prior art, the invention has the beneficial effects that:
1. the invention establishes a dynamic map of garbage target distribution based on the pose of the odometer module, reduces the monitoring pressure of managers and improves the working efficiency of cleaning personnel.
2. According to the method, a part of traditional convolution in darknet-53 is replaced by the deep separable convolution, the calculated amount of the model is greatly reduced, the garbage target can be identified more accurately based on the improved yolov3 target identification algorithm, the parameters of the model are fewer, the time for training and optimizing is shortened, the identification speed of the model is increased, and the requirement on hardware is lowered.
[ description of the drawings ]
FIG. 1 is a schematic structural view of the present invention;
FIG. 2 is a computing flow diagram of the odometer module;
FIG. 3 is a flow chart of the present invention for performing object recognition;
FIG. 4 is a flow chart of the present invention for establishing a point cloud map;
fig. 5 is a schematic structural diagram of the modified yolov3 model of the present invention.
[ detailed description of the invention ]
Reference will now be made in detail to the present preferred embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to like elements throughout.
As shown in fig. 1 to 5, a machine vision intelligent recognition system for a garbage target includes:
the odometer module 1 is used for calculating pose data of the robot;
the target recognition module 2 is used for searching the position of the junk target in the image by using a target recognition algorithm and calculating the position data and the depth data of the junk target;
and the mapping module 3 is used for receiving the pose data of the mileage meter processing module 1, the garbage target position data and the depth data of the target identification module 2, obtaining the coordinates of the garbage target in a world coordinate system, and splicing and constructing to obtain a point cloud map of the garbage target.
The odometer module 1 comprises a wheel type odometer 11 and a laser odometer 12, the laser odometer 12 comprises a 16-line laser radar, the wheel type odometer 11 comprises 2 photoelectric encoders, and the odometer module 1 estimates the pose of the robot in real time.
The target recognition module 2 comprises a high-definition camera.
The invention establishes a dynamic map of garbage target distribution based on the pose of the odometer module 1, reduces the monitoring pressure of managers and improves the working efficiency of cleaning personnel.
A machine vision intelligent identification method of a rubbish target comprises the following steps:
step A, calculating pose data:
a1, calibration of the wheeled odometer 11: collecting n data, each data including angular velocities W of two wheelsLAnd WRCalculating the distance b between the two wheels and the radius of the two wheels by a model-based method according to the duration of the data and the matching value of the laser radar;
a2, removing the motion distortion of the laser radar data: the industrial personal computer reads laser data, the STM32 uploads odometer integral data, whether the position and attitude data in an odometer queue are synchronous with the laser radar data in time is checked, if so, the processing is not carried out, otherwise, secondary interpolation is carried out on the position and attitude data, and the laser radar data is transformed according to the position and attitude data to obtain the laser radar data with motion distortion removed;
a3, using a front-end registration algorithm on the acquired lidar data: firstly, searching a matching point in the point cloud, then carrying out pose change and error calculation on the point cloud according to a matching calculation rotation matrix R and a translation matrix T, and continuously iterating until the error is smaller than a tolerance value.
A4, optimizing the pose graph based on a graph optimization technology;
step B, identifying a garbage target:
the target recognition module 2 finds the position of the junk target in the image by using a target recognition algorithm, and then issues position data and depth data of the junk target;
step C, establishing a point cloud map:
the mapping module 3 processes the data from the odometer module 1 and the target identification module 2, and performs coordinate transformation according to the following formula:
z＝d
wherein, (x, z) is the coordinate of the garbage target in the camera coordinate system, u is the position of the target in the image coordinate system, d is the distance between the target and the camera, fxAnd cxAll the parameters are internal parameters of the camera, coordinates of the target under a camera coordinate system are transformed by using the optimized pose to obtain coordinates of the target under a world coordinate system, point cloud splicing and filtering (an outlier removal filter and a down-sampling filter) are performed, and finally a point cloud map is spliced and constructed;
step D, establishing a training set and a testing set:
d1, refining the definition of the garbage target: selecting a plurality of waste products as definitions of garbage targets, specifically selecting 14 common waste products in communities as definitions of garbage targets, wherein the waste products comprise garbage bags, express packaging bags, take-out packing boxes, cat and dog excrement, orange peel, banana peel, shaddock peel, mashed apples, cigarette boxes, thick paper boards, waste newspapers, plastic beverage bottles, beer bottles and pop cans;
d2, collecting photographs of the waste: acquiring a corresponding sample image by adopting a field acquisition mode and a crawling mode;
d3, preprocessing of the image: cropping the collected image data and transforming to 416 x 416 standard sample data;
d4, amplification of sample data: sample data is expanded in the following way:
d4-1. magnification of width and height of sample image 1.5 times;
d4-2. width reduction 1/3 and height reduction 1/2 of the sample image, and ensuring that the image is a multiple of 32;
d4-3, brightness enhancement of the sample image;
d4-4, the brightness of the sample image is reduced;
d4-5, turning the sample image by 90 degrees or 180 degrees;
d4-6, adding noise to the sample image;
d5, manual labeling: labeling the image data in the sample set by using an image labeling tool imageLabel according to the definition in the step D1 to obtain label data (x, y, w, h, c), wherein (x, y) represents the center coordinates of the garbage target in the image, w and h represent the width and height of the garbage target respectively, c represents the category of the garbage target, c is 0 to represent the background, and c is 1 to 14 to sequentially represent the garbage targets defined in the step D1;
d6, partitioning of sample set: dividing a sample set into a training set and a test set according to a ratio of 98: 2;
step E, constructing an improved yolov3 model:
respectively creating a convolutional layer, a depth separable convolutional layer, an upsampling layer, a residual layer, a splicing layer and a prediction layer, wherein the improved yolov3 model is composed of the 6 basic levels, the depth separable convolutional layer is used for replacing a3 multiplied by 3 convolutional layer in an original yolov3 residual module, the upsampling layer uses a bilinear interpolation algorithm to expand a feature map, the residual layer adds the output of a lower layer with the output of a higher layer, and the splicing layer adds the output of the higher layer containing rich semantics with the output of a lower layer with higher resolutionFor superposition, the prediction layer contains 3 feature maps, each grid of each feature map also contains 3 anchors box with different aspect ratios, each anchor box is a1 × 20 vector and contains the following prediction information: t is tx,ty,tw,thThe bounding box contains the confidence of the object and the probability that the object belongs to the garbage target;
step F, training the network:
initializing the weight of the model by using a random initialization strategy, then, throwing training data into the model, predicting each grid of the layer feature diagram after forward propagation to obtain a prediction result of 3 x (4+1+15), and finally, calculating the error between the prediction result and the real result according to the following cost function formula:
calculating an error between a predicted value and a true value through the cost function, then reversely propagating the error, updating the weight and deviation of the network through a momentum gradient descent strategy, and stopping training when the loss value of the cost function is small and stable, namely, convergence;
step G, network testing:
test set data is lost into the model, a prediction result is obtained, and a confidence threshold of a boundary box is set to be ThscoresAnd (5) rejecting the prediction result with low confidence coefficient, setting a non-maximum suppression threshold value of 0.5, executing non-maximum suppression, preventing the same target from being identified in multiple ways, and finally outputting the corrected prediction result.
According to the method, a part of traditional convolution in darknet-53 is replaced by the deep separable convolution, the calculated amount of the model is greatly reduced, the garbage target can be identified more accurately based on the improved yolov3 target identification algorithm, the parameters of the model are fewer, the time for training and optimizing is shortened, the identification speed of the model is increased, and the requirement on hardware is lowered.
The above examples are merely preferred embodiments of the present invention, and other embodiments of the present invention are possible, such as a reasonable combination of the technical solutions described in the examples. Those skilled in the art can make equivalent changes or substitutions without departing from the spirit of the present invention, and such equivalent changes or substitutions are included in the scope set forth in the claims of the present application.
Claims (5)
1. The utility model provides a machine vision intelligent recognition system of rubbish target which characterized in that: comprises that
The odometer module (1) is used for calculating pose data of the robot;
the target recognition module (2) searches the position of the junk target in the image by using a target recognition algorithm and calculates the position data and the depth data of the junk target;
and the mapping module (3) is used for receiving and processing the pose data of the odometer module (1), the garbage target position data and the depth data of the target identification module (2) to obtain the coordinates of the garbage target under a world coordinate system, and splicing and constructing to obtain a point cloud map of the garbage target.
2. The machine vision intelligent recognition system of a spam target according to claim 1, wherein: the odometer module (1) comprises a wheeled odometer (11) and a laser odometer (12).
3. The machine vision intelligent recognition system of a spam target according to claim 1, wherein: the target recognition module (2) comprises a high-definition camera.
4. A machine vision intelligent identification method of a rubbish target is characterized by comprising the following steps:
step A, calculating pose data:
a1, calibration of wheeled odometer (11): collecting a plurality of data segments, each data segment comprising angular velocities W of two wheelsLAnd WRCalculating the distance b between the two wheels and the radius of the two wheels by a model-based method according to the duration of the data and the matching value of the laser radar;
a2, removing the motion distortion of the laser radar data: the industrial personal computer reads laser data, the STM32 uploads odometer integral data, whether the position and attitude data in an odometer queue are synchronous with the laser radar data in time is checked, if so, the processing is not carried out, otherwise, secondary interpolation is carried out on the position and attitude data, and the laser radar data is transformed according to the position and attitude data to obtain the laser radar data with motion distortion removed;
a3, using a front-end registration algorithm on the acquired lidar data: firstly, searching a matching point in the point cloud, then carrying out pose change and error calculation on the point cloud according to a matching calculation rotation matrix R and a translation matrix T, and continuously iterating until the error is smaller than a tolerance value.
A4, optimizing the pose graph based on a graph optimization technology;
step B, identifying a garbage target:
the target recognition module (2) finds the position of the junk target in the image by using a target recognition algorithm, and then releases the position data and the depth data of the junk target;
step C, establishing a point cloud map:
the mapping module (3) processes data from the odometer module (1) and the target identification module (2), and performs coordinate transformation according to the following formula:
z＝d
and transforming the coordinates of the target under the camera coordinate system by using the optimized pose to obtain the coordinates of the target under the world coordinate system, and finally splicing and constructing a point cloud map.
5. The machine vision intelligent recognition method of the junk target according to claim 4, further comprising the steps of:
step D, establishing a training set and a testing set:
d1, refining the definition of the garbage target: selecting a plurality of waste products as definitions of garbage targets;
d2, collecting photographs of the waste: acquiring a corresponding sample image by adopting a field acquisition mode and a crawling mode;
d3, preprocessing of the image: cropping the collected image data and transforming to 416 x 416 standard sample data;
d4, amplification of sample data: sample data is expanded in the following way:
d4-1. magnification of width and height of sample image 1.5 times;
d4-2. width reduction 1/3 and height reduction 1/2 of the sample image, and ensuring that the image is a multiple of 32;
d4-3, brightness enhancement of the sample image;
d4-4, the brightness of the sample image is reduced;
d4-5, turning the sample image by 90 degrees or 180 degrees;
d4-6, adding noise to the sample image;
d5, manual labeling: according to the definition in the step D1, labeling the image data in the sample set by using an image labeling tool to obtain label data (x, y, w, h, c);
d6, partitioning of sample set: dividing a sample set into a training set and a test set according to a ratio of 98: 2;
step E, constructing an improved yolov3 model:
respectively creating a convolutional layer, a depth separable convolutional layer, an upsampling layer, a residual layer, a splicing layer and a prediction layer, wherein the upsampling layer uses a bilinear interpolation algorithm to expand a feature map, the residual layer adds the output of a lower layer and the output of an upper layer, the splicing layer superposes the output of the upper layer containing rich semantics and the output of a lower layer with higher resolution, the prediction layer comprises 3 feature maps, each grid of each feature map also comprises 3 anchor boxes with different length-width ratios, and each anchor box is a1 × 20 vector and comprises the following prediction information: t is tx,ty,tw,thThe bounding box contains the confidence of the object and the probability that the object belongs to the garbage target;
step F, training the network:
initializing the weight of the model by using a random initialization strategy, then, throwing training data into the model, predicting each grid of the layer feature diagram after forward propagation to obtain a prediction result of 3 x (4+1+15), and finally, calculating the error between the prediction result and the real result according to the following cost function formula:
calculating an error between a predicted value and a true value through the cost function, then reversely propagating the error, updating the weight and deviation of the network through a momentum gradient descent strategy, and stopping training when the loss value of the cost function is small and stable, namely, convergence;
step G, network testing:
test set data is lost into the model, a prediction result is obtained, and a confidence threshold of a boundary box is set to be ThscoresAnd (5) rejecting the prediction result with low confidence coefficient, setting a non-maximum suppression threshold value of 0.5, executing non-maximum suppression, preventing the same target from being identified in multiple ways, and finally outputting the corrected prediction result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010924563.5A CN111914815A (en) | 2020-09-05 | 2020-09-05 | Machine vision intelligent recognition system and method for garbage target |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010924563.5A CN111914815A (en) | 2020-09-05 | 2020-09-05 | Machine vision intelligent recognition system and method for garbage target |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111914815A true CN111914815A (en) | 2020-11-10 |
Family
ID=73267259
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010924563.5A Pending CN111914815A (en) | 2020-09-05 | 2020-09-05 | Machine vision intelligent recognition system and method for garbage target |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111914815A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112560755A (en) * | 2020-12-24 | 2021-03-26 | 中再云图技术有限公司 | Target detection method for identifying urban exposed garbage |
| CN113033672A (en) * | 2021-03-29 | 2021-06-25 | 西安电子科技大学 | Multi-class optical image rotating target self-adaptive detection method based on feature enhancement |
| CN113436251A (en) * | 2021-06-24 | 2021-09-24 | 东北大学 | Pose estimation system and method based on improved YOLO6D algorithm |
| CN117421692A (en) * | 2023-12-18 | 2024-01-19 | 深圳火眼智能有限公司 | Garbage illegal delivery identification method, device and equipment for garbage delivery station |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110263675A (en) * | 2019-06-03 | 2019-09-20 | 武汉联一合立技术有限公司 | Garbage target recognition system and recognition method of community security robot |
| CN110796186A (en) * | 2019-10-22 | 2020-02-14 | 华中科技大学无锡研究院 | Dry and wet garbage identification and classification method based on improved YOLOv3 network |
| WO2020037960A1 (en) * | 2018-08-21 | 2020-02-27 | 深圳大学 | Sar target recognition method and apparatus, computer device, and storage medium |
| CN111360780A (en) * | 2020-03-20 | 2020-07-03 | 北京工业大学 | Garbage picking robot based on visual semantic SLAM |
-
2020
- 2020-09-05 CN CN202010924563.5A patent/CN111914815A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020037960A1 (en) * | 2018-08-21 | 2020-02-27 | 深圳大学 | Sar target recognition method and apparatus, computer device, and storage medium |
| CN110263675A (en) * | 2019-06-03 | 2019-09-20 | 武汉联一合立技术有限公司 | Garbage target recognition system and recognition method of community security robot |
| CN110796186A (en) * | 2019-10-22 | 2020-02-14 | 华中科技大学无锡研究院 | Dry and wet garbage identification and classification method based on improved YOLOv3 network |
| CN111360780A (en) * | 2020-03-20 | 2020-07-03 | 北京工业大学 | Garbage picking robot based on visual semantic SLAM |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112560755A (en) * | 2020-12-24 | 2021-03-26 | 中再云图技术有限公司 | Target detection method for identifying urban exposed garbage |
| CN112560755B (en) * | 2020-12-24 | 2022-08-19 | 中再云图技术有限公司 | Target detection method for identifying urban exposed garbage |
| CN113033672A (en) * | 2021-03-29 | 2021-06-25 | 西安电子科技大学 | Multi-class optical image rotating target self-adaptive detection method based on feature enhancement |
| CN113033672B (en) * | 2021-03-29 | 2023-07-28 | 西安电子科技大学 | Multi-class optical image rotation target self-adaptive detection method based on feature enhancement |
| CN113436251A (en) * | 2021-06-24 | 2021-09-24 | 东北大学 | Pose estimation system and method based on improved YOLO6D algorithm |
| CN113436251B (en) * | 2021-06-24 | 2024-01-09 | 东北大学 | Pose estimation system and method based on improved YOLO6D algorithm |
| CN117421692A (en) * | 2023-12-18 | 2024-01-19 | 深圳火眼智能有限公司 | Garbage illegal delivery identification method, device and equipment for garbage delivery station |
| CN117421692B (en) * | 2023-12-18 | 2024-04-09 | 深圳火眼智能有限公司 | Garbage illegal delivery identification method, device and equipment for garbage delivery station |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110263675B (en) | Garbage target identification system and method of community security robot | |
| CN111914815A (en) | Machine vision intelligent recognition system and method for garbage target | |
| CN111461245B (en) | Wheeled robot semantic mapping method and system fusing point cloud and image | |
| CN106845408B (en) | Street garbage identification method under complex environment | |
| CN107203781B (en) | End-to-end weak supervision target detection method based on significance guidance | |
| CN109858569A (en) | Multi-tag object detecting method, system, device based on target detection network | |
| CN102096821B (en) | Number plate identification method under strong interference environment on basis of complex network theory | |
| CN113076871B (en) | Fish shoal automatic detection method based on target shielding compensation | |
| CN113408584B (en) | RGB-D multi-modal feature fusion 3D target detection method | |
| CN112488025B (en) | Double-temporal remote sensing image semantic change detection method based on multi-modal feature fusion | |
| CN104156731A (en) | License plate recognition system based on artificial neural network and method | |
| CN110675421B (en) | Depth image collaborative segmentation method based on few labeling frames | |
| CN110032952B (en) | Road boundary point detection method based on deep learning | |
| CN112991534B (en) | Indoor semantic map construction method and system based on multi-granularity object model | |
| CN113469264A (en) | Construction method of automatic garbage classification model, garbage sorting method and system | |
| WO2022247045A1 (en) | Laser radar information-based mobile robot location re-identification method | |
| CN113065486A (en) | Floater identification method, system, equipment and medium suitable for river channel | |
| CN114022837A (en) | Station left article detection method and device, electronic equipment and storage medium | |
| CN111611956B (en) | Rail detection method and system for subway visual image | |
| CN115761674A (en) | Road edge positioning detection method, equipment and medium | |
| CN113129370B (en) | Semi-supervised object pose estimation method combining generated data and label-free data | |
| CN110853058B (en) | High-resolution remote sensing image road extraction method based on visual saliency detection | |
| CN113222025B (en) | Feasible region label generation method based on laser radar | |
| CN115359411B (en) | Substation environment understanding method based on improved deep Lab V < 3+ > network | |
| CN212624084U (en) | Machine vision intelligent recognition device of rubbish target |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |