CN117649638A - Automatic sorting method and system for corn haploids based on computer vision - Google Patents

Automatic sorting method and system for corn haploids based on computer vision Download PDF

Info

Publication number
CN117649638A
CN117649638A CN202311606186.0A CN202311606186A CN117649638A CN 117649638 A CN117649638 A CN 117649638A CN 202311606186 A CN202311606186 A CN 202311606186A CN 117649638 A CN117649638 A CN 117649638A
Authority
CN
China
Prior art keywords
seed
seeds
corn
seed metering
metering device
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311606186.0A
Other languages
Chinese (zh)
Inventor
和贤桃
朱晋葶
杨丽
张东兴
崔涛
张凯良
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Agricultural University
Original Assignee
China Agricultural University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Agricultural University filed Critical China Agricultural University
Priority to CN202311606186.0A priority Critical patent/CN117649638A/en
Publication of CN117649638A publication Critical patent/CN117649638A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/70—Labelling scene content, e.g. deriving syntactic or semantic representations
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A40/00—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production
    • Y02A40/10—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production in agriculture

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Multimedia (AREA)
  • Artificial Intelligence (AREA)
  • Software Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Computational Linguistics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • General Engineering & Computer Science (AREA)
  • Molecular Biology (AREA)
  • Data Mining & Analysis (AREA)
  • Biophysics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Pretreatment Of Seeds And Plants (AREA)

Abstract

The invention discloses a method and a system for automatically sorting corn haploids based on computer vision; the method comprises the following steps: preprocessing the images through cutting and the like, and respectively inputting the three-side images into a corn haploid recognition model to obtain classification results; if the classification results of the three-side images are all non-embryo surfaces, the embryo parts are not collected, a non-collection information screening air tap arranged in the seed metering disc is started, seeds are separated from the suction holes before entering the seed pushing area, and the seeds return to the seed filling area; if the three surfaces contain embryo surfaces, the seeds are separated from the suction holes of the seed metering disc under the action of the seed cleaning and scraping device, and are judged according to the classification result; when the seeds judged to be haploid fall, the seed sorting air tap starts to sort the seeds into a haploid seed box, and the seeds judged to be diploid fall freely into a diploid seed box. According to the invention, corn kernels do not need to be manually placed, so that the sorting process is more automatic, and the method is more suitable for image information acquisition of corn kernels with various shapes such as round kernels.

Description

Automatic sorting method and system for corn haploids based on computer vision
Technical Field
The invention belongs to the field of corn breeding and nondestructive testing, and particularly relates to a computer vision-based automatic sorting method and system for corn haploids.
Background
The haploid breeding technology is a rapid and efficient breeding way, the breeding period can be greatly shortened, but the natural induction rate of the corn haploid is about 0.1%, the artificial induction rate is only about 10%, and therefore, the rapid and nondestructive separation of haploid seeds from a large number of seeds is a key step for improving the haploid breeding efficiency.
The most effective and reliable method for identifying haploids is a genetic marking method, the method introduces a Navajo marker gene R-nj into corn seeds, marked haploid kernels only have purple color marks on endosperm parts, and diploid kernels have color marks on endosperm and embryo parts, so that haploid kernels can be rapidly and nondestructively sorted by identifying whether the embryo parts have color marks or not.
The method for automatically sorting the corn haploid grains comprises a machine vision method and a nuclear magnetic resonance method. The nuclear magnetic resonance method has the advantages of long identification time and expensive equipment; the machine vision method is fast, the equipment cost is lower, but the existing machine vision detection equipment is complex in structure and large in size, corn kernels are required to be placed manually in an embryo-facing mode, full-automatic sorting cannot be achieved, the requirement on the shape of the corn kernels is high, spherical seeds are difficult to stably place and detect, and the machine vision detection equipment is mainly applicable to horse-tooth seeds.
Meanwhile, because the haploid grain and the diploid grain are different in color marks of embryo surfaces, the embryo parts of the grains need to be manually placed upwards when pictures are collected, and therefore the efficiency of embryo picture collection needs to be improved urgently.
Disclosure of Invention
Aiming at the problem of how to realize automatic sorting of haploid grains in the background technology, the invention provides a method and a system for automatically sorting corn haploid based on computer vision, which can realize automation of the whole processes of feeding, detection and sorting, and simultaneously can obtain three images of corn grains at different angles through mirrors arranged on two sides in a single detection, thereby being applicable to corn grains with various shapes and improving sorting efficiency; the technical scheme of the method comprises the following steps:
step A, feeding haploid and diploid seeds to be detected into an air-suction type seed metering device through a feed hopper, wherein the seeds are adsorbed on a suction hole of a seed metering disc by single seeds and rotate along with the suction hole;
b, triggering a color industrial camera to collect pictures by using a photoelectric sensor, and simultaneously collecting three-side images of corn kernels passing through an image collecting area by using two plane mirrors arranged in the image collecting area;
step C, preprocessing the images through cutting and the like, and respectively inputting the three-side images into a corn haploid recognition model to obtain classification results;
step D, if the classification results of the three-side images are non-embryo surfaces, judging that embryo parts are not collected, starting a non-collection information screening air tap arranged in the seed metering disc, separating seeds from the suction holes before the seeds enter the seed pushing area, and returning the seeds to the seed filling area for carrying out next round of cyclic image collection;
if the three surfaces contain embryo surfaces, the seeds are separated from the suction holes of the seed metering disc under the action of the seed cleaning and scraping device, and are judged according to the classification result when the seed dropping opening drops; when the seeds judged to be haploid fall, the seed sorting air tap starts to sort the seeds into a haploid seed box, and the seeds judged to be diploid fall freely into a diploid seed box;
the corn haploid identification model building steps are as follows:
step 1, putting haploid and diploid corn seeds into an air suction type seed metering device, enabling the seeds to be adsorbed on suction holes of a seed metering disc by the single seeds, sequentially passing through an image acquisition area, arranging two plane mirrors at two sides of the image acquisition area, vertically arranging a color industrial camera on a front shell of the seed metering device, and simultaneously acquiring image information of different sides of one corn seed frame;
step 2, extracting an interested region, cutting an original image, and obtaining single-sided images of three single grains from one frame of image;
step 3, classifying and marking the extracted single-sided images of the seeds into three types of embryo surface haploids, embryo surface diploids and non-embryo surfaces, performing data enhancement treatment and size adjustment on the single-sided images, and classifying the images into a training set and a testing set according to the proportion;
step 4, constructing a convolutional neural network, inputting a training set into the convolutional neural network for training, extracting and classifying the characteristics of the training set, and obtaining a classification model after training;
and 5, using the test set as input of the convolutional neural network, and evaluating the performance of the classification model according to the result, so as to obtain the corn haploid identification model.
And selecting the region of interest, determining the coordinates and the size of the region of interest according to the plane mirror reflection imaging principle and the size of corn kernels.
The data enhancement method specifically comprises the steps of turning, rotating and adjusting brightness, wherein the data set is used for enhancing corn kernels, the size of an image is adjusted by a double interpolation method, and the image data set is subjected to image data set adjustment according to 7:3 to divide the training set and the test set.
The convolutional neural network comprises a convolutional layer, a batch normalization layer, a pooling layer and a full connection layer, and is classified by using a softmax classifier.
The original images of the collected haploids and diploid grains are at least 500, and a single-sided image of 1500 single grains is obtained by cutting;
each grain single-sided image is adjusted to be 100 x 100 in size by using a bicubic interpolation method;
the construction of the convolutional neural network in the step 4 comprises the following steps:
step 41, the input image is 100×100×3;
step 42, 4 convolutional layers (CONV), using 5*5 convolutional kernels, stride and padding being 1, the number of convolutional kernels being 32, 64, 128; the output structures of the convolution layers are 98 x 32, 47 x 64, 21 x 128 and 8 x 128 respectively;
the batch normalization layer is connected behind each convolution layer, the output size is unchanged, the batch normalization is to normalize the output of each layer, then the data characteristics before normalization are restored through the learned parameters in BN, and the batch normalization layer can accelerate the speed of network training and convergence;
4 pooling layers (POOL), using a maximum pooling operation, core size of 2 x 2;
3 full connection layers (FC) with characteristic numbers of 2048, 512 and 128 respectively, wherein a dropout layer is used in the full connection layers, so that overfitting is avoided;
step 43, selecting ReLu as an activation function;
step 44, the output layer uses a Softmax classifier to obtain three classifications;
step 45, using an RMSprop optimizer and a cross entropy loss function in the training process, wherein the initial learning rate is 0.0001, and the learning rate is attenuated to be 0.5 times of the original learning rate when the iteration times are increased by 100 times;
and 46, error back propagation, weight updating and storing an optimal model.
The invention also provides an automatic sorting system of the automatic sorting method of the corn haploids based on computer vision, and the technical scheme of the system comprises the following steps: the device comprises a feeding funnel, a color industrial camera, a haploid seed box, a diploid seed box, a fan, a seed sorting air tap, an air suction type seed metering device, a seed cleaning and scraping component and a non-acquisition information screening air tap, wherein the air suction type seed metering device is vertically arranged above a frame, the feeding funnel is arranged at one side of the air suction type seed metering device and is communicated with a seed filling area below the air suction type seed metering device, a seed metering disc in the air suction type seed metering device is arranged along a seed suction hole on the air suction type seed metering device, and the air suction chamber is connected with the fan to create a negative pressure environment; the seed sorting air tap and the non-acquisition information screening air tap are connected with an air compressor through independent valves, a driving motor fixed outside the front shell is connected with a seed metering disc through a transmission structure in a seed metering device, and a haploid seed box and a diploid seed box are arranged on the ground below the seed sorting air tap;
a seed charging area, a photoelectric sensor, an image acquisition area, a seed eliminating area without acquisition information and a seed dropping area with information are sequentially arranged along the rotation direction of a seed metering disc in the air suction type seed metering device, wherein the photoelectric sensor is arranged between a front shell and the seed metering disc and in front of the image acquisition area and is used for externally triggering a camera;
the position of the image acquisition area is provided with plane mirrors with two sides fixed on a plane mirror bracket, the color industrial camera is arranged at one side of the air suction seed metering device through a camera bracket, the lens of the color industrial camera is focused through an image acquisition port of the front shell and is opposite to the running track of the two plane mirrors and the suction hole, and the angles of the two plane mirrors enable the color industrial camera to acquire images of seeds on the suction hole in three directions;
the position of the no-acquisition information seed removing area is provided with a no-acquisition information screening air tap, the no-acquisition information screening air tap is opposite to the running track of the suction hole, and when no embryo part is acquired, seeds are sprayed back to the seed filling area for secondary acquisition;
the seed cleaning and scraping component is arranged at the position of the seed falling area with information seeds, the upper end of the seed cleaning and scraping component is clung to the seed metering disc, seeds sucked on the suction holes are scraped off the seed metering disc, and then the air suction type seed metering device is discharged from the seed falling port of the air suction type seed metering device; and a seed sorting air nozzle is arranged outside the seed falling opening, and when the seeds judged to be haploid fall, the seed sorting air nozzle starts to sort the seeds into a haploid seed box, and the seeds judged to be diploid fall freely into a diploid seed box.
The two plane mirrors are oppositely arranged at two sides of the suction hole, and the bottom edges are parallel.
The size of the plane mirror is at least 16 x 20mm.
The air nozzle without the collected information screening air nozzle is arranged outside the air suction type seed metering device, a screening air nozzle mounting hole is formed in the side wall of the air suction type seed metering device, and a nozzle without the collected information screening air nozzle extends into the air suction type seed metering device from the screening air nozzle mounting hole.
The invention has the beneficial effects that:
1. the air suction type seed metering device is used for carrying out single granulation on corn seeds, image acquisition is carried out, the corn seeds do not need to be manually placed, the sorting process is more automatic, and the air suction type seed metering device is more suitable for image information acquisition of corn seeds with various shapes such as round grains.
2. When the corn seed image is collected, the plane mirrors arranged on the two sides of the suction hole of the seed metering disc are utilized to collect three-side image information of the corn seed at the same position, the three-side image is used as input of the classification model, the property of the corn seed is comprehensively identified, the probability of collecting embryo information of the corn seed is improved, and the sorting efficiency is improved.
3. In the haploid sorting model, the acquired image corn weight is divided into three types, namely, embryo surface haploid and embryo surface diploid and non-embryo surface, if the three-surface image information does not contain embryo surface information, namely, the embryo surface information is non-embryo surface, seeds not acquired embryo surface information are screened out by utilizing a nozzle arranged in a seed metering disc, the seeds fall back to a seed filling area of the seed metering disc, are adsorbed to a seed metering disc suction hole again and are acquired in an image, and secondary manual feeding is not needed for the same seeds.
4. In the convolutional neural network model, a batch normalization layer is added after the convolutional layer to accelerate the training speed and improve the generalization capability of the network.
Drawings
Fig. 1 is a schematic diagram of an embodiment of an automated sorting system for corn haploids based on computer vision.
Fig. 2 is a schematic diagram of an internal structure of a seed metering device according to an embodiment of the present invention.
FIG. 3 is a schematic view of a mounting of a seed plate and a mirror in an embodiment of the present invention.
FIG. 4 is a flow chart of a sorting method according to an embodiment of the present invention.
Fig. 5 is a schematic diagram of a sorting method according to an embodiment of the present invention.
Wherein: 1-feeding hopper, 2-color industrial camera, 3-camera support, 4-air compressor, 5-haploid seed box, 6-diploid seed box, 7-fan, 8-seed sorting air cock, 9-air suction seed metering ware, 10-no information collection screening air cock, 901-front housing, 902-screening air cock mounting hole, 903-photoelectric sensor, 904-plane mirror, 905-seed suction air chamber, 906-driving motor, 907-seed cleaning and scraping component, 908-seed metering disc, 909-seed dropping mouth, 910-suction hole, 911-protrusion.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings.
In the embodiment of the present invention shown in fig. 1, the system includes: the seed sorting device comprises a feeding funnel 1, a color industrial camera 2, a haploid seed box 5, a diploid seed box 6, a fan 7, a seed sorting air tap 8, an air suction type seed metering device 9, a seed cleaning and scraping component 907 and a no-acquisition information screening air tap 10, wherein the air suction type seed metering device 9 is vertically arranged above a frame, the feeding funnel 1 is arranged on one side of the air suction type seed metering device 9 and is communicated with a seed filling area below the air suction type seed metering device 9, a seed metering disc 908 in the air suction type seed metering device 9 is arranged along the seed suction hole 910 on the air suction type seed metering device 9, and the air suction type seed metering device 9 is connected with the fan 7 through a seed suction air chamber 905 to create a negative pressure environment; the seed sorting air tap 8 and the non-collection information screening air tap 10 are connected with the air compressor 4 through independent valves, a driving motor 906 fixed outside the front shell 901 is connected with a seed metering disc 908 through a transmission structure in the seed metering device, and a haploid seed box 5 and a diploid seed box 6 are arranged on the ground below the seed sorting air tap 8;
a seed charging area, a photoelectric sensor, an image acquisition area, a seed eliminating area without acquisition information and a seed dropping area with information are sequentially arranged along the rotation direction of the seed metering disc 908 in the air suction type seed metering device 9, wherein the photoelectric sensor is arranged between the front shell and the seed metering disc and in front of the image acquisition area and is used for externally triggering a camera;
the position of the image acquisition area is provided with a plane mirror 904 with two sides fixed on a plane mirror bracket, the color industrial camera 2 is arranged on one side of the air suction seed sowing device 9 through a camera bracket 3, the focusing of a lens of the color industrial camera 2 is opposite to the running track of the two plane mirrors 904 and the suction hole 910 at the same time after passing through an image acquisition opening of the front shell 901, and the angles of the two plane mirrors 904 enable the color industrial camera 2 to acquire images of seeds on the suction hole 910 at three directions at the same time;
the position of the no-acquisition information seed removing area is provided with a no-acquisition information screening air tap 10, the no-acquisition information screening air tap 10 is opposite to the running track of the suction hole 910, and when no embryo is acquired, seeds are sprayed back to the seed filling area for secondary acquisition;
the seed cleaning and scraping component 907 is arranged at the position of the seed falling region with information seeds, the upper end of the seed cleaning and scraping component 907 is clung to the seed metering disc 908, seeds sucked on the suction hole 910 are scraped off the seed metering disc 908, and then the air suction type seed metering device 9 is discharged through the seed falling port 909 of the air suction type seed metering device 9; a seed sorting air tap 8 is arranged outside the seed falling opening 909, when the seeds judged to be haploid fall, the seed sorting air tap starts sorting the seeds into a haploid seed box 5, and the seeds judged to be diploid fall freely into a diploid seed box 6 (the opening time and the size of the seed sorting air tap 8 depend on the installation positions of the haploid seed box 5 and the diploid seed box 6);
in this embodiment, according to the size of the corn kernel, in order to acquire a more complete kernel image, two plane mirrors are relatively installed at two sides of the suction hole, the bottom edges are parallel, and are obliquely installed, and the size of the plane mirrors is at least 16 x 20mm; the camera can collect not only the actual face image of the grain face lens, but also the mirror image in the two plane mirrors, so that the possibility of collecting embryo face information is increased; the plane mirror is fixed in the plane mirror bracket, the plane mirror bracket is fixed in the air suction type seed sowing device 9, in order to make the corn seed side surface image collected by the plane mirror more complete, the seed sowing plate adopts a convex hole seed sowing plate with a convex 911 around the suction hole 910, and the mirror mounting position is provided with a groove;
in this embodiment, in order to prevent the start and stop of the air tap from interfering with the seed sucking effect in the air suction type seed metering device 9, the no-collection-information screening air tap 10 is arranged outside the air suction type seed metering device 9, the sidewall of the air suction type seed metering device 9 is provided with a screening air tap mounting hole 902, and the nozzle of the no-collection-information screening air tap 10 extends into the air suction type seed metering device 9 from the screening air tap mounting hole 902;
the sorting method of the system comprises the following steps:
1) Feeding haploid and diploid seeds to be detected into an air suction type seed metering device 9 through a feed hopper 1, driving a seed metering disc 908 to rotate by a driving motor 906, and adsorbing the seeds on a suction hole of the seed metering disc and rotating with the seeds under the action of disturbance of the seed metering disc and a negative pressure air chamber 905;
2) The kernels pass through a photoelectric sensor 903 to trigger a color industrial camera 2 to acquire a photo, and three-side images of the corn kernels passing through the area are acquired simultaneously by utilizing two plane mirrors 904 arranged in the image acquisition area;
3) Preprocessing the images through cutting and the like, and respectively inputting the three-side images into a corn haploid recognition model to obtain classification results;
4) If the classification results of the three-side images are all non-embryo surfaces, the embryo parts are not acquired, a non-acquisition information screening air tap arranged in the seed metering disc is started, seeds are separated from the suction holes before entering the seed pushing area, and the seeds return to the seed filling area for the next round of image cycle acquisition;
if the three surfaces contain embryo surfaces, the seeds are separated from the suction holes of the seed metering disc under the action of a seed cleaning and scraping device 907, and are judged according to the classification result when the seed falling port 909 falls; when the seeds judged to be haploid fall, the seed sorting air tap starts to sort the seeds into a haploid seed box 5, and the seeds judged to be diploid fall freely into a diploid seed box 6;
the system specifically uses a sorting method based on a constructed corn haploid identification model, and the construction steps of the corn haploid identification model are as follows:
step 1, putting haploid and diploid corn seeds into an air suction type seed metering device 9, enabling the seeds to be adsorbed on suction holes of a seed metering disc 908 by single seeds, sequentially passing through an image acquisition area, wherein two sides of the image acquisition area are provided with two plane mirrors 904, a color industrial camera 2 is vertically arranged on a front shell 901 of the seed metering device, and one frame of image can simultaneously acquire image information of 3 different sides of one corn seed;
step 2, extracting an interested region, cutting an original image, and obtaining single-sided images of three single grains from one frame of image; selecting a region of interest, determining the coordinates and the size of the region of interest according to the plane mirror reflection imaging principle and the size of corn kernels;
step 3, classifying and marking the extracted single-sided images of the seeds into three types of embryo surface haploids, embryo surface diploids and non-embryo surfaces, performing data enhancement treatment and size adjustment on the single-sided images, and classifying the images into a training set and a testing set according to the proportion; the data enhancement method specifically comprises the steps of turning, rotating and adjusting brightness, wherein the data set is used for enhancing corn kernels, the size of an image is adjusted by a double interpolation method, and the image data set is subjected to the following steps of: 3, dividing the training set and the testing set in proportion;
step 4, constructing a convolutional neural network, inputting a training set into the convolutional neural network for training, extracting and classifying the characteristics of the training set, and obtaining a classification model after training; the convolutional neural network comprises a convolutional layer, a batch normalization layer, a pooling layer and a full connection layer, and is classified by using a softmax classifier;
step 5, using the test set as input of a convolutional neural network, and evaluating the performance of the classification model according to the result so as to obtain a corn haploid identification model;
in order to acquire multi-surface images of corn kernels, sample kernels can be thrown into a seed metering device for image acquisition for two or more times, the acquired original images of haploids and diploid kernels are at least 500, and single-surface images of 1500 single-kernel kernels are obtained through cutting;
each grain single-sided image is adjusted to be 100 x 100 in size by using a bicubic interpolation method;
the construction of the convolutional neural network in the step 4 comprises the following steps:
step 41, the input image is 100×100×3;
step 42, 4 convolutional layers (CONV), using 5*5 convolutional kernels, stride and padding being 1, the number of convolutional kernels being 32, 64, 128; the output structures of the convolution layers are 98 x 32, 47 x 64, 21 x 128 and 8 x 128 respectively;
each convolution layer is followed by a batch normalization layer (batch Normalization layer, BN), the output size is unchanged, batch normalization is to normalize each layer of output, then the data characteristics before normalization are restored through the parameters learned in the BN, and the batch normalization layer can accelerate the speed of network training and convergence;
4 pooling layers (POOL), using a maximum pooling operation, core size of 2 x 2;
3 full connection layers (FC) with characteristic numbers of 2048, 512 and 128 respectively, wherein a dropout layer is used in the full connection layers, so that overfitting is avoided;
step 43, selecting ReLu as an activation function;
step 44, the output layer uses a Softmax classifier to obtain three classifications;
step 45, using an RMSprop optimizer and a cross entropy loss function in the training process, wherein the initial learning rate is 0.0001, and the learning rate is attenuated to be 0.5 times of the original learning rate when the iteration times are increased by 100 times;
and 46, error back propagation, weight updating and storing an optimal model.

Claims (10)

1. A computer vision-based automatic sorting method for corn haploids, which is characterized by comprising the following steps:
step A, feeding haploid and diploid seeds to be detected into an air-suction type seed metering device through a feed hopper, wherein the seeds are adsorbed on a suction hole of a seed metering disc by single seeds and rotate along with the suction hole;
b, triggering a color industrial camera to collect pictures by using a photoelectric sensor, and simultaneously collecting three-side images of corn kernels passing through an image collecting area by using two plane mirrors arranged in the image collecting area;
step C, preprocessing the images through cutting and the like, and respectively inputting the three-side images into a corn haploid recognition model to obtain classification results;
step D, if the classification results of the three-side images are non-embryo surfaces, judging that embryo parts are not collected, starting a non-collection information screening air tap arranged in the seed metering disc, separating seeds from the suction holes before the seeds enter the seed pushing area, and returning the seeds to the seed filling area for carrying out next round of cyclic image collection;
if the three surfaces contain embryo surfaces, the seeds are separated from the suction holes of the seed metering disc under the action of the seed cleaning and scraping device, and are judged according to the classification result when the seed dropping opening drops; when the seeds judged to be haploid fall, the seed sorting air tap starts to sort the seeds into a haploid seed box, and the seeds judged to be diploid fall freely into a diploid seed box;
the corn haploid identification model building steps are as follows:
step 1, putting haploid and diploid corn seeds into an air suction type seed metering device, enabling the seeds to be adsorbed on suction holes of a seed metering disc by the single seeds, sequentially passing through an image acquisition area, arranging two plane mirrors at two sides of the image acquisition area, vertically arranging a color industrial camera on a front shell of the seed metering device, and simultaneously acquiring image information of different sides of one corn seed frame;
step 2, extracting an interested region, cutting an original image, and obtaining single-sided images of three single grains from one frame of image;
step 3, classifying and marking the extracted single-sided images of the seeds into three types of embryo surface haploids, embryo surface diploids and non-embryo surfaces, performing data enhancement treatment and size adjustment on the single-sided images, and classifying the images into a training set and a testing set according to the proportion;
step 4, constructing a convolutional neural network, inputting a training set into the convolutional neural network for training, extracting and classifying the characteristics of the training set, and obtaining a classification model after training;
and 5, using the test set as input of the convolutional neural network, and evaluating the performance of the classification model according to the result, so as to obtain the corn haploid identification model.
2. The automatic sorting method of corn haploids based on computer vision according to claim 1, wherein the selection of the region of interest is determined according to the principle of plane mirror reflection imaging and the size determination of corn kernels, and the coordinates and the size of the region of interest are determined.
3. The automated sorting method of corn haploids based on computer vision as claimed in claim 1, characterized in that the method of data enhancement specifically comprises the adjustment of flip, rotation, brightness, for enhancing the data set of corn kernels, adjusting the image size by means of double interpolation, and the image data set is processed according to 7:3 to divide the training set and the test set.
4. The automated sorting method of corn haploids based on computer vision of claim 1, characterized in that the convolutional neural network comprises a convolutional layer, a batch normalization layer, a pooling layer, a fully connected layer, sorted using a softmax classifier.
5. The automatic sorting method of corn haploids based on computer vision as claimed in claim 1, characterized in that the original images of the collected haploids and diploid seeds are at least 500, and the single-sided images of 1500 single-grain seeds are obtained by cutting.
6. The automated sorting method of corn haploids based on computer vision of claim 1, characterized in that each grain single-sided image is sized to 100 x 100 using bicubic interpolation;
the construction of the convolutional neural network in the step 4 comprises the following steps:
step 41, the input image is 100×100×3;
step 42, 4 convolution layers, namely using convolution kernels of 5*5, wherein the steps and the filling are 1, and the number of the convolution kernels is 32, 64, 128 and 128; the output structures of the convolution layers are 98 x 32, 47 x 64, 21 x 128 and 8 x 128 respectively;
the batch normalization layer is connected behind each convolution layer, the output size is unchanged, the batch normalization is to normalize the output of each layer, then the data characteristics before normalization are restored through the learned parameters in BN, and the batch normalization layer can accelerate the speed of network training and convergence;
4 pooling layers, using maximum pooling operation, core size of 2 x 2;
3 full-connection layers, wherein the feature numbers are 2048, 512 and 128 respectively, and a dropout layer is used in the full-connection layers to avoid over-fitting;
step 43, selecting ReLu as an activation function;
step 44, the output layer uses a Softmax classifier to obtain three classifications;
step 45, using an RMSprop optimizer and a cross entropy loss function in the training process, wherein the initial learning rate is 0.0001, and the learning rate is attenuated to be 0.5 times of the original learning rate when the iteration times are increased by 100 times;
and 46, error back propagation, weight updating and storing an optimal model.
7. An automated sorting system for automated sorting of corn haploids based on computer vision as claimed in claim 1, characterized in that it comprises: the device comprises a feeding funnel (1), a color industrial camera (2), a haploid seed box (5), a diploid seed box (6), a fan (7), a seed sorting air tap (8), an air suction type seed metering device (9), a seed cleaning and scraping component (907) and a non-acquisition information screening air tap (10), wherein the air suction type seed metering device (9) is vertically arranged above a frame, the feeding funnel (1) is arranged on one side of the air suction type seed metering device (9) and is introduced into a seed filling area below the air suction type seed metering device (9), a seed sucking hole (910) on the air suction type seed metering device (9) is connected with the fan (7) through a seed sucking air chamber (905) along a seed metering disc (908) in the air suction type seed metering device (9), and a negative pressure environment is created; the seed sorting air tap (8) and the screening air tap (10) without collecting information are connected with the air compressor (4) through independent valves, a driving motor (906) fixed outside the front shell (901) is connected with the seed metering disc (908) through a transmission structure in the seed metering device, and the haploid seed box (5) and the diploid seed box (6) are arranged on the ground below the seed sorting air tap (8);
a seed charging area, a photoelectric sensor, an image acquisition area, an information-free seed removing area and an information seed dropping area are sequentially arranged along the rotation direction of a seed metering disc (908) in the air suction type seed metering device (9), wherein the photoelectric sensor is arranged between a front shell and the seed metering disc and in front of the image acquisition area and is used for externally triggering a camera;
the position of the image acquisition area is provided with a plane mirror (904) with two sides fixed on a plane mirror bracket, the color industrial camera (2) is arranged on one side of the air suction seed metering device (9) through a camera bracket (3), the focusing of a lens of the color industrial camera (2) is opposite to the running track of the two plane mirrors (904) and the suction hole (910) after passing through an image acquisition port of the front shell (901), and the angles of the two plane mirrors (904) enable the color industrial camera (2) to acquire images of seeds on the suction hole (910) in three directions at the same time;
the position of the no-acquisition information seed removing area is provided with a no-acquisition information screening air tap (10), the no-acquisition information screening air tap (10) is opposite to the running track of the suction hole (910), and when no embryo part is acquired, seeds are sprayed back to the seed filling area for secondary acquisition;
a seed cleaning and scraping component (907) is arranged at the position of the seed falling area with information seeds, the upper end of the seed cleaning and scraping component (907) is clung to the seed metering disc (908), seeds sucked on the suction hole (910) are scraped off the seed metering disc (908), and then the air suction type seed metering device (9) is discharged from the seed falling opening (909) of the air suction type seed metering device (9); a seed sorting air tap (8) is arranged outside the seed falling opening (909), when the seeds are judged to fall, the seed sorting air tap starts to sort the seeds into a haploid seed box (5), and the seeds are judged to fall freely into a diploid seed box (6).
8. The automatic sorting system for corn haploids based on computer vision as claimed in claim 7, characterized in that two plane mirrors are oppositely arranged at two sides of the suction hole, and the bottom edges are parallel.
9. The automated sorting system of computer vision-based corn haploids of claim 8, wherein the size of the mirror is at least 16 x 20mm.
10. The automatic sorting system for corn haploids based on computer vision according to claim 7, characterized in that the no-acquisition-information screening air tap (10) is arranged outside the air suction type seed metering device (9), a screening air tap mounting hole (902) is formed in the side wall of the air suction type seed metering device (9), and a nozzle of the no-acquisition-information screening air tap (10) extends into the air suction type seed metering device (9) from the screening air tap mounting hole (902).
CN202311606186.0A 2023-11-29 2023-11-29 Automatic sorting method and system for corn haploids based on computer vision Pending CN117649638A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311606186.0A CN117649638A (en) 2023-11-29 2023-11-29 Automatic sorting method and system for corn haploids based on computer vision

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311606186.0A CN117649638A (en) 2023-11-29 2023-11-29 Automatic sorting method and system for corn haploids based on computer vision

Publications (1)

Publication Number Publication Date
CN117649638A true CN117649638A (en) 2024-03-05

Family

ID=90045934

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311606186.0A Pending CN117649638A (en) 2023-11-29 2023-11-29 Automatic sorting method and system for corn haploids based on computer vision

Country Status (1)

Country Link
CN (1) CN117649638A (en)

Similar Documents

Publication Publication Date Title
CN106733701B (en) Potato multi-stage combination dysnusia detecting system and detection method
US8253054B2 (en) Apparatus and method for sorting plant material
JP7442462B2 (en) Seed sorting
Zhang et al. Separation of touching grain kernels in an image by ellipse fitting algorithm
CN103801520B (en) The automatic well-chosen stage division of shrimp and device
US20040141641A1 (en) Seed image analyzer
CN109876569B (en) Online detection device, system and method for purified gas index of bag-type dust remover
CN113820325B (en) Corn grain direct-harvest impurity-containing rate and breakage rate online detection system and method
CN109934297B (en) Rice seed test method based on deep learning convolutional neural network
CN1485616A (en) Fowl eggs quality non-destruction automatic detection grading apparatus and process
CN105806751A (en) On-line monitoring system and method for crushing of cereals in grain tank of combine harvester
Wan Kernel handling performance of an automatic grain quality inspection system
CN110575973B (en) Crop seed quality detection and screening system
CN107121436A (en) The Intelligent detecting method and identification device of a kind of silicon material quality
Aznan et al. Rice seed varieties identification based on extracted colour features using image processing and artificial neural network (ANN)
CN113228049A (en) Milk analyzer for classifying milk
Song et al. Design and experiment of a sorting system for haploid maize kernel
CN117649638A (en) Automatic sorting method and system for corn haploids based on computer vision
CN115900586B (en) Real-time monitoring device and real-time monitoring method for morphology of reclaimed sand particles
CN117115506A (en) Corn kernel crushing and identifying method and sorting device based on HSV threshold segmentation
CN116485766A (en) Grain imperfect grain detection and counting method based on improved YOLOX
US20230398576A1 (en) Method for identifying object to be sorted, sorting method, and sorting device
Gan et al. Design and testing of a machine-vision-based air-blow sorting platform for famous tea fresh leaves production
CN117745658A (en) Edible fungus quality detection and sorting method and system
WO2022182226A1 (en) Apparatus and method for automatically sorting, counting and marking oil palm seeds

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination