CN114037835A - Grain quality estimation method, device, equipment and storage medium - Google Patents
Grain quality estimation method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN114037835A CN114037835A CN202210025872.8A CN202210025872A CN114037835A CN 114037835 A CN114037835 A CN 114037835A CN 202210025872 A CN202210025872 A CN 202210025872A CN 114037835 A CN114037835 A CN 114037835A
- Authority
- CN
- China
- Prior art keywords
- grain
- quality
- seeds
- seed
- obtaining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 51
- 239000011159 matrix material Substances 0.000 claims description 34
- 238000006243 chemical reaction Methods 0.000 claims description 26
- 238000001514 detection method Methods 0.000 claims description 14
- 238000012545 processing Methods 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 6
- 230000000877 morphologic effect Effects 0.000 claims description 6
- 230000009466 transformation Effects 0.000 claims description 6
- 101100379079 Emericella variicolor andA gene Proteins 0.000 claims description 4
- 238000012216 screening Methods 0.000 abstract description 8
- 239000000463 material Substances 0.000 abstract description 4
- 230000008569 process Effects 0.000 abstract description 4
- 230000000875 corresponding effect Effects 0.000 description 18
- 238000005303 weighing Methods 0.000 description 12
- 230000000007 visual effect Effects 0.000 description 11
- 230000011218 segmentation Effects 0.000 description 7
- 238000013145 classification model Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 239000002245 particle Substances 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000000605 extraction Methods 0.000 description 3
- 238000005070 sampling Methods 0.000 description 3
- 238000012805 post-processing Methods 0.000 description 2
- 240000001931 Ludwigia octovalvis Species 0.000 description 1
- 206010039509 Scab Diseases 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 230000003628 erosive effect Effects 0.000 description 1
- 238000013467 fragmentation Methods 0.000 description 1
- 238000006062 fragmentation reaction Methods 0.000 description 1
- 230000003902 lesion Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 239000002994 raw material Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30242—Counting objects in image
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Geometry (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
Abstract
The invention discloses a method, a device, equipment and a storage medium for estimating the quality of grain seeds, wherein the method comprises the following steps: acquiring a first multi-view image of grain seeds to be estimated; obtaining a second multi-view image of each grain seed to be estimated according to the first multi-view image; obtaining a plurality of contour maps and category information of each grain seed according to the second multi-view image; and estimating the quality or the quality ratio of each type of grain seeds according to the plurality of profile maps and the type information of each grain seed. According to the method, the spatial position and the type of the grain seeds are detected, the quality of the single grain seeds is estimated according to the form information of the single grain seeds, the quality estimation of the single grain seeds is further realized, the estimation process can be realized without screening the grain seeds, and manpower and material resources are saved.
Description
Technical Field
The invention relates to the technical field of detection, in particular to a method, a device, equipment and a storage medium for estimating the quality of grain seeds.
Background
In a grain seed quality appearance detection task, for example, imperfect grain detection, the mass ratio of each category of imperfect grains needs to be calculated, and the mass of 1000 grain seeds needs to be measured for a thousand grains weight. The traditional detection method is to manually detect and select various perfect grains and imperfect grains, and then carry out weighing calculation on various grains for statistical operation. This statistical approach is labor intensive and inefficient.
At present, in the related art, imperfect particle detection is performed in a mode of combining machine identification and sorting to obtain an imperfect rate. However, in order to obtain the imperfect rate of the grain seeds, screening is not necessary for sample detection, and imperfect grains are removed.
Disclosure of Invention
The present invention is directed to solving, at least to some extent, one of the technical problems in the related art. Therefore, one purpose of the invention is to provide a grain seed quality estimation method, a grain seed quality estimation device, grain seed quality estimation equipment and a storage medium, wherein the grain seed quality estimation method, the grain seed quality estimation device, the grain seed quality estimation equipment and the storage medium can realize imperfect detection without screening.
Therefore, the embodiment of the invention provides a method for estimating the quality of grain seeds, which comprises the following steps:
acquiring a first multi-view image of grain seeds to be estimated;
obtaining a second multi-view image of each grain seed to be estimated according to the first multi-view image;
obtaining a plurality of contour maps and category information of each grain seed according to the second multi-view image;
and estimating the quality or the quality ratio of each type of grain seeds according to the plurality of profile maps and the type information of each grain seed.
Further, the obtaining a second multi-view image of each grain seed of the grain seeds to be estimated according to the first multi-view image includes:
acquiring the positions of marking points in each view angle image of the first multi-view angle image, and registering each view angle image of the first multi-view angle image according to the positions of the marking points;
performing target detection on the registered first multi-view images to obtain position information of each grain seed in the grain seeds to be estimated in each view image of the first multi-view images;
and cutting corresponding single grain seeds from each view angle image of the first multi-view-angle image according to the position information to obtain corresponding cutting small images, and taking all the cutting small images of each grain seed as second multi-view-angle images.
Further, the obtaining a plurality of contour maps of each grain seed grain according to the second multi-view image includes:
obtaining a plurality of mask images of each grain seed according to the second multi-view image;
and performing morphological processing on the mask images to obtain a plurality of contour maps of each grain seed.
Further, the estimating the quality or the quality ratio of each type of grain seeds according to the plurality of profile maps and the type information of each grain seed comprises:
calculating the areas of the plurality of contour maps of each grain seed;
obtaining the quality of each grain seed according to the area;
and obtaining the quality or the quality ratio of each type of grain seeds according to the quality of each grain seed and the corresponding type information.
Further, the obtaining the quality of each grain seed according to the area comprises:
obtaining the mass transformation coefficient R of each grain seed by the following formula:
wherein,Areais the area of a single grain seed,A max 、A h 、A m 、A l respectively a preset first area threshold value, a preset second area threshold value, a preset third area threshold value and a preset fourth area threshold value,R max 、R h 、R m 、R l are respectively andA max 、A h 、A m 、A l the method comprises the steps that a first mass conversion coefficient threshold value, a second mass conversion coefficient threshold value, a third mass conversion coefficient threshold value and a fourth mass conversion coefficient threshold value are preset correspondingly;
and obtaining the quality of each grain seed according to the mass conversion coefficient and the average quality of the normal grain seeds.
Further, the estimating the quality or the quality ratio of each type of grain seeds according to the plurality of profile maps and the type information of each grain seed comprises:
obtaining the total area of each type of grain seeds in the grain seeds to be estimated according to the plurality of profile maps and the type information of each grain seed;
and obtaining the quality or the mass ratio of each type of grain seeds according to the total area and a pre-established polynomial equation of the area and the quality.
Further, the grain kernel to be estimated is divided intomBatch sampling process, the method further comprising:
acquiring the quality of each batch of grain seeds;
obtaining the quantity of each type of grain seeds in each batch according to the type information of each grain seed;
constructing a relation matrix according to the quality of each batch of grain seeds and the quantity of each class of grain seeds;
and obtaining the quality or the quality ratio of each type of grain seeds according to the relation matrix.
Further, the relationship matrix is
，c 11 ,...,c 1n Represents the first batchnThe number of grain kernels of each of the categories,w 1 ,...,w m to representmThe obtaining of the quality or the quality ratio of each type of grain seeds according to the relationship matrix comprises the following steps:
judgment ofm=nWhether the result is true or not;
if so, constructing according to the relation matrixnFirst order equation, solving saidnObtaining the quality or the quality ratio of each type of grain seeds by the primary equation;
if not, adjusting the relation matrix according to the total mass or the number of the grain seeds of each batch to enable the number of rows and columns of the adjusted relation matrix to be equal, and obtaining the mass or the mass ratio of the grain seeds of each category according to the adjusted relation matrix.
In addition, in order to achieve the above object, an embodiment of the present invention further provides a device for estimating grain quality, including:
the device comprises an acquisition module, a storage module and a processing module, wherein the acquisition module is used for acquiring a first multi-view image of grain seeds to be estimated and obtaining a second multi-view image of each grain seed in the grain seeds to be estimated according to the first multi-view image;
the classification module is used for acquiring a plurality of contour maps and category information of each grain seed according to the second multi-view image;
and the estimation module is used for estimating the quality or the quality ratio of each type of grain seeds according to the plurality of profile maps and the type information of each grain seed.
In addition, in order to achieve the above object, an embodiment of the present invention further provides a grain seed quality estimation apparatus, including a memory and a processor; the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to implement the grain kernel quality estimation method.
In addition, in order to achieve the above object, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the method for estimating the quality of grain kernels is implemented.
According to the grain seed quality estimation method, the grain seed quality estimation device, the grain seed quality estimation equipment and the storage medium, the first multi-view image of the grain seeds to be estimated is obtained, the second multi-view image of each grain seed in the grain seeds to be estimated is obtained according to the first multi-view image, the plurality of contour maps and the category information of each grain seed are obtained according to the second multi-view image, and finally the quality or the quality ratio of each category of grain seeds is estimated according to the plurality of contour maps and the category information of each grain seed. The spatial position and the type of the grain seeds are detected, the quality of the single grain seeds is estimated according to the form information of the single grain seeds, the quality estimation of the single grain seeds is further realized, the estimation process can be realized without screening the grain seeds, and manpower and material resources are saved.
Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.
Drawings
Fig. 1 is a flowchart of a grain kernel quality estimation method according to a first embodiment of the present invention;
FIG. 2 is a schematic view of detecting the position information of grain seeds according to the present invention;
FIG. 3 is a schematic diagram illustrating mask image generation for grain kernels according to the present invention;
FIG. 4 is a flow chart of a method for estimating grain kernel quality according to a second embodiment of the present invention;
FIG. 5 is an overall flowchart of the grain kernel quality estimation method of the present invention;
FIG. 6 is a table of area and mass data for grain kernels of the present invention;
FIG. 7 is a table of area and mass data for worsted type grain kernels of the present invention;
FIG. 8 is a table of area and mass data for germinating grain kernels in accordance with the present invention;
FIG. 9 is a table of area and mass data for broken grain seeds according to the present invention;
FIG. 10 is a schematic illustration of the relationship between area and mass of four types of grain seeds in accordance with the present invention;
FIG. 11 is a flowchart illustrating a method for estimating grain seed quality according to a third embodiment of the present invention;
fig. 12 is a structural diagram of a grain seed quality estimation apparatus according to a fourth embodiment of the present invention;
fig. 13(a) - (d) are second multiview images of one example of the present invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are illustrative and intended to be illustrative of the invention and are not to be construed as limiting the invention.
The method, apparatus, device and storage medium for estimating grain seed quality according to the embodiments of the present invention are described with reference to fig. 1 to 13.
Fig. 1 is a flowchart of a grain kernel quality estimation method according to a first embodiment of the present invention. As shown in fig. 1, the method comprises the steps of:
and S10, acquiring a first multi-view image of the grain seeds to be estimated.
The grain seed number of the grain seeds to be estimated is larger than 1, and a first multi-view image of the grain seeds to be estimated can be acquired by using the multi-angle visual detection device. For example, the first multi-view image includes two view images, a front image and a back image, respectively.
And S20, obtaining a second multi-view image of each grain seed to be estimated according to the first multi-view image.
Specifically, the multi-view image of each grain seed can be acquired through the multi-view image of the grain seed to be estimated and the position information of each grain seed. For example, fig. 13(a), (b), (c), (d) show front and back images of four grain seeds, respectively.
And S30, obtaining a plurality of contour maps and category information of each grain seed according to the second multi-view image.
Specifically, the second multi-view image can be processed by utilizing a pre-obtained semantic segmentation model to obtain a plurality of contour maps of each grain seed; and processing the second multi-view image by using a pre-obtained classification model to obtain the class information of each grain seed.
The category information may include, but is not limited to, normal, lesion, worm damage, sprouting, and fragmentation, among others.
And S40, estimating the quality or the quality ratio of each type of grain seeds according to the plurality of profile maps and the type information of each grain seed.
Specifically, the quality of the corresponding grain seeds can be estimated according to the area and the category information of the plurality of profile maps of each grain seed, so that the quality estimation of each category of grain seeds can be realized.
In the embodiment, the profile map and the type of the grain seeds are detected, the area of the grain seeds is estimated according to the profile map, and the quality of the single grain seeds is estimated according to the area and the type of the single grain seeds, so that the quality estimation of the single grain seeds is realized. Compared with the related technology, the quality estimation of each type of grain seeds can be realized without screening the grain seeds.
Further, as shown in fig. 2, step S20 includes the following steps:
and S21, acquiring the positions of the marking points in the images of the various visual angles of the first multi-visual-angle image, and registering the images of the various visual angles of the first multi-visual-angle image according to the positions of the marking points.
Specifically, since the positions of the grain seeds in the multi-view images are different, in order to correspond the positions of each grain seed in each view image one by one, the multi-view images need to be registered. Taking two visual angle images as an example, firstly, when a first multi-visual angle image of grain seeds to be estimated is obtained, the grain seeds to be estimated can be placed on a transparent object placing plate, the object placing plate can be provided with mark points, and then the images of the grain seeds to be estimated can be collected through an upper image collecting device and a lower image collecting device of the object placing plate to obtain two visual angle images; then, positioning the mark points in the images of the two visual angles, and acquiring the pixel positions of the mark points in the images of the various visual angles; and finally, calculating a perspective transformation matrix of the two visual angle images according to the positions of the mark points, and further carrying out perspective transformation on one image so as to be registered and aligned with the reference image.
And S22, performing target detection on the registered first multi-view images to obtain the position information of each grain seed in the grain seeds to be estimated in each view image of the first multi-view images.
Specifically, the first multi-view image may be input into the target detection model, and the position information of all single grains in the grains to be estimated may be obtained through feature extraction and frame regression.
And S23, cutting the corresponding single grain seed from each view angle image of the first multi-view-angle image according to the position information to obtain corresponding cutting small images, and taking all the cutting small images of each grain seed as second multi-view-angle images.
Specifically, the position information of each grain seed detected in step S22box{x 0 ,y 0 ,x 1 ,y 1 }All the grain seeds can be cut from the first multi-view image, and the formula is as follows:
I
c
=I(x,y) x
1
>x>x
0
,y
1
>y>y
0
wherein,Iis a first multi-view image and is,I c in order to cut the small picture,（x，y）is the pixel coordinates in the first multi-view image,（x 0 ,y 0 ）、（x 1 ,y 1 ）respectively, two diagonal pixel coordinates of the cut thumbnail.
Further, as shown in fig. 3, step S30 includes the following steps:
and S31, obtaining a plurality of mask images of each grain seed according to the second multi-view image.
The first visual angle image is input into a semantic segmentation model, grain grains are distinguished from the background through feature extraction, and a mask image of a single grain is obtained, wherein the mask image can be regarded as the rough outline shape of the grain.
And S32, performing morphological processing on the mask images to obtain a plurality of contour maps of each grain seed.
It should be noted that, in order to obtain more accurate area information of a single grain seed and eliminate interference factors, referring to fig. 5, the embodiment performs morphological post-processing on an image of a mask image of each grain seed obtained after semantic segmentation, rejects a small connected domain according to the area of the connected domain, and retains the connected domain with the largest area, so as to obtain a more accurate grain seed profile map.
In an embodiment of the present invention, when the category information of each grain seed is obtained according to the second multi-view image, the second multi-view image may be input into a classification model obtained in advance to obtain a category to which each grain seed belongs.
In the embodiment, the multi-view images of all the grain seeds are cut from the whole multi-view image of the grain seeds to be estimated according to the position information of each grain seed, the profile map and the category of the grain seeds are acquired according to the multi-view image of each grain seed, the area of the grain seeds is obtained according to the profile map, and the quality of each grain seed is estimated according to the area and the category of each grain seed, so that the quality estimation of each grain seed can be realized without screening the grain seeds.
In an embodiment, referring to fig. 4, a second embodiment of the method for estimating the quality of grain kernels according to the present invention is provided according to the first embodiment, and step S40 includes the following steps:
and S41, calculating the areas of the plurality of contour maps of each grain seed.
Wherein, the profile of the grain kernel is assumed to beI m Width and height are respectivelyW、HThe contour map only contains one connected domain, and if the pixel value of the connected domain is 255, the area calculation formula of the connected domain is as follows:
wherein,I m (i,j)as a contour plot point(i,j)The pixel value of (c).
And S42, obtaining the quality of each grain seed according to the area.
Specifically, a piecewise function or an area-to-mass conversion curve may be used to calculate the mass of each grain seed particle.
And S43, obtaining the quality or the quality ratio of each type of grain seeds according to the quality of each grain seed and the corresponding type information.
And calculating the quality or the mass ratio of each type of grain seeds according to the quantity and the quality of the grain seeds in each type.
Further, in step S42, the obtaining of the quality of each grain seed according to the area of each grain seed may specifically include: obtaining the mass transformation coefficient R of each grain seed through the following formula; and obtaining the quality of each grain seed according to the mass conversion coefficient R and the average quality of the normal grain seeds.
Wherein,Areais the area of a single grain seed,A max 、A h 、A m 、A l respectively a predetermined first area thresholdA value, a second area threshold, a third area threshold, and a fourth area threshold,R max 、R h 、R m 、R l are respectively andA max 、A h 、A m 、A l corresponding preset first mass conversion coefficient threshold value, second mass conversion coefficient threshold value, third mass conversion coefficient threshold value and fourth mass conversion coefficient threshold valueR max 、R h 、R m 、R l Is an area ofA max 、A h 、A m 、A l The grain seed quality of the grain seed is equal to the average quality of normal grain seedsW base Relative coefficients of the standard;
it should be noted that the mass conversion coefficient is multiplied by the average mass of the normal grain seedsW base Obtaining the estimated quality of the grain seeds, wherein the formula is as follows:
W=R*W base 。
then, the total mass of each type of grain seeds is counted, and finally the mass or mass ratio of each type of grain seeds can be obtained.
In another embodiment, step S40 may include: obtaining the total area of each type of grain seeds in the grain seeds to be estimated according to the plurality of profile maps and the type information of each grain seed; and obtaining the quality or the quality ratio of each type of grain seeds according to the total area and a pre-established polynomial equation of the area and the quality of each type.
The process of establishing the polynomial equation of the area and the mass may include:
separating grains intomThe batch sampling processing is carried out on the raw materials,mis a positive integer greater than 1; obtaining a plurality of profile maps and category information of each grain seed in each batch; obtaining the average area of each type of grain seeds in a corresponding batch according to the plurality of profile maps and the type information of each grain seed,and obtaining the average mass corresponding to the average area obtained by weighing through a weighing device; and performing polynomial fitting according to the average area and the average mass of the m batches of grain seeds of each category to obtain a polynomial equation of the area and the mass corresponding to each category.
Specifically, for each grain seed category, the average area of a plurality of batches of grain seeds obtained by statistics is assumed to beX= [x1,x2,...,xn]Corresponding to an average mass ofY=[y1,y2,...,yn]Then a least squares method can be used to fit the polynomial to obtain the area to mass conversion relationship.
It should be noted that, when the above-mentioned mass or mass ratio is calculated, the area and mass of different types of grain seeds need to be counted in advance to obtain the data table of the area and mass of four types of grain seeds with scab, worm erosion, sprouting and crushing (see fig. 6, 7, 8 and 9) and the area and mass relationship diagram of four types of grain seeds (see fig. 10).
It can be seen that for each class, mass is positively correlated with the average area, with increasing mass as the area increases. The area span of the broken particles is the widest, and the mass change interval is the largest. The mass of the broken pieces will be lower at the same area. Based on the method, the quality of each grain seed can be estimated according to the profile obtained by the segmentation model and the class information obtained by the classification model.
In an embodiment, referring to fig. 11, a third embodiment of the grain seed quality estimation method according to the present invention is provided according to the first embodiment, where a weighing device is provided, the grain seed to be estimated is sampled and processed in m batches, and the method further includes the following steps:
and S111, obtaining the quality of each batch of grain seeds.
Specifically, the weighing device can be used to obtain the quality of each batch of grain seeds.
And S112, obtaining the quantity of each type of grain seeds in each batch according to the type information of each grain seed.
S113, constructing a relation matrix according to the quality of each batch of grain seeds and the quantity of each class of grain seeds.
Wherein the relationship matrix is
，c 11 ,...,c 1n Represents the first batchnThe number of grain kernels of each of the categories,w 1 ,...,w m to representmThe total mass of each batch of grain kernels in the batch.
And S114, obtaining the quality or the quality ratio of each type of grain seeds according to the relation matrix.
Further, step S114 may include the following steps:
judgment ofm=nWhether the result is true or not;
if so, constructing according to the relation matrixnFirst order equation of elements, solvingnObtaining the quality or the quality ratio of each type of grain seeds by the primary equation;
if not, adjusting the relation matrix according to the total mass or the number of the grain seeds of each batch to enable the number of rows and columns of the adjusted relation matrix to be equal, and obtaining the mass or the mass ratio of each type of grain seeds according to the adjusted relation matrix.
In particular, whenm=nTime, can be converted into solving onenThe equation of the prime order has unique equation solution for each category, and the quality of each single grain in each category can be obtained. When inm>nWhen the method is used, the matrixes need to be pruned and combined, namely samples with small total mass or total seed number are removed from the matrixes, or samples with small total mass or total seed number are combined until the requirements are metm=nThe conditions of (1). And the quality of each category is ensured to have a unique solution, so that the average quality of the single grains of each category can be accurately solved.
It should be noted that, under the condition of being provided with the weighing device, the total mass of each batch of grain seeds can be obtained, the mass of each type of grain seeds can be obtained through the relation matrix according to the total mass of each batch and the obtained quantity of the grain seeds, and various mass ratios can be obtained according to the mass of each type of grain seeds. If the weighing device is not arranged, the accuracy is low compared with the mass obtained under the condition of the weighing device because the obtained mass is obtained based on the preset average mass, but the accurate mass ratio of various types of grain seeds can be obtained, and based on the mass ratio, the mass obtained in the mode is not considered, and only the mass ratio obtained in the mode is considered, as shown in fig. 5.
In addition, referring to fig. 12, a fourth embodiment of the present invention further provides a device for estimating grain quality, the device comprising:
the acquisition module 10 is configured to acquire a first multi-view image of the grain seeds to be estimated, and obtain a second multi-view image of each grain seed in the grain seeds to be estimated according to the first multi-view image.
Specifically, the grain seed number of the grain seeds to be estimated is larger than 1, and a first multi-view image of the grain seeds to be estimated can be acquired by using the multi-angle visual detection device. The multi-view image of each grain seed can be acquired through the multi-view image of the grain seed to be estimated and the position information of each grain seed.
And the classification module 20 is configured to obtain a plurality of contour maps and category information of each grain seed according to the second multi-view image.
Specifically, the semantic segmentation model can be used for processing the second multi-view image to obtain a plurality of profile maps of each grain seed; and processing the second multi-view image by using the classification model to obtain the class information of each grain seed.
And the estimation module 30 is used for estimating the quality or the quality ratio of each type of grain seeds according to the plurality of profile maps and the type information of each grain seed.
Specifically, the quality of the grain seeds can be estimated according to the areas of the plurality of profile maps of each grain seed, so that the quality estimation of each category of grain seeds is realized.
In the embodiment, the profile map and the type of the grain seeds are detected, the area of the grain seeds is estimated according to the profile map, and the quality of the single grain seeds is estimated according to the area and the type of the single grain seeds, so that the quality estimation of the single grain seeds is realized. Compared with the related technology, the quality estimation of each type of grain seeds can be realized without screening the grain seeds.
Further, the obtaining module 10 may be specifically configured to: acquiring the positions of marking points in each view image of the first multi-view image, and registering each view image of the first multi-view image according to the positions of the marking points; performing target detection on the registered first multi-view images to obtain position information of each grain seed in the grain seeds to be estimated in each view image of the first multi-view images; and cutting the corresponding single grain seed from each view angle image of the first multi-view-angle image according to the position information to obtain a corresponding cutting small image, and taking all the cutting small images of each grain seed as a second multi-view-angle image.
Particularly, according to the position information of each grain seedbox{x 0 ,y 0 ,x 1 ,y 1 }All the grain seeds can be cut from the first multi-view image, and the formula is as follows:
I
c
=I(x,y) x
1
>x>x
0
,y
1
>y>y
0
wherein,Iis a first multi-view image and is,I c in order to cut the small picture,（x，y）is the pixel coordinates in the first multi-view image,（x 0 ,y 0 ）、（x 1 ,y 1 ）respectively, two diagonal pixel coordinates of the cut thumbnail.
Further, the classification module 20 is specifically operable to: obtaining a plurality of mask images of each grain seed according to the second multi-view image; performing morphological processing on the mask images to obtain a plurality of contour maps of each grain seed; and inputting the second multi-view image into a pre-obtained classification model to obtain the category of each grain seed.
The first visual angle image is input into a semantic segmentation model, grain grains are distinguished from the background through feature extraction, and a mask image of a single grain is obtained, wherein the mask image can be regarded as the rough outline shape of the grain.
It should be noted that, in order to obtain more accurate area information of a single grain seed and eliminate interference factors, the embodiment performs morphological post-processing on an image obtained by performing semantic segmentation on a mask image of each grain seed, rejects a small connected domain according to the area of the connected domain, and retains the connected domain with the largest area, so as to obtain a more accurate grain seed profile map.
In the embodiment, the multi-view images of all the grain seeds are cut from the whole multi-view image of the grain seeds to be estimated according to the position information of each grain seed, the profile map and the category of the grain seeds are acquired according to the multi-view image of each grain seed, the area of the grain seeds is obtained according to the profile map, and the quality of each grain seed is estimated according to the area and the category of each grain seed, so that the quality estimation of each grain seed can be realized without screening the grain seeds.
Further, the estimation module 30 is specifically operable to: calculating the areas of the plurality of contour maps of each grain seed, and obtaining the quality of each grain seed according to the areas; and obtaining the quality or the quality ratio of each type of grain seeds according to the quality of each grain seed and the corresponding type information.
Wherein, the profile of the grain kernel is assumed to beI m Width and height are respectivelyW、HThe contour map only contains one connected domain, and if the pixel value of the connected domain is 255, the area calculation formula of the connected domain is as follows:
wherein,I m (i,j)as a contour plot point(i,j)The pixel value of (c).
Specifically, a piecewise function or an area-to-mass conversion curve may be used to calculate the mass of each grain seed particle.
As an example, the mass conversion coefficient R of each grain seed can be obtained by the following formula, and the mass of each grain seed can be obtained according to the mass conversion coefficient and the average mass of normal grain seeds.
Wherein,Areais the area of a single grain seed,A max 、A h 、A m 、A l respectively a preset first area threshold value, a preset second area threshold value, a preset third area threshold value and a preset fourth area threshold value,R max 、R h 、R m 、R l are respectively andA max 、A h 、A m 、A l the method comprises the steps that a first mass conversion coefficient threshold value, a second mass conversion coefficient threshold value, a third mass conversion coefficient threshold value and a fourth mass conversion coefficient threshold value are preset correspondingly;
after obtaining the mass transformation coefficient, multiplying by the average massW base Obtaining the estimated quality of the grain seeds, wherein the formula is as follows:
W=R*W base 。
in this embodiment, the estimation module 30 is further specifically configured to: obtaining the total area of each type of grain seeds in the grain seeds to be estimated according to the plurality of profile maps and the type information of each grain seed; and obtaining the quality or the quality ratio of each type of grain seeds according to the total area and a pre-established polynomial equation of the area and the quality of each type.
Specifically, when a polynomial equation of the area and the mass is established, batch sampling treatment can be carried out on the grain seeds to obtain the average area of each type of grain seeds in multiple batches of grain seedsX=[x1,x2,...,xn]And the average mass corresponding thereto isY=[y1,y2,...,yn]Then a least squares method can be used to fit the polynomial to obtain the area to mass conversion relationship.
In an embodiment, in the case that a weighing device is provided, the obtaining module 10 may further obtain the mass of each batch of grain seeds. The estimation module 30 is further specifically configured to obtain the number of each batch of each type of grain seeds according to the type information of each grain seed, and construct a relationship matrix according to the quality of each batch of grain seeds and the number of each batch of each type of grain seeds; and obtaining the quality or the quality ratio of each type of grain seeds according to the relation matrix.
Wherein the relationship matrix is
，c 11 ,...,c 1n Represents the first batchnThe number of grain kernels of each of the categories,w 1 ,...,w m to representmThe total mass of each batch of grain kernels in the batch.
Further, obtaining the quality or the quality ratio of each type of grain seeds according to the relationship matrix may include:
judgment ofm=nWhether the result is true or not;
if so, constructing according to the relation matrixnFirst order equation of elements, solvingnObtaining the quality or the quality ratio of each type of grain seeds by the primary equation;
if not, adjusting the relation matrix according to the total mass or the number of the grain seeds of each batch to enable the number of rows and columns of the adjusted relation matrix to be equal, and obtaining the mass or the mass ratio of each type of grain seeds according to the adjusted relation matrix.
In particular, whenm=nTime, can be converted into solving onenThe equation of the prime order has unique equation solution for each category, and the quality of each single grain in each category can be obtained. When inm>nWhen the method is used, the matrixes need to be pruned and combined, namely samples with small total mass or total seed number are removed from the matrixes, or samples with small total mass or total seed number are combined until the requirements are metm=nThe conditions of (1). And the quality of each category is ensured to have a unique solution, so that the average quality of the single grains of each category can be accurately solved.
It should be noted that, under the condition of being provided with the weighing device, the total mass of each batch of grain seeds can be obtained, the mass of each type of grain seeds can be obtained through the relation matrix according to the total mass of each batch and the obtained quantity of the grain seeds, and various mass ratios can be obtained according to the mass of each type of grain seeds. If the weighing device is not arranged, the accuracy is low compared with the mass obtained under the condition of the weighing device because the obtained mass is obtained based on the preset average mass, but the accurate mass ratio of various types of grain seeds can be obtained, and based on the mass ratio, the mass obtained in the mode is not considered, and only the mass ratio obtained in the mode is considered, as shown in fig. 5.
In addition, the fifth embodiment of the present invention further provides a grain seed quality estimation device, which includes a memory and a processor; the processor runs the program corresponding to the executable program code by reading the executable program code stored in the memory, so as to realize the grain kernel quality estimation method.
In addition, a sixth embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the method for estimating the quality of grain kernels is implemented.
It should be noted that the logic and/or steps represented in the flowcharts or otherwise described herein, such as an ordered listing of executable instructions that can be considered to implement logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
It should be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
In the description of the present invention, it is to be understood that the terms "central," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," "circumferential," and the like are used in the orientations and positional relationships indicated in the drawings for convenience in describing the invention and to simplify the description, and are not intended to indicate or imply that the referenced device or element must have a particular orientation, be constructed and operated in a particular orientation, and are not to be considered limiting of the invention.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "a plurality" means at least two, e.g., two, three, etc., unless specifically limited otherwise.
In the present invention, unless otherwise expressly stated or limited, the terms "mounted," "connected," "secured," and the like are to be construed broadly and can, for example, be fixedly connected, detachably connected, or integrally formed; can be mechanically or electrically connected; they may be directly connected or indirectly connected through intervening media, or they may be connected internally or in any other suitable relationship, unless expressly stated otherwise. The specific meanings of the above terms in the present invention can be understood by those skilled in the art according to specific situations.
In the present invention, unless otherwise expressly stated or limited, the first feature "on" or "under" the second feature may be directly contacting the first and second features or indirectly contacting the first and second features through an intermediate. Also, a first feature "on," "over," and "above" a second feature may be directly or diagonally above the second feature, or may simply indicate that the first feature is at a higher level than the second feature. A first feature being "under," "below," and "beneath" a second feature may be directly under or obliquely under the first feature, or may simply mean that the first feature is at a lesser elevation than the second feature.
Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and that variations, modifications, substitutions and alterations can be made to the above embodiments by those of ordinary skill in the art within the scope of the present invention.
Claims (11)
1. A method of estimating grain seed quality, the method comprising:
acquiring a first multi-view image of grain seeds to be estimated;
obtaining a second multi-view image of each grain seed to be estimated according to the first multi-view image;
obtaining a plurality of contour maps and category information of each grain seed according to the second multi-view image;
and estimating the quality or the quality ratio of each type of grain seeds according to the plurality of profile maps and the type information of each grain seed.
2. The method of claim 1, wherein obtaining a second multi-view image of each grain seed to be estimated according to the first multi-view image comprises:
acquiring the positions of marking points in each view angle image of the first multi-view angle image, and registering each view angle image of the first multi-view angle image according to the positions of the marking points;
performing target detection on the registered first multi-view images to obtain position information of each grain seed in the grain seeds to be estimated in each view image of the first multi-view images;
and cutting corresponding single grain seeds from each view angle image of the first multi-view-angle image according to the position information to obtain corresponding cutting small images, and taking all the cutting small images of each grain seed as second multi-view-angle images.
3. The method of claim 1, wherein the obtaining the plurality of profile maps of each grain seed according to the second multi-view image comprises:
obtaining a plurality of mask images of each grain seed according to the second multi-view image;
and carrying out morphological processing on the mask image to obtain a plurality of contour maps of each grain seed.
4. The method of claim 3, wherein the estimating the quality or the quality ratio of each type of grain seeds according to the plurality of profile maps and the type information of each grain seed comprises:
calculating the areas of the plurality of contour maps of each grain seed;
obtaining the quality of each grain seed according to the area;
and obtaining the quality or the quality ratio of each type of grain seeds according to the quality of each grain seed and the corresponding type information.
5. The method of claim 4, wherein the obtaining the quality of each grain seed according to the area comprises:
obtaining the mass transformation coefficient of each grain seed by the following formulaR：
Wherein,Areais the area of a single grain seed,A max 、A h 、A m 、A l respectively a preset first area threshold value, a preset second area threshold value, a preset third area threshold value and a preset fourth area threshold value,R max 、R h 、R m 、R l are respectively andA max 、A h 、A m 、A l the method comprises the steps that a first mass conversion coefficient threshold value, a second mass conversion coefficient threshold value, a third mass conversion coefficient threshold value and a fourth mass conversion coefficient threshold value are preset correspondingly;
and obtaining the quality of each grain seed according to the mass conversion coefficient and the average quality of the normal grain seeds.
6. The method of claim 3, wherein the estimating the quality or the quality ratio of each type of grain seeds according to the plurality of profile maps and the type information of each grain seed comprises:
obtaining the total area of each type of grain seeds in the grain seeds to be estimated according to the plurality of profile maps and the type information of each grain seed;
and obtaining the quality or the quality ratio of each type of grain seeds according to the total area and a pre-established polynomial equation of the area and the quality of each type.
7. The method of claim 1, wherein the grain kernels to be estimated are sampled in m batches, and the method further comprises:
acquiring the quality of each batch of grain seeds;
obtaining the quantity of each type of grain seeds in each batch according to the type information of each grain seed;
constructing a relation matrix according to the quality of each batch of grain seeds and the quantity of each class of grain seeds;
and obtaining the quality or the quality ratio of each type of grain seeds according to the relation matrix.
8. The method of estimating grain seed quality of claim 7, wherein the relationship matrix is
，c 11 ,...,c 1n Represents the first batchnThe number of grain kernels of each of the categories,w 1 ,...,w m to representmThe obtaining of the quality or the quality ratio of each type of grain seeds according to the relationship matrix comprises the following steps:
judgment ofm=nWhether the result is true or not;
if so, constructing according to the relation matrixnFirst order equation, solving saidnObtaining the quality or the quality ratio of each type of grain seeds by the primary equation;
if not, adjusting the relation matrix according to the total mass or the number of the grain seeds of each batch to enable the number of rows and columns of the adjusted relation matrix to be equal, and obtaining the mass or the mass ratio of the grain seeds of each category according to the adjusted relation matrix.
9. A grain seed quality estimation device, the device comprising:
the device comprises an acquisition module, a storage module and a processing module, wherein the acquisition module is used for acquiring a first multi-view image of grain seeds to be estimated and obtaining a second multi-view image of each grain seed in the grain seeds to be estimated according to the first multi-view image;
the classification module is used for acquiring a plurality of contour maps and category information of each grain seed according to the second multi-view image;
and the estimation module is used for estimating the quality or the quality ratio of each type of grain seeds according to the plurality of profile maps and the type information of each grain seed.
10. A grain seed quality estimation apparatus, comprising a memory, a processor; wherein the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to implement the grain kernel quality estimation method according to any one of claims 1 to 8.
11. A computer-readable storage medium, on which a computer program is stored, wherein the computer program, when executed by a processor, implements the method of estimating quality of grain kernels according to any of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210025872.8A CN114037835B (en) | 2022-01-11 | 2022-01-11 | Grain quality estimation method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210025872.8A CN114037835B (en) | 2022-01-11 | 2022-01-11 | Grain quality estimation method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114037835A true CN114037835A (en) | 2022-02-11 |
| CN114037835B CN114037835B (en) | 2022-04-22 |
Family
ID=80141627
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210025872.8A Active CN114037835B (en) | 2022-01-11 | 2022-01-11 | Grain quality estimation method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114037835B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114066967A (en) * | 2022-01-11 | 2022-02-18 | 安徽高哲信息技术有限公司 | Training method and device of volume estimation model and volume estimation method and device |
| CN116893127A (en) * | 2023-09-11 | 2023-10-17 | 中储粮成都储藏研究院有限公司 | Grain appearance quality index detector |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108256134A (en) * | 2016-12-29 | 2018-07-06 | 航天信息股份有限公司 | A kind of method and device for selecting grain location mode |
| US20190073759A1 (en) * | 2017-09-05 | 2019-03-07 | Vibe Imaging Analytics Ltd. | System and method for automated grain inspection and analysis of results |
| CN112560749A (en) * | 2020-12-23 | 2021-03-26 | 安徽高哲信息技术有限公司 | Crop analysis system and analysis method |
| CN112560748A (en) * | 2020-12-23 | 2021-03-26 | 安徽高哲信息技术有限公司 | Crop shape analysis subsystem and method |
| CN112580540A (en) * | 2020-12-23 | 2021-03-30 | 安徽高哲信息技术有限公司 | Artificial intelligent crop processing system and method |
| CN113109240A (en) * | 2021-04-08 | 2021-07-13 | 国家粮食和物资储备局标准质量中心 | Method and system for determining imperfect grains of grains implemented by computer |
-
2022
- 2022-01-11 CN CN202210025872.8A patent/CN114037835B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108256134A (en) * | 2016-12-29 | 2018-07-06 | 航天信息股份有限公司 | A kind of method and device for selecting grain location mode |
| US20190073759A1 (en) * | 2017-09-05 | 2019-03-07 | Vibe Imaging Analytics Ltd. | System and method for automated grain inspection and analysis of results |
| CN112560749A (en) * | 2020-12-23 | 2021-03-26 | 安徽高哲信息技术有限公司 | Crop analysis system and analysis method |
| CN112560748A (en) * | 2020-12-23 | 2021-03-26 | 安徽高哲信息技术有限公司 | Crop shape analysis subsystem and method |
| CN112580540A (en) * | 2020-12-23 | 2021-03-30 | 安徽高哲信息技术有限公司 | Artificial intelligent crop processing system and method |
| CN113109240A (en) * | 2021-04-08 | 2021-07-13 | 国家粮食和物资储备局标准质量中心 | Method and system for determining imperfect grains of grains implemented by computer |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114066967A (en) * | 2022-01-11 | 2022-02-18 | 安徽高哲信息技术有限公司 | Training method and device of volume estimation model and volume estimation method and device |
| CN116893127A (en) * | 2023-09-11 | 2023-10-17 | 中储粮成都储藏研究院有限公司 | Grain appearance quality index detector |
| CN116893127B (en) * | 2023-09-11 | 2023-12-08 | 中储粮成都储藏研究院有限公司 | Grain appearance quality index detector |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114037835B (en) | 2022-04-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114037835B (en) | Grain quality estimation method, device, equipment and storage medium | |
| EP2191439B1 (en) | Method for digital image analysis of maize | |
| CN116664559B (en) | Machine vision-based memory bank damage rapid detection method | |
| CN115661021A (en) | Defect detection method, device, equipment and storage medium | |
| CN114529613A (en) | Method for extracting characteristic point high-precision coordinates of circular array calibration plate | |
| CN116468687A (en) | Scratch defect detection method and device, storage medium and electronic equipment | |
| CN110715918B (en) | Single-kernel corn starch content Raman hyperspectral classification method | |
| CN116523898A (en) | Tobacco phenotype character extraction method based on three-dimensional point cloud | |
| CN114937038A (en) | Remote sensing image quality evaluation method oriented to usability | |
| CN112361977A (en) | Linear distance measuring method based on weight distribution | |
| Sako et al. | Computer image analysis and classification of giant ragweed seeds | |
| CN114638958A (en) | Multi-feature fusion ternary positive electrode material roughness extraction method and device | |
| CN110622651A (en) | Method for detecting quality of sweet corn | |
| CN112927287B (en) | Phenotype data analysis method of target object, storage medium and terminal | |
| CN116434066B (en) | Deep learning-based soybean pod seed test method, system and device | |
| CN106898010A (en) | Particle copies the method and device planted | |
| CN115311505B (en) | Silkworm cocoon classification method and purchase system based on cloud service big data platform | |
| CN110580495A (en) | automatic analysis method for leaf area and leaf surface anthracnose lesion number of pear | |
| CN108875825B (en) | Granary pest detection method based on image blocking | |
| CN114067105B (en) | Grain density estimation method, storage medium, and grain density estimation apparatus | |
| CN116245879A (en) | Glass substrate flatness evaluation method and system | |
| CN109084721B (en) | Method and apparatus for determining a topographical parameter of a target structure in a semiconductor device | |
| CN116152220A (en) | Seed counting and size measuring method based on machine vision | |
| JP5644655B2 (en) | Edible fruit inspection method, inspection apparatus and program | |
| Sainis et al. | Image analysis of wheat grains developed in different environments and its implications for identification |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |