CN112229836B - Detection device and detection method for rice filled grains and rice empty grains - Google Patents
Detection device and detection method for rice filled grains and rice empty grains Download PDFInfo
- Publication number
- CN112229836B CN112229836B CN202010970495.6A CN202010970495A CN112229836B CN 112229836 B CN112229836 B CN 112229836B CN 202010970495 A CN202010970495 A CN 202010970495A CN 112229836 B CN112229836 B CN 112229836B
- Authority
- CN
- China
- Prior art keywords
- image
- empty
- camera
- grains
- rice
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 235000013339 cereals Nutrition 0.000 title claims abstract description 130
- 241000209094 Oryza Species 0.000 title claims abstract description 75
- 235000007164 Oryza sativa Nutrition 0.000 title claims abstract description 74
- 235000009566 rice Nutrition 0.000 title claims abstract description 74
- 238000001514 detection method Methods 0.000 title claims abstract description 17
- 239000002245 particle Substances 0.000 claims description 86
- 230000006835 compression Effects 0.000 claims description 40
- 238000007906 compression Methods 0.000 claims description 40
- 238000001914 filtration Methods 0.000 claims description 25
- 238000000034 method Methods 0.000 claims description 24
- 239000007787 solid Substances 0.000 claims description 21
- 230000007246 mechanism Effects 0.000 claims description 15
- 238000003825 pressing Methods 0.000 claims description 12
- 239000012535 impurity Substances 0.000 claims description 11
- 238000005286 illumination Methods 0.000 claims description 7
- 230000000295 complement effect Effects 0.000 claims description 6
- 230000000712 assembly Effects 0.000 claims description 4
- 238000000429 assembly Methods 0.000 claims description 4
- 210000005069 ears Anatomy 0.000 claims description 4
- 230000002452 interceptive effect Effects 0.000 claims 1
- 238000005516 engineering process Methods 0.000 abstract description 2
- 230000001360 synchronised effect Effects 0.000 abstract description 2
- 238000004590 computer program Methods 0.000 description 6
- 238000000926 separation method Methods 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 4
- XAGFODPZIPBFFR-UHFFFAOYSA-N aluminium Chemical compound [Al] XAGFODPZIPBFFR-UHFFFAOYSA-N 0.000 description 2
- 229910052782 aluminium Inorganic materials 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000005484 gravity Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 2
- 241000196324 Embryophyta Species 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000009395 breeding Methods 0.000 description 1
- 230000001488 breeding effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000003749 cleanliness Effects 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000002156 mixing Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 239000011800 void material Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration using local operators
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/90—Dynamic range modification of images or parts thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/187—Segmentation; Edge detection involving region growing; involving region merging; involving connected component labelling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N2021/8466—Investigation of vegetal material, e.g. leaves, plants, fruits
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30242—Counting objects in image
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Analytical Chemistry (AREA)
- Life Sciences & Earth Sciences (AREA)
- Chemical & Material Sciences (AREA)
- Health & Medical Sciences (AREA)
- Biochemistry (AREA)
- General Health & Medical Sciences (AREA)
- Immunology (AREA)
- Pathology (AREA)
- Geometry (AREA)
- Quality & Reliability (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
Abstract
The invention discloses a detection device and a detection method for rice filled grains and rice empty grains, wherein the device comprises a base, a backlight light source part, a camera mounting bracket, a camera and a computer; the backlight light source component is arranged on the base in the middle; the camera mounting bracket is arranged on the base; the camera is adjustably arranged on the camera mounting bracket and is connected with the computer, and the lens of the camera is right opposite to the center of the light emitting surface of the backlight light source part. The invention utilizes the machine vision technology to realize the rapid synchronous detection and counting of the real grains and the empty grains under the condition that the real grain group, the empty grain group and the branch group coexist in the same image, and the real grain number, the empty grain number and the maturing rate are automatically obtained.
Description
Technical Field
The invention relates to a detection device and a detection method for rice filled grains and rice empty grains, and belongs to the field of machine vision.
Background
The rice is an important grain crop in China, the improvement of the rice yield is an important target of rice breeding, the rice setting rate is closely related to the rice yield and is an important rice phenotypic parameter, the setting rate refers to the ratio of the number of solid grains to the number of total grains, the number of total grains is equal to the sum of the number of solid grains and the number of empty grains, and the accurate measurement of the setting rate provides a scientific basis for screening high-yield and high-quality new rice varieties.
At present, the following two methods are mainly used for detecting rice full grains and rice empty grains:
1) and (4) a manual counting method. The traditional counting method of the rice solid grains and the rice empty grains is that a worker firstly threshes rice ears, then separates the solid grains and the rice empty grains obtained by threshing with an air separation device or a water separation device, and finally, the solid grains and the rice empty grains are manually counted respectively. This method has the following disadvantages: firstly, the whole process consumes a large amount of labor force, and the cost of human resources is high; secondly, the air separation device can not stably separate the solid particles from the empty particles, so that the phenomenon of mixing the solid particles with the empty particles after air separation still exists, which brings difficulty to subsequent accurate counting; thirdly, the operation of separating solid particles from empty particles by using a water separation device is complicated, a large amount of time is consumed, and the counting efficiency is low; fourth, counting errors are easily introduced due to fatigue and inattention of the operator.
2) An automatic counting method of a rice grain counter. The staff can count the real grain automatically with the help of the rice grain counting instrument. This method has the following disadvantages: firstly, the cleanliness of the solid particle group to be counted is required to be high, branch impurities cannot be mixed in, and otherwise, obvious counting errors can be caused; secondly, the rice grain counting instrument can not count empty grains.
Disclosure of Invention
In view of the above, the present invention provides a device and a method for detecting rice full grains and empty grains, which utilize machine vision technology to realize rapid and synchronous detection and counting of the full grains and the empty grains under the condition that a full grain group, an empty grain group and a branch group coexist in the same image, and automatically obtain the number of the full grains, the number of the empty grains and the maturing rate.
The first purpose of the invention is to provide a device for detecting rice filled grains and rice empty grains.
The second purpose of the invention is to provide a method for detecting rice filled grains and rice empty grains.
It is a third object of the invention to provide a computer apparatus.
It is a fourth object of the present invention to provide a storage medium.
The first purpose of the invention can be achieved by adopting the following technical scheme:
a detection device for rice filled grains and rice empty grains comprises a base, a backlight light source component, a camera mounting bracket, a camera and a computer;
the backlight light source component is arranged on the base in the middle;
the camera mounting bracket is arranged on the base;
the camera is adjustably arranged on the camera mounting bracket and is connected with the computer, and the lens of the camera is right opposite to the center of the light emitting surface of the backlight light source part.
Further, the light emitting surface of the backlight light source part can place all particles generated by threshing single rice ears without overlapping;
the light emitting surface of the backlight light source part is parallel to the horizontal plane, and the backlight light source part is fixedly connected with the base.
Furthermore, the camera mounting bracket comprises a bracket vertical rod, a bracket longitudinal rod and a camera fixing mechanism;
the support vertical rod is fixedly connected with the base;
the vertical rod of the bracket is fixedly connected with the vertical rod of the bracket;
the camera fixing mechanism is fixedly connected with the bracket longitudinal rod;
the camera is adjustably fixed to the camera fixing mechanism.
Furthermore, the camera fixing mechanism comprises a compression screw supporting seat and an adjusting and fixing component;
the compression screw supporting seat is fixedly connected with the vertical rod of the bracket;
the two adjusting and fixing assemblies are oppositely arranged on two sides of the compression screw supporting seat;
each adjusting and fixing assembly comprises a camera compression screw, a rubber block and a knob, the camera compression screw is respectively in threaded connection with one side of a compression screw supporting seat, and two ends of the camera compression screw are respectively positioned on the inner side and the outer side of the compression screw supporting seat; the rubber block is fixedly connected with one end of the camera compression screw rod, which is positioned at the inner side of the compression screw rod supporting seat; the knob is fixedly connected with one end of the camera compression screw rod, which is positioned outside the compression screw rod supporting seat;
for the two adjusting and fixing components, the camera compression screw is continuously screwed in through the knob until the rubber blocks are extruded to the two side faces of the camera, so that the camera is fixed.
The second purpose of the invention can be achieved by adopting the following technical scheme:
a method for detecting rice filled grains and rice empty grains, which comprises the following steps:
acquiring a first image including a complete boundary line, a real particle group, an empty particle group and a branch group of a light emitting surface of the backlight light source part;
carrying out graying, filtering and binarization processing on the first image in sequence to obtain a second image;
performing complement operation on the second image to obtain a third image;
identifying a complete boundary line of a light emitting surface of the backlight light source part in the third image, and filtering interference impurities outside the boundary line to obtain a fourth image;
identifying and filtering all branches in the fourth image to obtain a fifth image;
and identifying the empty particles in the fifth image according to the notch characteristics formed by the unclosed empty particle glumes, and respectively calculating the number of the empty particles, the number of the actual particles and the seed setting rate.
Further, the identifying a complete boundary line of the light emitting surface of the backlight light source component in the third image, and filtering out interference impurities outside the boundary line to obtain a fourth image specifically includes:
finding out all contours in the second image, and solving the respective areas of all the contours;
sorting the areas of all the contours in the second image to find out the contour with the largest area; wherein, the outline with the largest area corresponds to the complete boundary line of the light-emitting surface of the backlight light source part;
and filling the interior of the outline with the largest area, and performing AND operation on the filled outline and the third image to filter out interference impurities outside the boundary line.
Further, the identifying and filtering out all branches in the fourth image to obtain a fifth image specifically includes:
finding out all connected domains in the fourth image, and solving the respective length-width ratio and area of all the connected domains;
sorting the length-width ratios of all connected domains in size, solving the median of the length-width ratios, and taking the median as the length-width ratio of a single grain;
sorting the areas of all connected domains in size, solving the median of the areas, and taking the median as the area value of a single grain;
traversing each connected domain in the fourth image, analyzing the aspect ratio value and the area value of the connected domain, judging whether the connected domain corresponds to the branch and peduncle connected domain according to the length-width ratio of single grain and the area value of the single grain, and if so, filtering the connected domain from the fourth image.
Further, according to the length-width ratio of single grain and the area value of single grain, judge whether this connected domain corresponds branch and stalk connected domain, specifically include:
if the length-width ratio of the connected domain is greater than 1.5 times of the length-width ratio of a single grain, or the area value of the connected domain is less than 0.5 times of the area value of the single grain, the connected domain is considered to correspond to the branch connected domain; otherwise, the connected domain is not corresponding to the branch connected domain.
Further, the identifying of the empty particles in the fifth image according to the notch feature formed by the unclosed empty particle glumes respectively calculates the number of the empty particles, the number of the solid particles and the seed setting rate, and specifically includes:
finding out all connected domains in the fifth image, and solving the total number of the connected domains of the fifth image;
solving the approximate polygon outline corresponding to all connected domains in the fifth image and filling the inside of the outline; the approximate polygon corresponding to the empty particle connected domain can fit a gap formed by unclosed glumes to obtain an approximate polygon image;
solving convex hull outlines corresponding to all the approximate polygons respectively and filling the interiors of the outlines to obtain convex hull images;
subtracting the approximate polygon image from the convex hull image, and then filtering to obtain a convex hull image reflecting the gap characteristics of the empty particles;
finding out all connected domains of the salient images, and solving the total number of the connected domains of the salient images;
and respectively calculating the empty particle number, the actual particle number and the seed setting rate according to the total number of the connected domains of the fifth image and the total number of the connected domains of the notch image.
Further, the number of empty grains, the number of filled grains, and the maturing rate are respectively calculated according to the total number of connected domains of the fifth image and the total number of connected domains of the notch image, as follows:
number of empty particles ═ D
Number of particles ═ C-D
Wherein C represents the total number of connected components of the fifth image, and D represents the total number of connected components of the saliency image.
The third purpose of the invention can be achieved by adopting the following technical scheme:
a computer device comprises a processor and a memory for storing a program executable by the processor, wherein the processor executes the program stored in the memory to realize the rice grain detecting method.
The fourth purpose of the invention can be achieved by adopting the following technical scheme:
a storage medium storing a program which, when executed by a processor, implements the above-described method for detecting rice filled grain and empty grain.
Compared with the prior art, the invention has the following beneficial effects:
the invention is provided with the backlight light source component which can provide proper illumination for image acquisition and place the particle group to be counted, the camera is adjustably arranged on the camera mounting bracket, and the lens of the camera can be enabled to face the center of the light emitting surface of the backlight light source component by adjusting the position of the camera.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the structures shown in the drawings without creative efforts.
Fig. 1 is a schematic structural view of a device for detecting rice filled grains and rice empty grains in embodiment 1 of the present invention.
Fig. 2 is a schematic view of a base structure in embodiment 1 of the present invention.
Fig. 3 is a schematic structural diagram of a backlight light source unit according to embodiment 1 of the present invention.
Fig. 4 is a schematic structural view of a camera mounting bracket according to embodiment 1 of the present invention.
Fig. 5 is a schematic structural view of a camera fixing mechanism according to embodiment 1 of the present invention.
Fig. 6 is a block diagram of a computer according to embodiment 1 of the present invention.
FIG. 7 is a flowchart of a method for detecting rice filled grains and rice empty grains according to example 1 of the present invention.
Fig. 8 is a flowchart of acquiring a first image according to embodiment 1 of the present invention.
Fig. 9 is a flowchart of acquiring a fourth image according to embodiment 1 of the present invention.
Fig. 10 is a flowchart of acquiring a fifth image according to embodiment 1 of the present invention.
Fig. 11 is a flowchart of calculating the number of empty grains, the number of actual grains, and the set percentage in example 1 of the present invention.
The system comprises a base 1, a non-slip supporting foot 101, a base cross beam 102, a base longitudinal beam 103, a backlight light source part 2, a luminous surface 201, a camera mounting support 3, a support vertical rod 301, a support longitudinal rod 302, a camera fixing mechanism 303, a compression screw rod supporting seat 3031, a camera compression screw rod 3032, a rubber block 3033, a knob 3034, a camera 4, a system bus 5, a processor 6, an input unit 7, a display unit 8, a network interface 9, a nonvolatile storage medium 10 and an internal storage 11.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer and more complete, the technical solutions in the embodiments of the present invention will be described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some embodiments of the present invention, but not all embodiments, and all other embodiments obtained by a person of ordinary skill in the art without creative efforts based on the embodiments of the present invention belong to the protection scope of the present invention.
Example 1:
as shown in fig. 1, the present embodiment provides a device for detecting rice filled grain and rice empty grain, which comprises a base 1, a backlight light source part 2, a camera mounting bracket 3, a camera 4 and a computer (not shown in the figure).
As shown in fig. 1 and 2, the base 1 is used for supporting the whole detection device and preventing the detection device from sliding, the base 1 is placed on a static horizontal plane and comprises anti-skid supporting legs 101, a base cross beam 102 and a base longitudinal beam 103, wherein the base cross beam 102 and the base longitudinal beam 103 are both aluminum profiles.
Furthermore, the anti-skid supporting feet 101 are fixedly connected with the base cross beam 102, and the base cross beam 102 is fixedly connected with the base longitudinal beam 103; specifically, the number of the anti-skid supporting legs 101 is four, the number of the base beams 102 is two, two of the anti-skid supporting legs 101 are fastened and connected with two ends of one of the base beams 102, the other two anti-skid supporting legs 101 are fastened and connected with two ends of the other base beam 102, and the anti-skid supporting legs 101 can be fastened and connected with the base beams 102 through nuts and washers; the number of the base longitudinal beams 103 is four, one base cross beam 102 is fastened with one end of each of the four base longitudinal beams 103, the other base cross beam 102 is fastened with the other end of each of the four base longitudinal beams 103, and the base cross beam 102 can be fastened with the base longitudinal beams 103 through corner connectors, screws and nuts.
As shown in fig. 1 and fig. 3, the backlight source component 2 is used for providing proper illumination for image acquisition and placing the particle group to be counted, and is centrally installed on the base 1, and is fastened and connected with the base 1 through screws and nuts, the light emitting surface 201 of the backlight source component 2 is large enough to place all particles generated by threshing of rice ears from a single plant without overlapping, and the light emitting surface 201 of the backlight source component 2 is parallel to the horizontal plane, so as to ensure that all particles placed on the light emitting surface 201 do not leave the light emitting surface due to the movement of gravity.
The camera mounting bracket 3 is used for mounting a camera 4 and adjusting the camera 4 to a proper position for shooting, and the camera mounting bracket 3 is mounted on the base 1; the camera 4 is adjustably installed on the camera mounting bracket 3 and connected to the computer through a data line, a lens of the camera 4 faces the center of the light emitting surface 201 of the backlight light source part 2, and the camera 4 is used for capturing an image and transmitting the image to the computer.
As shown in fig. 1, 2 and 4, the camera mounting bracket 3 includes a bracket vertical rod 301, a bracket longitudinal rod 302 and a camera fixing mechanism 303, wherein the bracket vertical rod 301 and the bracket longitudinal rod 302 are both made of aluminum profiles, the bracket vertical rod 301 is fastened and connected with the base 1, and the bracket vertical rod 301 can be fastened and connected with the base beam 102 through an angle code, a screw and a nut; the support longitudinal rod 302 is fixedly connected with the support vertical rod 301, and the support longitudinal rod 302 can be fixedly connected with the support vertical rod 301 through an angle code, a screw and a nut; the camera fixing mechanism 303 is fixedly connected with the bracket longitudinal rod 302, and the camera fixing mechanism 303 can be fixedly connected with the bracket longitudinal rod through screws; the camera 4 is adjustably fixed to the camera fixing mechanism 303.
As shown in fig. 1, 4 and 5, the camera fixing mechanism 303 includes a compression screw support 3031 and two adjustment fixing assemblies, the compression screw support 3031 is fastened to the bracket longitudinal rod 302, and the two adjustment fixing assemblies have the same structure and are oppositely disposed on the left and right sides of the compression screw support 3031.
Further, each adjusting and fixing assembly comprises a camera compression screw 3032, a rubber block 3033 and a knob 3034, the camera compression screw 3032 is respectively in threaded connection with one side of a compression screw supporting seat 3031, and two ends of the camera compression screw 3032 are respectively positioned at the inner side and the outer side of the compression screw supporting seat 3031; the rubber block 3033 is fixedly connected with one end of the camera compression screw 3032, which is positioned at the inner side of the compression screw supporting seat 3031; the knob 3034 is fixedly connected with one end of the camera compression screw 3032, which is positioned on the outer side of the compression screw supporting seat 3031.
For two adjusting and fixing components, the camera pressing screw 3032 is continuously screwed in through the knob 3034 until the rubber block 3033 is extruded on two side surfaces of the camera 4, so that friction force in the opposite direction of the gravity borne by the camera 4 and the like can be generated to fix the camera 4, and it can be understood that the left camera pressing screw 3032 is screwed in a little bit, the right camera pressing screw 3032 is screwed in a little bit, so that the horizontal position of the camera 4 is between the center of the pressing screw supporting seat 3031 and the left side of the pressing screw supporting seat 3031, the left camera pressing screw 3032 is screwed in a little bit, the right camera pressing screw 3032 is screwed in a little bit, the horizontal position of the camera 4 is between the center of the pressing screw supporting seat 3031 and the right side of the pressing screw supporting seat 3031, and the left camera pressing screw 3032 and the right camera pressing screw supporting seat are screwed in the same displacement, the horizontal position of the camera 4 can be adjusted to the center of the compression screw support base 3031, and the height position of the camera 4 can be adjusted only by moving the camera 4 relative to the rubber block 3033 and then fixing the camera 4 through the knob 3034.
The computer can run the detection software of the rice full grain and the rice empty grain to receive, store and process the images sent by the camera and calculate the number of the full grain, the number of the empty grain and the maturing rate; the computer has a structure as shown in fig. 6, and includes a processor 6, a memory, an input unit 7, a display unit 8, and a network interface 9 connected via a system bus 5; the processor 6 is used for providing calculation and control capability, the memory includes a nonvolatile storage medium 10 and an internal memory 11, the nonvolatile storage medium 10 stores an operating system, a computer program and a database, the internal memory 11 provides an environment for the operating system and the computer program in the nonvolatile storage medium 10 to run, and the computer program realizes the functions of the computer when being executed by the processor 6.
As shown in fig. 7, the present embodiment further provides a method for detecting rice filled grains and rice empty grains, which is mainly implemented by the computer, and includes the following steps:
s101, acquiring a first image including a complete boundary line, a real particle group, an empty particle group and a branch group of a light emitting surface of the backlight light source component.
As shown in fig. 8, the step S101 specifically includes:
s1011, threshing the single rice ear, and collecting the threshed solid grains, empty grains and branch stalks.
S1012, the particle group including the solid particles, the empty particles, and the branches is spread over the light emitting surface of the backlight light source unit without overlapping.
Wherein, the particle position does not exceed the light-emitting surface range of the backlight light source part, and the total number of the branches is less than the total number of the rice grains.
And S1013, observing a real-time picture obtained by shooting by the camera, adjusting the horizontal position of the camera until the center of the picture is right opposite to the center of the light emitting surface of the backlight light source, and adjusting the height position of the camera until the picture comprises a complete boundary line, a solid particle group, an empty particle group and a branch group of the light emitting surface of the backlight light source component.
S1014, fixing the camera, and then, all the detection does not need to adjust the position of the camera again.
And S1015, starting the backlight source, adjusting the illumination intensity until the camera can acquire clear images, and then detecting without adjusting the illumination intensity of the backlight source again.
S1016, clicking a start detection button of the rice real grain and empty grain detection software to enable the camera to acquire a current picture image, wherein the image is a first image comprising a complete boundary line of a light emitting surface of the backlight light source part, a real grain group, an empty grain group and a branch group.
And S102, sequentially carrying out graying, filtering and binarization processing on the first image to obtain a second image.
A. Carrying out graying processing based on the following formula to obtain a grayscale image:
I(x,y)＝0.114×B(x,y)+0.587×G(x,y)+0.299×R(x,y) (1)
wherein I (x, y) is a gray value of a pixel of the gray-scale image at the coordinate point (x, y), B (x, y) is a blue component value of the pixel of the first image at the coordinate point (x, y) in the RGB color space, G (x, y) is a green component value of the pixel of the first image at the coordinate point (x, y) in the RGB color space, and R (x, y) is a red component value of the pixel of the first image at the coordinate point (x, y) in the RGB color space.
B. The grayscale image is filtered based on the following method, and a filtered grayscale image is obtained.
And solving the median of all pixel gray values in the neighborhood of each pixel point in the gray map, and replacing the original gray value of the pixel point by the median.
C. Carrying out binarization processing on the filtered gray level image based on the following modes to obtain a second image:
where H (x, y) represents the grayscale value of the pixel of the filtered grayscale image at the coordinate point (x, y), F (x, y) represents the grayscale value of the pixel of the second image at the coordinate point (x, y), and T represents the threshold determined by the maximum inter-class variance method.
And S103, performing complement operation on the second image to obtain a third image.
And performing complement operation on the second image based on the following formula to obtain a third image:
M(x,y)＝255-F(x,y) (3)
where F (x, y) represents the grayscale value of the pixel of the second image at the coordinate point (x, y), and M (x, y) represents the grayscale value of the pixel of the third image at the coordinate point (x, y).
And S104, identifying a complete boundary line of the light emitting surface of the backlight light source component in the third image, and filtering interference impurities except the boundary line to obtain a fourth image.
As shown in fig. 9, the step S104 specifically includes:
s1041, finding out all contours in the second image, and calculating the respective areas of all the contours.
S1042, sorting the areas of all the contours in the second image in size, and finding out the contour with the largest area.
Wherein the outline having the largest area corresponds to the entire boundary line of the light emitting surface of the backlight light source section.
And S1043, filling the interior of the outline with the largest area, and performing AND operation on the filled outline and the third image to filter out interference impurities outside the boundary line.
And S105, identifying and filtering all branches in the fourth image to obtain a fifth image.
As shown in fig. 10, the step S105 specifically includes:
s1051, all connected domains in the fourth image are searched, and the respective aspect ratio and area of all connected domains are obtained.
S1052, sorting the length-width ratios of all the connected domains, calculating the median of the length-width ratios, and taking the median as the length-width ratio A of the single grain.
S1053, sorting the areas of all connected domains, calculating the median of the areas, and regarding the median as the area value B of a single grain.
S1054, traversing each connected domain in the fourth image, analyzing the length-width ratio value and the area value of the connected domain, judging whether the connected domain corresponds to the branch-stem connected domain according to the length-width ratio A of the single grain and the area value B of the single grain, and if so, filtering the connected domain from the fourth image.
Wherein, according to the length-width ratio A of single grain and the area value B of single grain, judge whether this connected domain corresponds branch and stalk connected domain, specifically do: if the length-width ratio of the connected domain is greater than 1.5 times of the length-width ratio A of a single grain, or the area value of the connected domain is less than 0.5 times of the area value B of the single grain, the connected domain is considered to correspond to the branch-stem connected domain; otherwise, the connected domain is not corresponding to the branch connected domain.
And S106, identifying the empty particles in the fifth image according to the notch characteristics formed by the unclosed empty particle glumes, and respectively calculating the number of the empty particles, the number of the solid particles and the seed setting rate.
As shown in fig. 11, the step S106 specifically includes:
s1061, finding out all connected domains in the fifth image, and solving the total number C of the connected domains of the fifth image.
And S1062, obtaining the approximate polygon outline corresponding to all the connected domains in the fifth image and filling the inside of the outline.
And the approximate polygon corresponding to the empty particle connected domain can fit a gap formed by unclosed glumes to obtain an approximate polygon image.
And S1063, obtaining convex hull outlines corresponding to all the approximate polygons respectively, and filling the interiors of the outlines to obtain convex hull images.
And S1064, subtracting the approximate polygon image from the convex hull image, and then filtering to obtain a convex hull image reflecting the void notch characteristics.
S1065, finding all connected domains of the notch image, and solving the total number D of the connected domains of the notch image.
S1066, respectively calculating the number of empty grains, the number of actual grains and the maturing rate according to the total number C of the connected domains of the fifth image and the total number D of the connected domains of the notch image, as follows:
number of empty grain ═ D (4)
Number of grains ═ C-D (5)
Example 2:
the present embodiment provides a storage medium, which is a computer-readable storage medium, and stores a computer program, and when the computer program is executed by a processor, the method for detecting rice filled grains and rice empty grains of embodiment 1 is implemented as follows:
acquiring a first image including a complete boundary line, a real particle group, an empty particle group and a branch group of a light emitting surface of the backlight light source part;
carrying out graying, filtering and binarization processing on the first image in sequence to obtain a second image;
performing complement operation on the second image to obtain a third image;
identifying a complete boundary line of a light emitting surface of the backlight light source part in the third image, and filtering interference impurities outside the boundary line to obtain a fourth image;
identifying and filtering all branches in the fourth image to obtain a fifth image;
and identifying the empty particles in the fifth image according to the notch characteristics formed by the unclosed empty particle glumes, and respectively calculating the number of the empty particles, the number of the actual particles and the seed setting rate.
It should be noted that the computer readable storage medium of the present embodiment may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this embodiment, however, a computer readable signal medium may include a propagated data signal with a computer readable program embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable storage medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program embodied on the computer readable storage medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
In summary, the invention is provided with the backlight light source component, the backlight light source component can provide proper illumination for image acquisition and place the particle group to be counted, the camera is adjustably arranged on the camera mounting bracket, and the lens of the camera can be enabled to face the center of the light emitting surface of the backlight light source component by adjusting the position of the camera.
The above description is only for the preferred embodiments of the present invention, but the protection scope of the present invention is not limited thereto, and any person skilled in the art can substitute or change the technical solution and the inventive concept of the present invention within the scope of the present invention.
Claims (10)
1. A detection device for rice filled grains and rice empty grains is characterized by comprising a base, a backlight light source component, a camera mounting bracket, a camera and a computer;
the backlight light source component is arranged on the base in the middle;
the camera mounting bracket is arranged on the base;
the camera is adjustably arranged on the camera mounting bracket and is connected with the computer;
during detection, acquiring a first image including a complete boundary line, a real particle group, an empty particle group and a branch group of a light emitting surface of the backlight light source part; carrying out graying, filtering and binarization processing on the first image in sequence to obtain a second image; performing complement operation on the second image to obtain a third image; identifying a complete boundary line of a light emitting surface of the backlight light source part in the third image, and filtering interference impurities outside the boundary line to obtain a fourth image; identifying and filtering all branches in the fourth image to obtain a fifth image; identifying the empty particles in the fifth image according to the notch characteristics formed by the unclosed empty particle glumes, and respectively calculating the number of the empty particles, the number of the actual particles and the seed setting rate;
the acquiring of the first image including the complete boundary line, the real particle group, the empty particle group and the branch group of the light emitting surface of the backlight light source part specifically includes: threshing the single rice ear, and collecting the threshed solid grains, empty grains and branch stalks; spreading particle groups including solid particles, empty particles and branches on the light-emitting surface of the backlight light source part in a non-overlapping mode, wherein the positions of the particles do not exceed the range of the light-emitting surface of the backlight light source part, and the total number of the branches is smaller than that of the rice grains; adjusting the horizontal position of the camera until the center of the picture is opposite to the center of the light emitting surface of the backlight light source part, and adjusting the height position of the camera until the picture comprises a complete boundary line, a solid particle group, an empty particle group and a branch group of the light emitting surface of the backlight light source part; fixing a camera; starting a backlight source, and adjusting the illumination intensity until a clear image can be acquired by the camera; and enabling the camera to acquire a current picture image, wherein the image is a first image comprising a complete boundary line of a light emitting surface of the backlight light source part, a real particle group, an empty particle group and a branch group.
2. The apparatus for detecting rice grains in real and empty as claimed in claim 1, wherein the light emitting surface of the backlight light source unit is capable of placing all the grains generated from threshing rice ears of a single rice plant without overlapping;
the light emitting surface of the backlight light source part is parallel to the horizontal plane, and the backlight light source part is fixedly connected with the base.
3. The device for detecting the presence of solid and empty rice kernels according to any one of claims 1-2, wherein the camera mounting bracket comprises a bracket vertical rod, a bracket longitudinal rod and a camera fixing mechanism;
the support vertical rod is fixedly connected with the base;
the vertical rod of the bracket is fixedly connected with the vertical rod of the bracket;
the camera fixing mechanism is fixedly connected with the bracket longitudinal rod;
the camera is adjustably fixed to the camera fixing mechanism.
4. The device for detecting the presence of rice grains in real rice and rice grains in empty rice as claimed in claim 3, wherein said camera fixing mechanism comprises a pressing screw supporting base and an adjusting and fixing component;
the compression screw supporting seat is fixedly connected with the vertical rod of the bracket;
the two adjusting and fixing assemblies are oppositely arranged on two sides of the compression screw supporting seat;
each adjusting and fixing assembly comprises a camera compression screw, a rubber block and a knob, the camera compression screw is respectively in threaded connection with one side of a compression screw supporting seat, and two ends of the camera compression screw are respectively positioned on the inner side and the outer side of the compression screw supporting seat; the rubber block is fixedly connected with one end of the camera compression screw rod, which is positioned at the inner side of the compression screw rod supporting seat; the knob is fixedly connected with one end of the camera compression screw rod, which is positioned outside the compression screw rod supporting seat;
for the two adjusting and fixing components, the camera compression screw is continuously screwed in through the knob until the rubber blocks are extruded to the two side faces of the camera, so that the camera is fixed.
5. A method for detecting rice full grains and empty grains is characterized by comprising the following steps:
acquiring a first image including a complete boundary line, a real particle group, an empty particle group and a branch group of a light emitting surface of the backlight light source part;
carrying out graying, filtering and binarization processing on the first image in sequence to obtain a second image;
performing complement operation on the second image to obtain a third image;
identifying a complete boundary line of a light emitting surface of the backlight light source part in the third image, and filtering interference impurities outside the boundary line to obtain a fourth image;
identifying and filtering all branches in the fourth image to obtain a fifth image;
identifying the empty particles in the fifth image according to the notch characteristics formed by the unclosed empty particle glumes, and respectively calculating the number of the empty particles, the number of the actual particles and the seed setting rate;
the acquiring of the first image including the complete boundary line, the real particle group, the empty particle group and the branch group of the light emitting surface of the backlight light source part specifically includes:
threshing the single rice ear, and collecting the threshed solid grains, empty grains and branch stalks;
spreading particle groups including solid particles, empty particles and branches on the light-emitting surface of the backlight light source part in a non-overlapping mode, wherein the positions of the particles do not exceed the range of the light-emitting surface of the backlight light source part, and the total number of the branches is smaller than that of the rice grains;
adjusting the horizontal position of the camera until the center of the picture is opposite to the center of the light emitting surface of the backlight light source part, and adjusting the height position of the camera until the picture comprises a complete boundary line, a solid particle group, an empty particle group and a branch group of the light emitting surface of the backlight light source part;
fixing a camera;
starting a backlight source, and adjusting the illumination intensity until a clear image can be acquired by the camera;
and enabling the camera to acquire a current picture image, wherein the image is a first image comprising a complete boundary line of a light emitting surface of the backlight light source part, a real particle group, an empty particle group and a branch group.
6. The method for detecting rice filled grains and rice empty grains according to claim 5, wherein the step of identifying a complete boundary line of a light emitting surface of the backlight light source component in the third image and filtering out interfering impurities outside the boundary line to obtain a fourth image specifically comprises the steps of:
finding out all contours in the second image, and solving the respective areas of all the contours;
sorting the areas of all the contours in the second image to find out the contour with the largest area; wherein, the outline with the largest area corresponds to the complete boundary line of the light-emitting surface of the backlight light source part;
and filling the interior of the outline with the largest area, and performing AND operation on the filled outline and the third image to filter out interference impurities outside the boundary line.
7. The method for detecting rice filled grains and rice empty grains according to claim 5, wherein the identifying and filtering out all branches and stalks in the fourth image to obtain a fifth image specifically comprises:
finding out all connected domains in the fourth image, and solving the respective length-width ratio and area of all the connected domains;
sorting the length-width ratios of all connected domains in size, solving the median of the length-width ratios, and taking the median as the length-width ratio of a single grain;
sorting the areas of all connected domains in size, solving the median of the areas, and taking the median as the area value of a single grain;
traversing each connected domain in the fourth image, analyzing the aspect ratio value and the area value of the connected domain, judging whether the connected domain corresponds to the branch and peduncle connected domain according to the length-width ratio of single grain and the area value of the single grain, and if so, filtering the connected domain from the fourth image.
8. The method for detecting rice filled grains and rice empty grains according to claim 7, wherein the step of judging whether the connected domain corresponds to the branch-stem connected domain according to the length-width ratio of a single grain and the area value of the single grain specifically comprises the steps of:
if the length-width ratio of the connected domain is greater than 1.5 times of the length-width ratio of a single grain, or the area value of the connected domain is less than 0.5 times of the area value of the single grain, the connected domain is considered to correspond to the branch connected domain; otherwise, the connected domain is not corresponding to the branch connected domain.
9. The method for detecting rice grains according to any one of claims 5 to 8, wherein the step of identifying the empty grains in the fifth image according to the notch feature formed by the unclosed empty grain glumes respectively comprises the steps of:
finding out all connected domains in the fifth image, and solving the total number of the connected domains of the fifth image;
solving the approximate polygon outline corresponding to all connected domains in the fifth image and filling the inside of the outline; the approximate polygon corresponding to the empty particle connected domain can fit a gap formed by unclosed glumes to obtain an approximate polygon image;
solving convex hull outlines corresponding to all the approximate polygons respectively and filling the interiors of the outlines to obtain convex hull images;
subtracting the approximate polygon image from the convex hull image, and then filtering to obtain a convex hull image reflecting the gap characteristics of the empty particles;
finding out all connected domains of the salient images, and solving the total number of the connected domains of the salient images;
and respectively calculating the empty particle number, the actual particle number and the seed setting rate according to the total number of the connected domains of the fifth image and the total number of the connected domains of the notch image.
10. The method of claim 9, wherein the number of empty grains, the number of filled grains and the maturing rate are calculated according to the total number of connected components in the fifth image and the total number of connected components in the gap image, respectively, as follows:
number of empty particles ═ D
Number of particles ═ C-D
Wherein C represents the total number of connected components of the fifth image, and D represents the total number of connected components of the saliency image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010970495.6A CN112229836B (en) | 2020-09-15 | 2020-09-15 | Detection device and detection method for rice filled grains and rice empty grains |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010970495.6A CN112229836B (en) | 2020-09-15 | 2020-09-15 | Detection device and detection method for rice filled grains and rice empty grains |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112229836A CN112229836A (en) | 2021-01-15 |
| CN112229836B true CN112229836B (en) | 2022-02-15 |
Family
ID=74117116
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010970495.6A Active CN112229836B (en) | 2020-09-15 | 2020-09-15 | Detection device and detection method for rice filled grains and rice empty grains |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112229836B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115100447B (en) * | 2022-07-18 | 2024-05-17 | 浙江大学 | Rice setting rate measuring device and method based on thermal infrared image registration fusion |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2009045035A1 (en) * | 2007-10-01 | 2009-04-09 | Rural Development Administration | White and brown rice appearance characteristics measurement system and method |
| CN101905215A (en) * | 2010-07-22 | 2010-12-08 | 华中科技大学 | Digitalized paddy rice seed testing machine |
| CN104021369A (en) * | 2014-04-30 | 2014-09-03 | 南京农业大学 | Grain counting method for spike of single rice based on digital image processing technology |
| CN104268890A (en) * | 2014-10-13 | 2015-01-07 | 扬州大学 | Method for calculating rice maturing rate |
| CN105975966A (en) * | 2016-04-21 | 2016-09-28 | 南京农业大学 | Rice grain mildew nondestructive test method |
| WO2019054235A1 (en) * | 2017-09-13 | 2019-03-21 | キヤノン株式会社 | Information processing device, information processing method, and program |
| CN109682817A (en) * | 2019-02-22 | 2019-04-26 | 哈尔滨工程大学 | Degree of whiteness detection device and method based on computer vision technique |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101306583B1 (en) * | 2011-11-08 | 2013-09-10 | 대원강업주식회사 | A measurement equipment and method for shape measure of curved spring |
| CN203204266U (en) * | 2013-03-26 | 2013-09-18 | 东莞市彤光电子科技有限公司 | Image capturing device for liquid crystal displayer backlight source visual detection |
| JP6861986B2 (en) * | 2016-11-21 | 2021-04-21 | 株式会社ブレイン | Article identification device with indirect illumination light source |
| CN206862904U (en) * | 2017-03-17 | 2018-01-09 | 苏州玻色智能科技有限公司 | A kind of vision-based detection experiment porch |
| CN210465273U (en) * | 2019-08-02 | 2020-05-05 | 深圳市优尼可智能装备有限公司 | Camera detection mechanism of 3C auxiliary material automated inspection mark machine |
-
2020
- 2020-09-15 CN CN202010970495.6A patent/CN112229836B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2009045035A1 (en) * | 2007-10-01 | 2009-04-09 | Rural Development Administration | White and brown rice appearance characteristics measurement system and method |
| CN101905215A (en) * | 2010-07-22 | 2010-12-08 | 华中科技大学 | Digitalized paddy rice seed testing machine |
| CN104021369A (en) * | 2014-04-30 | 2014-09-03 | 南京农业大学 | Grain counting method for spike of single rice based on digital image processing technology |
| CN104268890A (en) * | 2014-10-13 | 2015-01-07 | 扬州大学 | Method for calculating rice maturing rate |
| CN105975966A (en) * | 2016-04-21 | 2016-09-28 | 南京农业大学 | Rice grain mildew nondestructive test method |
| WO2019054235A1 (en) * | 2017-09-13 | 2019-03-21 | キヤノン株式会社 | Information processing device, information processing method, and program |
| CN109682817A (en) * | 2019-02-22 | 2019-04-26 | 哈尔滨工程大学 | Degree of whiteness detection device and method based on computer vision technique |
Non-Patent Citations (1)
| Title |
|---|
| "Classification of Bulk Samples of Cereal Grains using Machine Vision";S. Majumdar, D. S. Jayas;《Journal of Agricultural Engineering Research》;19991231;第73卷(第1期);35-47 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112229836A (en) | 2021-01-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20110221906A1 (en) | Multiple Camera System for Automated Surface Distress Measurement | |
| DE69329820T2 (en) | MEAT IMAGE ANALYSIS. | |
| US9330337B2 (en) | Color analytics for a digital image | |
| CN105701805B (en) | A kind of pork intramuscular fat content lossless detection method based on computer vision | |
| CN112229836B (en) | Detection device and detection method for rice filled grains and rice empty grains | |
| US8249342B1 (en) | Color analytics for a digital image | |
| CN105959585A (en) | Multi-grade backlight detection method and device | |
| CN110160750B (en) | LED display screen visual detection system, detection method and detection device | |
| CN105405145B (en) | A kind of granule number grain method based on image Segmentation Technology | |
| CN109035225B (en) | Design quality evaluation method for automobile brake pad appearance quality inspection lighting system | |
| CN112070168A (en) | Coal sorting method and device | |
| CN113518182B (en) | Cucumber phenotype characteristic measuring method based on raspberry pie | |
| CN114286014A (en) | Image database acquisition method | |
| CN108303427B (en) | Household damaged degree detection system based on image procossing | |
| CN214097211U (en) | Transparent plate glass's defect detecting device | |
| US20150312481A1 (en) | Vision systems and methods for analysing images taken by image sensors | |
| CN107300360A (en) | A kind of shaft size fast algorithm of detecting of rain fed crops seed three | |
| CN106023223A (en) | Orange fruit size describing method and organic fruit size grading method | |
| US20190302004A1 (en) | Adaptive Method for a Light Source | |
| MacCormac | On-line image processing for tobacco grading in Zimbabwe | |
| CN105027732B (en) | The method and system of corn ear test | |
| Nishikawa et al. | Area properties of television pictures | |
| CN110765905A (en) | Method and device for measuring specific gravity of impurities contained in grains harvested by combine harvester | |
| JP2016061646A (en) | Color headlight tester and light-dark branch line search method | |
| EP3751460A1 (en) | Exposure compensating method and camera apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |