CN109359531A - Fruit harvesting area automatic positioning method facing natural scene - Google Patents
Fruit harvesting area automatic positioning method facing natural scene Download PDFInfo
- Publication number
- CN109359531A CN109359531A CN201811062603.9A CN201811062603A CN109359531A CN 109359531 A CN109359531 A CN 109359531A CN 201811062603 A CN201811062603 A CN 201811062603A CN 109359531 A CN109359531 A CN 109359531A
- Authority
- CN
- China
- Prior art keywords
- fruit
- pixel
- centre
- region
- neighborhood
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 235000013399 edible fruits Nutrition 0.000 title claims abstract description 113
- 238000003306 harvesting Methods 0.000 title claims abstract description 52
- 238000000034 method Methods 0.000 title claims abstract description 48
- 238000001514 detection method Methods 0.000 claims abstract description 15
- 238000012549 training Methods 0.000 claims abstract description 6
- 239000000284 extract Substances 0.000 claims description 27
- 238000000605 extraction Methods 0.000 claims description 22
- 238000010586 diagram Methods 0.000 claims description 19
- 238000012545 processing Methods 0.000 claims description 13
- 239000003086 colorant Substances 0.000 claims description 11
- 238000006243 chemical reaction Methods 0.000 claims description 9
- 230000011218 segmentation Effects 0.000 claims description 8
- 230000003321 amplification Effects 0.000 claims description 6
- 238000003199 nucleic acid amplification method Methods 0.000 claims description 6
- 241000238370 Sepia Species 0.000 claims description 5
- 230000003628 erosive effect Effects 0.000 claims description 5
- 230000005484 gravity Effects 0.000 claims description 5
- 238000013507 mapping Methods 0.000 claims description 5
- 238000003711 image thresholding Methods 0.000 claims description 4
- 238000012804 iterative process Methods 0.000 claims description 4
- 230000008569 process Effects 0.000 claims description 3
- 239000006185 dispersion Substances 0.000 claims description 2
- 241001629511 Litchi Species 0.000 abstract description 37
- 230000002708 enhancing effect Effects 0.000 abstract description 2
- 230000000694 effects Effects 0.000 description 19
- 238000005286 illumination Methods 0.000 description 12
- 230000010339 dilation Effects 0.000 description 4
- 230000003014 reinforcing effect Effects 0.000 description 4
- 238000001914 filtration Methods 0.000 description 3
- 238000003384 imaging method Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 235000020971 citrus fruits Nutrition 0.000 description 2
- 239000012636 effector Substances 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 2
- 230000004807 localization Effects 0.000 description 2
- 239000002420 orchard Substances 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 206010036590 Premature baby Diseases 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000002146 bilateral effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000877 morphologic effect Effects 0.000 description 1
- 230000000422 nocturnal effect Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000013138 pruning Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000002187 spin decoupling employing ultra-broadband-inversion sequences generated via simulated annealing Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/64—Three-dimensional objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/77—Retouching; Inpainting; Scratch removal
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30181—Earth observation
- G06T2207/30188—Vegetation; Agriculture
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to a natural scene-oriented automatic positioning method for a fruit harvesting area, which comprises the following steps: extracting litchi fruit areas in the training data, counting weighted tricolor brightness distribution characteristics of the litchi fruit areas as objective brightness reference, and enhancing weighted tricolor brightness components by adopting an iterative Retinex algorithm; extracting the fruit region with enhanced brightness by combining the corrected color difference map, thresholding treatment and mathematical morphology; reconstructing hue components in an HSI color space through the position relation of local neighborhood pixels and hue information, and extracting branch regions according to hue distribution characteristics; the branch framework is extracted by adopting a thinning algorithm, the key angular points on the framework are extracted through angular point detection and the mode distribution rule of angular point neighborhood pixels, and the litchi fruit harvesting area is automatically positioned by combining the relative position constraint of the fruits and the branches and the spatial distribution characteristic of the key angular points. The method can improve the adaptivity and the accuracy of automatic positioning of the litchi fruit harvesting area in a natural scene.
Description
Technical field
The present invention relates to IT application to agriculture, precision agriculture, machine vision and technical field of image processing, and in particular to a kind of
Fruit harvesting region automatic positioning method towards natural scene.
Background technique
Modern orchard gradually appears the status that management cost is constantly incremented by, manual labor constantly successively decreases, close with labour
Picking fruit mode based on collection type gradually shows trend (the Gongal A., Amatya of sustainable developability reduction
S.,Karkee M.,et al,2015.Sensors and systems for fruit detection and localiza
tion.Comput.Electron.Agric.116,8-19).The intelligent harvesting equipment of fruit is a kind of traditional artificial side of change
The automated job of formula is equipped, and the critical components such as mobile platform, control system, the detection of woods fruit and end effector are generally included,
For step by step input into the automatic recovery operations of a variety of mountain area woods fruits, the main problem that it is faced has been nature into slice at present
The reliable detection with harvesting point/region accurate positionin in fruit region under part.
Fruit detection generallys use machine vision and image processing techniques with harvesting point/zone location.For natural scene
Influence of the middle forest fruit regional imaging quality vulnerable to illumination condition, (Xu L.M., the Lv J.D., 2018.Recognition such as Xu
method for apple fruit based on SUSAN and PCNN.Multimed.Tools Appl.77(6),
7205-7219) using the fruit image acquired in the homographic filtering method processing weaker situation of illumination.Wang etc. (Wang C.L.,
Lee W.S,Zou X.J.,et al,2018.Detection and counting of immature green citrus
fruit based on the Local Binary Patterns(LBP)feature using illumination-
Normalized images.Precision Agric.17 (6), 678-697) use the Retinex algorithm based on bilateral filtering
Improve the overall brightness in fruit region in image.But the above method does not consider the INTELLIGENT IDENTIFICATION of input picture illumination property,
Processing brightness is higher or the more uniform image of Luminance Distribution is easy to appear " overexposure " phenomenon or changes the original in fruit region
There is hue information, subsequent fruit region detection and harvesting point/zone location accuracy may be influenced, so being not suitable for certainly
Woods fruit image under dynamic processing natural scene.Zhuang etc. (Zhuang J.J., Luo S.M., Hou C.J., et al,
2018.Detection of orchard citrus fruits using a monocular machine vision-
based method for automatic fruit picking applications.Comput.Electron.Agric.
152,64-73) the even problem of uneven illumination of fruit area image has been automatically processed using local block homographic filtering method, but should
Method includes more adjustment parameter, may need to readjust when environmental factor significantly changes, treatment process is relatively complicated.
(Wang C.L., Zou X.J., Tang Y.C., et al, the 2016.Localisation of litchi in an such as Wang
unstructured environment using binocular stereo vision.Biosyst.Eng.145,39-51)
In conjunction with K- means Method and the method for registering based on tag template extracts the red litchi fruits in binocular vision image sequence
There is preferable detection effect in region to the lesser woods fruit image of backcolor difference.Point is harvested for the lichee under pneumatic environment
Orientation problem, (Xiong J.T., He Z.L., Lin R., et al, the 2018.Visual positioning such as Xiong
technology of picking robots for dynamic litchi clusters with disturbance.Co
Mput.Electron.Agric.151,226-237) the tone distributed knowledge based on litchi fruits region, using clustering algorithm from
Mature litchi fruits region and its mass center are extracted in the chrominance component of HSI image, through pendulum criterion from binocular vision image
Harvesting point is estimated, but automatic positioning error is larger, is suitable for the wider end effector of beta pruning job area.To avoid environment
Influence of the factor to lichee harvesting spot placement accuracy, Xiong etc. (Xiong J.T., Lin R., Liu Z., et al,
2018.The recognition of litchi clusters and the calculation of picking point
In a nocturnal natural environment.Biosyst.Eng.166,44-57) using based on LED orientation light filling
Nighttime imaging system, non-woods fruit in night scenes image and interference limb region, knot are filtered out by fuzzy C-means clustering
It closes Otsu Threshold segmentation and Harris corner detection approach orients the harvesting point of litchi fruits, this method is mainly for the treatment of preceding back
The more significant night woods fruit image of scape difference.
Although currently based on the litchi fruits of machine vision and image processing techniques harvesting point/region automatic positioning method
Achieved certain effect, but in the continually changing natural scene of environmental factor, more reliable, more accurate harvesting point or
Region automatic positioning method is harvested to explore there is still a need for further.
Summary of the invention
The present invention provides one kind towards natural field to overcome at least one defect (deficiency) described in the above-mentioned prior art
The fruit harvesting region automatic positioning method of scape, it is intended to improve fruit detection and harvesting zone location based on machine vision technique
The adaptivity and accuracy of system.
To achieve the purpose of the present invention, it is achieved using following technical scheme:
A kind of fruit harvesting region automatic positioning method towards natural scene, comprising the following steps:
(1) the weighting three primary colors luminance component V for extracting current RGB image I, utilizes modified R X chromaticity difference diagram (Modified
Red and Green/Blue (X) Chromatic Mapping, MRXCM) method generates modified R X (Red&Green/ from I
Blue) chromaticity difference diagram Cm, from CmMiddle extraction part fruit region RoIs (Regions of Interest) calculates RoIs Luminance Distribution
Statistical nature, Retinex algorithm processing V is iteratively used according to statistical nature and reconstructs I, when iterative process terminates
The enhanced RGB image I ' of brightness is exported by color space conversion model；
(2) modified R X chromaticity difference diagram C is therefrom extracted by MRXCM methodm', to Cm' execution imagethresholding obtains
Cm＂, and C is handled using mathematical Morphology Algorithmm＂ therefrom extracts potential mature or immature fruit region Of；
(3) the chrominance component H ' for extracting I ', measures the positional relationship of local neighborhood pixel and the centre of neighbourhood, and it is adjacent to measure part
The hue information difference of domain pixel and the centre of neighbourhood establishes the weight distribution model of local neighborhood centering heart pixel tonal value, leads to
It crosses and the centre of neighbourhood pixel by noise pollution is isolated, reconstructs chrominance component H ' in a manner of weighted meanc, to H 'cExecute image threshold
Value is divided to obtain Hc＂ therefrom extracts potential sepia branch region Os；
(4) O is extractedsSkeleton Qs', detect Qs' on angle point and extract it is relevant to branch bifurcated attribute key angle point,
Utilize O under gravityfAnd Qs' relative position constraint and crucial angle point spatial characteristics, positioning fruit harvesting point distribution
Section.
Further, step (1) the weighting three primary colors luminance component V for extracting current RGB image I, specifically, with becoming
Amount R, G and B respectively indicate the grey scale pixel value of the red, green and blue Color Channel of I, calculate weighting three primary colors brightness point according to the following formula
Measure V:
V=α R+ β G+ γ B
Further, step (1) is described utilizes modified R X chromaticity difference diagram (Modified Red and Green/Blue (X)
Chromatic Mapping, MRXCM) method generates modified R X (Red&Green/Blue) chromaticity difference diagram C from Im, specifically, root
The red color tone or green hue shown according to fruit region calculates the monochromatic ratio of R and G or R and B as scale factor, and weights
Into conventional red green difference figure or reddish blue difference figure, modified chromaticity difference diagram C is generatedm。
Further, step (1) is described from CmMiddle extraction part fruit region RoIs (Regions of Interest),
The statistical nature for calculating RoIs Luminance Distribution, specifically, to CmIt executes Threshold segmentation and erosion operation obtains bianry image, extract
Local fruit region RoIs calculates the average brightness value M of mask images institute overlay area in V as mask imagesRoIs。
Further, step (1) is described iteratively uses Retinex algorithm processing V according to statistical nature and reconstructs
I exports the enhanced RGB image I ' of brightness by color space conversion model when iterative process is terminated, specifically, extracting more
The three primary colors weighted luminance component of a training image counts the average brightness value M in fruit regiontrain, and execute following steps:
S1. judge whether MRoIs≥Mtrain, if so then execute step S2, S3 is thened follow the steps if not；
S2. I as the enhanced RGB image I ' of brightness and is exported into I '；
S3. enhanced luminance component V ' is obtained using Retinex algorithm, passes through RGB/HSI color space conversion model
The tone H and color saturation S of current RGB image are extracted, V ' is merged to generate the enhanced RGB image I ' of brightness, extracts again
The average M of I 'RoIs' judges whether MRoIs' >=MtrainIf otherwise continuing to execute step S3, if continuing to execute step S4；
S4. I ' is exported.
Preferably, step (2) is described handles C using mathematical Morphology Algorithmm＂ therefrom extracts potential mature or prematurity
Fruit region Of, specifically, successively using the morphological erosion operation based on coarse scale structures member, based on small scale structures member
Dilation operation and holes filling handle Cm＂, using 8- connectivity criteria from treated CmPotential fruit region O is extracted in ＂f。
Further, step (3) positional relationship for measuring local neighborhood pixel and the centre of neighbourhood, specifically, leading
Domain center is (xc,yc) m × n neighborhood in, coordinates computed be (xi,yi) ith pixel and centre of neighbourhood distance dis
((xi,yi),(xc,yc)), the positional relationship w of ith pixel and the centre of neighbourhood is obtained according to the following formulapi:
wpi=exp {-λPmax(dis((xi,yi),(xc,yc)))}
In formula, λpFor position amplification coefficient, i=1,2 ..., m × n.
Further, step (3) the hue information difference for measuring local neighborhood pixel and the centre of neighbourhood, specifically,
In field, centre coordinate is (xc,yc) m × n neighborhood in, calculate neighborhood in coordinate be (xi,yi) ith pixel tone value
hiWith neighborhood tone intermediate value hmDifference diff (hi,hm), and the neighborhood territory pixel tone value standard deviation sigma in addition to central pointc, according to
The different information w of following formula calculating ith pixel and the centre of neighbourhoodHi:
In formula, λHFor tone difference amplification coefficient, i=1,2 ..., m × n.
Further, step (3) the weight distribution model for establishing local neighborhood centering heart pixel tonal value, by every
From by noise pollution centre of neighbourhood pixel, chrominance component H ' is reconstructed in a manner of weighted meanc, specifically, passing through wpiAnd wHi
Point multiplication operation distribution ith pixel weight wciIf centre of neighbourhood tone value hcWith hmDeviation is larger, then field center is arranged
Weight be 0, be otherwise provided as 1, calculate the weighted average of each pixel tonal value in neighborhood, the weight as centre of neighbourhood pixel
Structure tone value h 'c, the chrominance component H ' reconstructed after all pixels of H ' is traversed based on above-mentioned processc。
Preferably, step (3) is described to H 'cIt executes imagethresholding and obtains Hc＂ therefrom extracts potential sepia
Branch region Os, specifically, according to branch hue information distribution setting segmentation threshold in multiple training images, to H 'cExecute image
Threshold division obtains Hc＂, using 8- connectivity criteria from HcPotential branch region O is extracted in ＂s。
Further, step (4) the extraction OsSkeleton Qs', detect Qs' on angle point and extract with branch bifurcated category
Property relevant crucial angle point, specifically, thinning algorithm is utilized to extract OsSkeleton Qs', it is extracted using Harris Corner Detection Algorithm
Qs' on whole angle points utilize crotch angle point adjacent according to the bifurcated feature between fruit branches different in the case of more fruit adhesions
The mode distribution rule and skeleton connectivity of domain pixel judge whether crotch angle point belongs to crucial angle point and extract crucial angle point.
Further, step (4) is described utilizes O under gravityfAnd Qs' relative position constraint and crucial angle point sky
Between distribution character, position fruit harvesting point distributed area, specifically, according to fruit position of centre of gravity under gravity lower than its connect
Strong connectedness feature between branch, fruit carpopodium and branch calculates carpopodium and branch join domain ArMiddle all pixels space is sat
Target statistical nature, and combine the statistical nature of crucial angular coordinate extract only with OfBranch skeleton with connectivity extracts
Harvesting central point of the crucial angle point as fruit on the skeleton far from carpopodium, with ArMiddle pixel and angled key space of points column coordinate
Dispersion degree as deviate harvesting point regional standard it is poor, according to harvesting central point and regional standard difference position fruit harvesting point
Distributed area.
Compared with prior art, the beneficial effect of technical solution of the present invention is: (1) iterative Retinex algorithm can be automatic
The overall brightness distribution that input RGB image is adjusted according to the light characteristic in litchi fruits region, is improving image overall brightness
The color information for remaining to maintain each region of original image when distribution, has preferable scene adaptive, is suitable for natural environment
Under image preprocessing；(2) simultaneously using in the positional relationship of local neighborhood pixel and hue information reconstruct HSI color space
Chrominance component can not only eliminate noise jamming, moreover it is possible to restore the practical hue information of centre of neighbourhood pixel to greatest extent, favorably
In the reliability for improving branch extracted region；(3) it is distributed based on Harris Corner Detection Algorithm and the mode of angle point neighborhood territory pixel
Rule Extraction key angle point, is avoided that the interference of noise angle point to a certain extent, be conducive to improve litchi fruits harvesting point or
Harvest the positional accuracy in region.
Detailed description of the invention
Fig. 1 is the implementation example figure of the fruit harvesting region automatic positioning method process towards natural scene in the present embodiment.
Fig. 2 a is the image embodiment figure of a secondary illumination abundance.
Fig. 2 b is the adaption brightness reinforcing effect implementation example figure of image shown in Fig. 2 a.
Fig. 2 c is the weaker image embodiment figure of a secondary illumination.
Fig. 2 d is the adaption brightness reinforcing effect implementation example figure of image shown in Fig. 2 c.
Fig. 3 a is the litchi fruits extraction effect implementation example figure of image shown in Fig. 2 a.
Fig. 3 b is the litchi fruits extraction effect implementation example figure of image shown in Fig. 2 c.
Fig. 4 a be the centre of neighbourhood not by noise pollution when tone value reconstruction result implementation example figure.
Fig. 4 b be the centre of neighbourhood by noise pollution when tone value reconstruction result implementation example figure.
Fig. 5 a is the lichee branch extraction effect implementation example figure of image shown in Fig. 2 a.
Fig. 5 b is the lichee branch extraction effect implementation example figure of image shown in Fig. 2 c.
Fig. 6 a is that the litchi fruits of image shown in Fig. 2 a harvest point location effect example figure.
Fig. 6 b is that the litchi fruits of image shown in Fig. 2 c harvest point location effect example figure.
Specific embodiment
The attached figures are only used for illustrative purposes and cannot be understood as limitating the patent；
In order to better illustrate this embodiment, the certain components of attached drawing have omission, zoom in or out, and do not represent actual product
Size；
To those skilled in the art, it is to be understood that certain known features and its explanation, which may be omitted, in attached drawing
's.
In the description of the present invention, unless otherwise indicated, the meaning of " plurality " is two or more.
The following further describes the technical solution of the present invention with reference to the accompanying drawings and examples.
Embodiment
As shown in Figure 1, a kind of fruit harvesting region automatic positioning method towards natural scene, comprising the following steps:
(1) the weighting three primary colors luminance component V for extracting current RGB image I, utilizes modified R X chromaticity difference diagram (Modified
Red and Green/Blue (X) Chromatic Mapping, MRXCM) method generates modified R X (Red&Green/ from I
Blue) chromaticity difference diagram Cm, from CmMiddle extraction part fruit region RoIs (Regions of Interest) calculates RoIs Luminance Distribution
Statistical nature, according to statistical nature export the enhanced RGB image I ' of brightness；
(2) modified R X chromaticity difference diagram C is therefrom extracted by MRXCM methodm', to Cm' execution imagethresholding obtains
Cm＂, and C is handled using mathematical Morphology Algorithmm＂ therefrom extracts potential mature or immature fruit region Of；
(3) the chrominance component H ' for extracting I ', measures the positional relationship of local neighborhood pixel and the centre of neighbourhood, and it is adjacent to measure part
The hue information difference of domain pixel and the centre of neighbourhood establishes the weight distribution model of local neighborhood centering heart pixel tonal value, leads to
It crosses and the centre of neighbourhood pixel by noise pollution is isolated, reconstructs chrominance component H ' in a manner of weighted meanc, to H 'cExecute image threshold
Value segmentation, therefrom extracts potential sepia branch region Os；
(4) O is extractedsSkeleton Qs', detect Qs' on angle point and extract it is relevant to branch bifurcated attribute key angle point,
Utilize O under gravityfAnd Qs' relative position constraint and crucial angle point spatial characteristics, positioning fruit harvesting point distribution
Section.
The imaging effect in fruit region is easy to be influenced by illumination condition in visual pattern, it is therefore necessary to examine in fruit
The illumination compensation surveyed and first carry out image before harvesting point location, and the hue information in fruit region cannot occur significantly to change after compensating
Become, and and brightness value higher input picture uniform for luminance information, is not then considered as illumination compensation, otherwise will appear " mistake
Degree exposure " phenomenon and cause significantly changing for the original hue information in fruit region.For this purpose, the present invention passes through iterative Retinex
Algorithm, it is automatic to enhance according to the Luminance Distribution feature in fruit region in input image lightness information and illumination abundance/homogeneous image
The brightness of input picture.
By taking litchi fruits as an example.
When algorithm starts, if current RGB image I is input RGB image Ic.When compensating the luminance information of I, in order to the greatest extent may be used
The original hue information of image can not be changed, the weighting three primary colors luminance component V of I is extracted according to formula (1).
V=α R+ β G+ γ B
Wherein, R, G and B respectively indicate the grey scale pixel value of the red, green and blue Color Channel of I, and s.t. indicates constraint condition.
Using red color tone litchi fruits region detection as target in the present embodiment, and three parameters being arranged in formula (1) be respectively α=
0.6, β=0.3, γ=0.1, to protrude the grayscale information of red component in image.
In order to isolate the background area different from fruit hue information, setting red component (R) and green (G) from I
Or the ratio of blue component (B), as impact factor, the factor value is higher, illustrates the difference between fruit and other background areas
It is different bigger, and be weighted in conventional color difference drawing method, obtain the calculating of the modified R X chromaticity difference diagram as shown in formula (2)~(4)
Model extracts chromaticity difference diagram C from Im。
Cm=λ × R-X (2)
λ=R/X (3)
In order to extract red color tone litchi fruits region, β > γ is set in the present embodiment.
C is divided using Otsu algorithmmTo obtain corresponding bianry imageIn order to guarantee that the region of subsequent extracted is
Litchi fruits part is corroded by the circular configuration member that radius is more than 8 pixels in the present embodimentIt is secondary corresponding to obtain one
The mask images in fruit region, the luminance picture in litchi fruits region is extracted by the point multiplication operation of mask images and V, and is calculated
The average value M of the luminance pictureRoIs.On the other hand, it is manually chosen from the training image data comprising red color tone litchi fruits
Fruit region, and calculate the weighting three primary colors luminance component and average brightness value M in these regionstrain。
Compare MRoIsAnd Mtrain:
If MRoIs≥Mtrain, then not executing the brightness based on Retinex algorithm to V enhances, directly output I '=Ic；
If MRoIs< Mtrain, then executing the brightness based on Retinex algorithm to V enhances, and obtains new luminance component V ', passes through
RGB/HSI color space conversion model extraction IcChrominance component H and color saturation component S, and merge V ' generation brightness enhancing
RGB image I ' afterwards, enables I againc=I ' continues according to above-mentioned workflow management IcMRoIs, enhance I in an iterative mannercIt is bright
Degree, until meeting condition MRoIs≥MtrainWhen, export the enhanced RGB image I ' of brightness.
Fig. 2 a~Fig. 2 d gives with above-mentioned iterative manner using the effect contrast figure of Retinex algorithm before and after the processing.Figure
2a is the image of a secondary illumination abundance, and brightness reinforcing effect is as shown in Figure 2 b, and Fig. 2 c is the weaker image of a secondary illumination, bright
It is as shown in Figure 2 d to spend reinforcing effect.
By formula (2)~(4) from the middle extraction modified R X chromaticity difference diagram C of I 'm', using Otsu algorithm to Cm' execute thresholding
Segmentation obtains bianry image Cm＂ successively uses the erosion operation based on coarse scale structures member (radius can be used for 4 pixels
Circular configuration member), dilation operation based on small scale structures member (radius can be used first for the circular configuration of 3 pixels) and hole
Filling processing Cm＂, by 8- connectivity criteria from treated Cm" in extract potential fruit region, and be labeled as Of.Above-mentioned " big ruler
" big " and " small " in very little structural elements ", " small scale structures member " is in processing CmThe erosion operation and dilation operation carried out when ＂
What the structural elements of use were compared.Fig. 3 gives through treated litchi fruits extracted region effect picture, and Fig. 3 a is
The litchi fruits extraction effect implementation example figure of image shown in Fig. 2 a, Fig. 3 b are that the litchi fruits extraction effect of image shown in Fig. 2 c is real
Apply example diagram.
In order to guarantee the accurate extraction in sepia branch region, the present invention is given birth to again using local hue information restructing algorithm
At filtered chrominance component, to eliminate the noise jamming in input picture.By RGB/HSI color space conversion model from I '
Middle extraction chrominance component H '.In any m × n neighborhood of H ', if the coordinate and tone value of the centre of neighbourhood are respectively (xc,yc) and
hc, the coordinate and tone value respectively (x of i-th of (i=1,2 ..., m × n) pixeli,yi) and hi, i-th is determined according to formula (5)
A pixel (xi,yi) with the positional relationship w of the centre of neighbourhoodpi。
wpi=exp {-λpmax(dis((xi,yi),(xc,yc)))},(xi,yi)≠(xc,yc) (5)
Wherein, λpFor position amplification coefficient, λ is taken in the present embodimentp=2；dis((xi,yi),(xc,yc)) it is metric point
(xi,yi) and (xc,yc) the distance between function namely ith pixel taken in the present embodiment at a distance from the centre of neighbourhood chessboard away from
From i.e. dis ((xi,yi),(xc,yc))=max (| xi-xc|,|yi-yc|)。
In addition, another key factor for influencing the estimation of m × n centre of neighbourhood hue information is the tone value of neighborhood territory pixel, it is
Reduce influence of the centre of neighbourhood polluted by noise to its hue information reconstruction result, the present invention, which measures, removes center in neighborhood
Pixel tone standard deviation sigma outside pointcAnd calculating field intermediate value hm, local neighborhood pixel (x is measured by formula (6) and (7)i,yi)
With hmDifference on hue information, result queue wHi。
Wherein, λHFor tone difference amplification coefficient, λ is taken in the present embodimentH=1；diff(hi,hc) be ith pixel color
Tone pitch hiWith the difference of neighborhood tone intermediate value hm, diff (h is taken in the present embodimenti,hc)=| | hi-hc||2, NcFor except point (xc,yc)
Outer neighborhood territory pixel set.
Since the tone value of centre of neighbourhood pixel can be influenced by the position of m × n neighborhood territory pixel and tone value simultaneously, this
It invents while considering wpiAnd wHiCarry out reconstruction point (xc,yc) hue information, allow it closer to practical tone value, adopted in the present embodiment
Neighborhood territory pixel (x is formed with the mode of dot producti,yi) weight, i.e. wci=wpi×wHi.Particularly, point (xc,yc) before reconstitution
Tone may be interfered by noise, so for (xi,yi)=(xc,yc) situation, if hcWith hmDeviation is larger, then w is arrangedci
=0, otherwise w is setci=1.Finally the weighted mean of m × n neighborhood hue information is calculated as point (x according to formula (8)c,yc)
Reconstruction result.
After whole pixels to H ' complete above-mentioned calculating, the reconstructed image H ' of H ' can be obtainedc.Fig. 4 gives m × n
The centre of neighbourhood pixel tonal value reconstruct numerical operation result figure, Fig. 4 a be centre of neighbourhood pixel not by noise pollution when tone
Be worth reconstruction result, Fig. 4 b be centre of neighbourhood pixel by noise pollution when tone value reconstruction result, take m=5 in the present embodiment,
N=5.Each grid in Fig. 4 a and Fig. 4 b represents a pixel, and the first row numerical value in grid indicates the point before reconstitution
Tone value, the numerical value in the second row bracket indicates contribution weight of the hue information of the point in restructuring procedure, neighborhood midpoint
In the third line numerical value indicate the tone value of point after reconstitution.
Litchi branch strip area is manually extracted from training image data, and counts these areas after being transformed into HSI color space
The average tone value h in domainavg, as to H 'cCarry out the segmentation threshold of binary conversion treatment, H 'cMiddle tone value is less than or equal to havgPicture
Element is set to prospect, is otherwise set to background, and potential branch region O is extracted in the foreground image obtained by as 8- connectivity criterias。
Fig. 5 gives the effect picture of litchi branch strip area extraction, and Fig. 5 a is the lichee branch extraction effect embodiment of image shown in Fig. 2 a
Figure, Fig. 5 b are the lichee branch extraction effect implementation example figure of image shown in Fig. 2 c.
O is handled using thinning algorithms, extract the skeleton O of litchi branch strip areas', O is detected by Harris algorithms' in
All angle points (set its quantity as k).In order to judge whether certain angle point is located at branch bifurcation, all angle points are traversed, for jth
(j=1,2 ..., k) a angle point, if its neighborhood territory pixel meets mode distribution rule shown in formula (9) simultaneously, by angle point j
Coordinate be added to angled key point set AcIn, otherwise delete angle point j.
Wherein, N8(Pi) indicate that 8 neighborhoods of angle point j (do not include the pixel quantity that value is 1 in angle point j), P2、P4、P6With
P8For the 4 neighborhood territory pixel values of angle point j, and they are located at top, right, lower section and the left of angle point j.
To OfMorphological dilations operation (radius can be used for the circular configuration of 3 pixels member) is executed, O is obtainedf', by Os′
With Of' region superposition is carried out, therefrom search for the join domain A of litchi fruits and branchr, and calculate ArMiddle all pixels point row is sat
Target mean value Pr1With the mean value P of column coordinater2.Then according to the opposite position of litchi fruits under gravity and litchi branch strip area
It sets and adhesion relation, calculates AcThe mean value P of middle key angle point row coordinatec1With the mean value P of column coordinatec2If AcFor empty set or Pr1
< Pc1, from Os' middle rejecting and Of' do not have connectivity branch region, extract only with OfBranch skeleton with connectivity.Most
Afterwards, from AcThe middle the smallest angle point of search row coordinate value, and centered on the angle point, with Pr2With Pc2Between absolute difference be standard
Difference determines the harvesting point distributed area of litchi fruits.Fig. 6 a, Fig. 6 b give the effect picture of litchi fruits harvesting point location, figure
6a is that the litchi fruits of image shown in Fig. 2 a harvest point location effect example figure, and Fig. 6 b is the litchi fruits of image shown in Fig. 2 c
Harvest point location effect example figure.
The same or similar label correspond to the same or similar components；
Positional relationship is described in attached drawing only for illustration, should not be understood as the limitation to this patent；
Obviously, the above embodiment of the present invention be only to clearly illustrate example of the present invention, and not be pair
The restriction of embodiments of the present invention.For those of ordinary skill in the art, may be used also on the basis of the above description
To make other variations or changes in different ways.There is no necessity and possibility to exhaust all the enbodiments.It is all this
Made any modifications, equivalent replacements, and improvements etc., should be included in the claims in the present invention within the spirit and principle of invention
Protection scope within.
Claims (10)
1. a kind of fruit harvesting region automatic positioning method towards natural scene, which comprises the following steps:
(1) the weighting three primary colors luminance component V for extracting current RGB image I, utilizes modified R X chromaticity difference diagram (Modified Red
And Green/Blue (X) Chromatic Mapping, MRXCM) method generates modified R X (Red&Green/Blue) from I
Chromaticity difference diagram Cm, from CmMiddle extraction part fruit region RoIs (Regions of Interest) calculates the system of RoIs Luminance Distribution
Feature is counted, Retinex algorithm processing V is iteratively used according to statistical nature and reconstructs I, is passed through when iterative process terminates
Color space conversion model exports the enhanced RGB image I ' of brightness；
(2) modified R X chromaticity difference diagram C is therefrom extracted by MRXCM methodm', to Cm' execution imagethresholding obtains Cm＂, and
C is handled using mathematical Morphology Algorithmm＂, from CmPotential mature or immature fruit region O is extracted in ＂f；
(3) the chrominance component H ' for extracting I ', measures the positional relationship of local neighborhood pixel and the centre of neighbourhood, measures local neighborhood picture
Element and the hue information difference of the centre of neighbourhood, establish the weight distribution model of local neighborhood centering heart pixel tonal value, by every
From by noise pollution centre of neighbourhood pixel, chrominance component H is reconstructed in a manner of weighted meanc', to Hc' execute image threshold
Segmentation obtains Hc＂ therefrom extracts potential sepia branch region Os；
(4) O is extractedsSkeleton Qs', detect Qs' on angle point and extract crucial angle point relevant to branch bifurcated attribute, utilization
O under gravityfAnd Qs' relative position constraint and crucial angle point spatial characteristics, position fruit harvesting point distributed area
Between.
2. the fruit harvesting region automatic positioning method according to claim 1 towards natural scene, which is characterized in that step
Suddenly (1) the weighting three primary colors luminance component V for extracting current RGB image I, specifically, respectively indicating I's with variable R, G and B
The grey scale pixel value of red, green and blue Color Channel calculates weighting three primary colors luminance component V according to the following formula:
V=α R+ β G+ γ B
3. the fruit harvesting region automatic positioning method according to claim 1 towards natural scene, which is characterized in that step
Suddenly (1) it is described using modified R X chromaticity difference diagram (Modified Red and Green/Blue (X) Chromatic Mapping,
MRXCM) method generates modified R X (Red&Green/Blue) chromaticity difference diagram C from Im, specifically, shown according to fruit region
Red color tone or green hue calculate the monochromatic ratio of R and G or R and B as scale factor, and are weighted to conventional red green difference figure
Or in reddish blue difference figure, modified chromaticity difference diagram C is generatedm。
4. the fruit harvesting region automatic positioning method according to claim 1 towards natural scene, which is characterized in that step
Suddenly (1) is described from CmMiddle extraction part fruit region RoIs (Regions of Interest) calculates the system of RoIs Luminance Distribution
Feature is counted, specifically, to CmIt executes Threshold segmentation and erosion operation obtains bianry image, extract local fruit region RoIs conduct
Mask images, and calculate the average brightness value M of mask images institute overlay area in VRoIs。
5. the fruit harvesting region automatic positioning method according to claim 4 towards natural scene, which is characterized in that step
Suddenly (1) is described iteratively uses Retinex algorithm processing V according to statistical nature and reconstructs I, logical when iterative process terminates
The color space conversion model output enhanced RGB image I ' of brightness is crossed, specifically, the three primary colors for extracting multiple training images add
Luminance component is weighed, the average brightness value M in fruit region is countedtrain, and execute following steps:
S1. judge whether MRoIs≥Mtrain, if so then execute step S2, S3 is thened follow the steps if not；
S2. I as the enhanced RGB image I ' of brightness and is exported into I '；
S3. enhanced luminance component V ' is obtained using Retinex algorithm, passes through RGB/HSI color space conversion model extraction
The tone H and color saturation S of current RGB image merge V ' to generate the enhanced RGB image I ' of brightness, extract I's ' again
Average MRoIs' judges whether MRoIs' >=MtrainIf otherwise continuing to execute step S3, if continuing to execute step S4；
S4. I ' is exported.
6. the fruit harvesting region automatic positioning method according to claim 1 towards natural scene, which is characterized in that step
Suddenly (3) positional relationship for measuring local neighborhood pixel and the centre of neighbourhood, specifically, being (x at field centerc,yc) m × n
In neighborhood, coordinates computed is (xi,yi) ith pixel and centre of neighbourhood distance dis ((xi,yi),(xc,yc)), under
Formula obtains the positional relationship w of ith pixel and the centre of neighbourhoodpi:
wpi=exp {-λPmax(dis((xi,yi),(xc,yc)))}
In formula, λpFor position amplification coefficient, i=1,2 ..., m × n.
7. the fruit harvesting region automatic positioning method according to claim 1 or 6 towards natural scene, feature exist
In step (3) the hue information difference for measuring local neighborhood pixel and the centre of neighbourhood, specifically, in field centre coordinate
For (xc,yc) m × n neighborhood in, calculate neighborhood in coordinate be (xi,yi) ith pixel tone value hiIn neighborhood tone
Value hmDifference diff (hi,hm), and the neighborhood territory pixel tone value standard deviation sigma in addition to central pointc, calculate according to the following formula i-th
The different information w of pixel and the centre of neighbourhoodHi:
(xi,yi)≠(xc,yc)
In formula, λHFor tone difference amplification coefficient, i=1,2 ..., m × n.
8. the fruit harvesting region automatic positioning method according to claim 7 towards natural scene, which is characterized in that step
Suddenly (3) the weight distribution model for establishing local neighborhood centering heart pixel tonal value, by the way that the neighborhood by noise pollution is isolated
Center pixel reconstructs chrominance component H ' in a manner of weighted meanc, specifically, passing through wpiAnd wHiPoint multiplication operation distribute i-th
The weight w of pixelciIf centre of neighbourhood tone value hcWith hmDeviation is larger, then the weight that field center is arranged is 0, is otherwise arranged
It is 1, calculates the weighted average of each pixel tonal value in neighborhood, the reconstruct tone value h ' as centre of neighbourhood pixelc, based on upper
State the chrominance component H ' reconstructed after all pixels of process traversal H 'c。
9. the fruit harvesting region automatic positioning method according to claim 1 towards natural scene, which is characterized in that step
Suddenly (4) the extraction OsSkeleton Qs', detect Qs' on angle point and extract crucial angle point relevant to branch bifurcated attribute, have
Body is to extract O using thinning algorithmsSkeleton Qs', Q is extracted using Harris Corner Detection Algorithms' on whole angle points, root
According to the bifurcated feature between fruit branches different in the case of more fruit adhesions, rule are distributed using the mode of crotch angle point neighborhood territory pixel
Then judge whether crotch angle point belongs to crucial angle point and extract crucial angle point with skeleton connectivity.
10. the fruit harvesting region automatic positioning method according to claim 1 towards natural scene, which is characterized in that
Step (4) is described to utilize O under gravityfAnd Qs' relative position constraint and crucial angle point spatial characteristics, position fruit
Real harvesting point distributed area, specifically, connecting branch, fruit carpopodium and branch lower than it according to fruit position of centre of gravity under gravity
Strong connectedness feature between item calculates carpopodium and branch join domain ArThe statistical nature of middle all pixels space coordinate, and tie
Close crucial angular coordinate statistical nature extract only with OfBranch skeleton with connectivity extracts separate carpopodium on the skeleton
Harvesting central point of the crucial angle point as fruit, with ArMiddle pixel and the dispersion degree of angled key space of points column coordinate are as deviation
The regional standard for harvesting point is poor, positions fruit harvesting point distributed area according to harvesting central point and regional standard difference.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811062603.9A CN109359531B (en) | 2018-09-12 | 2018-09-12 | Fruit harvesting area automatic positioning method facing natural scene |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811062603.9A CN109359531B (en) | 2018-09-12 | 2018-09-12 | Fruit harvesting area automatic positioning method facing natural scene |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109359531A true CN109359531A (en) | 2019-02-19 |
| CN109359531B CN109359531B (en) | 2021-12-14 |
Family
ID=65350966
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811062603.9A Active CN109359531B (en) | 2018-09-12 | 2018-09-12 | Fruit harvesting area automatic positioning method facing natural scene |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109359531B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112541860A (en) * | 2019-09-23 | 2021-03-23 | 深圳开阳电子股份有限公司 | Skin color beautifying correction method and device |
| CN118058074A (en) * | 2024-02-20 | 2024-05-24 | 广东若铂智能机器人有限公司 | Method for judging burst interference in string-type fruit picking process |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101726251A (en) * | 2009-11-13 | 2010-06-09 | 江苏大学 | Automatic fruit identification method of apple picking robot on basis of support vector machine |
| CN103727877A (en) * | 2013-12-20 | 2014-04-16 | 北京农业信息技术研究中心 | Fruit identifying and locating method, device and system |
| CN104700404A (en) * | 2015-03-02 | 2015-06-10 | 中国农业大学 | Fruit location identification method |
| CN105373794A (en) * | 2015-12-14 | 2016-03-02 | 河北工业大学 | Vehicle license plate recognition method |
| CN105701829A (en) * | 2016-01-16 | 2016-06-22 | 常州大学 | Bagged green fruit image segmentation method |
| US9462749B1 (en) * | 2015-04-24 | 2016-10-11 | Harvest Moon Automation Inc. | Selectively harvesting fruits |
| CN106326894A (en) * | 2016-08-31 | 2017-01-11 | 西南交通大学 | Adverse state detection method of transverse pins of rotation double lugs of high-speed rail overhead contact line equipment |
| CN106327443A (en) * | 2016-08-24 | 2017-01-11 | 电子科技大学 | Night image enhancement method based on improved genetic algorithm |
| CN108319973A (en) * | 2018-01-18 | 2018-07-24 | 仲恺农业工程学院 | Detection method for citrus fruits on tree |
-
2018
- 2018-09-12 CN CN201811062603.9A patent/CN109359531B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101726251A (en) * | 2009-11-13 | 2010-06-09 | 江苏大学 | Automatic fruit identification method of apple picking robot on basis of support vector machine |
| CN103727877A (en) * | 2013-12-20 | 2014-04-16 | 北京农业信息技术研究中心 | Fruit identifying and locating method, device and system |
| CN104700404A (en) * | 2015-03-02 | 2015-06-10 | 中国农业大学 | Fruit location identification method |
| US9462749B1 (en) * | 2015-04-24 | 2016-10-11 | Harvest Moon Automation Inc. | Selectively harvesting fruits |
| CN105373794A (en) * | 2015-12-14 | 2016-03-02 | 河北工业大学 | Vehicle license plate recognition method |
| CN105701829A (en) * | 2016-01-16 | 2016-06-22 | 常州大学 | Bagged green fruit image segmentation method |
| CN106327443A (en) * | 2016-08-24 | 2017-01-11 | 电子科技大学 | Night image enhancement method based on improved genetic algorithm |
| CN106326894A (en) * | 2016-08-31 | 2017-01-11 | 西南交通大学 | Adverse state detection method of transverse pins of rotation double lugs of high-speed rail overhead contact line equipment |
| CN108319973A (en) * | 2018-01-18 | 2018-07-24 | 仲恺农业工程学院 | Detection method for citrus fruits on tree |
Non-Patent Citations (5)
| Title |
|---|
| CHENGLIN WANG等: "A robust fruit image segmentation algorithm against varying illumination for vision system of fruit harvesting robot", 《OPTIK》 * |
| 刘阳: "自然环境下目标物的高速图像检测算法研究", 《中国博士学位论文全文数据库 信息科技辑》 * |
| 李毅 等: "尺度变化的Retinex红外图像增强", 《液晶与显示》 * |
| 沈甜: "苹果采摘机器人重叠果实快速动态识别及定位研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
| 熊俊涛 等: "基于Retinex图像增强的不同光照条件下的成熟荔枝识别", 《农业工程学报》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112541860A (en) * | 2019-09-23 | 2021-03-23 | 深圳开阳电子股份有限公司 | Skin color beautifying correction method and device |
| CN118058074A (en) * | 2024-02-20 | 2024-05-24 | 广东若铂智能机器人有限公司 | Method for judging burst interference in string-type fruit picking process |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109359531B (en) | 2021-12-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Dias et al. | Multispecies fruit flower detection using a refined semantic segmentation network | |
| Hu et al. | Automatic detection of single ripe tomato on plant combining faster R-CNN and intuitionistic fuzzy set | |
| CN107578418B (en) | Indoor scene contour detection method fusing color and depth information | |
| CN105718945B (en) | Apple picking robot night image recognition method based on watershed and neural network | |
| CN108319973B (en) | Detection method for citrus fruits on tree | |
| CN104598908B (en) | A kind of crops leaf diseases recognition methods | |
| Liu et al. | A method of segmenting apples at night based on color and position information | |
| CN108573276A (en) | A kind of change detecting method based on high-resolution remote sensing image | |
| CN105574514B (en) | The raw tomato automatic identifying method in greenhouse | |
| CN108537239A (en) | A kind of method of saliency target detection | |
| CN110120042A (en) | A kind of crop map based on SLIC super-pixel and automatic threshold segmentation is as pest and disease damage method for extracting region | |
| Salazar-Gonzalez et al. | Retinal blood vessel segmentation via graph cut | |
| Ji et al. | In-field automatic detection of maize tassels using computer vision | |
| CN102542560B (en) | Method for automatically detecting density of rice after transplantation | |
| CN110599507A (en) | Tomato identification and positioning method and system | |
| Changhui et al. | Overlapped fruit recognition for citrus harvesting robot in natural scenes | |
| CN109359531A (en) | Fruit harvesting area automatic positioning method facing natural scene | |
| Thenmozhi et al. | A novel plant leaf ailment recognition method using image processing algorithms | |
| CN111882573A (en) | Cultivated land plot extraction method and system based on high-resolution image data | |
| CN118351468A (en) | Crop pest and disease damage identification method and system based on unmanned aerial vehicle | |
| Patil | Pomegranate fruit diseases detection using image processing techniques: a review | |
| Tran et al. | Automatic dragon fruit counting using adaptive thresholds for image segmentation and shape analysis | |
| Hashim et al. | An intelligent classification model for rubber seed clones based on shape features through imaging techniques | |
| CN117475373A (en) | Tea garden pest and disease damage identification and positioning method and system based on binocular vision | |
| Li et al. | Image processing for crop/weed discrimination in fields with high weed pressure |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |