CN103279762A - Judging method for common growth form of fruit under natural environment - Google Patents
Judging method for common growth form of fruit under natural environment Download PDFInfo
- Publication number
- CN103279762A CN103279762A CN2013101883464A CN201310188346A CN103279762A CN 103279762 A CN103279762 A CN 103279762A CN 2013101883464 A CN2013101883464 A CN 2013101883464A CN 201310188346 A CN201310188346 A CN 201310188346A CN 103279762 A CN103279762 A CN 103279762A
- Authority
- CN
- China
- Prior art keywords
- fruit
- image
- leaves
- branches
- blocked
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The invention discloses a judging method for the common growth form of a fruit under natural environment, which concretely comprises the an image collecting step of collecting a fruit image based on binocular vision in real time; a target object extracting step of processing the collected image to extract the fruit and leaves in the image; a target object depth computing step of obtaining the depth information of a target object area by adopting a combined match and depth correcting model; and a fruit growth form judging step of finally determining the category to which the fruit belongs by adopting a method of being coarse and fine in sequence to judge the fruit growth form. As to picking robots for picking fruits such as apples and citrus, the judging method for the common growth form of the fruit under natural environment can enable the robots to automatically finish the classification of the fruit with different growth forms in the collected image, thus finishing the precise identification on the fruit by adopting a corresponding method.
Description
Technical field
The present invention relates to the common growthform decision method of fruit under a kind of physical environment, Taobao relates to a kind of common growthform decision method to class fruits such as apple, oranges and tangerines.
Background technology
For picking robot, because the non-structured characteristics of natural working environment, existing much influence the accurately factors of identification of fruit, wherein the fruit adjacency, overlappingly block that to block with branches and leaves be principal element.The fruit adjacency is exactly that two or more fruits abut against together mutually.Fruit is overlapping to be blocked, and as its name suggests, looks from vision sensor image acquisition direction exactly, and two or more fruits overlap but not necessarily contact and cause rear fruit part to be blocked by the place ahead fruit.Obviously, fruit is overlapping to be blocked and comprises fruit in abutting connection with (fruit and can be regarded as the fruit of broad sense by adjacency overlapping), and in order to narrate convenience, being generally called this class fruit is the overlapping fruit that blocks.It then is to block from the vision sensor image acquisition direction fruit that is caused by tree branch and leaf of looking that branches and leaves block fruit, and it can be divided into the fruit edge again and be blocked to be blocked by branches and leaves with fruit by branches and leaves and be divided into two or polylith and namely seriously blocked fruit two classes by branches and leaves.They all are the common growthforms of fruit, its recognition methods all is different naturally, yet there is not the common growthform decision method of a kind of perfect fruit, so picking robot can't self-adaptation adopts correlation method to finish the accurate identification of different growthform fruits, the sluggish practicalization of picking robot.
Summary of the invention
Common growthform decision method above shortcomings at fruit in the prior art, the invention provides the common growthform decision method of fruit under a kind of physical environment, can make picking robot finish the classification of gathering different growthform fruits in the image automatically, thereby can adopt correlation method to realize its accurate identification, for the practicability that promotes picking robot plays an important role.
Technical scheme of the present invention is:
The common growthform decision method of fruit under a kind of physical environment specifically may further comprise the steps:
1) image acquisition step: based on the real-time collecting fruit image of binocular vision.
2) destination object extraction step: at first adopt the pre-service of adaptive wiener filter method image; Next employing is peeled off dividing method layer by layer based on the dynamic threshold of color characteristic the garbage in the pretreatment image is removed; Adopt cluster segmentation algorithm based on color characteristic and textural characteristics to obtain fruit, branch and leaf in the image then, wherein the Contourlet transform method is adopted in the extraction of textural characteristics.Cutting apart the cut apart fragment of back in the image then adopts based on the noise-eliminating method of textural characteristics and removes, the minimum boundary rectangle method of employing level is at last confined connected regions all in the image, extract the isolated area in the supplement image in each rectangle, superpose to repair hole by isolated area image and original image.
3) destination object depth calculation step: adopt combinations matches and depth correction model determination to go out the depth information of connected region in each minimum boundary rectangle based on binocular vision, remove for the zone that exceeds outside the picking robot depth of implements, in addition, this depth information also is used for subsequent treatment.
4) fruit growth form determination step: adopting the smart decision method in thick back earlier, is that benchmark determines roughly that connected region is single fruit or the overlapping fruit that blocks in the fruit image with different depth apart from fruit length and width and area data interval range at first namely.
If single fruit, adopt how much its place rectangles of computing method fast detecting and other rectangle to have or not to overlap or have or not other rectangle in certain area coverage: if exist coincidence regional or other rectangle arranged, employing detects to overlap based on the zone of stem and leaf plot picture mapping detection method and has or not in the zone between zone or two rectangles that branches and leaves are further rough judges that belonging to category-A still is C class fruit, if slightly be judged to the category-A fruit, adopt the dual scanning line emargintion to detect and detect the method that combines based on the zone mapping of stem and leaf plot picture and judge that accurately belonging to category-A still is the category-B fruit, if slightly be judged to C class fruit, then adopt with quadrat method and judge that accurately belonging to the C class still is D class fruit; If do not have to overlap zone or other rectangle, then directly adopt the dual scanning line emargintion to detect and detect the method that combines based on the zone mapping of stem and leaf plot picture and judge that accurately belonging to category-A still is the category-B fruit.
Slightly declaring the result for beginning is overlapping accurate determination step and method of blocking fruit, and it is equal to the accurate determination step of above-mentioned single fruit and method, and different is final accurately result of determination, i.e. E class or F class or G class or H class fruit.
Beneficial effect of the present invention is:
For class picking fruit machine people such as apple, oranges and tangerines, the common growthform decision method of fruit can make it finish the classification of gathering different growthform fruits in the image automatically under a kind of physical environment of the present invention, thereby can adopt correlation method to finish its accurate identification.
Description of drawings
Fig. 1 is the general flow chart of the common growthform decision method of fruit under a kind of physical environment of the present invention;
Fig. 2 is the process flow diagram of destination object extraction step among the present invention;
Fig. 3 is the process flow diagram of fruit growth form determination step among the present invention.
Embodiment
Below in conjunction with accompanying drawing the present invention is described in further detail.
The steps flow chart of the common growthform decision method of fruit specifically comprises the steps: as shown in Figure 1 under a kind of physical environment of the present invention
(1) image acquisition step
The collection of image except subsequent extracted goes out the two-dimensional signal of destination object, also will be obtained the depth information of destination object based on binocular vision system.
(2) destination object extraction step
This step implementing procedure as shown in Figure 2.The at first polytrope of illumination under the physical environment is having a strong impact on the segmentation effect of image, so this step adopts the pre-service of adaptive wiener filter method image, to eliminate the following noise of being gathered in the image of different illumination conditions such as high light, the low light level.
In the image except fruit, branches and leaves information; also sky may be arranged; (orchard is retaining for the preservation of soil moisture for the orchard mulch film; improve the fruit color index; usually can covering with plastic film) etc. garbage; and sky is together interlaced with the fruit tree branches and leaves, earlier it is removed from pretreatment image so the employing of this step is peeled off dividing method layer by layer based on the dynamic threshold of color characteristic.
Although there is bigger color distinction in the image between fruit, the branches and leaves, but when target is similar to background color, only utilize color characteristic intactly the fruit target to be split, so-called over-segmentation or less divided phenomenon can occur, so this step adopts cluster segmentation algorithm based on color characteristic and textural characteristics to obtain fruit, branch and leaf in the image.Here the method for Contourlet conversion is adopted in the extraction of textural characteristics.By utilizing Contourlet conversion high-frequency sub-band matrix of coefficients, choose the gradient energy of high-frequency sub-band all directions as proper vector.Gradient energy can characterize the inherent continuity of texture image well.
Cut apart to exist inevitably in the image of back and cut apart fragment, so this step adopts de-noising based on textural characteristics in the fruit, branch and the leaf image that split (for target image, non-target information all can be described as noise) method, to guarantee the pure property of target information, be applied to better in the follow-up fruit growth form differentiation and go.
Also can there be hole phenomenon in various degree in the image after cutting apart inevitably, its operation times needs manual intervention to traditional mathematical morphology hole fill method because pore size differs, therefore this step is confined connected regions all in the image according to the minimum boundary rectangle method of the at first employing level of actual conditions of successive image processing, extract the isolated area in the supplement image in each rectangle then, superpose to repair hole by isolated area image and original image.
(3) destination object depth calculation step
Some objective fruit position of possibility has exceeded the depth of implements of picking robot in the fruit image, need not carry out subsequent treatment again, therefore this step adopts combinations matches and depth correction model determination to go out the depth information of connected region in each minimum boundary rectangle based on binocular vision, remove for the zone that exceeds outside the picking robot depth of implements, in addition, this depth information also is used for subsequent treatment.
(4) fruit growth form determination step
The smart decision method in thick back is earlier adopted in the judgement of fruit growth form, determine at first roughly namely in the fruit image that connected region is single fruit or the overlapping fruit that blocks in the minimum boundary rectangle, judge accurately progressively then this connected region belongs to following which classification.
A. single nothing is blocked fruit;
B. single only edge is blocked fruit by branches and leaves;
C. single but seriously blocked fruit by branches and leaves;
D. single but blocked seriously simultaneously by branches and leaves that the edge is also blocked fruit by branches and leaves;
E. the overlapping fruit that blocks;
F. overlapping and the edge is blocked fruit by branches and leaves;
G. overlapping and seriously blocked fruit by branches and leaves;
H. the overlapping while is seriously blocked by branches and leaves and the edge is also blocked fruit by branches and leaves.
This step implementing procedure as shown in Figure 3, fruit growth form determination step specifically comprises the steps:
Step 1: different depth is measured apart from the wide data of fruit length and width and area and branch footpath, the statistical study measurement result, with it as background data base.Be benchmark with different depth in the database apart from fruit length and width and area data interval range, come in the rough process decision chart picture that the connected region fruit is that single fruit still is the overlapping fruit that blocks in the minimum boundary rectangle.
Step 2: slightly declare the result for Step 1, adopt how much its place rectangles of computing method fast detecting and other rectangle to have or not to overlap or have or not other rectangle in certain area coverage (getting the footpath wide region of branch under this depth distance).Overlap the zone or other rectangle is arranged if exist, change Step 3 over to and further judge; If do not have the zone of coincidence or other rectangle, change Step 4 over to and further judge.
Step 3: overlap the zone or other rectangle is arranged if exist, whether employing detects to overlap based on the zone of stem and leaf plot picture mapping detection method and has or not branches and leaves to exist between zone or two rectangles in the zone, divided to determine two rectangle connected region fruits seriously to be blocked by branches and leaves.If there are branches and leaves to exist, then can determine roughly and belong to C or G class fruit (even Step 1 slightly declares the result and is single fruit, then belongs to C class fruit; Otherwise, belong to G class fruit.Following being equal to, repeat no more.）。If the existence of no branches and leaves, then can determine roughly and belong to A or E class fruit.
Step 4: if do not have the zone of coincidence or other rectangle, adopt dual scanning line (horizontal scanning line and vertical scan line) emargintion detection algorithm that place rectangle connected region is detected.If there is emargintion, adopt the zone mapping detection method based on the stem and leaf plot picture to detect again, detecting has branches and leaves to exist in the zone, thereby can accurately judge and belong to category-B or F class fruit.If the non-flanged breach then can accurately be judged to belong to A or E class fruit.
Step 5: slightly declare the result for Step 3, adopt Step 4 methods to detect the fruit edge and whether be blocked, belong to D or H class fruit and A or E class fruit thereby can accurately determine.
The above only is preferred embodiment of the present invention, not in order to limit the present invention.All any modifications of doing within the spirit and principles in the present invention, be equal to and replace and improvement etc., all should be included within protection scope of the present invention.
Claims (4)
1. the common growthform decision method of fruit under the physical environment specifically may further comprise the steps:
(1) image acquisition step: based on the real-time collecting fruit image of binocular vision;
(2) destination object extraction step: the image of gathering is handled, extracted fruit and branches and leaves in the image;
(3) destination object depth calculation step: adopt combinations matches and depth correction model to obtain its depth information to the destination object zone;
(4) fruit growth form determination step: the smart decision method in thick back is earlier adopted in the judgement of fruit growth form, determines finally it belongs to following which classification:
A. single nothing is blocked fruit;
B. single only edge is blocked fruit by branches and leaves;
C. single but seriously blocked fruit by branches and leaves;
D. single but blocked seriously simultaneously by branches and leaves that the edge is also blocked fruit by branches and leaves;
E. the overlapping fruit that blocks;
F. overlapping and the edge is blocked fruit by branches and leaves;
G. overlapping and seriously blocked fruit by branches and leaves;
H. the overlapping while is seriously blocked by branches and leaves and the edge is also blocked fruit by branches and leaves.
2. the common growthform decision method of fruit under a kind of physical environment according to claim 1 is characterized in that: at first adopt the pre-service of adaptive wiener filter method image in the step (2); Next employing is peeled off dividing method layer by layer based on the dynamic threshold of color characteristic the garbage in the pretreatment image is removed; Adopt cluster segmentation algorithm based on color characteristic and textural characteristics to obtain fruit, branch and leaf in the image then, wherein the Contourlet transform method is adopted in the extraction of textural characteristics; Cutting apart the cut apart fragment of back in the image then adopts based on the noise-eliminating method of textural characteristics and removes, the minimum boundary rectangle method of employing level is at last confined connected regions all in the image, extract the isolated area in the supplement image in each rectangle, superpose to repair hole by isolated area image and original image.
3. the common growthform decision method of fruit under a kind of physical environment according to claim 1, it is characterized in that: adopt combinations matches and depth correction model determination to go out the depth information of connected region in each minimum boundary rectangle based on binocular vision in the step (3), remove for the zone that exceeds outside the picking robot depth of implements, in addition, this depth information also is used for subsequent treatment.
4. the common growthform decision method of fruit under a kind of physical environment according to claim 1, it is characterized in that: adopting the smart decision method in thick back earlier in the step (4), is that benchmark determines roughly that connected region is single fruit or the overlapping fruit that blocks in the fruit image with different depth apart from fruit length and width and area data interval range at first namely;
If single fruit, adopt how much its place rectangles of computing method fast detecting and other rectangle to have or not to overlap or have or not other rectangle in certain area coverage: if exist coincidence regional or other rectangle arranged, employing detects to overlap based on the zone of stem and leaf plot picture mapping detection method and has or not in the zone between zone or two rectangles that branches and leaves are further rough judges that belonging to category-A still is C class fruit, if slightly be judged to the category-A fruit, adopt the dual scanning line emargintion to detect and detect the method that combines based on the zone mapping of stem and leaf plot picture and judge that accurately belonging to category-A still is the category-B fruit, if slightly be judged to C class fruit, then adopt with quadrat method and judge that accurately belonging to the C class still is D class fruit; If do not have to overlap zone or other rectangle, then directly adopt the dual scanning line emargintion to detect and detect the method that combines based on the zone mapping of stem and leaf plot picture and judge that accurately belonging to category-A still is the category-B fruit;
Slightly declaring the result for beginning is overlapping accurate determination step and method of blocking fruit, and its accurate determination step and method with single fruit is identical, and its difference is the final result who accurately judges, i.e. E class or F class or G class or H class fruit.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310188346.4A CN103279762B (en) | 2013-05-21 | 2013-05-21 | Common growth form of fruit decision method under a kind of physical environment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310188346.4A CN103279762B (en) | 2013-05-21 | 2013-05-21 | Common growth form of fruit decision method under a kind of physical environment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103279762A true CN103279762A (en) | 2013-09-04 |
| CN103279762B CN103279762B (en) | 2016-04-13 |
Family
ID=49062277
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310188346.4A Active CN103279762B (en) | 2013-05-21 | 2013-05-21 | Common growth form of fruit decision method under a kind of physical environment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103279762B (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104299261A (en) * | 2014-09-10 | 2015-01-21 | 深圳大学 | Three-dimensional imaging method and system for human body |
| CN104700404A (en) * | 2015-03-02 | 2015-06-10 | 中国农业大学 | Fruit location identification method |
| CN105719282A (en) * | 2016-01-16 | 2016-06-29 | 常州大学 | Fruit, branch and leaf region obtaining method of red apple images in garden |
| CN107966944A (en) * | 2017-11-30 | 2018-04-27 | 贵州财经大学 | Smart greenhouse zone control system and subregion picking method |
| CN108901540A (en) * | 2018-06-28 | 2018-11-30 | 重庆邮电大学 | Fruit tree light filling and fruit thinning method based on artificial bee colony fuzzy clustering algorithm |
| CN110017778A (en) * | 2019-04-25 | 2019-07-16 | 广州富港万嘉智能科技有限公司 | The dimension measurement method and peeling method of melon and fruit, corresponding device and storage medium |
| CN110197129A (en) * | 2019-05-09 | 2019-09-03 | 常州大学 | A kind of fruits and vegetables growthform visual recognition method under natural environment |
| CN111045373A (en) * | 2019-12-31 | 2020-04-21 | 北京黎明文仪家具有限公司 | Furniture monitoring system and method based on cloud service |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101726251A (en) * | 2009-11-13 | 2010-06-09 | 江苏大学 | Automatic fruit identification method of apple picking robot on basis of support vector machine |
| CN102113434A (en) * | 2011-01-14 | 2011-07-06 | 江苏大学 | Picking method of picking robot under fruit oscillation condition |
-
2013
- 2013-05-21 CN CN201310188346.4A patent/CN103279762B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101726251A (en) * | 2009-11-13 | 2010-06-09 | 江苏大学 | Automatic fruit identification method of apple picking robot on basis of support vector machine |
| CN102113434A (en) * | 2011-01-14 | 2011-07-06 | 江苏大学 | Picking method of picking robot under fruit oscillation condition |
Non-Patent Citations (1)
| Title |
|---|
| 吕继东: "苹果采摘机器人视觉测量与避障控制研究", 《万方数据企业知识服务平台》, 26 April 2013 (2013-04-26), pages 26 - 92 * |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104299261A (en) * | 2014-09-10 | 2015-01-21 | 深圳大学 | Three-dimensional imaging method and system for human body |
| US10019838B2 (en) | 2014-09-10 | 2018-07-10 | Shenzhen University | Human body three-dimensional imaging method and system |
| CN104700404A (en) * | 2015-03-02 | 2015-06-10 | 中国农业大学 | Fruit location identification method |
| CN104700404B (en) * | 2015-03-02 | 2018-03-02 | 中国农业大学 | A kind of fruit positioning identifying method |
| CN105719282A (en) * | 2016-01-16 | 2016-06-29 | 常州大学 | Fruit, branch and leaf region obtaining method of red apple images in garden |
| CN105719282B (en) * | 2016-01-16 | 2018-06-08 | 常州大学 | A kind of orchard mcintosh image fruit branches and leaves area obtaining method |
| CN107966944A (en) * | 2017-11-30 | 2018-04-27 | 贵州财经大学 | Smart greenhouse zone control system and subregion picking method |
| CN108901540A (en) * | 2018-06-28 | 2018-11-30 | 重庆邮电大学 | Fruit tree light filling and fruit thinning method based on artificial bee colony fuzzy clustering algorithm |
| CN110017778A (en) * | 2019-04-25 | 2019-07-16 | 广州富港万嘉智能科技有限公司 | The dimension measurement method and peeling method of melon and fruit, corresponding device and storage medium |
| CN110017778B (en) * | 2019-04-25 | 2021-07-06 | 广州富港万嘉智能科技有限公司 | Melon and fruit size measuring method and peeling method, corresponding device and storage medium |
| CN110197129A (en) * | 2019-05-09 | 2019-09-03 | 常州大学 | A kind of fruits and vegetables growthform visual recognition method under natural environment |
| CN111045373A (en) * | 2019-12-31 | 2020-04-21 | 北京黎明文仪家具有限公司 | Furniture monitoring system and method based on cloud service |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103279762B (en) | 2016-04-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103310218B (en) | A kind of overlap blocks fruit precise recognition method | |
| CN103279762A (en) | Judging method for common growth form of fruit under natural environment | |
| CN103295018A (en) | Method for precisely recognizing fruits covered by branches and leaves | |
| CN103336946B (en) | A kind of cluster shape tomato recognition methods based on binocular stereo vision | |
| CN109636784B (en) | Image saliency target detection method based on maximum neighborhood and super-pixel segmentation | |
| CN102622755B (en) | Plant limb identification method | |
| CN103870808A (en) | Finger vein identification method | |
| Sunny et al. | An efficient citrus canker detection method based on contrast limited adaptive histogram equalization enhancement | |
| CN104318546A (en) | Multi-scale analysis-based greenhouse field plant leaf margin extraction method and system | |
| CN103400134A (en) | Non-contact method for extracting region of interest of finger vein sample | |
| CN105719275A (en) | Parallel combination image defect segmentation method | |
| Feng et al. | A separating method of adjacent apples based on machine vision and chain code information | |
| Nagar et al. | Pest detection on leaf using image processing | |
| Ren et al. | Effective SAR sea ice image segmentation and touch floe separation using a combined multi-stage approach | |
| CN104636716B (en) | Green fruit recognition methods | |
| CN101788685B (en) | Remote sensing earthquake damage information extracting and digging method based on pixels | |
| Zhang et al. | Cherry recognition in natural environment based on the vision of picking robot | |
| CN104408407A (en) | Night identification method of apple harvesting robot | |
| Lv et al. | Method to acquire regions of fruit, branch and leaf from image of red apple in orchard | |
| Mohammed Amean et al. | Automatic plant branch segmentation and classification using vesselness measure | |
| CN107194320A (en) | A kind of greenhouse green pepper picking robot target identification method based on image characteristic analysis | |
| CN105844641A (en) | Adaptive threshold segmentation method in dynamic environment | |
| CN115731257A (en) | Leaf form information extraction method based on image | |
| Sun et al. | A vision system based on TOF 3D imaging technology applied to robotic citrus harvesting | |
| Nejati et al. | Kiwifruit detection in challenging conditions |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20201225 Address after: Building 7, no.8-2, Dutou street, Daitou Town, Liyang City, Changzhou City, Jiangsu Province Patentee after: Liyang Chang Technology Transfer Center Co.,Ltd. Address before: Gehu Lake Road Wujin District 213164 Jiangsu city of Changzhou province No. 1 Patentee before: CHANGZHOU University |
|
| TR01 | Transfer of patent right |