CN114258781B - Morphology and color space-based strawberry stem picking point positioning method - Google Patents
Morphology and color space-based strawberry stem picking point positioning method Download PDFInfo
- Publication number
- CN114258781B CN114258781B CN202210009044.5A CN202210009044A CN114258781B CN 114258781 B CN114258781 B CN 114258781B CN 202210009044 A CN202210009044 A CN 202210009044A CN 114258781 B CN114258781 B CN 114258781B
- Authority
- CN
- China
- Prior art keywords
- fruit
- picking
- coordinates
- area
- point
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Harvesting Machines For Specific Crops (AREA)
- Image Processing (AREA)
Abstract
The invention relates to a method for positioning strawberry fruit stem picking points based on morphology and color space, and belongs to the field of mobile robot target identification and positioning. The method comprises the following steps: s1: identifying the positions of strawberry fruits and fruit stalks by using a Mask RCNN network; s2: adopting HSV color space, customizing a strawberry fruit maturity judging rule, judging the maturity of the identified fruit area, reserving a mature fruit area, and discarding an immature fruit area; s3: firstly calculating the gravity center of a target fruit, then calculating an included angle generated by the gravity center of the fruit and the fruit stalks, and judging the membership of the fruit stalks and the fruit by adopting a self-defined fruit stalk picking point positioning method; if the membership is met, calculating the optimal picking point position of the mature fruit corresponding to the fruit stalks. The invention can realize accurate picking point positioning among strawberries in different forms, improves the recognition precision and reduces the calculation cost.
Description
Technical Field
The invention belongs to the field of mobile robot target identification and positioning, relates to a deep neural network model multi-target detection and geometric morphology positioning method, and particularly relates to a morphology and color space-based strawberry stem picking point positioning method for an intelligent mobile picking robot.
Background
Due to the continuous and strong strawberry industry and the rapid development of modern agriculture, strawberry automatic picking robots are widely used, and the main aims of the current strawberry picking robots to improve strawberry picking efficiency and enhance strawberry picking accuracy are achieved.
At present, two main methods exist for positioning the picking points of strawberries by an intelligent mobile automatic picking robot. The first is that the vision system uses machine learning typical algorithms to identify picking points, for example using SVM (support vector machine), RVM (correlation vector machine), etc. The method identifies the strawberry fruits in the image through a classification algorithm, then utilizes the artificial experience to manually and directly define the fruit stalks and the picking point positions thereof, and the picking point positions are usually positioned above the identified strawberry fruits. Although the method has high recognition speed, as only fruits are recognized and fruit stalks are not really recognized, the positions of the fruit stalks and picking points are directly inferred through experience, and the accuracy of the algorithm recognition result is low. When a complex fruit scene is encountered, the method has larger positioning deviation on the fruit stalks and the picking points, so that the fruits are cut or the fruits are damaged in the falling process after the fruit stalks are cut. Another mainstream method is to identify only the fruit stalks in the image, and then calculate the center of gravity of the fruit stalks to define picking point positions. Although the method accurately identifies the fruit stalks, the relationship between the fruit stalks and the fruits is eliminated. In the actual picking process, the method is easy to pick immature fruits and has great loss. Strawberries belong to the berry class of fruits, and are usually picked by a gripper or a cutting robot due to the soft nature of the fruits. The shape of the strawberries has the characteristic of diversity, and the identification result of the classification algorithm is the minimum rectangle of the area where the fruits or fruit stalks are located. The hand-grabbing robot cannot confirm the specific outline of the fruits and the fruit stalks, and the fruits are easy to damage in the picking process. The method for locating the fruit stalks and picking points by using the manual experience values has no universality. The cutting robots often cut fruit stalks of other fruits in an empty or staggered way, so that the capability and fruit resources of the picking robots are greatly wasted, and the farm picking efficiency cannot be improved.
In order to solve the above problems, a method capable of accurately and automatically picking strawberries is needed.
Disclosure of Invention
Therefore, the invention aims to provide the morphological and color space-based strawberry stem picking point positioning method for the intelligent mobile picking robot, which solves the difficulty of deep learning in application on stem identification and greatly improves the intelligent agricultural fruit picking efficiency by matching with a correct picking point positioning algorithm. The invention applies the multi-task target detection deep neural network to the recognition of strawberry fruits and fruit stalks, can accurately recognize the shapes of the strawberries and the fruit stalks, and improves the positioning accuracy of the robot to the fruit stalk picking points.
In order to achieve the above purpose, the present invention provides the following technical solutions:
a strawberry fruit stalk picking point positioning method based on morphology and color space specifically comprises the following steps:
s1: identifying the positions of strawberry fruits and fruit stalks by using a Mask RCNN network;
s2: adopting HSV color space, customizing a strawberry fruit maturity judging rule, judging the maturity of the identified fruit area, reserving a mature fruit area, and discarding an immature fruit area;
s3: firstly calculating the gravity center of a target fruit, then calculating an included angle generated by the gravity center of the fruit and the fruit stalks, and judging the membership of the fruit stalks and the fruit by adopting a self-defined fruit stalk picking point positioning method; if the membership is met, calculating the optimal picking point position of the mature fruit corresponding to the fruit stalks.
Further, in step S2, the rule for determining the maturity of the strawberry fruit is defined, which specifically includes: and defining scales for three channels H, S, V by using HSV color space, dividing H value distribution intervals of red, white and green, analyzing and calculating pixel values of red, white and green and total distribution values in a fruit area, and finally calculating maturity attribution of the values in a custom fuzzy rule.
Further, in step S2, the H value distribution intervals of the red, white, and green are divided into:
red: h E (330,22)
Green: h E (75,155)
White: h E (45,60)
Wherein, red and green are divided in the non-extreme ranges of S and V, namely S is less than or equal to 0.1, and V is more than or equal to 0.2;
the red, white and green pixel values and the overall distribution value D in the fruit area are analyzed and calculated as follows:
wherein H is C D is the number of red, green or white pixels to be determined C The total number of pixels in the fruit area; when calculating D, if the value D of the currently calculated color is located in the interval [67%,100%]The state of D for this color is defined as L; similarly, if the value D of the currently calculated color is within the interval [34%,66 ]]The state of D for this color is defined as M; the value D of the currently calculated color lies in the interval [0,33%]The state of D for this color is defined as S'.
Further, in step S3, the center of gravity of the target fruit is calculated, including: acquiring information of a fruit area judged to be mature, and dividing the area into a limited triangle; then the center of gravity of the triangle formed by every two adjacent points is calculated, and so on, the irregular polygon is decomposed into a limited number of triangles.
Further, in step S3, the center of gravity of the target fruit is calculated, specifically including: acquiring coordinate information of a fruit area, selecting any point outside the fruit area as an origin point P (0, 0) of the two-dimensional coordinate system, establishing the two-dimensional coordinate system, and taking each pixel point as a coordinate point; two adjacent points A (x 1 ,y 1 ) And B (x) 2 ,y 2 ) Is connected with the origin to construct a triangle, and the three vertex positions of the triangle are A (x 1 ,y 1 )，B(x 2 ,y 2 ) P (0, 0), area of triangle is:
the G-coordinate of the center of gravity of the triangle is:
by analogy, the fruit area X can be divided into stacks X of n triangles 1 ，X 2 ，…，X n The gravity center of the triangles is G i Area is S i The center of gravity G (X, y) coordinates of the fruit area X are:
wherein, (G) ix ,G iy ) Representing the barycentric coordinates of the ith triangle.
Further, in step S3, determining the membership of the fruit stalks and the fruits specifically includes: converting the obtained fruit stalk image pixel information into coordinates of two-dimensional coordinates; obtaining coordinates of the highest point and the lowest point of the fruit stalk area by adopting a traversing method; calculate the highest point M (x) 1 ,y 1 ) Minimum point N (x 2 ,y 2 ) And (3) judging that the fruit stalks are the corresponding fruit pickable fruit stalks when the included angles with the horizontal straight line where the gravity center G (x, y) of the fruit is located are smaller than 180 degrees.
Further, in step S3, calculating the optimal picking point position of the mature fruit corresponding to the fruit stalk specifically includes: after the center of gravity of the fruit is obtained and the fruit stalk membership is judged, connecting the highest point M (x 1 ,y 1 ) And the lowest point N (x 2 ,y 2 ) Calculating the coordinate of the midpoint Q of the two-point connecting line:defining Q as an optimal picking point; the horizontal straight line range where the midpoint is located is the optimal picking or cutting range of the picking stage robot.
Further, in step S3, the calculated optimal picking point of the fruit stalks is converted from the pixel coordinates to the camera coordinates corresponding to the intelligent mobile robot, so as to obtain the coordinates of the camera corresponding to the optimal picking point of the fruit stalks, which specifically includes:
(1) Acquiring calculated pixel coordinates of the picking points: q= (u, v);
(2) Converting the pixel coordinates of the picking points into image plane coordinates;
let the physical dimensions of each pixel in the u-axis and v-axis directions be d x And d y The following formula is obtained:
wherein d x And d y Representing the actual size of pixels on a camera photosensitive chip, and connecting a pixel coordinate system with a real-size coordinate system; u (u) 0 And v 0 Representing the center of the image plane;
obtaining the image plane coordinates of the picking points through the matrix change: p= (x, y);
(3) Converting the image plane coordinates of the picking points into camera coordinates of the picking robot;
camera parameters are known: camera focal length f, camera optical axis: the z-axis, camera coordinates are: p (P) c ＝(x w ,y w ,z w ) The method comprises the steps of carrying out a first treatment on the surface of the The conversion relationship between the image plane coordinates and the camera coordinates is as follows:
wherein z is c The position coordinates of the actual points of the camera are known by manual measurement; x is x c And y is c Is the axis of the coordinate plane in which the camera is located. Thus, the specific position of the picking point in the camera coordinate system is obtained.
The invention has the beneficial effects that: when the strawberry peduncles are identified and the peduncles are picked and positioned, compared with the prior art, the method provided by the invention can accurately identify the target object by dividing the fruit and the peduncles. For picking strawberries, the identification precision is greatly improved, and the fruit is positioned more accurately by the picking robot. Meanwhile, due to the fruit stem picking point positioning algorithm, in an actual use scene, the picking range meeting the picking requirement can be obtained. Finally, the invention can overcome the defect of different picking point calculation methods caused by strawberries in different forms, provides a general calculation mode, and improves the general capability and the picking efficiency of the intelligent mobile picking robot.
The invention can also realize picking point positioning among strawberries in different forms, and reduce the calculation cost. The robot picking device is beneficial to improving the accuracy of picking points, saving hardware resources of the robot, and adapting the robot to picking work of various scenes.
Additional advantages, objects, and features of the invention will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the invention. The objects and other advantages of the invention may be realized and obtained by means of the instrumentalities and combinations particularly pointed out in the specification.
Drawings
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in the following preferred detail with reference to the accompanying drawings, in which:
FIG. 1 is a block diagram of the whole flow of a method for positioning a picking point of a strawberry fruit stalk;
FIG. 2 is a block diagram of a multi-tasking deep neural network;
FIG. 3 is a flow chart for judging the maturity of a fruit area;
FIG. 4 is a schematic diagram of fruit stem and fruit membership judgment;
fig. 5 is a schematic diagram of a specific positioning method of the fruit stem picking point positioning algorithm.
Detailed Description
Other advantages and effects of the present invention will become apparent to those skilled in the art from the following disclosure, which describes the embodiments of the present invention with reference to specific examples. The invention may be practiced or carried out in other embodiments that depart from the specific details, and the details of the present description may be modified or varied from the spirit and scope of the present invention. It should be noted that the illustrations provided in the following embodiments merely illustrate the basic idea of the present invention by way of illustration, and the following embodiments and features in the embodiments may be combined with each other without conflict.
Referring to fig. 1 to 5, fig. 1 shows a method for positioning a strawberry stem picking point based on morphology and color space, which specifically comprises the following steps:
step 1: the segmented image is acquired using a Mask RCNN multitasking object detection network.
Taking the strawberry image as a target image, and inputting the target image into a Mask RCNN network; and obtaining the identification result which contains two classifications of fruits and fruit stalks and meets the expectations.
Based on a deep neural network of multi-target task detection, target detection and example segmentation can be combined, a plurality of target objects can be identified at the same time, and the identification result is a specific outline of fruits and fruit stalks; meanwhile, the deep neural network based on multi-target task detection is quick in recognition time, high in accuracy, and extremely friendly to the mobile robot with the priority of resource utilization rate, and recognition results of all target objects are not mutually interfered. Aiming at the hand-grabbing type picking robot, the network method can accurately identify the edge outline of the fruit, so that the damage to the fruit in the picking process is avoided; for cutting type picking robots, the network method can accurately position picking points or picking ranges, and the fruit is prevented from being accidentally injured in the cutting process.
Step 2: and judging the maturity of the identified fruits. The process comprises the following 3 steps.
(1) Acquiring all fruit and fruit stalk information output by a network;
(2) The fruit information is converted into HSV color space, the pixel values of red, white and green and the distribution situation thereof are calculated, and it is known that in the HSV color space, H E [0,360], S E [0,255], V E [0,255], because the HSV color space is close to the perception of colors by human eyes, the HSV color space has no strict color interval division, H represents hue, namely various colors perceived by human eyes, S represents saturation, is the color shade perceived by human eyes, V is brightness, and the saturation influences the brightness perceived by human eyes. Thus, the color is routinely distinguished, largely depending on the hue. Therefore, default S and V are in the non-extreme range (non-black), and the H value distribution intervals for dividing red, white, and green are respectively:
red: h E (330,22)
Green: h E (75,155)
White: h E (45,60)
Wherein, red and green are divided in the non-extreme ranges of S and V, namely S is less than or equal to 0.1, and V is more than or equal to 0.2.
According to daily experience, the white color of the immature strawberry fruit is approaching off-white, namely, the white color doped with yellow color, and V epsilon (0.9,1) is judged when the white color is judged.
The HSV values are converted to calculated values, and the converted HSV ranges are as follows.
H∈[0,180],S∈[0,255],V∈[0,255]
In the conversion process, the fractional part was computationally discarded and partially combined, so that the obtained red, white, and green values were approximately distributed as shown in table 1.
TABLE 1 HSV color space color channel distribution Condition
As shown in table 1, white, red and green are selected as the distinguished color channels, and the distribution ratio of the HSV values in the actual fruit area information in the three color intervals is calculated; and then, the duty ratios of the three color channels are corresponding to the fuzzy rule, and the maturity of the three color channels is judged. For example, if the total number of the obtained fruit area is 100, the number of pixels determined to be red is calculated to be 32, the number of pixels determined to be green is 30, and the number of pixels determined to be white is 27, the percentage of three colors to the total number of pixels is calculated to determine that red is S, green is S, and white is S, then the fruit area maturity is immature. The specific fuzzy rules are shown in table 2.
TABLE 2 fuzzy rule
The color value interval corresponding to L, M, S' is defined, and the specific values are shown in table 3 below.
TABLE 3L, M, S' distribution intervals
Namely:
wherein H is C D is the number of red/green/white pixels to be determined C The total number of pixels in the fruit area.
(3) And reserving the mature fruit information and discarding the immature fruit information according to the custom fuzzy rule.
Step 3: center of gravity calculation was performed on the identified mature fruit. The process comprises the following 2 steps.
(1) Acquiring target area pixel information of ripe fruits and fruit stalks detected by a network and judged by ripeness;
(2) And converting the pixel information of the fruit area into coordinate information, and calculating and determining the gravity center position. As known from botanicals, the peduncles of plants are mostly located above the center of gravity of the fruits. The fruits identified by example segmentation are in irregular shapes, and the gravity center position cannot be directly calculated. Firstly, acquiring coordinate information of a fruit area, selecting any point outside the fruit area as an origin point P (0, 0) of the two-dimensional coordinate system, establishing the two-dimensional coordinate system, and taking each pixel point as a coordinate point. Two adjacent points A (x 1 ,y 1 ) And B (x) 2 ,y 2 ) Is connected with the origin to construct a triangle, and the three vertex positions of the triangle are A (x 1 ,y 1 )，B(x 2 ,y 2 ) P (0, 0), area of triangle is:
the G-coordinate of the center of gravity of the triangle is:
by analogy, the fruit area X can be divided into stacks X of n triangles 1 ，X 2 ，…，X n The gravity center of the triangles is G i Area is S i The center of gravity G (X, y) coordinates of the fruit area X are:
wherein, (G) ix ,G iy ) Representing the barycentric coordinates of the ith triangle.
Step 4: judging whether the identified fruit stalks are the cuttable fruit stalks corresponding to the fruits, and if so, calculating the optimal picking points of the identified fruit stalks. The process includes 5 steps.
(1) Judging whether the fruit stalks are the fruit stalks which can be picked corresponding to the target fruits. Firstly, acquiring position information of a fruit stalk region, and acquiring coordinates of a lowest point and a highest point of the fruit stalk region by adopting a traversing method;
(2) Calculate the highest point M (x 1 ,y 1 ) And the lowest point N (x 2 ,y 2 ) Included angle with the horizontal straight line where the center of gravity of the fruit is located. Connecting the lowest point, the highest point and the calculated center of gravity of the fruit, and calculating an included angle between the calculated center of gravity and a horizontal straight line where the center of gravity is located;
(3) Calculating an included angle, and judging whether the fruit corresponds to the fruit stem or not; when the two included angles between the highest point and the lowest point of the fruit stalks and the center of gravity of the fruit are smaller than 180 degrees, judging that the fruit stalks are the fruit stalks corresponding to the fruit meeting morphological requirements;
(4) And (5) confirming the fruit stem picking points. The highest point M (x) 1 ,y 1 ) And the lowest point N (x 2 ,y 2 ) Calculating coordinates of a midpoint Q of the line segment:the midpoint is the optimal picking point of the fruit stalks. In an actual scene, most of fruit stalks are in a curved irregular shape, the calculated picking points are not necessarily positioned on the actual fruit stalks, and a horizontal straight line where the picking points are positioned is taken as an optimal picking cutting range;
(5) And calculating the camera coordinates of the picking robot corresponding to the picking points in actual picking.
a) Acquiring calculated pixel coordinates of the picking points: q= (u, v).
b) The pixel coordinates of the picking points are converted into image plane coordinates.
Let the physical dimensions of each pixel in the u-axis and v-axis directions be d x And d y The following formula can be obtained:
wherein d x And d y Representing the actual size of pixels on a camera photosensitive chip, and connecting a pixel coordinate system with a real-size coordinate system; u (u) 0 And v 0 Representing the center of the image plane.
Obtaining the image plane coordinates of the picking points through the matrix change: p= (x, y).
c) And converting the image plane coordinates of the picking points into camera coordinates of the picking robot.
Camera parameters are known: f (focal length of camera)) The optical center of the camera is located on the axis: the z-axis, camera coordinates are: p (P) c ＝(x w ,y w ,z w ) The method comprises the steps of carrying out a first treatment on the surface of the The conversion relationship between the image plane coordinates and the camera coordinates is as follows:
wherein z is c The position coordinates of the actual points of the camera are known by manual measurement; x is x c And y is c Is the axis of the coordinate plane in which the camera is located. Thus, the specific position of the picking point in the camera coordinate system is obtained.
The embodiment designs a multi-task detection depth convolution neural network and an optimal fruit stalk picking point positioning method capable of realizing real-time strawberry fruit and fruit stalk identification on mobile robot edge computing equipment. Based on the existing multi-task target detection deep neural network, the method can solve the difficulty of deep learning in application to fruit stem identification, and greatly improves the efficiency of intelligent agricultural fruit picking by matching with a correct picking point positioning algorithm. Therefore, the invention applies the multi-task target detection deep neural network to the recognition of the strawberry fruits and the fruit stalks, can accurately recognize the shapes of the strawberries and the fruit stalks, and improves the positioning accuracy of the robot to the fruit stalk picking points.
Finally, it is noted that the above embodiments are only for illustrating the technical solution of the present invention and not for limiting the same, and although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications and equivalents may be made thereto without departing from the spirit and scope of the present invention, which is intended to be covered by the claims of the present invention.
Claims (6)
1. The strawberry fruit stem picking point positioning method based on morphology and color space is characterized by comprising the following steps of:
s1: identifying the positions of strawberry fruits and fruit stalks by using a Mask RCNN network;
s2: adopting HSV color space, customizing a strawberry fruit maturity judging rule, judging the maturity of the identified fruit area, reserving a mature fruit area, and discarding an immature fruit area;
s3: firstly calculating the gravity center of a target fruit, then calculating an included angle generated by the gravity center of the fruit and the fruit stalks, and judging the membership of the fruit stalks and the fruit by adopting a self-defined fruit stalk picking point positioning method; if the membership is met, calculating the optimal picking point position of the mature fruit corresponding to the fruit stalks;
calculating the center of gravity of the target fruit, comprising: acquiring information of a fruit area judged to be mature, and dividing the area into a limited triangle; then calculating the gravity centers of triangles formed by every two adjacent points, and so on, decomposing the irregular polygon into a limited triangle;
judging the membership of the fruit stalks and the fruits, which comprises the following steps: converting the obtained fruit stalk image pixel information into coordinates of two-dimensional coordinate system points; obtaining coordinates of the highest point and the lowest point of the fruit stalk area by adopting a traversing method; calculate the highest point M (x) 1 ,y 1 ) Minimum point N (x 2 ,y 2 ) And (3) judging that the fruit stalks are the corresponding fruit pickable fruit stalks when the included angles with the horizontal straight line where the gravity center G (x, y) of the fruit is located are smaller than 180 degrees.
2. The method for positioning strawberry stalk picking points according to claim 1, wherein in step S2, the rule for determining the maturity of the strawberry fruit is customized, specifically comprising: and defining scales for three channels H, S, V by using HSV color space, dividing H value distribution intervals of red, white and green, analyzing and calculating pixel values of red, white and green and total distribution values in a fruit area, and finally calculating maturity attribution of the values in a custom fuzzy rule.
3. The method according to claim 2, wherein in step S2, the H-value distribution intervals of red, white and green are divided into:
red: h E (330,22)
Green: h E (75,155)
White: h E (45,60)
Wherein, red and green are divided in the non-extreme ranges of S and V, namely S is less than or equal to 0.1, and V is more than or equal to 0.2;
the red, white and green pixel values and the overall distribution value D in the fruit area are analyzed and calculated as follows:
wherein H is C D is the number of red, green or white pixels to be determined C The total number of pixels in the fruit area; if the value D of the currently calculated color is within the interval [67%,100%]The state of D for this color is defined as L; similarly, if the value D of the currently calculated color is within the interval [34%,66%]The state of D for this color is defined as M; the value D of the currently calculated color lies in the interval [0,33%]The state of D for this color is defined as S'.
4. The method according to claim 1, wherein in step S3, the center of gravity of the target fruit is calculated, specifically comprising: acquiring coordinate information of a fruit area, selecting any point outside the fruit area as an origin point P (0, 0) of the two-dimensional coordinate system, establishing the two-dimensional coordinate system, and taking each pixel point as a coordinate point; two adjacent points A (x 1 ,y 1 ) And B (x) 2 ,y 2 ) Is connected with the origin to construct a triangle, and the three vertex positions of the triangle are A (x 1 ,y 1 )，B(x 2 ,y 2 ) P (0, 0), area of triangle is:
the G-coordinate of the center of gravity of the triangle is:
similarly, the fruit area X is divided into n superimposed triangles X 1 ，X 2 ，…，X n The gravity center of the triangles is G i Area is S i The center of gravity G (X, y) coordinates of the fruit area X are:
wherein, (G) ix ,G iy ) Representing the barycentric coordinates of the ith triangle.
5. The method according to claim 1, wherein in step S3, the optimal picking point position of the ripe fruit corresponding to the fruit stem is calculated, and the method specifically comprises: after the center of gravity of the fruit is obtained and the fruit stalk membership is judged, connecting the highest point M (x 1 ,y 1 ) And the lowest point N (x 2 ,y 2 ) Calculating the coordinate of the midpoint Q of the two-point connecting line: q=12 (x) 2 -x 1 ,y 2 -y 1 ) Defining Q as the optimal picking point; the horizontal straight line range where the midpoint is located is the optimal picking or cutting range of the picking stage robot.
6. The method according to claim 5, wherein in step S3, the calculated optimal picking point of the fruit stalks is converted from the pixel coordinates to the camera coordinates corresponding to the intelligent mobile robot, so as to obtain the coordinates of the camera corresponding to the optimal picking point of the fruit stalks, and specifically comprising:
(1) Acquiring calculated pixel coordinates of the picking points: q= (u, v);
(2) Converting the pixel coordinates of the picking points into image plane coordinates;
let the physical dimensions of each pixel in the u-axis and v-axis directions be d x And d y The following formula is obtained:
wherein d x And d y Representing the actual size of pixels on a camera photosensitive chip, and connecting a pixel coordinate system with a real-size coordinate system; u (u) 0 And v 0 Representing the center of the image plane;
obtaining the image plane coordinates of the picking points through the matrix change: p= (x, y);
(3) Converting the image plane coordinates of the picking points into camera coordinates of the picking robot;
camera parameters are known: camera focal length f, camera optical axis: the z-axis, camera coordinates are: p (P) c ＝(x w ,y w ,z w ) The method comprises the steps of carrying out a first treatment on the surface of the The conversion relationship between the image plane coordinates and the camera coordinates is as follows:
wherein z is c The position coordinates of the actual points of the camera; x is x c And y is c Is the axis of the coordinate plane in which the camera is located.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210009044.5A CN114258781B (en) | 2022-01-06 | 2022-01-06 | Morphology and color space-based strawberry stem picking point positioning method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210009044.5A CN114258781B (en) | 2022-01-06 | 2022-01-06 | Morphology and color space-based strawberry stem picking point positioning method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114258781A CN114258781A (en) | 2022-04-01 |
| CN114258781B true CN114258781B (en) | 2023-07-21 |
Family
ID=80832447
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210009044.5A Active CN114258781B (en) | 2022-01-06 | 2022-01-06 | Morphology and color space-based strawberry stem picking point positioning method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114258781B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115250744B (en) * | 2022-07-29 | 2023-09-15 | 四川启睿克科技有限公司 | Multi-angle strawberry picking system and method |
| CN117152544B (en) * | 2023-10-31 | 2024-03-15 | 锐驰激光(深圳)有限公司 | Tea-leaf picking method, equipment, storage medium and device |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2012110256A (en) * | 2010-11-22 | 2012-06-14 | National Agriculture & Food Research Organization | Fruit stem cutter |
| CN109711325A (en) * | 2018-12-25 | 2019-05-03 | 华南农业大学 | A kind of mango picking point recognition methods |
| CN112990103A (en) * | 2021-04-16 | 2021-06-18 | 苏州大学 | String mining secondary positioning method based on machine vision |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5119392B2 (en) * | 2007-02-26 | 2013-01-16 | 井関農機株式会社 | Fruit harvesting robot and strawberry cultivation facility |
| JP5213038B2 (en) * | 2008-08-28 | 2013-06-19 | 独立行政法人農業・食品産業技術総合研究機構 | Strawberry quality measuring method and strawberry quality measuring device |
| CN109271867B (en) * | 2018-08-20 | 2021-11-02 | 浙江荣亚工贸有限公司 | Automatic judgment method for strawberry maturity |
| CN111079530A (en) * | 2019-11-12 | 2020-04-28 | 青岛大学 | Mature strawberry identification method |
| CN111666883B (en) * | 2020-06-08 | 2023-04-07 | 佛山科学技术学院 | Grape picking robot target identification and fruit stalk clamping and cutting point positioning method |
| CN111684924A (en) * | 2020-06-23 | 2020-09-22 | 王孟超 | Strawberry picking robot based on raspberry group |
-
2022
- 2022-01-06 CN CN202210009044.5A patent/CN114258781B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2012110256A (en) * | 2010-11-22 | 2012-06-14 | National Agriculture & Food Research Organization | Fruit stem cutter |
| CN109711325A (en) * | 2018-12-25 | 2019-05-03 | 华南农业大学 | A kind of mango picking point recognition methods |
| CN112990103A (en) * | 2021-04-16 | 2021-06-18 | 苏州大学 | String mining secondary positioning method based on machine vision |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114258781A (en) | 2022-04-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114258781B (en) | Morphology and color space-based strawberry stem picking point positioning method | |
| CN105718945B (en) | Apple picking robot night image recognition method based on watershed and neural network | |
| Wang et al. | Localisation of litchi in an unstructured environment using binocular stereo vision | |
| EP1655620B1 (en) | Obstacle detection using stereo vision | |
| CN114170228B (en) | Computer image edge detection method | |
| CN114067206B (en) | Spherical fruit identification positioning method based on depth image | |
| CN113255434B (en) | Apple identification method integrating fruit characteristics and deep convolutional neural network | |
| CN111783693A (en) | Intelligent identification method of fruit and vegetable picking robot | |
| CN107993224A (en) | A kind of object detection localization method based on circle marker thing | |
| CN109584258A (en) | Meadow Boundary Recognition method and the intelligent mowing-apparatus for applying it | |
| CN101996317A (en) | Method and device for identifying markers in human body | |
| CN111369497B (en) | Walking type tree fruit continuous counting method and device | |
| CN111612797B (en) | Rice image information processing system | |
| He et al. | Zanthoxylum bungeanum fruit detection by adaptive thresholds in HSV space for an automatic picking system | |
| CN109886062A (en) | A kind of camellia oleifera fruit flower identification positioning system | |
| CN117237354A (en) | Visual detection method for defects of textile clothes | |
| CN116740097A (en) | Image processing method | |
| CN111079530A (en) | Mature strawberry identification method | |
| CN112183230A (en) | Identification and central point positioning method for pears in natural pear orchard environment | |
| Yu et al. | Design and implementation of an automatic peach-harvesting robot system | |
| CN115512119A (en) | Greenhouse fresh jujube maturity recognition and picking system based on artificial intelligence | |
| CN111340897A (en) | Color identification method based on fuzzy mathematical similarity | |
| CN115601690A (en) | Edible fungus environment detection method based on intelligent agriculture | |
| CN114842469A (en) | Self-adaptive identification method and system for mature fruits | |
| Chen et al. | Measurement of the distance from grain divider to harvesting boundary based on dynamic regions of interest |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |