CN114258781A - Strawberry stem picking point positioning method based on morphology and color space - Google Patents

Strawberry stem picking point positioning method based on morphology and color space Download PDF

Info

Publication number
CN114258781A
CN114258781A CN202210009044.5A CN202210009044A CN114258781A CN 114258781 A CN114258781 A CN 114258781A CN 202210009044 A CN202210009044 A CN 202210009044A CN 114258781 A CN114258781 A CN 114258781A
Authority
CN
China
Prior art keywords
fruit
picking
stem
strawberry
coordinates
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202210009044.5A
Other languages
Chinese (zh)
Other versions
CN114258781B (en
Inventor
郑太雄
赵思远
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chongqing University of Post and Telecommunications
Original Assignee
Chongqing University of Post and Telecommunications
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chongqing University of Post and Telecommunications filed Critical Chongqing University of Post and Telecommunications
Priority to CN202210009044.5A priority Critical patent/CN114258781B/en
Publication of CN114258781A publication Critical patent/CN114258781A/en
Application granted granted Critical
Publication of CN114258781B publication Critical patent/CN114258781B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Landscapes

  • Harvesting Machines For Specific Crops (AREA)
  • Image Processing (AREA)

Abstract

The invention relates to a strawberry fruit stem picking point positioning method based on morphology and color space, and belongs to the field of target identification and positioning of mobile robots. The method comprises the following steps: s1: identifying positions of strawberry fruits and fruit stalk areas by using a Mask RCNN network; s2: defining a strawberry fruit maturity judgment rule by self by adopting an HSV color space, judging the maturity of the identified fruit region, reserving a mature fruit region, and abandoning an immature fruit region; s3: a self-defined fruit stem picking point positioning method is adopted, the gravity center of a target fruit is calculated, then the included angle formed by the gravity center of the fruit and the fruit stem is calculated, and the membership relationship between the fruit stem and the fruit is judged; and if the membership is satisfied, calculating the optimal picking point position of the mature fruit corresponding to the fruit stem. The invention can realize accurate picking point positioning among strawberries with different forms, improve the identification precision and reduce the calculation cost.

Description

Strawberry stem picking point positioning method based on morphology and color space
Technical Field
The invention belongs to the field of target identification and positioning of mobile robots, relates to a deep neural network model multi-target detection and geometric morphology positioning method, and particularly relates to a morphological and color space-based strawberry fruit stem picking point positioning method for an intelligent mobile picking robot.
Background
Due to the continuous growth of the strawberry industry and the rapid development of modern agriculture, the automatic strawberry picking robot is widely used, and the improvement of the strawberry picking efficiency and the enhancement of the strawberry picking accuracy become main targets for the development of the existing strawberry picking robot.
At present, two main methods are mainly used for positioning the picking points of strawberries by an intelligent mobile automatic picking robot. The first is that the vision system uses machine learning typical algorithms to identify picking points, such as using SVM (support vector machine), RVM (correlation vector machine), etc. The method identifies strawberry fruits in the image through a classification algorithm, and then manually and directly defines fruit stalks and picking point positions thereof by utilizing manual experience, wherein the picking point positions are usually positioned above the identified strawberry fruits. Although the method has high identification speed, the fruit stalks are only identified and not really identified, and the positions of the fruit stalks and picking points are directly deduced through experience, so that the algorithm identification result has low precision. When a complex fruit scene is encountered, the method has large positioning deviation on fruit stems and picking points, so that the fruits are cut or the fruit stems are damaged in the process of falling off. Another mainstream method is to identify only the fruit stalks in the image and then calculate the gravity center of the fruit stalks to define the picking point position. Although the method accurately identifies the fruit stalks, the method is separated from the relationship between the fruit stalks and the fruits. In the actual picking process, the method is easy to pick immature fruits and has large loss. Strawberries belong to the berry class fruits, and are generally picked by grippers or cutting type robots due to the soft nature of the fruits. The strawberry shape has the characteristic of diversity, and the identification result of the classification algorithm is the minimum rectangle of the area where the fruit or the fruit stalk is located. The gripper robot cannot confirm the specific contours of the fruits and the fruit stalks, and the fruits are easily damaged in the picking process. The method for positioning the fruit stalks and picking points by artificial experience values does not have universality. The cutting robot often cuts or miscut the fruit stalks of other fruits, greatly wastes the ability and the fruit resources of the picking robot, and can not improve the efficiency of farm picking.
In order to solve the above problems, a method capable of accurately and automatically picking strawberries is needed.
Disclosure of Invention
In view of the above, the invention aims to provide a method for positioning strawberry fruit stem picking points based on morphology and color space for an intelligent mobile picking robot, which solves the difficulty of applying deep learning to fruit stem identification, and greatly improves the efficiency of intelligent agricultural fruit picking by matching with a correct picking point positioning algorithm. The invention applies the multitask target detection deep neural network to the identification of strawberry fruits and fruit stalks, can accurately identify the shapes of strawberries and fruit stalks, and improves the positioning precision of the robot on fruit stalk picking points.
In order to achieve the purpose, the invention provides the following technical scheme:
a strawberry stalk picking point positioning method based on morphology and color space specifically comprises the following steps:
s1: identifying positions of strawberry fruits and fruit stalk areas by using a Mask RCNN network;
s2: defining a strawberry fruit maturity judgment rule by self by adopting an HSV color space, judging the maturity of the identified fruit region, reserving a mature fruit region, and abandoning an immature fruit region;
s3: a self-defined fruit stem picking point positioning method is adopted, the gravity center of a target fruit is calculated, then the included angle formed by the gravity center of the fruit and the fruit stem is calculated, and the membership relationship between the fruit stem and the fruit is judged; and if the membership is satisfied, calculating the optimal picking point position of the mature fruit corresponding to the fruit stem.
Further, in step S2, customizing a judgment rule of the maturity of the strawberry fruit specifically includes: defining scales for the three channels H, S, V by using an HSV color space, dividing H-value distribution intervals of red, white and green, analyzing and calculating pixel values of red, white and green and overall distribution values in a fruit region, and finally calculating maturity attribution of the values in a self-defined fuzzy rule.
Further, in step S2, the H-number distribution intervals of red, white, and green divided are:
red: h e (330, 22)
Green: h epsilon (75,155)
White: h epsilon (45, 60)
Wherein, the red and the green are divided in the non-extreme range of S and V, namely S is less than or equal to 0.1, and V is more than or equal to 0.2;
analyzing and calculating the red, white and green pixel values and the overall distribution value D in the fruit area as follows:
Figure BDA0003458214580000021
wherein HCNumber of pixels for red, green or white to be determined, dCThe total number of pixels in the fruit area; when calculating D, if the value D of the current calculated color is in the interval [ 67%, 100%]The state of D of the color is defined as L; similarly, if the current color value D is within the interval [ 34%, 66% ]]The state of D for this color is defined as M; the value D of the currently calculated color lies in the interval [0, 33%]The state of D for this color is defined as S'.
Further, in step S3, calculating the center of gravity of the target fruit includes: acquiring information of a fruit region judged to be ripe, and dividing the region into a limited number of triangles; and then calculating the gravity center of a triangle formed by every two adjacent points, and so on to decompose the irregular polygon into a finite number of triangles.
Further, in step S3, calculating the center of gravity of the target fruit specifically includes: acquiring coordinate information of a fruit area, selecting any point outside the fruit area as an origin point P (0,0) of the two-dimensional coordinate system, establishing the two-dimensional coordinate system, and taking each pixel point as a coordinate point; two adjacent points A (x) are selected1,y1) And B (x)2,y2) Connected with the origin to construct a triangle, and the three vertex positions of the triangle are A (x)1,y1)，B(x2,y2) P (0,0), the area of the triangle is:
Figure BDA0003458214580000031
the barycentric G coordinate of the triangle is:
Figure BDA0003458214580000032
Figure BDA0003458214580000033
by analogy, the fruit region X can be divided into n triangular superposed X1，X2，…，XnThe center of gravity of these triangles is GiArea is SiThe barycentric G (X, y) coordinates of the fruit region X are:
Figure BDA0003458214580000034
Figure BDA0003458214580000035
wherein (G)ix,Giy) Representing the barycentric coordinates of the ith triangle.
Further, in step S3, determining the membership relationship between the fruit stalks and the fruits specifically includes: converting the obtained pixel information of the fruit stem image into two-dimensional coordinates which are two-dimensional coordinate system points; obtaining coordinates of the highest point and the lowest point of the fruit stalk region by adopting a traversal method; calculating the highest point M (x) of fruit stalks1,y1) Lowest point N (x)2,y2) And when the included angle between the horizontal straight line and the gravity center G (x, y) of the fruit is smaller than 180 degrees, judging the fruit stem as a corresponding fruit and picking the fruit stem.
Further, in step S3, calculating an optimal picking point position of the ripe fruit corresponding to the fruit stem, specifically including: after the gravity center of the fruit is obtained and the membership of the fruit stem is judged, the highest point M (x) of the fruit stem is connected in the membership fruit stem area1,y1) And the lowest point N (x)2,y2) And calculating the coordinate of the midpoint Q of the two connecting lines:
Figure BDA0003458214580000036
defining Q as an optimal picking point;the horizontal straight line range of the midpoint is the optimal picking or cutting range of the robot in the picking period.
Further, in step S3, the calculated optimal picking point of the fruit stem is converted from the pixel coordinate to the camera coordinate corresponding to the intelligent mobile robot, so as to obtain the coordinate of the camera corresponding to the optimal picking point of the fruit stem, which specifically includes:
(1) acquiring the pixel coordinates of the calculated picking points: q ═ u, v;
(2) converting the pixel coordinates of the picking points into image plane coordinates;
let the physical size of each pixel in the u-axis and v-axis directions be dxAnd dyThe following formula is obtained:
Figure BDA0003458214580000041
Figure BDA0003458214580000042
wherein d isxAnd dyThe actual size of the pixel on the camera photosensitive chip is represented and used for connecting a pixel coordinate system and a real size coordinate system; u. of0And v0Representing the center of the image plane;
Figure BDA0003458214580000043
obtaining the image plane coordinates of the picking points through the matrix change: p ═ x, y;
(3) converting the image plane coordinates of the picking points into camera coordinates of the picking robot;
the known camera parameters are: focal length f of the camera, axis of the optical center of the camera: z-axis, camera coordinates are: pc＝(xw,yw,zw) (ii) a The image plane coordinates are transformed into camera coordinates as follows:
Figure BDA0003458214580000044
wherein z iscThe position coordinates of the actual points of the camera can be known through manual measurement; x is the number ofcAnd ycIs the axis of the coordinate plane in which the camera is located. Thus, the specific position of the picking point in the camera coordinate system is obtained.
The invention has the beneficial effects that: when orienting to strawberry stem identification and stem picking point positioning, compared with the prior method, the method adopts example segmentation of fruits and stems, and can accurately identify the target object. For the picking of strawberries, the identification precision is greatly improved, so that the picking robot can more accurately position the fruits. Meanwhile, due to the fruit stem picking point positioning algorithm, a picking range meeting picking requirements can be obtained in an actual use scene. Finally, the invention can overcome the defects of different picking point calculation methods caused by strawberries with different forms, provides a universal calculation mode and improves the universal capability and picking efficiency of the intelligent mobile picking robot.
The invention can realize the positioning of picking points among strawberries with different forms, thereby reducing the calculation cost. The method is beneficial to improving the accuracy of picking points, and can save the hardware resources of the robot, and the robot is suitable for picking work in various scenes.
Additional advantages, objects, and features of the invention will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the invention. The objectives and other advantages of the invention may be realized and attained by the means of the instrumentalities and combinations particularly pointed out hereinafter.
Drawings
For the purposes of promoting a better understanding of the objects, aspects and advantages of the invention, reference will now be made to the following detailed description taken in conjunction with the accompanying drawings in which:
FIG. 1 is a block diagram of the overall flow of the strawberry stem picking point positioning method of the present invention;
FIG. 2 is a block diagram of a multitasking objective deep neural network flow;
FIG. 3 is a flowchart of judging the maturity of a fruit region;
FIG. 4 is a schematic view of the determination of membership of fruit stalks and fruits;
fig. 5 is a schematic diagram of a specific positioning method of a fruit stem picking point positioning algorithm.
Detailed Description
The embodiments of the present invention are described below with reference to specific embodiments, and other advantages and effects of the present invention will be easily understood by those skilled in the art from the disclosure of the present specification. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention. It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present invention in a schematic way, and the features in the following embodiments and examples may be combined with each other without conflict.
Referring to fig. 1 to 5, fig. 1 shows a positioning method for picking points of strawberry stalks based on morphology and color space, which specifically includes the following steps:
step 1: segmented images were acquired using Mask RCNN multitask object detection network.
Taking the strawberry image as a target image, and inputting the target image into a Mask RCNN network; and obtaining a recognition result which is expected and comprises two classifications of fruits and fruit stalks.
The deep neural network based on multi-target task detection can combine target detection with instance segmentation, can simultaneously identify a plurality of target objects, and the identification result is the specific contour of fruits and fruit stalks; meanwhile, the deep neural network based on multi-target task detection is quick in identification time and high in accuracy, identification results of all target objects are not interfered with one another, and the method is extremely friendly to a mobile robot with a priority on resource utilization rate. Aiming at the picking robots of grippers, the network method can accurately identify the edge outline of the fruit, and damage to the fruit in the picking process is avoided; for the cutting picking robot, the network method can accurately position the picking point position or picking range, and avoid accidentally injuring fruits in the cutting process.
Step 2: and judging the maturity of the identified fruits. The process comprises the following 3 steps.
(1) Acquiring all fruits and fruit stems output by a network;
(2) the fruit information is converted into an HSV color space, the pixel values of red, white and green and the distribution conditions of the pixel values are calculated, H belongs to [0,360], S belongs to [0,255] and V belongs to [0,255] in the HSV color space, because the HSV color space is close to the perception of human eyes to colors, the HSV color space is not strictly divided into color intervals, H represents hue, namely various colors perceived by human eyes, S represents saturation, the depth of the colors perceived by the human eyes is light, V represents brightness, and the brightness perceived by the human eyes is influenced by the saturation. Therefore, color is recognized on a daily basis, depending largely on the hue. Therefore, by default, S and V are in non-extreme ranges (not black), and H value distribution areas for dividing red, white and green are respectively:
red: h e (330, 22)
Green: h epsilon (75,155)
White: h epsilon (45, 60)
Wherein, the red and the green are divided in the non-extreme range of S and V, namely S is less than or equal to 0.1, and V is more than or equal to 0.2.
According to daily experience, the white color of the immature strawberry fruit approaches to beige, namely, the white color is mixed with yellow, and when the white color is judged, V is (0.9, 1).
The HSV values are converted to calculated values, and the converted HSV ranges are as follows.
H∈[0,180],S∈[0,255],V∈[0,255]
In the conversion process, the fractional part is discarded computationally and combined with the fractional part, so that the obtained red, white and green values are roughly distributed in the interval shown in table 1.
TABLE 1 HSV COLOR SPACE COLOR CHANNEL DISTRIBUTION
Figure BDA0003458214580000061
As shown in table 1, white, red and green are selected as the distinguishing color channels, and the distribution ratio of the HSV value in the actual fruit region information in the three color intervals is calculated; then, the ratio of the three color channels is associated with the fuzzy rule to judge the maturity. For example, the obtained fruit area has a total of 100 pixels, and if the calculated number of pixels determined to be red is 32, the calculated number of pixels determined to be green is 30, and the calculated number of pixels determined to be white is 27, the percentage of the total number of pixels in the three colors is calculated, and the obtained fruit area is determined to be immature, the obtained fruit area is determined to be green and white. The specific fuzzy rule is shown in table 2.
TABLE 2 fuzzy rules
Figure BDA0003458214580000071
The specific numerical values of the color numerical value interval corresponding to the custom L, M, S' are shown in the following table 3.
TABLE 3L, M, S' distribution Range
Figure BDA0003458214580000072
Namely:
Figure BDA0003458214580000081
wherein HCNumber of red \ green \ white pixels to be judged, dCThe total number of pixels in the fruit area.
(3) According to the user-defined fuzzy rule, the information of the mature fruits is reserved, and the information of the immature fruits is discarded.
And step 3: and performing gravity center calculation on the identified mature fruits. The process comprises the following 2 steps.
(1) Acquiring target area pixel information of mature fruits and fruit stalks detected by a network and judged by maturity;
(2) and converting the pixel information of the fruit area into coordinate information, and calculating and determining the gravity center position. According to the plant science, the fruit stalks of the plants are mostly on the upper side of the gravity center of the fruits. Most of fruits identified by example segmentation are irregular in shape, and the gravity center position cannot be directly calculated. Firstly, coordinate information of a fruit area is obtained, any point outside the fruit area is selected as an origin point P (0,0) of the two-dimensional coordinate system, the two-dimensional coordinate system is established, and each pixel point is used as a coordinate point. Two adjacent points A (x) are selected1,y1) And B (x)2,y2) Connected with the origin to construct a triangle, and the three vertex positions of the triangle are A (x)1,y1)，B(x2,y2) P (0,0), the area of the triangle is:
Figure BDA0003458214580000082
the barycentric G coordinate of the triangle is:
Figure BDA0003458214580000083
Figure BDA0003458214580000084
by analogy, the fruit region X can be divided into n triangular superposed X1，X2，…，XnThe center of gravity of these triangles is GiArea is SiThe barycentric G (X, y) coordinates of the fruit region X are:
Figure BDA0003458214580000085
Figure BDA0003458214580000086
wherein (G)ix,Giy) Is shown asBarycentric coordinates of the i triangles.
And 4, step 4: judging whether the identified fruit stalks are corresponding to the fruit and can be cut, and if the fruit stalks are corresponding to the fruit, calculating the optimal picking point. The process comprises 5 steps.
(1) Judging whether the fruit stalks are corresponding to the target fruits or not, and picking the fruit stalks. Firstly, acquiring position information of a fruit stalk region, and obtaining coordinates of two points, namely a lowest point and a highest point, of the fruit stalk region by adopting a traversal method;
(2) calculate the highest point M (x)1,y1) And the lowest point N (x)2,y2) And the horizontal straight line of the center of gravity of the fruit. Connecting the lowest point, the highest point and the calculated center of gravity of the fruit, and calculating an included angle between the center of gravity and a horizontal straight line where the center of gravity is located;
(3) calculating an included angle, and judging whether the fruit stem corresponds to the fruit; when two included angles between the highest point and the lowest point of the fruit stem and the center of gravity of the fruit are both less than 180 degrees, judging the fruit stem to be the fruit stem corresponding to the fruit which meets the morphological requirement;
(4) and (5) confirming the picking points of the fruit stalks. Highest point M (x) of connecting fruit stalks1,y1) And the lowest point N (x)2,y2) And calculating the coordinate of the midpoint Q of the line segment:
Figure BDA0003458214580000091
the middle point is the best picking point of the fruit stalks. In an actual scene, fruit stalks are mostly in irregular curved shapes, the calculated picking points are not necessarily positioned on the real fruit stalks, and the horizontal straight line where the picking points are positioned is taken as the optimal cutting range for picking;
(5) and calculating the camera coordinates of the picking robot corresponding to the picking points in actual picking.
a) Acquiring the pixel coordinates of the calculated picking points: q ═ u, v.
b) And converting the pixel coordinates of the picking points into image plane coordinates.
Let the physical size of each pixel in the u-axis and v-axis directions be dxAnd dyThe following formula can be obtained:
Figure BDA0003458214580000092
Figure BDA0003458214580000093
wherein d isxAnd dyThe actual size of the pixel on the camera photosensitive chip is represented and used for connecting a pixel coordinate system and a real size coordinate system; u. of0And v0Representing the center of the image plane.
Figure BDA0003458214580000094
Obtaining the image plane coordinates of the picking points through the matrix change: p ═ x, y.
c) And converting the image plane coordinates of the picking points into the camera coordinates of the picking robot.
The known camera parameters are: f (focal length of camera), axis of the optical center of the camera: z-axis, camera coordinates are: pc＝(xw,yw,zw) (ii) a The image plane coordinates are transformed into camera coordinates as follows:
Figure BDA0003458214580000101
wherein z iscThe position coordinates of the actual points of the camera can be known through manual measurement; x is the number ofcAnd ycIs the axis of the coordinate plane in which the camera is located. Thus, the specific position of the picking point in the camera coordinate system is obtained.
The embodiment designs a multi-task detection depth convolution neural network and a fruit stalk optimal picking point positioning method which can realize real-time strawberry fruit and fruit stalk identification on mobile robot edge computing equipment. Based on the existing multi-task target detection deep neural network, the method can solve the difficulty of deep learning in application of fruit stem identification, and greatly improve the fruit picking efficiency of intelligent agriculture by matching with a correct picking point positioning algorithm. Therefore, the invention applies the multitask target detection deep neural network to the identification of strawberry fruits and fruit stalks, can accurately identify the shapes of strawberries and fruit stalks, and improves the positioning precision of the robot on the fruit stalk picking points.
Finally, the above embodiments are only intended to illustrate the technical solutions of the present invention and not to limit the present invention, and although the present invention has been described in detail with reference to the preferred embodiments, it will be understood by those skilled in the art that modifications or equivalent substitutions may be made on the technical solutions of the present invention without departing from the spirit and scope of the technical solutions, and all of them should be covered by the claims of the present invention.

Claims (8)

1. A strawberry stalk picking point positioning method based on morphology and color space is characterized by comprising the following steps:
s1: identifying positions of strawberry fruits and fruit stalk areas by using a Mask RCNN network;
s2: defining a strawberry fruit maturity judgment rule by self by adopting an HSV color space, judging the maturity of the identified fruit region, reserving a mature fruit region, and abandoning an immature fruit region;
s3: a self-defined fruit stem picking point positioning method is adopted, the gravity center of a target fruit is calculated, then the included angle formed by the gravity center of the fruit and the fruit stem is calculated, and the membership relationship between the fruit stem and the fruit is judged; and if the membership is satisfied, calculating the optimal picking point position of the mature fruit corresponding to the fruit stem.
2. The strawberry stem picking point positioning method according to claim 1, wherein in step S2, customizing the judgment rule of the maturity of strawberry fruit specifically comprises: defining scales for the three channels H, S, V by using an HSV color space, dividing H-value distribution intervals of red, white and green, analyzing and calculating pixel values of red, white and green and overall distribution values in a fruit region, and finally calculating maturity attribution of the values in a self-defined fuzzy rule.
3. The strawberry stem picking point positioning method according to claim 2, wherein in step S2, the H-number distribution intervals of red, white and green are divided as follows:
red: h e (330, 22)
Green: h epsilon (75,155)
White: h epsilon (45, 60)
Wherein, the red and the green are divided in the non-extreme range of S and V, namely S is less than or equal to 0.1, and V is more than or equal to 0.2;
analyzing and calculating the red, white and green pixel values and the overall distribution value D in the fruit area as follows:
Figure FDA0003458214570000011
wherein HCNumber of pixels for red, green or white to be determined, dCThe total number of pixels in the fruit area; if the value D of the current calculated color is within the interval of [ 67%, 100%]The state of D of the color is defined as L; similarly, if the value D of the current calculated color is within the interval of [ 34%, 66%]The state of D for this color is defined as M; the value D of the current calculated color is in the interval [0, 33%]The state of D for this color is defined as S'.
4. The strawberry stem picking point positioning method of claim 3, wherein in the step S3, calculating the gravity center of the target fruit comprises: acquiring information of a fruit region judged to be ripe, and dividing the region into a limited number of triangles; and then calculating the gravity center of a triangle formed by every two adjacent points, and so on to decompose the irregular polygon into a finite number of triangles.
5. The strawberry stem picking point positioning method according to claim 1 or 4, wherein in the step S3, calculating the gravity center of the target fruit specifically comprises: acquiring coordinate information of a fruit area, selecting any point outside the fruit area as an origin point P (0,0) of the two-dimensional coordinate system, establishing the two-dimensional coordinate system, and taking each pixel point as a seatMarking points; two adjacent points A (x) are selected1,y1) And B (x)2,y2) Connected with the origin to construct a triangle, and the three vertex positions of the triangle are A (x)1,y1)，B(x2,y2) P (0,0), the area of the triangle is:
Figure FDA0003458214570000021
the barycentric G coordinate of the triangle is:
Figure FDA0003458214570000022
Figure FDA0003458214570000023
by analogy, the fruit area X is divided into n triangular superposed X1，X2，…，XnThe center of gravity of these triangles is GiArea is SiThe barycentric G (X, y) coordinates of the fruit region X are:
Figure FDA0003458214570000024
Figure FDA0003458214570000025
wherein (G)ix,Giy) Representing the barycentric coordinates of the ith triangle.
6. The method for positioning the picking points of the strawberry stalks and the fruits according to claim 1, wherein in the step S3, the step of judging the membership relationship between the stalks and the fruits specifically comprises the following steps: converting the obtained pixel information of the stem image into two-dimensional coordinate system pointsCoordinates; obtaining coordinates of the highest point and the lowest point of the fruit stalk region by adopting a traversal method; calculating the highest point M (x) of fruit stalks1,y1) Lowest point N (x)2,y2) And when the included angle between the horizontal straight line and the gravity center G (x, y) of the fruit is smaller than 180 degrees, judging the fruit stem as a corresponding fruit and picking the fruit stem.
7. The strawberry stem picking point positioning method according to claim 6, wherein in step S3, calculating the optimal picking point position of the mature fruit corresponding to the stem specifically comprises: after the gravity center of the fruit is obtained and the membership of the fruit stem is judged, the highest point M (x) of the fruit stem is connected in the membership fruit stem area1,y1) And the lowest point N (x)2,y2) And calculating the coordinate of the midpoint Q of the two connecting lines:
Figure FDA0003458214570000031
defining Q as an optimal picking point; the horizontal straight line range of the midpoint is the optimal picking or cutting range of the robot in the picking period.
8. The strawberry stem picking point positioning method according to claim 7, wherein in step S3, the step of converting the calculated optimal stem picking point from pixel coordinates into camera coordinates corresponding to the intelligent mobile robot to obtain coordinates of the camera corresponding to the optimal stem picking point comprises:
(1) acquiring the pixel coordinates of the calculated picking points: q ═ u, v;
(2) converting the pixel coordinates of the picking points into image plane coordinates;
let the physical size of each pixel in the u-axis and v-axis directions be dxAnd dyThe following formula is obtained:
Figure FDA0003458214570000032
Figure FDA0003458214570000033
wherein d isxAnd dyThe actual size of the pixel on the camera photosensitive chip is represented and used for connecting a pixel coordinate system and a real size coordinate system; u. of0And v0Representing the center of the image plane;
Figure FDA0003458214570000034
obtaining the image plane coordinates of the picking points through the matrix change: p ═ x, y;
(3) converting the image plane coordinates of the picking points into camera coordinates of the picking robot;
the known camera parameters are: focal length f of the camera, axis of the optical center of the camera: z-axis, camera coordinates are: pc＝(xw,yw,zw) (ii) a The image plane coordinates are transformed into camera coordinates as follows:
Figure FDA0003458214570000035
wherein z iscPosition coordinates of actual points of the camera; x is the number ofcAnd ycIs the axis of the coordinate plane in which the camera is located.
CN202210009044.5A 2022-01-06 2022-01-06 Morphology and color space-based strawberry stem picking point positioning method Active CN114258781B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210009044.5A CN114258781B (en) 2022-01-06 2022-01-06 Morphology and color space-based strawberry stem picking point positioning method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210009044.5A CN114258781B (en) 2022-01-06 2022-01-06 Morphology and color space-based strawberry stem picking point positioning method

Publications (2)

Publication Number Publication Date
CN114258781A true CN114258781A (en) 2022-04-01
CN114258781B CN114258781B (en) 2023-07-21

Family

ID=80832447

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210009044.5A Active CN114258781B (en) 2022-01-06 2022-01-06 Morphology and color space-based strawberry stem picking point positioning method

Country Status (1)

Country Link
CN (1) CN114258781B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115250744A (en) * 2022-07-29 2022-11-01 四川启睿克科技有限公司 Multi-angle strawberry picking system and method
CN117152544A (en) * 2023-10-31 2023-12-01 锐驰激光(深圳)有限公司 Tea-leaf picking method, equipment, storage medium and device

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2008206438A (en) * 2007-02-26 2008-09-11 Iseki & Co Ltd Fruit harvesting robot
JP2010054342A (en) * 2008-08-28 2010-03-11 National Agriculture & Food Research Organization Method and apparatus for measuring quality of strawberry
JP2012110256A (en) * 2010-11-22 2012-06-14 National Agriculture & Food Research Organization Fruit stem cutter
CN109271867A (en) * 2018-08-20 2019-01-25 浙江荣亚工贸有限公司 A kind of Strawberry ripening degree automatic judging method
CN109711325A (en) * 2018-12-25 2019-05-03 华南农业大学 A kind of mango picking point recognition methods
CN111079530A (en) * 2019-11-12 2020-04-28 青岛大学 Mature strawberry identification method
CN111666883A (en) * 2020-06-08 2020-09-15 佛山科学技术学院 Grape picking robot target identification and fruit stalk clamping and cutting point positioning method
CN111684924A (en) * 2020-06-23 2020-09-22 王孟超 Strawberry picking robot based on raspberry group
CN112990103A (en) * 2021-04-16 2021-06-18 苏州大学 String mining secondary positioning method based on machine vision

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2008206438A (en) * 2007-02-26 2008-09-11 Iseki & Co Ltd Fruit harvesting robot
JP2010054342A (en) * 2008-08-28 2010-03-11 National Agriculture & Food Research Organization Method and apparatus for measuring quality of strawberry
JP2012110256A (en) * 2010-11-22 2012-06-14 National Agriculture & Food Research Organization Fruit stem cutter
CN109271867A (en) * 2018-08-20 2019-01-25 浙江荣亚工贸有限公司 A kind of Strawberry ripening degree automatic judging method
CN109711325A (en) * 2018-12-25 2019-05-03 华南农业大学 A kind of mango picking point recognition methods
CN111079530A (en) * 2019-11-12 2020-04-28 青岛大学 Mature strawberry identification method
CN111666883A (en) * 2020-06-08 2020-09-15 佛山科学技术学院 Grape picking robot target identification and fruit stalk clamping and cutting point positioning method
CN111684924A (en) * 2020-06-23 2020-09-22 王孟超 Strawberry picking robot based on raspberry group
CN112990103A (en) * 2021-04-16 2021-06-18 苏州大学 String mining secondary positioning method based on machine vision

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115250744A (en) * 2022-07-29 2022-11-01 四川启睿克科技有限公司 Multi-angle strawberry picking system and method
CN115250744B (en) * 2022-07-29 2023-09-15 四川启睿克科技有限公司 Multi-angle strawberry picking system and method
CN117152544A (en) * 2023-10-31 2023-12-01 锐驰激光(深圳)有限公司 Tea-leaf picking method, equipment, storage medium and device
CN117152544B (en) * 2023-10-31 2024-03-15 锐驰激光(深圳)有限公司 Tea-leaf picking method, equipment, storage medium and device

Also Published As

Publication number Publication date
CN114258781B (en) 2023-07-21

Similar Documents

Publication Publication Date Title
Li et al. Detection of fruit-bearing branches and localization of litchi clusters for vision-based harvesting robots
CN114258781B (en) Morphology and color space-based strawberry stem picking point positioning method
CN105718945B (en) Apple picking robot night image recognition method based on watershed and neural network
Wang et al. Localisation of litchi in an unstructured environment using binocular stereo vision
EP1655620B1 (en) Obstacle detection using stereo vision
CN111418349B (en) Intelligent fruit picking robot and method for realizing fruit picking
CN112561985B (en) Hedgerow nursery stock trimming and centering method based on binocular vision
CN114067309B (en) Sweet pepper identification and picking sequence determination method based on multi-view three-dimensional reconstruction
CN113255434B (en) Apple identification method integrating fruit characteristics and deep convolutional neural network
CN112990103A (en) String mining secondary positioning method based on machine vision
CN114260895A (en) Method and system for determining grabbing obstacle avoidance direction of mechanical arm of picking machine
CN114067206A (en) Spherical fruit identification and positioning method based on depth image
CN115560754A (en) Visual navigation method based on weed removal
Hu et al. Recognition and localization of strawberries from 3D binocular cameras for a strawberry picking robot using coupled YOLO/Mask R-CNN
CN109886062A (en) A kind of camellia oleifera fruit flower identification positioning system
CN117152544B (en) Tea-leaf picking method, equipment, storage medium and device
CN113065562B (en) Crop ridge row extraction and dominant route selection method based on semantic segmentation network
CN111369497B (en) Walking type tree fruit continuous counting method and device
CN111612797B (en) Rice image information processing system
CN111079530A (en) Mature strawberry identification method
CN111897333A (en) Robot walking path planning method
CN115512119A (en) Greenhouse fresh jujube maturity recognition and picking system based on artificial intelligence
Yu et al. Design and implementation of an automatic peach-harvesting robot system
Bachche et al. Distinction of green sweet peppers by using various color space models and computation of 3 dimensional location coordinates of recognized green sweet peppers based on parallel stereovision system
CN115601690A (en) Edible fungus environment detection method based on intelligent agriculture

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant