CN109348865B - Cinnabar orange picking robot and picking method thereof - Google Patents
Cinnabar orange picking robot and picking method thereof Download PDFInfo
- Publication number
- CN109348865B CN109348865B CN201811345265.XA CN201811345265A CN109348865B CN 109348865 B CN109348865 B CN 109348865B CN 201811345265 A CN201811345265 A CN 201811345265A CN 109348865 B CN109348865 B CN 109348865B
- Authority
- CN
- China
- Prior art keywords
- fixed
- cinnabar
- steering engine
- orange
- camera
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- A—HUMAN NECESSITIES
- A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
- A01D—HARVESTING; MOWING
- A01D46/00—Picking of fruits, vegetables, hops, or the like; Devices for shaking trees or shrubs
- A01D46/30—Robotic devices for individually picking crops
-
- A—HUMAN NECESSITIES
- A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
- A01D—HARVESTING; MOWING
- A01D46/00—Picking of fruits, vegetables, hops, or the like; Devices for shaking trees or shrubs
- A01D46/22—Baskets or bags attachable to the picker
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P60/00—Technologies relating to agriculture, livestock or agroalimentary industries
- Y02P60/12—Technologies relating to agriculture, livestock or agroalimentary industries using renewable energies, e.g. solar water pumping
Abstract
The invention discloses a cinnabar orange picking robot and a picking method thereof. At present, the domestic cinnabar orange picking operation is basically finished manually, and belongs to typical labor-intensive work. The invention relates to a cinnabar orange picking robot which comprises a travelling mechanism, a rotating mechanism, a lifting mechanism, a telescopic mechanism, a picking mechanism and a camera. The travelling mechanism comprises a travelling motor, a travelling wheel and a universal wheel. The camera is arranged on the bottom plate. The rotation mechanism includes a rotation plate and a rotation driving member. The lifting mechanism comprises an upper supporting plate, a lower supporting frame, a lifting frame, an optical axis and a lifting driving piece. The telescopic mechanism comprises a telescopic plate, a telescopic rod group, a third gear, a fourth gear and a second steering engine. The picking mechanism comprises a baffle, a fixed blade, a driving blade and a third steering engine. According to the invention, the cinnabar orange positions in the image are identified in a mode of combining graying, binarization, median filtering and convolutional neural network, so that the identification accuracy is improved.
Description
Technical Field
The invention belongs to the technical field of agricultural machinery, and particularly relates to a cinnabar orange picking robot and a picking method thereof.
Background
In the planting process of the cinnabar orange, the quality of the harvesting operation directly influences the follow-up links of storage, processing, sales and the like of the cinnabar orange, thereby finally influencing the market price and economic benefit. However, due to the complexity of the picking operation environment and operation, the automation degree of the orange picking operation is still low, and the current domestic orange picking operation is basically finished manually, thus the orange picking operation belongs to typical labor-intensive work. This greatly increases the production cost of the cinnabar orange. Therefore, it is important to design a cinnabar orange picking device with high automation degree.
Disclosure of Invention
The invention aims to provide a cinnabar orange picking robot and a picking method thereof.
The invention relates to a cinnabar orange picking robot which comprises a travelling mechanism, a rotating mechanism, a lifting mechanism, a telescopic mechanism, a picking mechanism and a camera. The travelling mechanism comprises a bottom plate, a travelling motor, travelling wheels and universal wheels. Both traveling wheels are supported on the bottom surface of the base plate. The universal wheel is arranged on the bottom surface of the bottom plate. The two travelling wheels are driven by two travelling motors respectively. The camera is arranged on the bottom plate. The rotating mechanism comprises a rotating plate and a rotating driving piece. The rotating plate is arranged above the bottom plate and forms a revolute pair with the bottom plate. The rotating plate is driven by a rotation driving member. The lifting mechanism comprises an upper supporting plate, a lower supporting frame, a lifting frame, an optical axis and a lifting driving piece. The bottom of optical axis is all fixed with the lower carriage, and the top is all fixed with last backup pad. The lower support frame is fixed with the rotating plate. The lifting frame and the optical axis form a sliding pair and are driven by a lifting driving piece.
The telescopic mechanism comprises a telescopic plate, a telescopic rod group, a third gear, a fourth gear and a second steering engine. The telescopic rod group comprises a first connecting rod, a second connecting rod, a third connecting rod and a fourth connecting rod. And the inner ends of the first connecting rod and the second connecting rod are respectively fixed with a third gear and are hinged with the lifting frame. The two third gears are meshed. The outer ends of the first connecting rod and the second connecting rod are respectively hinged with one ends of the third connecting rod and the fourth connecting rod. The other ends of the third connecting rod and the fourth connecting rod are respectively fixed with a fourth gear and are hinged with the expansion plate. The two fourth gears are meshed. One of the third gears is driven by the second steering engine.
The picking mechanism comprises a baffle, a fixed blade, a driving blade and a third steering engine. And the third steering engine and the baffle are fixed on the expansion plate. The inner end of the driving blade is fixed with the output shaft of the third steering engine. The outer end of the fixed blade is fixed with the baffle, and the inner end of the fixed blade and the output shaft of the third steering engine form a revolute pair.
The picking method of the cinnabar orange picking robot specifically comprises the following steps:
step one, starting a camera, and shooting a picture of a cinnabar orange tree to obtain an original color image. And carrying out graying treatment on the original color image to obtain a gray scale image. Gray value Gray of pixel of ith row and jth column in Gray map i,j The expression of (2) is shown as the formula (1):
Gray i,j ＝0.072169B i,j +0.715160G i,j +0.212671R i,j (1)
in the formula (1), B i,j Blue channel brightness for the pixel of the ith row and jth column in the original color image; g i,j Green channel brightness for the pixel of the ith row and jth column in the original color image; r is R i,j The red channel brightness for the pixel of the ith row and jth column in the original color image.
And step two, binarizing the gray level map to obtain a black-and-white map. Pixel value a of ith row and jth column pixels in black-and-white image i,j The expression of (2) is shown as the following formula:
in the formula (2), the er is more than or equal to 150 and less than or equal to 180.
And thirdly, median filtering is carried out on the black-and-white image, so that all pixels in a white pixel group with the pixel number smaller than x multiplied by y on the black-and-white image are changed into black, and a filtered black-and-white image is obtained. X is more than or equal to 3 and less than or equal to 5, y is more than or equal to 3 and less than or equal to 5; the white pixel group is composed of white pixels connected together. The remaining n white pixel groups in the black and white map are filtered.
And fourthly, selecting n undetermined identification areas from the original color image circle. The n undetermined identification areas on the original color image correspond to the n white pixel groups in the filtered black-and-white image in position respectively.
Step five: and sequentially intercepting all the undetermined identification areas on the original color image to obtain n undetermined identification images. And sequentially introducing n Zhang Daiding identification images into a convolutional neural network, and judging whether cinnabar orange exists in each identification image to be determined. M of n undetermined identification areas of the original color image are used as effective identification areas.
Step six, k=1, 2, …, m, steps seven and eight are performed in sequence.
Step seven, calculating the distance L between the cinnabar orange corresponding to the kth effective identification area and the camera k 。L k The expression of (2) is shown in the formula (3).
In the formula (3), K z For the distance calculation coefficient, the expression is as shown in formula (4):
in the formula (4), R m The Apix is the average diameter of the cinnabar orange and the number of one row of pixel points in a photo taken by the camera; θ is one half of the camera view angle.
Step eight, calculating the coordinate value (x) of the center position of the cinnabar orange corresponding to the kth effective identification area in a plane rectangular coordinate system with the camera as the origin of coordinates and the x-z plane perpendicular to the axis of the camera k ,y k ,z k )，x k 、y k 、z k The relation of (2) is shown in the formula (5).
In the formula (5), u xk The abscissa of the geometric center point of the kth effective identification area on the original color image; v yk The ordinate of the geometric center point of the kth effective identification area on the original color image; dx is an abscissa displacement coefficient, and the expression is shown in the formula (6); dy is an ordinate displacement coefficient, and the expression is shown as a formula (7);
in the formulas (6) and (7), L p Is the focal length of the camera.
Step nine: assign 1 to k.
Step ten: the first steering engine drives the rotating plate to rotate; the lifting motor drives the lifting frame to move up and down. The second steering engine rotates to drive the expansion plate to horizontally move, so that the fruit stalks of the kth cinnabar orange are positioned between the driving blade and the fixed blade.
Step eleven: and the third steering engine drives the driving blade to rotate, and the driving blade and the fixed blade cut fruit stalks of the kth cinnabar orange and enter a step twelve.
Step twelve: and the third steering engine drives the driving blade to reset. If k is less than m, increasing k by 1, and repeating steps ten and eleven; otherwise, step thirteenth is entered.
Step thirteen: the travelling mechanism drives the bottom plate to shift positions, and then the steps one to twelve are repeatedly executed.
Further, the cinnabar orange picking robot also comprises a camera overturning assembly. The camera tripod head component comprises a tripod head base, a first tripod head, a second tripod head, a fourth steering engine and a fifth steering engine. The cradle head base is fixed on the bottom plate. The first tripod head and the tripod head base form a revolute pair with a public axis arranged vertically. And the fourth steering engine is fixed on the holder base. The output shaft of the fourth steering engine is fixed with the first cloud platform. The second cloud platform and the first cloud platform constitute the revolute pair that the public axis level set up. And the fifth steering engine is fixed on the first cloud platform frame. And an output shaft of the fifth steering engine is fixed with the second cloud platform. The camera is fixed on the second cloud platform.
Further, the rotary driving piece comprises a first steering engine, a first gear and a second gear. The first steering engine is fixed on the bottom plate. The first gear and the second gear are respectively fixed with the first steering engine output shaft and the rotating plate. The first gear is meshed with the second gear. The lifting driving piece comprises a belt wheel, a driving belt and a lifting motor. The two belt wheels are respectively supported on the upper supporting plate and the lower supporting frame and are connected through a transmission belt. The lifting motor is fixed on the lower supporting frame. The output shaft of the lifting motor is fixed with a belt wheel positioned on the lower supporting frame.
Further, both traveling motors are fixed on the bottom surface of the bottom plate. The output shafts of the two travelling motors are respectively fixed with the two travelling wheels. The second steering engine is fixed on the lifting plate. The output shaft of the second steering engine is fixed with one of the third gears.
Further, the pitch diameters of the two third gears and the two fourth gears are equal.
Further, the outer ends of the fixed blade and the driving blade face to one side far away from the telescopic mechanism.
Further, the bottom surface of expansion plate is kept away from telescopic machanism's that end is fixed with and collects the string bag. The collecting net bag is positioned under the outer ends of the fixed blade and the driving blade.
Further, the camera adopts a visual module with the model of OpenMV3CamM 7. R is R m The value of (2) is 33mm; apix has a value of 320; the value of theta is 57.5 degrees; l (L) p The value of (2) is 2.8mm.
The invention has the beneficial effects that:
1. according to the invention, through the design of the telescopic mechanism, the cinnabar orange picking in a larger range can be realized under the condition that the travelling mechanism does not move, so that the number of the cinnabar oranges which can be picked by single image processing is more, and the picking efficiency is improved.
2. According to the invention, the cinnabar orange positions in the image are identified in a mode of combining graying, binarization, median filtering and convolutional neural network, so that the identification accuracy is improved.
3. The invention can calculate the space position coordinates of each cinnabar orange and improves the positioning efficiency of the picking mechanism.
Drawings
FIG. 1 is a schematic diagram of the overall structure of the present invention;
FIG. 2 is a schematic diagram of a combination of a running mechanism and a rotating mechanism according to the present invention;
FIG. 3 is a schematic view of a lifting mechanism according to the present invention;
FIG. 4 is a schematic diagram of a combination of a telescoping mechanism and a picking mechanism according to the present invention;
fig. 5 is a schematic diagram of the distance between the camera and the cinnabar orange in the invention.
Detailed Description
The invention is further described below with reference to the accompanying drawings.
As shown in fig. 1, the cinnabar orange picking robot comprises a travelling mechanism 5, a rotating mechanism 4, a lifting mechanism 1, a telescopic mechanism 3, a picking mechanism 2, a camera overturning assembly and a camera.
As shown in fig. 1 and 2, the traveling mechanism 5 includes a base plate, two traveling motors 24, traveling wheels 22, and universal wheels 23. Two travelling wheels are coaxially arranged and respectively supported on two sides of the bottom surface of the bottom plate. Both universal wheels 23 are mounted at the tail end of the bottom surface of the bottom plate. Both travel motors 24 are fixed to the bottom surface of the base plate. The output shafts of the two traveling motors 24 are fixed to the two traveling wheels 29, respectively.
As shown in fig. 1 and 2, the rotation mechanism 4 includes a rotation plate 28 and a rotation driver. The rotating plate 28 is disposed above the bottom plate in parallel and forms a revolute pair with the bottom plate. The rotary drive comprises a first steering engine 25, a first gear 27 and a second gear 26. The first steering engine 25 is fixed on the bottom plate. The first gear 27 and the second gear 26 are respectively fixed with the output shaft of the first steering engine 25 and the rotating plate 28. The first gear 27 is meshed with the second gear 26. The rotation of the first steering engine can drive the rotation plate 28 to rotate.
As shown in fig. 1 and 3, the lifting mechanism 1 includes an upper support plate 12, a lower support frame 7, a lifting frame 8, an optical axis 10, and a lifting drive. The bottom ends of the four optical axes 10 are fixed with the lower supporting frame 7, and the top ends are fixed with the upper supporting plate 12. The lower support frame 7 and the rotating plate 28 are fixed through aluminum columns. The lifting frame 8 and the four optical axes 10 form a sliding pair through linear bearings 17. The lifting drive comprises a pulley 11, a drive belt 9 and a lifting motor 6. Two pulleys 11 are supported on the upper support plate 12 and the lower support frame 7, respectively, and are connected by a transmission belt 9. The lifting motor is fixed on the lower supporting frame 7. The output shaft of the lifting motor 6 is fixed with a belt wheel 11 positioned on the lower supporting frame 7. The lifting frame 8 is driven to slide along the optical axis 10 by the rotation of the lifting motor.
As shown in fig. 1 and 4, the telescopic mechanism 3 comprises a telescopic plate 16, a telescopic rod group 15, a third gear 13, a fourth gear and a second steering engine. The telescopic link group 15 includes a first link, a second link, a third link, and a fourth link. The inner ends of the first connecting rod and the second connecting rod are respectively fixed with a third gear 13 and are hinged with the edge of the lifting frame 8. The two third gears 13 mesh. The outer ends of the first connecting rod and the second connecting rod are respectively hinged with one ends of the third connecting rod and the fourth connecting rod. The other ends of the third connecting rod and the fourth connecting rod are respectively fixed with a fourth gear and are hinged with the expansion plate 16. The two fourth gears are meshed. The pitch diameters of the two third gears 13 and the two fourth gears are equal. The second steering engine is fixed on the lifting plate. The output shaft of the second steering engine is fixed with one of the third gears 13.
As shown in fig. 1 and 4, picking mechanism 2 includes a baffle 18, a stationary blade 19, a driving blade 20, and a third steering engine 21. The third steering engine 21 and the baffle 18 are both fixed on the expansion plate 16. The inner end of the driving blade 20 is fixed with the output shaft of the third steering engine 21. The outer end of the fixed blade 19 is fixed with the baffle 18, and the inner end forms a revolute pair with the output shaft of the third steering engine 21. The outer ends of the fixed blade 19 and the active blade 20 face to the side far away from the telescopic mechanism 3. The rotation energy of the third steering engine 21 drives the driving blade 20 to turn over towards the fixed blade 19, so that shearing action is realized. The bottom surface of the expansion plate 16 is fixed with a collecting net bag at the end far away from the expansion mechanism 3. The collecting net bag is positioned under the outer ends of the fixed blade 19 and the driving blade 20
The camera tripod head component comprises a tripod head base, a first tripod head, a second tripod head, a fourth steering engine and a fifth steering engine. The cradle head base is fixed on the bottom plate. The first tripod head and the tripod head base form a revolute pair with a public axis arranged vertically. And the fourth steering engine is fixed on the holder base. The output shaft of the fourth steering engine is fixed with the first cloud platform. The second cloud platform and the first cloud platform constitute the revolute pair that the public axis level set up. And the fifth steering engine is fixed on the first cloud platform frame. And an output shaft of the fifth steering engine is fixed with the second cloud platform. The camera is fixed on the second cloud platform. The camera adopts a visual module with the model of OpenMV3CamM 7.
The picking method of the cinnabar orange picking robot specifically comprises the following steps:
step one, starting a camera, and shooting a picture of a cinnabar orange tree to obtain an original color image. And carrying out graying treatment on the original color image to obtain a gray scale image, and preventing the influence of overexposure. Gray value Gray of pixel of ith row and jth column in Gray map i,j The expression of (2) is shown as the formula (1):
Gray i,j ＝0.072169B i,j +0.715160G i,j +0.212671R i,j (1)
in the formula (1), B i,j B value (i.e. blue channel brightness) for the pixel of the ith row and jth column in the original color image; g i,j The G value (i.e., green channel brightness) for the pixel in the ith row and jth column in the original color image; r is R i,j R value (i.e., red channel brightness) for the pixel in the ith row and jth column in the original color image.
And step two, binarizing the gray level map to obtain a black-and-white map. Pixel value a of ith row and jth column pixels in black-and-white image i,j The expression of (2) is shown as the following formula:
in the formula (2), 150.ltoreq.er.ltoreq.180, and in this embodiment, er is preferably 152.
If a is i,j =255, the ith row and jth column pixels in the black-and-white image are black; otherwise, the ith row and jth column pixels in the black-and-white image are white. At this time, the cinnabar orange which is orange in the original color image becomes white, and the rest colors become black.
And thirdly, median filtering is carried out on the black-and-white image, so that all pixels in a white pixel group with the pixel number smaller than x multiplied by y on the black-and-white image are changed into black, and a filtered black-and-white image is obtained. x=4, y=4; the white pixel group is composed of white pixels connected together. At this time, n white pixel groups remain in the filtered black-and-white image (the number of pixels in the remaining white pixel groups is larger than x×y).
And fourthly, selecting n undetermined identification areas from the original color image circle. The n undetermined identification areas on the original color image correspond to the n white pixel groups in the filtered black-and-white image in position respectively.
Step five: and sequentially intercepting all the undetermined identification areas on the original color image to obtain n undetermined identification images. And sequentially introducing n Zhang Daiding identification images into the trained convolutional neural network, and judging whether cinnabar orange exists in each pending identification image. M of n undetermined identification areas of the original color image are used as effective identification areas.
Step six, k=1, 2, …, m, steps seven and eight are performed in sequence.
Step seven, calculating the distance L between the cinnabar orange corresponding to the kth effective identification area and the camera k 。L k The expression of (2) is shown in the formula (3).
In the formula (3), K z For the distance calculation coefficient, the expression is as shown in formula (4):
in the formula (4), R m The average diameter of the cinnabar orange is 33mm, apix is the number of one row of pixel points in a photo taken by a camera, and the value is 320; θ is one half of the camera view angle, and its value is 57.5 °, thereby obtaining K z Has a value of 6727.
The principle of building the formula (3) and the formula (4) is shown in fig. 5, in which, L p is the focal length of the camera; />And taking the camera as an end point, and enabling two sides to be one half of the angle tangential to the cinnabar orange corresponding to the kth effective identification area. Thereby can push out +.> Since the radius distribution of the orange basically meets the normal distribution, the invention uses R m Instead of the diameter of the picked cinnabar orange, more than 95 percent of the cinnabar orange can be successfully picked.
Step eight, calculating the coordinate value (x) of the center position of the cinnabar orange corresponding to the kth effective identification area in a plane rectangular coordinate system with the camera as the origin of coordinates and the x-z plane perpendicular to the axis of the camera k ,y k ,z k )，x k 、y k 、z k The relation of (2) is shown in the formula (5).
In the formula (5), u xk Is the abscissa of the geometric center point of the kth effective identification area on the original color image (i.e., the geometric center point of the kth effective identification area is located at the kth position of the original color image xk On column pixels); v yk Is the ordinate of the geometric center point of the kth effective identification area on the original color image (i.e., the geometric center point of the kth effective identification area is located at the kth v of the original color image yk On the row of pixels); dx is an abscissa displacement coefficient, and the expression is shown in the formula (6); dy is an ordinate displacement coefficient, and the expression is shown as a formula (7);
in the formulas (6) and (7), L p The focal length of the camera is 2.8mm.
Step nine: assign 1 to k.
Step ten: the first steering engine drives the rotating plate 28 to rotate; the lifting motor drives the lifting frame 8 to move up and down. The second steering engine rotates to drive the expansion plate 16 to horizontally move. The active blade 20 and the fixed blade 19 move to the coordinate point (x k ,y k ,z k ) At the coordinates, the fruit stalks of the kth cinnabar orange are located between the active blade 20 and the fixed blade 19.
Step eleven: the third steering engine 21 drives the driving blade 20 to rotate, and the driving blade 20 and the fixed blade 19 cut fruit stems of the kth cinnabar orange, so that the kth cinnabar orange falls into the net bag, and the step twelve is entered.
Step twelve: the third steering engine 21 drives the driving blade 20 to reset. If k is less than m, increasing k by 1, and repeating steps ten and eleven; otherwise, step thirteenth is entered.
Step thirteen: the traveling mechanism 5 drives the floor transfer position, and thereafter, the steps one to twelve are repeatedly performed.
The convolutional neural network mentioned in the step six comprises three parts of a convolutional layer, a downsampling layer and a full-connection layer; the convolution layer carries out convolution with the filter to obtain s pieces of first feature images. The downsampling layer is also called a pooling layer and is used for receiving s pieces of first characteristic diagrams input from the convolution layer and correspondingly outputting s pieces of second characteristic diagrams. And in the full-connection layer, processing the s second feature images by adopting a variable weight error cost function, returning parameters such as weight and the like, and further obtaining whether the input image contains the cinnabar orange.
Claims (8)
1. A cinnabar orange picking robot comprises a travelling mechanism, a rotating mechanism, a lifting mechanism, a telescopic mechanism, a picking mechanism and a camera; the method is characterized in that: the travelling mechanism comprises a bottom plate, a travelling motor, travelling wheels and universal wheels; both travelling wheels are supported on the bottom surface of the bottom plate; the universal wheels are arranged on the bottom surface of the bottom plate; the two travelling wheels are respectively driven by two travelling motors; the camera is arranged on the bottom plate; the rotating mechanism comprises a rotating plate and a rotating driving piece; the rotating plate is arranged above the bottom plate and forms a revolute pair with the bottom plate; the rotating plate is driven by a rotating driving piece; the lifting mechanism comprises an upper supporting plate, a lower supporting frame, a lifting frame, an optical axis and a lifting driving piece; the bottom ends of the optical axes are fixed with the lower supporting frame, and the top ends of the optical axes are fixed with the upper supporting plate; the lower support frame is fixed with the rotating plate; the lifting frame and the optical axis form a sliding pair and are driven by a lifting driving piece;
the telescopic mechanism comprises a telescopic plate, a telescopic rod group, a third gear, a fourth gear and a second steering engine; the telescopic rod group comprises a first connecting rod, a second connecting rod, a third connecting rod and a fourth connecting rod; the inner ends of the first connecting rod and the second connecting rod are respectively fixed with a third gear and are hinged with the lifting frame; two third gears are meshed; the outer ends of the first connecting rod and the second connecting rod are respectively hinged with one ends of the third connecting rod and the fourth connecting rod; the other ends of the third connecting rod and the fourth connecting rod are respectively fixed with a fourth gear and are hinged with the expansion plate; the two fourth gears are meshed; one of the third gears is driven by the second steering engine;
the picking mechanism comprises a baffle, a fixed blade, a driving blade and a third steering engine; the third steering engine and the baffle are both fixed on the expansion plate; the inner end of the driving blade is fixed with an output shaft of the third steering engine; the outer end of the fixed blade is fixed with the baffle, and the inner end of the fixed blade and the output shaft of the third steering engine form a revolute pair;
the picking method of the cinnabar orange picking robot comprises the following steps:
starting a camera, and shooting a picture of a cinnabar orange tree to obtain an original color image; carrying out graying treatment on the original color image to obtain a gray scale image; gray value Gray of pixel of ith row and jth column in Gray map i,j The expression of (2) is shown as the formula (1):
Gray i,j ＝0.072169B i,j +0.715160G i,j +0.212671R i,j (1)
in the formula (1), B i,j Blue channel brightness for the pixel of the ith row and jth column in the original color image; g i,j Green channel brightness for the pixel of the ith row and jth column in the original color image; r is R i,j Red channel brightness for the pixel of the ith row and jth column in the original color image;
step two, binarizing the gray level map to obtain a black-and-white map; pixel value a of ith row and jth column pixels in black-and-white image i,j The expression of (2) is shown as the following formula:
in the formula (2), the er is more than or equal to 150 and less than or equal to 180;
thirdly, median filtering is carried out on the black-and-white image, so that all pixels in a white pixel group with the pixel number smaller than x multiplied by y on the black-and-white image are changed into black, and a filtered black-and-white image is obtained; x is more than or equal to 3 and less than or equal to 5, y is more than or equal to 3 and less than or equal to 5; the white pixel group is composed of white pixels connected together; filtering the remaining n white pixel groups in the black-and-white map;
step four, selecting n undetermined identification areas from the original color image circle; n to-be-determined identification areas on the original color image correspond to n white pixel groups in the filtered black-and-white image in position respectively;
step five: sequentially intercepting all undetermined identification areas on an original color image to obtain n undetermined identification images; sequentially importing n Zhang Daiding identification images into a convolutional neural network, and judging whether cinnabar orange exists in each identification image to be determined; m of n undetermined identification areas of the original color image are used as effective identification areas;
step six, k=1, 2, …, m, steps seven and eight are sequentially executed;
step seven, calculating the distance L between the cinnabar orange corresponding to the kth effective identification area and the camera k ；L k The expression of (2) is shown as a formula (3);
in the formula (3), K z For the distance calculation coefficient, the expression is as shown in formula (4):
in the formula (4), R m The Apix is the average diameter of the cinnabar orange and the number of one row of pixel points in a photo taken by the camera; θ is one half of the camera view angle;
step eight, calculating the coordinate value (x) of the center position of the cinnabar orange corresponding to the kth effective identification area in a plane rectangular coordinate system with the camera as the origin of coordinates and the x-z plane perpendicular to the axis of the camera k ,y k ,z k )，x k 、y k 、z k The relation of (2) is shown as a formula (5);
in the formula (5), u xk The abscissa of the geometric center point of the kth effective identification area on the original color image; v yk The ordinate of the geometric center point of the kth effective identification area on the original color image; dx is an abscissa displacement coefficient, and the expression is shown in the formula (6); dy is an ordinate displacement coefficient, and the expression is shown as a formula (7);
in the formulas (6) and (7), L p Is the focal length of the camera;
step nine: assigning 1 to k;
step ten: the first steering engine drives the rotating plate to rotate; the lifting motor drives the lifting frame to move up and down; the second steering engine rotates to drive the expansion plate to horizontally move, so that the fruit stalks of the kth cinnabar orange are positioned between the driving blade and the fixed blade;
step eleven: the third steering engine drives the driving blade to rotate, and the driving blade and the fixed blade cut fruit stalks of the kth cinnabar orange and enter a step twelve;
step twelve: the third steering engine drives the active blade to reset; if k is less than m, increasing k by 1, and repeating steps ten and eleven; otherwise, enter step thirteenth;
step thirteen: the travelling mechanism drives the bottom plate to shift positions, and then the steps one to twelve are repeatedly executed.
2. The cinnabar orange picking robot as defined in claim 1, wherein: the camera overturning assembly is also included; the camera tripod head component comprises a tripod head base, a first tripod head, a second tripod head, a fourth steering engine and a fifth steering engine; the cradle head base is fixed on the bottom plate; the first tripod head and the tripod head base form a revolute pair with a public axis arranged vertically; the fourth steering engine is fixed on the cradle head base; an output shaft of the fourth steering engine is fixed with the first cloud platform frame; the second cloud platform and the first cloud platform form a revolute pair with a common axis arranged horizontally; the fifth steering engine is fixed on the first cloud platform frame; an output shaft of the fifth steering engine is fixed with the second cloud platform frame; the camera is fixed on the second cloud platform.
3. The cinnabar orange picking robot as defined in claim 1, wherein: the rotary driving piece comprises a first steering engine, a first gear and a second gear; the first steering engine is fixed on the bottom plate; the first gear and the second gear are respectively fixed with the first steering engine output shaft and the rotating plate; the first gear is meshed with the second gear; the lifting driving piece comprises a belt wheel, a driving belt and a lifting motor; the two belt wheels are respectively supported on the upper supporting plate and the lower supporting frame and are connected through a transmission belt; the lifting motor is fixed on the lower supporting frame; the output shaft of the lifting motor is fixed with a belt wheel positioned on the lower supporting frame.
4. The cinnabar orange picking robot as defined in claim 1, wherein: the two travelling motors are fixed on the bottom surface of the bottom plate; the output shafts of the two travelling motors are respectively fixed with the two travelling wheels; the second steering engine is fixed on the lifting plate; the output shaft of the second steering engine is fixed with one of the third gears.
5. The cinnabar orange picking robot as defined in claim 1, wherein: the pitch circle diameters of the two third gears and the two fourth gears are equal.
6. The cinnabar orange picking robot as defined in claim 1, wherein: the outer ends of the fixed blade and the driving blade face to one side far away from the telescopic mechanism.
7. The cinnabar orange picking robot as defined in claim 1, wherein: the bottom surface of the expansion plate is fixed with a collecting net bag at the end far away from the expansion mechanism; the collecting net bag is positioned under the outer ends of the fixed blade and the driving blade.
8. The cinnabar orange picking robot as defined in claim 1, wherein: the camera adopts a visual module with the model of OpenMV3CamM 7; r is R m The value of (2) is 33mm; apix has a value of 320; the value of theta is 57.5 degrees; l (L) p The value of (2) is 2.8mm.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811345265.XA CN109348865B (en) | 2018-11-13 | 2018-11-13 | Cinnabar orange picking robot and picking method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811345265.XA CN109348865B (en) | 2018-11-13 | 2018-11-13 | Cinnabar orange picking robot and picking method thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109348865A CN109348865A (en) | 2019-02-19 |
| CN109348865B true CN109348865B (en) | 2023-08-29 |
Family
ID=65344873
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811345265.XA Active CN109348865B (en) | 2018-11-13 | 2018-11-13 | Cinnabar orange picking robot and picking method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109348865B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023235922A1 (en) * | 2022-06-06 | 2023-12-14 | Ripe Robotics Pty Ltd | Produce picking device and method |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102165880A (en) * | 2011-01-19 | 2011-08-31 | 南京农业大学 | Automatic-navigation crawler-type mobile fruit picking robot and fruit picking method |
| CN103538644A (en) * | 2013-10-14 | 2014-01-29 | 上海大学 | Robot with functions of rolling motion and foot walking |
| CN104318266A (en) * | 2014-10-19 | 2015-01-28 | 温州大学 | Image intelligent analysis processing early warning method |
| CN104751117A (en) * | 2015-01-26 | 2015-07-01 | 江苏大学 | Lotus seedpod target image recognition method for picking robot |
| CN106169081A (en) * | 2016-06-29 | 2016-11-30 | 北京工业大学 | A kind of image classification based on different illumination and processing method |
| CN206274663U (en) * | 2016-08-31 | 2017-06-23 | 南昌航空大学 | A kind of monopod video camera |
| WO2017181127A1 (en) * | 2016-04-15 | 2017-10-19 | The Regents Of The University Of California | Robotic plant care systems and methods |
| CN108513803A (en) * | 2018-06-14 | 2018-09-11 | 梧州学院 | Mandarin orange picks collection device |
| EP3373203A1 (en) * | 2017-03-09 | 2018-09-12 | Panasonic Corporation | Equipment of localization of peduncle and method of localization of peduncle |
| CN108764286A (en) * | 2018-04-24 | 2018-11-06 | 电子科技大学 | The classifying identification method of characteristic point in a kind of blood-vessel image based on transfer learning |
| CN209572474U (en) * | 2018-11-13 | 2019-11-05 | 杭州电子科技大学 | A kind of vermillion orange picking robot |
-
2018
- 2018-11-13 CN CN201811345265.XA patent/CN109348865B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102165880A (en) * | 2011-01-19 | 2011-08-31 | 南京农业大学 | Automatic-navigation crawler-type mobile fruit picking robot and fruit picking method |
| CN103538644A (en) * | 2013-10-14 | 2014-01-29 | 上海大学 | Robot with functions of rolling motion and foot walking |
| CN104318266A (en) * | 2014-10-19 | 2015-01-28 | 温州大学 | Image intelligent analysis processing early warning method |
| CN104751117A (en) * | 2015-01-26 | 2015-07-01 | 江苏大学 | Lotus seedpod target image recognition method for picking robot |
| WO2017181127A1 (en) * | 2016-04-15 | 2017-10-19 | The Regents Of The University Of California | Robotic plant care systems and methods |
| CN106169081A (en) * | 2016-06-29 | 2016-11-30 | 北京工业大学 | A kind of image classification based on different illumination and processing method |
| CN206274663U (en) * | 2016-08-31 | 2017-06-23 | 南昌航空大学 | A kind of monopod video camera |
| EP3373203A1 (en) * | 2017-03-09 | 2018-09-12 | Panasonic Corporation | Equipment of localization of peduncle and method of localization of peduncle |
| CN108764286A (en) * | 2018-04-24 | 2018-11-06 | 电子科技大学 | The classifying identification method of characteristic point in a kind of blood-vessel image based on transfer learning |
| CN108513803A (en) * | 2018-06-14 | 2018-09-11 | 梧州学院 | Mandarin orange picks collection device |
| CN209572474U (en) * | 2018-11-13 | 2019-11-05 | 杭州电子科技大学 | A kind of vermillion orange picking robot |
Non-Patent Citations (1)
| Title |
|---|
| 侯贵洋 ; 赵桂杰 ; 王璐瑶 ; .草莓采摘机器人图像识别系统研究.软件.2018,(第06期),全文. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109348865A (en) | 2019-02-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN1302277C (en) | Field crops nutrition diagnostic method | |
| CN108271533B (en) | Automatic climbing picking device and picking method thereof | |
| CN109348865B (en) | Cinnabar orange picking robot and picking method thereof | |
| CN106471803A (en) | Imaging system and method | |
| CN102495066A (en) | High-light-spectrum transmission image collecting system and bean-pod nondestructive testing method based on system | |
| CN209927732U (en) | Binocular camera structure for grain detection imaging | |
| CN104992446B (en) | The image split-joint method of non-linear illumination adaptive and its realize system | |
| CN105954021B (en) | The detection method of spiral bevel gear circular tooth contact in a kind of automotive rear axle differential gear | |
| CN208119434U (en) | A kind of camera shooting unmanned plane automatically extracted for UHVDC Transmission Lines conducting wire | |
| CN210741400U (en) | Plant phenotype 3D model reconstruction photography platform | |
| CN113812262B (en) | Tea-oil camellia fruit picking robot based on machine vision | |
| CN105844264B (en) | It is a kind of based on the recognition methods of tree peony fruit image of the oil of stress | |
| CN113965727A (en) | Plant growth full-period-oriented mobile phenotype information acquisition platform and method | |
| CN106218941A (en) | A kind of crawler type robot palletizer | |
| CN112098415B (en) | Nondestructive testing method for quality of waxberries | |
| CN116508493B (en) | Gantry crawler type tea picking robot and picking method thereof | |
| CN206260027U (en) | Camera assembly and electronic equipment | |
| CN214947353U (en) | Full-automatic cleaning device of pipeline inspection robot | |
| CN215179704U (en) | Multi-angle plant species intelligent recognition device | |
| CN111441193B (en) | Dinner plate forming and processing device | |
| CN209572474U (en) | A kind of vermillion orange picking robot | |
| CN205665005U (en) | Detection apparatus for spiral bevel gear flank of tooth contact zone among automobile rear drive differential mechanism | |
| CN216771539U (en) | Fruit quality comprehensive grading device based on machine vision and X-ray | |
| CN217407120U (en) | Harvester threshing cylinder with adjustable rotating speed | |
| CN217348221U (en) | Crop identification is with high spectrum detection unmanned aerial vehicle of descending buffering speed reduction |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |