CN114359546B - Day lily maturity identification method based on convolutional neural network - Google Patents
Day lily maturity identification method based on convolutional neural network Download PDFInfo
- Publication number
- CN114359546B CN114359546B CN202111650998.6A CN202111650998A CN114359546B CN 114359546 B CN114359546 B CN 114359546B CN 202111650998 A CN202111650998 A CN 202111650998A CN 114359546 B CN114359546 B CN 114359546B
- Authority
- CN
- China
- Prior art keywords
- neural network
- day lily
- image
- maturity
- point
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 241000756137 Hemerocallis Species 0.000 title claims abstract description 180
- 238000000034 method Methods 0.000 title claims abstract description 69
- 238000013527 convolutional neural network Methods 0.000 title claims abstract description 28
- 238000013528 artificial neural network Methods 0.000 claims abstract description 83
- 238000012549 training Methods 0.000 claims abstract description 37
- 238000013507 mapping Methods 0.000 claims abstract description 17
- 230000004913 activation Effects 0.000 claims abstract description 15
- 238000004364 calculation method Methods 0.000 claims abstract description 8
- 230000008569 process Effects 0.000 claims description 21
- 239000013598 vector Substances 0.000 claims description 18
- 238000005070 sampling Methods 0.000 claims description 12
- 230000001186 cumulative effect Effects 0.000 claims description 9
- 238000001514 detection method Methods 0.000 claims description 6
- 238000012795 verification Methods 0.000 claims description 6
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 claims description 3
- 238000012935 Averaging Methods 0.000 claims description 3
- 238000003062 neural network model Methods 0.000 claims description 3
- 238000010606 normalization Methods 0.000 claims description 3
- 238000012360 testing method Methods 0.000 claims description 3
- 210000000707 wrist Anatomy 0.000 claims description 3
- 238000013102 re-test Methods 0.000 claims description 2
- 238000002372 labelling Methods 0.000 abstract description 4
- 230000006870 function Effects 0.000 description 31
- 238000003306 harvesting Methods 0.000 description 6
- 230000000007 visual effect Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 235000013399 edible fruits Nutrition 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 238000012545 processing Methods 0.000 description 3
- 235000002566 Capsicum Nutrition 0.000 description 2
- 241000196324 Embryophyta Species 0.000 description 2
- 235000007688 Lycopersicon esculentum Nutrition 0.000 description 2
- 241000758706 Piperaceae Species 0.000 description 2
- 240000003768 Solanum lycopersicum Species 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 244000298697 Actinidia deliciosa Species 0.000 description 1
- 235000009436 Actinidia deliciosa Nutrition 0.000 description 1
- 208000000044 Amnesia Diseases 0.000 description 1
- 208000031091 Amnestic disease Diseases 0.000 description 1
- 206010061218 Inflammation Diseases 0.000 description 1
- 244000141359 Malus pumila Species 0.000 description 1
- 208000004880 Polyuria Diseases 0.000 description 1
- 235000021307 Triticum Nutrition 0.000 description 1
- 244000098338 Triticum aestivum Species 0.000 description 1
- 240000008042 Zea mays Species 0.000 description 1
- 235000005824 Zea mays ssp. parviglumis Nutrition 0.000 description 1
- 235000002017 Zea mays subsp mays Nutrition 0.000 description 1
- 230000006986 amnesia Effects 0.000 description 1
- 238000000137 annealing Methods 0.000 description 1
- 235000021016 apples Nutrition 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 235000005822 corn Nutrition 0.000 description 1
- 238000005260 corrosion Methods 0.000 description 1
- 230000007797 corrosion Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000035619 diuresis Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000023597 hemostasis Effects 0.000 description 1
- 230000036039 immunity Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
Landscapes
- Image Analysis (AREA)
Abstract
The invention discloses a day lily maturity identification method based on a convolutional neural network, which comprises the following steps: collecting image data of daylily and labeling characteristic points of the daylily; modifying the output layer, the activation function and the loss function of the YOLOv3 neural network; training on the acquired image dataset using the co dataset pre-trained neural network parameters; predicting the characteristic points of the day lily by using the trained neural network to obtain coordinates of the characteristic points in an image coordinate system; obtaining space coordinates by utilizing coordinates of the feature points in an image coordinate system and a depth image thereof; calculating the length characteristics of the day lily according to the space coordinates; the maturity of the day lily is obtained through the mapping relation between the maturity of the day lily and the length characteristics thereof. The method and the device have the advantages that the position and length information of the day lily are obtained in a mode of combining the neural network identification characteristic points with the three-dimensional information extracted by the depth camera, the maturity of the day lily is judged through the length, the identification rate is high, and the calculation cost is low.
Description
Technical Field
The invention relates to the technical field of intelligent daylily maturity identification, in particular to a daylily maturity identification method based on a convolutional neural network.
Background
The day lily, also called day lily and amnesia, is a plant of the genus Hemerocallis of the family Alternaceae. There are a large number of plants in Qinling mountain in south China, hunan China, shanxi Kong, jiangsu and Zhejiang provinces. It not only can be eaten, but also has various medical values such as hemostasis, anti-inflammation, heat clearing, diuresis promoting and the like.
At present, most of agricultural harvesting machines in China are still large manually operated machines, such as wheat and corn harvesting. Such a machine can only be used for harvesting hard crops, which need to be re-planted every year, without fear of the crops being broken up by the machine during harvesting. For crops such as tomatoes, peppers and the like, large harvesting machines are not suitable because tomatoes are soft and are very easily damaged by the machine in the picking process, while peppers grow on trees and are not suitable for use. For this situation, many developers develop intelligent robots for harvesting such fruits, and in the development process of the intelligent robots, the most critical technology is the research of visual recognition technology and the research of intelligent obstacle avoidance and path planning algorithm. For the vision picking algorithm, researchers usually detect targets by using a clustering algorithm, a Hough detection algorithm and the like and denoise the targets by using an image processing algorithm such as expansion, corrosion and the like. Although the visual processing algorithm is mature gradually, a good recognition effect can be obtained, the visual processing algorithm can only detect targets with obvious color and gradient characteristics, and the accuracy is low under the condition that the color and gradient characteristics are not obvious. The color characteristics of fruits and branches of day lily are not obvious, and the day lily is picked at night frequently, so that the difficulty of visual identification of a picking robot is increased.
Although the conventional visual recognition algorithm has been developed for many years, the accuracy is still lower under the condition that the agricultural picking is complex, so that many researchers apply the neural network algorithm to the agricultural picking. The most widely applied target detection algorithm predicts and positions the target through a convolutional neural network, has high precision and robustness, and has practical significance in the identification of round crops such as kiwi fruits, apples and the like. However, the algorithm adopts the anchor frame to carry out identification frame selection on the target, and the identification effect on the daylily fruits in the long strip shape is poor, because the pixels of the daylily only occupy a small part of the whole anchor frame. Therefore, there is a need for an image recognition method for daylily feature points.
Disclosure of Invention
The invention aims to provide a day lily maturity identification method based on a convolutional neural network, which is used for predicting feature points of day lily, extracting length features by using the obtained feature points through a 3D camera and finally obtaining the maturity and position of the day lily, so that the problems that the gradient and color features are not obvious in the traditional visual identification method can not be identified, and the problem that the shape of the day lily is not suitable for a target detection method generally by using an anchor frame can be solved.
In order to achieve the above object, the present invention provides the following solutions:
a day lily maturity identification method based on convolutional neural network comprises the following steps:
s1, acquiring images of day lily at night and day time by using a 3D camera based on TOF, marking characteristic points of the day lily in the acquired images by using a marking program, and storing an image dataset and a label thereof;
s2, constructing a convolutional neural network, and modifying an output layer, an activation function and a loss function of the YOLOv3 neural network, wherein the output layer is modified into a vector of x 24; the feature point coordinates that are not in the image are set to (-0.1 ), and the activation function is the leak-ReLU function:
the loss function is defined as:
wherein x is i And y i Respectively represent the coordinate prediction results of the neural network,and->Respectively representing the coordinate true values in the data set, C i And->Representing confidence and true values of the predictions, M i And->Respectively representing whether the predicted target is blocked; s represents the final predicted tensor side length of the model, and B represents 3 size categories responsible for predicting the target; the first row of cumulative terms represents the loss function of node position in the image, +>Representing the predicted point in the image; the second row of accumulated terms represents the position loss function of nodes not in the image, which weights are respectively lambda cooin And lambda (lambda) coonoin ，/>Indicating that the predicted point is not in the image; the third row of cumulative terms is a loss function of the probability of the target being present in the anchor frame, +.>Indicating that a target exists in the grid responsible for prediction; the last row of cumulative terms represents the loss function of the probability that the object is occluded +.>Representing the objectIs blocked with loss weight lambda mask ；
S3, training on the image data set acquired in the step S1 by using the neural network parameters pre-trained by the COCO data set;
s4, capturing image data and a depth map of the day lily by using a 3D camera based on TOF in the picking process, and predicting feature points of the day lily by using the neural network trained in the step S3 to obtain coordinates of the feature points in an image coordinate system;
s5, obtaining the space coordinates of the feature points by utilizing the coordinates of the feature points in the image coordinate system and the depth images thereof;
s6, calculating the distance between the characteristic points through the obtained space coordinates of the characteristic points, namely the length characteristics of the day lily;
and S7, obtaining the maturity of the day lily through the mapping relation between the maturity of the day lily and the length characteristics of the day lily, and simultaneously, using the space coordinates of the characteristic points to represent the position of the day lily to finish the identification and positioning of the day lily.
Further, in the step S1, the image of the day lily at night and day is collected by using a 3D camera based on TOF, the feature points of the day lily in the collected image are marked by using a marking program, and the image dataset and the label thereof are saved, specifically including:
1) When the day lily data set is collected, the time periods are uniformly distributed, namely the number of pictures in each time period is close to that in 24 hours a day;
2) When an image is acquired, the height of a 3D camera based on TOF is kept between 500mm and 1300mm, and the distance is the height range of day lily growth and is also the camera setting height during picking;
3) Randomly segmenting the picked image into 3 parts: training set, verification set, test set.
Further, in the step S2, a convolutional neural network is constructed, and an output layer, an activation function and a loss function of the YOLOv3 neural network are modified, which specifically includes:
1) Adjusting a neural network output layer:
the neural network has two outputs which are respectively used for predicting daylily targets with different sizes in the image, the dimensions of the output layers are respectively 13-12-2 and 26-12-2, and in order to initialize by using training parameters of YOLOv3, the neural network only changes a last 1*1 convolution layer;
13×13 and 26×26 in the output layer dimension of the neural network represent the number of grids divided in two outputs, where each grid is responsible for predicting two targets, and the two targets are represented by ten-dimensional vectors, that is, the probability that a target exists in the grid, the probability that the target is blocked, and the coordinates of four feature points of the target, and if the center point of the labeled real label falls within the grid, the grid is responsible for predicting the target, and the calculation formula of the center point of the target is as follows:
wherein, in the formula, x i And y i Respectively representing the image coordinate prediction results of the center points in the ith grid, wherein n represents the side length of the grid;
2) Expression mode of outputting feature point coordinates
The coordinate expression method is adopted as a polar coordinate expression method, the sitting angle of the grid responsible for predicting the point is set as a coordinate origin, the polar coordinate position of a certain characteristic point relative to the coordinate origin is (theta, r), and the coordinate system is normalized to obtain the prediction label of the final output layer, and the normalization method is as follows:
wherein w represents the size of the input image, and the predicted label of the output layer of a certain characteristic point is (theta ', r');
3) Activation function of neural network
For daylily targets with characteristic points not in the image, the real position label marks the coordinates of the characteristic points not in the image as a polar coordinate system (0, -0.1), and negative values are output to predict the daylily targets with the characteristic points not in the image; in order to make the output value negative, the activation function of the neural network is a Leaky-ReLU, and when the characteristic points are calculated and are not in the center point of the daylily target in the image, the characteristic points which are not in the image are ignored, and the average value of the coordinates of other characteristic points is calculated;
4) Loss function of neural network
In the neural network for identifying the characteristic points of the day lily, the neural network comprises probability loss, blocked probability loss and position coordinate loss of the characteristic points, wherein the position coordinate loss of the characteristic points is divided into coordinate loss of the characteristic points in the image and position loss of the characteristic point coordinates not in the image, larger weight is allocated to the position loss in the image, small weight is allocated to the position loss of the characteristic points not in the image, and finally a neural network loss function is obtained;
5) Network structure
The network structure is the structure corresponding to the first two output layers of the YOLOv3 network structure, namely the output obtained by 32 times downsampling and 16 times downsampling, the output dimension of the convolution layer of the last layer 1*1 of the network is modified to be 20, namely the neural network output layer, and the network structure branch corresponding to 8 times downsampling is deleted, namely the final neural network structure.
Further, in the step S3, training is performed on the image dataset acquired in the step S1 by using the neural network parameters pre-trained by the COCO dataset, which specifically includes:
1) Acquiring a network weight of the YOLOv3 as initialization data of the neural network, wherein the method is to download YOLOv3 training data in an open source website, and the data is required to be trained and stored by using a pytorch, so that a COCO data set can be subjected to target detection;
2) Extracting the corresponding weights of the first two outputs in the network parameters by using a pytorch, namely, the outputs obtained by 32 times downsampling and 16 times downsampling, deleting the weight of the last 1*1 convolution layer, then matching the weight with the established neural network model, and randomly initializing the network parameters of the last convolution layer to obtain the initialization weights of all layers of the neural network;
3) And (3) finishing the training process of the neural network by using the training set, verifying by using the verification set after finishing one-time training of all the training sets, preventing the network from being over-fitted, and finishing the training of the network by using an Adam optimizer in a pytorch in the training process and setting a hyper-parameter reference YOLOv 3.
Further, in the step S4, capturing image data and a depth map of day lily by using a 3D camera based on TOF during picking, specifically including:
the 3D camera based on TOF is arranged at the wrist of the picking manipulator, three depth images and photos are required to be acquired for identification in the picking process, the daylily to be picked and the position of the daylily are determined by photographing for the first time, after the manipulator moves to the vicinity of the daylily, the daylily is positioned again at the moment, photographing before picking is performed for the third time, retest accuracy is performed, photographing and calibration are repeated if the position of the manipulator is not at a preset position, and picking is performed on the target daylily if the position of the manipulator is just.
Further, in step S5, the coordinates of the feature points in the image coordinate system and the depth image thereof are used to obtain the spatial coordinates of the feature points, specifically:
and calculating three-dimensional coordinates of the feature points by using the identified feature points and the depth image, wherein the point A is taken as an example, and the calculation formula is as follows:
wherein x is A' ,y A' Is the horizontal and vertical coordinates of the feature point in the image coordinate system, namely the feature point coordinate identified by the neural network, f is the focal length of the camera, and is obtained by calibrating the camera, d A For the distance measured by the TOF-based 3D camera, the three-dimensional coordinates of the A point in the camera coordinate system in the image coordinate system are obtained as (x A ,y A ,z A )。
Further, in step S6, the distance between the feature points, that is, the length feature of day lily, is calculated according to the obtained spatial coordinates of the feature points, which specifically includes:
1) The distance between the two feature points, namely the distance between the point A and the point B, is calculated by the three-dimensional coordinates obtained in the step S5 as follows:
wherein, the length (A, B) is the distance between the point A and the point B, namely the length characteristic of the day lily;
2) Setting 4 characteristic points to divide day lily into 3 sections, wherein the length characteristic of the day lily is the sum of 3 sections of distances, and the first characteristic point is at the bottommost part of the day lily, namely the position needing to be broken off during picking; the second characteristic point is at the intersection point position of the flower stalks and the flower buds of the day lily; the third characteristic point is positioned at the middle position of the second characteristic point and the fourth characteristic point; the fourth feature point is located at the top end of day lily.
Further, in step S7, the maturity of the day lily is obtained through the mapping relationship between the maturity of the day lily and the length characteristics thereof, and meanwhile, the position of the day lily is represented by the space coordinates of the characteristic points, so as to complete the identification and positioning of the day lily, which specifically comprises:
the method for obtaining the mapping relation by researching the mapping relation between the maturity and the length characteristics of the day lily and obtaining the maturity of the day lily by using a linear interpolation method comprises the following steps:
selecting 100 daylily which starts growing of the same variety, periodically measuring and recording the length of the daylily, continuously sampling until the daylily is mature in a sampling period of 0.5 day, finally obtaining 100 vectors with different dimensions, correspondingly removing the length information of the overlong vectors with earlier sampling time from back to front according to the sampling time, averaging 100 data sampled each time to obtain a final vector, wherein each data of the vector corresponds to one maturity, recording the length corresponding to each maturity, and obtaining the mapping relation between the maturity and the length through a linear interpolation method.
According to the specific embodiment provided by the invention, the invention discloses the following technical effects: according to the day lily maturity identification method based on the convolutional neural network, the characteristic points of the day lily are predicted by using the convolutional neural network, and the length characteristics are extracted by using the obtained characteristic points through a 3D camera, so that the method for finally obtaining the maturity and the position of the day lily is achieved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the prior art, the drawings that are needed in the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic flow chart of a day lily maturity identification method based on a convolutional neural network according to an embodiment of the invention;
FIG. 2 is a schematic diagram of a neural network architecture according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of an output layer structure according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of calculating three-dimensional coordinates of feature points according to an embodiment of the present invention;
fig. 5 is a schematic diagram of day lily feature point positions according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The invention aims to provide a day lily maturity identification method based on a convolutional neural network, which is a method for predicting day lily characteristic points by using the convolutional neural network, extracting length characteristics by using the obtained characteristic points through a 3D camera and finally obtaining the maturity and the position of the day lily.
In order that the above-recited objects, features and advantages of the present invention will become more readily apparent, a more particular description of the invention will be rendered by reference to the appended drawings and appended detailed description.
The application scene of the embodiment of the invention is a day lily planting base, the day lily in the base is a day lily of a single variety, and the length characteristic shape is relatively stable. First, image information of day lily is collected, and the process is not limited to the collection in the above-mentioned base, but image data can be obtained by means of network downloading, video extraction and the like. And labeling the collected data, and storing labeling results. Meanwhile, an experiment of the mapping relation between the length characteristics and the maturity is carried out on the base, and related data are recorded, processed and stored. The neural network shown in fig. 2 is trained using the saved data and the labels thereof. Predicting the feature points of day lily by using the network obtained through training, obtaining a depth image by using a 3D camera based on TOF, extracting the corresponding depth of the feature points on the depth image, and calculating the three-dimensional coordinates of the feature points.
As shown in fig. 1, the day lily maturity identification method based on convolutional neural network provided by the invention comprises the following steps:
s1, acquiring images of day lily at night and day time by using a 3D camera based on TOF, marking characteristic points of the day lily in the acquired images by using a marking program, and storing an image dataset and a label thereof;
the method specifically comprises the following steps:
1) When the day lily data set is collected, the time periods are uniformly distributed, namely the number of pictures in each time period is close to that in 24 hours a day;
2) When an image is acquired, the height of a 3D camera based on TOF is kept between 500mm and 1300mm, and the distance is the height range of day lily growth and is also the camera setting height during picking;
3) Randomly segmenting the picked image into 3 parts: training set, verification set, test set.
S2, constructing a convolutional neural network, and modifying an output layer, an activation function and a loss function of the YOLOv3 neural network, wherein the output layer is modified into a vector of x 24; the feature point coordinates that are not in the image are set to (-0.1 ), and the activation function is the leak-ReLU function:
the loss function is defined as:
wherein x is i And y i Respectively represent the coordinate prediction results of the neural network,and->Respectively representing the coordinate true values in the data set, C i And->Representing confidence and true values of the predictions, M i And->Respectively representing whether the predicted target is blocked; s represents the final predicted tensor side length of the model, and B represents 3 size categories responsible for predicting the target; the first row of cumulative terms represents the loss function of node position in the image, +>Representing the predicted point in the image; the second row of accumulated terms represents the position loss function of nodes not in the image, which weights are respectively lambda cooin And lambda (lambda) coonoin ，/>Indicating that the predicted point is not in the image; the third row of cumulative terms is a loss function of the probability of the target being present in the anchor frame, +.>Indicating that a target exists in the grid responsible for prediction; the last row of cumulative terms represents the loss function of the probability that the object is occluded +.>Indicating that the target is occluded, the loss weight is lambda mask ；
S3, training on the image data set acquired in the step S1 by using the neural network parameters pre-trained by the COCO data set;
s4, capturing image data and a depth map of the day lily by using a 3D camera based on TOF in the picking process, and predicting feature points of the day lily by using the neural network trained in the step S3 to obtain coordinates of the feature points in an image coordinate system;
s5, obtaining the space coordinates of the feature points by utilizing the coordinates of the feature points in the image coordinate system and the depth images thereof;
s6, calculating the distance between the characteristic points through the obtained space coordinates of the characteristic points, namely the length characteristics of the day lily;
and S7, obtaining the maturity of the day lily through the mapping relation between the maturity of the day lily and the length characteristics of the day lily, and simultaneously, using the space coordinates of the characteristic points to represent the position of the day lily to finish the identification and positioning of the day lily.
The network structure finally obtained in step S2 is shown in fig. 2, and specific details of the network structure are as follows:
1) Adjusting a neural network output layer:
the neural network has two outputs which are respectively used for predicting daylily targets with different sizes in the image, and referring to the structure of the Yolov3 neural network, the neural network can completely refer to the first two outputs of the Yolov3, namely the outputs obtained by 32 times of downsampling and 16 times of downsampling.
Since the neural network ultimately identifies feature points of daylily instead of a target which is not a prediction frame, the dimension and meaning of the output layer also change accordingly. The output layer dimensions of the neural network are 13×13×12×2 and 26×26×12×2, and the neural network changes only the last 1*1 convolutional layer in order to initialize with the training parameters of YOLOv 3.
13 x 13 and 26 x 26 in the neural network output layer dimension represent the number of grids divided in the two outputs, respectively. Each grid is responsible for predicting two targets, and the two targets are respectively represented by ten-dimensional vectors, namely, the probability that the target exists in the grid, the probability that the target is blocked and the coordinates of four characteristic points of the target. If the center point of the labeled real label falls within the grid, the grid is responsible for predicting the target, and the center point of the target is calculated as follows:
wherein x is i And y i Respectively representing the image coordinate prediction results of the center points in the ith grid, wherein n represents the side length of the grid;
2) Expression mode of outputting feature point coordinates
In order to enhance the final prediction accuracy, it is necessary to design the relationship between the expression of the coordinates of the output feature points and the grid responsible for predicting the points. The invention adopts a coordinate expression method as a polar coordinate expression method, the sitting upper angle of a grid responsible for predicting the point is set as a coordinate origin, the polar coordinate position of a certain characteristic point relative to the coordinate origin is (theta, r), and the coordinate system is normalized to obtain the prediction label of the final output layer, and the normalization method is as follows:
wherein w represents the size of the input image, and the prediction label of the output layer of a certain feature point is (θ ', r'), as shown in fig. 3;
3) Activation function of neural network
In the prediction process of the network, feature points are often not in the image, and for the daylily targets, the real position labels label the coordinates of points which are not in the image as a polar coordinate system (0, -0.1), so that the neural network needs to output negative values to predict the targets containing the points which are not in the image; in order to make the output value negative, the activation function of the neural network is a leakage-ReLU, and when calculating the center points of the targets, the points which are not in the image are ignored to calculate the average value of other point coordinates;
4) Loss function of neural network
The training process of the neural network is a process that the neural network loss function tends to 0, so that the reasonable setting of the loss function affects the accuracy of final identification and the training speed of the neural network. The neural network for identifying the characteristic points of the day lily comprises probability loss, blocked probability loss and position coordinate loss of the characteristic points. The position coordinate loss of the feature points is further divided into the coordinate loss of the feature points in the image and the position loss of the feature point coordinates not in the image. A larger weight is allocated to the position loss in the image, and a smaller weight is allocated to the position loss of the feature points not in the image, so that the neural network loss function as claimed in claim one is finally obtained.
5) Network structure
As shown in fig. 2, the network structure is a structure corresponding to the first two output layers of the YOLOv3 network structure, that is, outputs obtained by 32 times of downsampling and 16 times of downsampling correspond to y1 and y2 in fig. 2, respectively. And modifying the output dimension of the last 1*1 convolution layer of the network to 20, namely the neural network output layer, and deleting the network structure branch corresponding to 8 times of downsampling to obtain the final neural network structure.
In the step S3, training is performed on the image dataset acquired in the step S1 by using the neural network parameters pre-trained by the COCO dataset, which specifically includes:
1) Acquiring a network weight of the YOLOv3 as initialization data of the neural network, wherein the method is to download YOLOv3 training data in an open source website, and the data is required to be trained and stored by using a pytorch, so that a COCO data set can be subjected to target detection;
2) Extracting the corresponding weights of the first two outputs in the network parameters by using a pytorch, namely, the outputs obtained by 32 times downsampling and 16 times downsampling, deleting the weight of the last 1*1 convolution layer, then matching the weight with the established neural network model, and randomly initializing the network parameters of the last convolution layer to obtain the initialization weights of all layers of the neural network;
3) And (3) finishing the training process of the neural network by using the training set, verifying by using the verification set after finishing one-time training of all the training sets, preventing the network from being over-fitted, and finishing the training of the network by using an Adam optimizer in a pytorch in the training process and setting a hyper-parameter reference YOLOv 3. Specifically, the super parameter settings are shown in table 1.
TABLE 1 super parameter settings
| Learning rate | 0.00258 | Cosine annealing super parameter | 0.17 |
| Momentum of learning rate | 0.779 | Weight decay coefficient | 0.00058 |
In step S4, capturing image data and a depth map of day lily by using a 3D camera based on TOF during picking, specifically including:
the TOF camera is arranged at the wrist of the picking manipulator, and three depth images and photos are required to be acquired for identification in the picking process. The day lily to be picked and the position thereof are determined by photographing for the first time, and after the mechanical arm moves to the vicinity of the day lily by photographing for the second time, the day lily is positioned again at the moment, so that the precision is higher than that of the first time. The third photographing is a photographing before picking, the purpose of the photographing is retesting accuracy, photographing and calibration are repeated if the position of the manipulator is not at the preset position, and picking is carried out on the target day lily if the position of the manipulator is just right. The same method as the step S1 is adopted to collect images in the day lily picking process, namely, the TOF 3D camera is used for collecting images to identify and position characteristic points of the day lily.
The TOF camera works according to the following principle: the infrared emitter installed on the camera emits infrared rays forwards, the receiver is installed on the camera, the infrared rays are reflected back when the infrared rays are emitted on an object, and the distance between the camera and a target point is judged by calculating the flight time of the infrared rays. When the camera emits infrared rays to different angles and calculates the flight time, a depth image of the camera can be obtained.
In step S5, as shown in fig. 4, the coordinates of the feature points in the image coordinate system and the depth image thereof are used to obtain the spatial coordinates of the feature points, specifically:
the three-dimensional coordinates of the feature point are calculated by using the identified feature point and the depth image, as shown in fig. 4, taking the point a as an example, the calculation formula is as follows:
wherein x 'is' A ,y' A Is the horizontal and vertical coordinates of the feature point in the image coordinate system, namely the feature point coordinate identified by the neural network, f is the focal length of the camera, and is obtained by calibrating the camera, d A For the distance measured by the TOF-based 3D camera, the three-dimensional coordinates of the A point in the camera coordinate system in the image coordinate system are obtained as (x A ,y A ,z A )。
In step S6, the distance between the feature points, that is, the length feature of day lily, is calculated according to the obtained spatial coordinates of the feature points, which specifically includes:
1) The distance between the two feature points, namely the distance between the point A and the point B, is calculated by the three-dimensional coordinates obtained in the step S5 as follows:
wherein, the length (A, B) is the distance between the point A and the point B, namely the length characteristic of the day lily;
2) As shown in fig. 5, since the fruit of day lily is not necessarily a straight line, but is a curve in most cases, 4 feature points are provided to divide day lily into 3 segments, and the length feature of day lily is the sum of 3 segments of distances. Wherein the first point is at the bottommost part of the day lily, namely the position which needs to be broken off during picking; the second point is positioned at the intersection point of the flower stalks and the flower buds of the day lily; the third point is positioned at the middle position between the second point and the fourth point; the fourth point is located at the top of day lily.
In step S7, the maturity of the day lily is obtained through the mapping relationship between the maturity of the day lily and the length characteristics thereof, and meanwhile, the position of the day lily is represented by the space coordinates of the characteristic points, so that the identification and the positioning of the day lily are completed, and the method specifically comprises the following steps:
the method for obtaining the mapping relation by researching the mapping relation between the maturity and the length characteristics of the day lily and obtaining the maturity of the day lily by using a linear interpolation method comprises the following steps:
selecting 100 daylily which starts growing of the same variety, periodically measuring and recording the length of the daylily, continuously sampling until the daylily is mature in a sampling period of 0.5 day, finally obtaining 100 vectors with different dimensions, correspondingly removing the length information of the overlong vectors with earlier sampling time from back to front according to the sampling time, averaging 100 data sampled each time to obtain a final vector, wherein each data of the vector corresponds to one maturity, recording the length corresponding to each maturity, and obtaining the mapping relation between the maturity and the length through a linear interpolation method.
The error generated by this embodiment is mainly composed of two parts, including an error generated by the resolution of the camera and an error generated by the measurement depth. The maximum error can be obtained by:
in the formula e x 、e y 、e z The values of (2) are:
in the formula e x' And d is the distance between the TOF camera and the target point, and f is the focal length obtained by calibrating the camera.
The maximum error that can be obtained by calculation for the method at different distances is shown in table 2:
TABLE 2
| Distance (mm) | Maximum error (mm) |
| 150 | 2.1 |
| 250 | 2.8 |
| 350 | 3.7 |
| 450 | 4.6 |
| 550 | 5.5 |
| 650 | 6.5 |
| 750 | 7.4 |
| 850 | 8.4 |
| 950 | 9.3 |
| 1050 | 10.3 |
As can be seen from table 2, the maximum error of the length feature recognized by the present invention is 10.3mm, and the distance between the camera and the recognition feature point is about 1m, and the error is within the allowable range for the first recognition. As the mechanical arm approaches the daylily to be picked in the picking process, the identification error is gradually reduced, so that the invention can ensure the accuracy of identification length characteristics and positioning.
In conclusion, the day lily maturity identification method based on the convolutional neural network provided by the invention has the characteristics that 1) the improved noise immunity of the neural network is very strong, the characteristic points of the day lily are identified by using the convolutional neural network, the robustness of the neural network is good, the accuracy is high and the like; 2) By predicting the characteristic points of the day lily, compared with a semantic segmentation algorithm for directly making the whole image, the method has the advantages of high labeling speed and low calculation force requirement; 3) Only predicting the characteristic points of the daylily, and finally, generating no point cloud data, thereby reducing the requirement on the performance of a computer and reducing the production cost of the picking robot; 4) The maturity is identified by utilizing the length characteristics of the daylily, and when the daylily of different varieties is handled, the identification mode can be switched between the different varieties only by changing the length setting, so that the daylily picking device has the practical meaning and popularization meaning of identifying and picking; 5) The neural network is trained according to the image characteristics of day 1 and 24 hours of day lily, the recognition effect is good in the daytime and at night, positioning is completed in the recognition process, meanwhile, the growth direction of day lily can be obtained, and the neural network has a direct guiding function on planning a picking path of a picking robot.
The principles and embodiments of the present invention have been described herein with reference to specific examples, the description of which is intended only to assist in understanding the methods of the present invention and the core ideas thereof; also, it is within the scope of the present invention to be modified by those of ordinary skill in the art in light of the present teachings. In view of the foregoing, this description should not be construed as limiting the invention.
Claims (8)
1. A day lily maturity identification method based on a convolutional neural network is characterized by comprising the following steps:
s1, acquiring images of day lily at night and day time by using a 3D camera based on TOF, marking characteristic points of the day lily in the acquired images by using a marking program, and storing an image dataset and a label thereof;
s2, improving a convolutional neural network, and modifying an output layer, an activation function and a loss function of the YOLOv3 neural network, wherein the output layer is modified into a vector of x 24; the feature point coordinates that are not in the image are set to (-0.1 ), and the activation function is the leak-ReLU function:
the loss function is defined as:
wherein x is i And y i Respectively represent the coordinate prediction results of the neural network,and->Respectively representing the coordinate true values in the data set, C i And->Representing confidence and true values of the predictions, M i And->Respectively representing whether the predicted target is blocked; s represents the final predicted tensor side length of the model, and B represents 3 size categories responsible for predicting the target; the first row of cumulative terms represents the loss function of node position in the image, +>Representing the predicted point in the image; the second row of accumulated terms represents the position loss function of nodes not in the image, which weights are respectively lambda cooin And lambda (lambda) coonoin ，/>Indicating that the predicted point is not in the image; the third row of cumulative terms is a loss function of the probability of the target being present in the anchor frame, +.>Indicating that a target exists in the grid responsible for prediction; the last row of cumulative terms represents the loss function of the probability that the object is occluded +.>Indicating that the target is occluded, the loss weight is lambda mask ；
S3, training on the image data set acquired in the step S1 by using a neural network pre-trained by the COCO data set;
s4, capturing image data and a depth map of the day lily by using a 3D camera based on TOF in the picking process, and predicting feature points of the day lily by using the neural network trained in the step S3 to obtain coordinates of the feature points in an image coordinate system;
s5, obtaining the space coordinates of the feature points by utilizing the coordinates of the feature points in the image coordinate system and the depth images thereof;
s6, calculating the distance between the characteristic points through the obtained space coordinates of the characteristic points, namely the length characteristics of the day lily;
and S7, obtaining the maturity of the day lily through the mapping relation between the maturity of the day lily and the length characteristics of the day lily, and simultaneously, using the space coordinates of the characteristic points to represent the position of the day lily to finish the identification and positioning of the day lily.
2. The day lily maturity recognition method based on the convolutional neural network according to claim 1, wherein in the step S1, images of day lily at night and day are collected by using a 3D camera based on TOF, feature points of day lily in the collected images are marked by using a marking program, and an image dataset and a label thereof are saved, specifically comprising:
1) When the day lily data set is collected, the time periods are uniformly distributed, namely the number of pictures in each time period is close to that in 24 hours a day;
2) When an image is acquired, the height of a 3D camera based on TOF is kept between 500mm and 1300mm, and the distance is the height range of day lily growth and is also the camera setting height during picking;
3) Randomly segmenting the picked image into 3 parts: training set, verification set, test set.
3. The daylily maturity identification method based on the convolutional neural network according to claim 2, wherein in the step S2, a convolutional neural network is constructed, and an output layer, an activation function and a loss function of the YOLOv3 neural network are modified, specifically comprising:
1) Adjusting a neural network output layer:
the neural network has two outputs which are respectively used for predicting daylily targets with different sizes in the image, the dimensions of the output layers are respectively 13-12-2 and 26-12-2, and in order to initialize by using training parameters of YOLOv3, the neural network only changes a last 1*1 convolution layer;
13×13 and 26×26 in the output layer dimension of the neural network represent the number of grids divided in two outputs, where each grid is responsible for predicting two targets, and the two targets are represented by ten-dimensional vectors, that is, the probability that a target exists in the grid, the probability that the target is blocked, and the coordinates of four feature points of the target, and if the center point of the labeled real label falls within the grid, the grid is responsible for predicting the target, and the calculation formula of the center point of the target is as follows:
wherein x is i And y i Respectively representing the image coordinate prediction results of the center points in the ith grid, wherein n represents the side length of the grid;
2) Expression mode of outputting feature point coordinates
The coordinate expression method is adopted as a polar coordinate expression method, the sitting angle of the grid responsible for predicting the point is set as a coordinate origin, the polar coordinate position of a certain characteristic point relative to the coordinate origin is (theta, r), and the coordinate system is normalized to obtain the prediction label of the final output layer, and the normalization method is as follows:
wherein w represents the size of the input image, and the predicted label of the output layer of a certain characteristic point is (theta ', r');
3) Activation function of neural network
For daylily targets with characteristic points not in the image, the real position label marks the coordinates of the characteristic points not in the image as a polar coordinate system (0, -0.1), and negative values are output to predict the daylily targets with the characteristic points not in the image; in order to make the output value negative, the activation function of the neural network is a Leaky-ReLU, and when the characteristic points are calculated and are not in the center point of the daylily target in the image, the characteristic points which are not in the image are ignored, and the average value of the coordinates of other characteristic points is calculated;
4) Loss function of neural network
In the neural network for identifying the characteristic points of the day lily, the neural network comprises probability loss, blocked probability loss and position coordinate loss of the characteristic points, wherein the position coordinate loss of the characteristic points is divided into coordinate loss of the characteristic points in the image and position loss of the characteristic point coordinates not in the image, larger weight is allocated to the position loss in the image, small weight is allocated to the position loss of the characteristic points not in the image, and finally a neural network loss function is obtained;
5) Network structure
The network structure is the structure corresponding to the first two output layers of the YOLOv3 network structure, namely the output obtained by 32 times downsampling and 16 times downsampling, the output dimension of the convolution layer of the last layer 1*1 of the network is modified to be 20, namely the neural network output layer, and the network structure branch corresponding to 8 times downsampling is deleted, namely the final neural network structure.
4. The daylily maturity recognition method based on the convolutional neural network according to claim 3, wherein in the step S3, training is performed on the image dataset acquired in the step S1 by using the co dataset pre-trained neural network, specifically comprising:
1) Acquiring a network weight of the YOLOv3 as initialization data of the neural network, wherein the method is to download YOLOv3 training data in an open source website, and the data is required to be trained and stored by using a pytorch, so that a COCO data set can be subjected to target detection;
2) Extracting the corresponding weights of the first two outputs in the network parameters by using a pytorch, namely, the outputs obtained by 32 times downsampling and 16 times downsampling, deleting the weight of the last 1*1 convolution layer, then matching the weight with the established neural network model, and randomly initializing the network parameters of the last convolution layer to obtain the initialization weights of all layers of the neural network;
3) And (3) finishing the training process of the neural network by using the training set, verifying by using the verification set after finishing one-time training of all the training sets, preventing the network from being over-fitted, and finishing the training of the network by using an Adam optimizer in a pytorch in the training process and setting a hyper-parameter reference YOLOv 3.
5. The daylily maturity recognition method based on the convolutional neural network according to claim 4, wherein in the step S4, the image data and the depth map of the daylily are captured by using a 3D camera based on TOF during picking, specifically comprising:
the 3D camera based on TOF is arranged at the wrist of the picking manipulator, three depth images and photos are required to be acquired for identification in the picking process, the daylily to be picked and the position of the daylily are determined by photographing for the first time, after the manipulator moves to the vicinity of the daylily, the daylily is positioned again at the moment, photographing before picking is performed for the third time, retest accuracy is performed, photographing and calibration are repeated if the position of the manipulator is not at a preset position, and picking is performed on the target daylily if the position of the manipulator is just.
6. The day lily maturity recognition method based on convolutional neural network according to claim 1, wherein in step S5, the coordinates of the feature points in the image coordinate system and the depth image thereof are utilized to obtain the spatial coordinates of the feature points, specifically:
and calculating three-dimensional coordinates of the feature points by using the identified feature points and the depth image, wherein the point A is taken as an example, and the calculation formula is as follows:
wherein x is A' ,y A' Is the horizontal and vertical coordinates of the feature point in the image coordinate system, namely the feature point coordinate identified by the neural network, f is the focal length of the camera, and is obtained by calibrating the camera, d A For the distance measured by the TOF-based 3D camera, the three-dimensional coordinates of the A point in the camera coordinate system in the image coordinate system are obtained as (x A ,y A ,z A )。
7. The day lily maturity recognition method based on convolutional neural network according to claim 6, wherein in step S6, the distance between feature points, i.e. the length feature of day lily, is calculated by the obtained spatial coordinates of the feature points, specifically comprising:
1) The distance between the two feature points, namely the distance between the point A and the point B, is calculated by the three-dimensional coordinates obtained in the step S5 as follows:
wherein, the length (A, B) is the distance between the point A and the point B, namely the length characteristic of the day lily;
2) Setting 4 characteristic points to divide day lily into 3 sections, wherein the length characteristic of the day lily is the sum of 3 sections of distances, and the first characteristic point is at the bottommost part of the day lily, namely the position needing to be broken off during picking; the second characteristic point is at the intersection point position of the flower stalks and the flower buds of the day lily; the third characteristic point is positioned at the middle position of the second characteristic point and the fourth characteristic point; the fourth feature point is located at the top end of day lily.
8. The day lily maturity identification method based on convolutional neural network according to claim 7, wherein in step S7, the day lily maturity is obtained through the mapping relation between the day lily maturity and its length characteristics, and the location of the day lily is represented by the space coordinates of the characteristic points, so as to complete the day lily identification and positioning, specifically comprising:
the method for obtaining the mapping relation by researching the mapping relation between the maturity and the length characteristics of the day lily and obtaining the maturity of the day lily by using a linear interpolation method comprises the following steps:
selecting 100 daylily which starts growing of the same variety, periodically measuring and recording the length of the daylily, continuously sampling until the daylily is mature in a sampling period of 0.5 day, finally obtaining 100 vectors with different dimensions, correspondingly removing the length information of the overlong vectors with earlier sampling time from back to front according to the sampling time, averaging 100 data sampled each time to obtain a final vector, wherein each data of the vector corresponds to one maturity, recording the length corresponding to each maturity, and obtaining the mapping relation between the maturity and the length through a linear interpolation method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111650998.6A CN114359546B (en) | 2021-12-30 | 2021-12-30 | Day lily maturity identification method based on convolutional neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111650998.6A CN114359546B (en) | 2021-12-30 | 2021-12-30 | Day lily maturity identification method based on convolutional neural network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114359546A CN114359546A (en) | 2022-04-15 |
| CN114359546B true CN114359546B (en) | 2024-03-26 |
Family
ID=81104152
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111650998.6A Active CN114359546B (en) | 2021-12-30 | 2021-12-30 | Day lily maturity identification method based on convolutional neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114359546B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115690521A (en) * | 2022-11-30 | 2023-02-03 | 仲恺农业工程学院 | Cabbage mustard maturity identification method |
| CN116267226B (en) * | 2023-05-16 | 2023-07-28 | 四川省农业机械研究设计院 | Mulberry picking method and device based on intelligent machine vision recognition of maturity |
| CN117094997B (en) * | 2023-10-18 | 2024-02-02 | 深圳市睿阳精视科技有限公司 | Flower opening degree detection and evaluation method |
| CN117273869B (en) * | 2023-11-21 | 2024-02-13 | 安徽农业大学 | Intelligent agricultural product pushing method, system, device and medium based on user data |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108009525A (en) * | 2017-12-25 | 2018-05-08 | 北京航空航天大学 | A kind of specific objective recognition methods over the ground of the unmanned plane based on convolutional neural networks |
| CN110930387A (en) * | 2019-11-21 | 2020-03-27 | 中原工学院 | Fabric defect detection method based on depth separable convolutional neural network |
| AU2020100705A4 (en) * | 2020-05-05 | 2020-06-18 | Chang, Jiaying Miss | A helmet detection method with lightweight backbone based on yolov3 network |
| CN112446388A (en) * | 2020-12-05 | 2021-03-05 | 天津职业技术师范大学(中国职业培训指导教师进修中心) | Multi-category vegetable seedling identification method and system based on lightweight two-stage detection model |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111739078B (en) * | 2020-06-15 | 2022-11-18 | 大连理工大学 | Monocular unsupervised depth estimation method based on context attention mechanism |
-
2021
- 2021-12-30 CN CN202111650998.6A patent/CN114359546B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108009525A (en) * | 2017-12-25 | 2018-05-08 | 北京航空航天大学 | A kind of specific objective recognition methods over the ground of the unmanned plane based on convolutional neural networks |
| CN110930387A (en) * | 2019-11-21 | 2020-03-27 | 中原工学院 | Fabric defect detection method based on depth separable convolutional neural network |
| AU2020100705A4 (en) * | 2020-05-05 | 2020-06-18 | Chang, Jiaying Miss | A helmet detection method with lightweight backbone based on yolov3 network |
| CN112446388A (en) * | 2020-12-05 | 2021-03-05 | 天津职业技术师范大学(中国职业培训指导教师进修中心) | Multi-category vegetable seedling identification method and system based on lightweight two-stage detection model |
Non-Patent Citations (3)
| Title |
|---|
| 基于ReLU函数的卷积神经网络的花卉识别算法;郭子琰;舒心;刘常燕;李雷;;计算机技术与发展;20180208(第05期);全文 * |
| 基于卷积神经网络的花朵品种的识别;杨静亚;李景霞;王振宇;程海;;黑龙江大学工程学报;20191225(第04期);全文 * |
| 基于感兴趣区域的机器人抓取系统;马世超;孙磊;何宏;郭延华;;科学技术与工程;20200418(第11期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114359546A (en) | 2022-04-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114359546B (en) | Day lily maturity identification method based on convolutional neural network | |
| Li et al. | Detection of fruit-bearing branches and localization of litchi clusters for vision-based harvesting robots | |
| CN109146948B (en) | Crop growth phenotype parameter quantification and yield correlation analysis method based on vision | |
| CN109785337B (en) | In-column mammal counting method based on example segmentation algorithm | |
| CN109934121B (en) | Orchard pedestrian detection method based on YOLOv3 algorithm | |
| Lu et al. | Canopy-attention-YOLOv4-based immature/mature apple fruit detection on dense-foliage tree architectures for early crop load estimation | |
| Yu et al. | Automatic image-based detection technology for two critical growth stages of maize: Emergence and three-leaf stage | |
| CN109255302A (en) | Object recognition methods and terminal, mobile device control method and terminal | |
| Gong et al. | Robotic harvesting of the occluded fruits with a precise shape and position reconstruction approach | |
| CN112200854A (en) | Leaf vegetable three-dimensional phenotype measurement method based on video image | |
| Paturkar et al. | Non-destructive and cost-effective 3D plant growth monitoring system in outdoor conditions | |
| Paturkar et al. | 3D reconstruction of plants under outdoor conditions using image-based computer vision | |
| Xiang et al. | Field‐based robotic leaf angle detection and characterization of maize plants using stereo vision and deep convolutional neural networks | |
| Xiang et al. | Measuring stem diameter of sorghum plants in the field using a high-throughput stereo vision system | |
| Badeka et al. | Harvest crate detection for grapes harvesting robot based on YOLOv3 model | |
| Jin et al. | Far-near combined positioning of picking-point based on depth data features for horizontal-trellis cultivated grape | |
| Rao | Design of automatic cotton picking robot with Machine vision using Image Processing algorithms | |
| Zhao et al. | Automatic sweet pepper detection based on point cloud images using subtractive clustering | |
| Mohammadi et al. | Estimation of leaf area in bell pepper plant using image processing techniques and artificial neural networks | |
| He et al. | Visual recognition and location algorithm based on optimized YOLOv3 detector and RGB depth camera | |
| US20240161325A1 (en) | Improved vision-based measuring | |
| He et al. | Detecting and localizing strawberry centers for robotic harvesting in field environment | |
| Yu et al. | Optimized faster R-CNN for fruit detection of strawberry harvesting robot | |
| CN114782455A (en) | Cotton row center line image extraction method for agricultural machine embedded equipment | |
| Mu et al. | Mask R-CNN based king flowers identification for precise apple pollination |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |