CN115271200A - Intelligent continuous picking system for famous and high-quality tea - Google Patents

Intelligent continuous picking system for famous and high-quality tea Download PDF

Info

Publication number
CN115271200A
CN115271200A CN202210876003.6A CN202210876003A CN115271200A CN 115271200 A CN115271200 A CN 115271200A CN 202210876003 A CN202210876003 A CN 202210876003A CN 115271200 A CN115271200 A CN 115271200A
Authority
CN
China
Prior art keywords
module
tea
picking
point
coordinates
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202210876003.6A
Other languages
Chinese (zh)
Other versions
CN115271200B (en
Inventor
朱立学
张智浩
林桂潮
张世昂
官金炫
陈品岚
陈定贤
陈嘉辉
钟凯勇
蔡柳坚
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhongkai University of Agriculture and Engineering
Original Assignee
Zhongkai University of Agriculture and Engineering
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhongkai University of Agriculture and Engineering filed Critical Zhongkai University of Agriculture and Engineering
Priority to CN202210876003.6A priority Critical patent/CN115271200B/en
Publication of CN115271200A publication Critical patent/CN115271200A/en
Application granted granted Critical
Publication of CN115271200B publication Critical patent/CN115271200B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
    • G06Q10/047—Optimisation of routes or paths, e.g. travelling salesman problem
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/02—Agriculture; Fishing; Forestry; Mining
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/762—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07—Target detection

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Business, Economics & Management (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Artificial Intelligence (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Human Resources & Organizations (AREA)
  • Economics (AREA)
  • Strategic Management (AREA)
  • Marketing (AREA)
  • General Business, Economics & Management (AREA)
  • Tourism & Hospitality (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Quality & Reliability (AREA)
  • Agronomy & Crop Science (AREA)
  • Animal Husbandry (AREA)
  • Mining & Mineral Resources (AREA)
  • Development Economics (AREA)
  • Game Theory and Decision Science (AREA)
  • Marine Sciences & Fisheries (AREA)
  • Operations Research (AREA)
  • Primary Health Care (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Image Processing (AREA)

Abstract

The invention provides an intelligent coherent picking system for famous and high-quality tea, which comprises a target identification module, a path planning module, a picking execution module and an operation control module, wherein the target identification module is used for identifying the tea; and the operation control module is respectively communicated and interacted with the target identification module, the path planning module and the picking execution module. The target recognition module obtains picking points by constructing a tea tender shoot image data set, carrying out YooloV 5 network model improvement and fitting a minimum external cuboid of a tea tender shoot three-dimensional point cloud. The system realizes accurate positioning of tea tender shoots through the visual recognition system, realizes intelligent and mechanical tea picking through the picking robot, does not need a large amount of labor force, and is high in picking efficiency and high in famous tea yield.

Description

Intelligent continuous picking system for famous and high-quality tea
Technical Field
The invention relates to the technical field of tea picking, in particular to an intelligent coherent picking system for famous and high-quality tea.
Background
In recent years, china vigorously develops the planting and production of famous tea, so that the income of tea farmers is obviously increased. The famous and high-quality tea is picked mainly by adopting a manual or handheld tea picking machine, the manual picking is high in selectivity and high in picking accuracy, the damage of tea is not easily caused, but a large amount of labor force is wasted by the manual picking, and the picking efficiency is low; the handheld tea plucking machine is low in cost, high in efficiency and capable of effectively saving labor, but the handheld tea plucking machine is mostly in a 'one-knife cutting' reciprocating cutting mode, tea leaves are influenced by illumination, temperature and humidity, irrigation modes and the like, the growth heights of tea tender shoots are different, meanwhile, the tea tender shoots are small in size, different in shape and dense in spatial distribution, and the 'one-knife cutting' reciprocating cutting mode is easy to cause problems of missing picking, wrong picking, even tea leaf breakage and the like. In addition, because the tea layers obtained by the existing handheld tea plucking machine are uneven, subsequent classification and screening still need to be carried out manually, and the labor cost and the time of the tea plucking process are increased.
Disclosure of Invention
Aiming at the problems in the prior art, the invention aims to provide an intelligent continuous picking system for famous tea, which realizes the accurate positioning of tea tender shoots through a visual recognition system, realizes intelligent and mechanical tea picking through a picking robot, does not need a large amount of labor force, and has high picking efficiency and famous tea yield.
The purpose of the invention is realized by the following technical scheme:
the utility model provides a pick system is linked up to intelligence for famous and high-quality tea which characterized in that: the system comprises a target identification module, a path planning module, a picking execution module and an operation control module; the operation control module is respectively in communication interaction with the target identification module, the path planning module and the picking execution module;
the target identification module is used for acquiring picking points of tea tender shoots;
the path planning module is used for planning a traveling path of the picking execution module, and specifically comprises the following components: the operation control module inputs the picking points obtained by the target identification module and the positions of the current picking execution modules into the path planning module, and the path planning module fits the three-dimensional coordinates of the picking execution modules with the three-dimensional coordinates of the picking points to obtain the shortest and optimal picking path;
the picking execution module comprises a motion mechanical arm and an end effector, and the motion mechanical arm moves the end effector to each picking point according to a picking path planned in the path planning module; the end effector comprises a shearing part and a telescopic part, picking of tea tender shoots is achieved through closing of the shearing part, and collection of tea is achieved through contraction of the telescopic part and opening of the shearing part.
For further optimization, the specific steps of the target identification module for acquiring picking points of tea tender shoots are as follows:
s10, constructing a tea tender shoot image data set; s20, in the data set of the step S10, constructing a characteristic diagram with rich semantic information through a Bi-Directional characteristic Pyramid Network (BiFPN) and an effective Channel Attention mechanism (ECA), so as to improve a YoloV5 Network model, obtain an improved YoloV5 Network model and detect small-size tea tender shoots; s30, obtaining a tea three-dimensional point cloud based on the training result of the improved YooloV 5 network model in the step S20; then screening out three-dimensional point clouds of tender buds of the tea leaves from the three-dimensional point clouds of the tea leaves; finally fitting a cuboid externally connected with the minimum tea tender shoot to obtain the accurate position and picking point of the tea tender shoot;
the YoloV5 network model comprises a backhaul module, a Neck module and a Head module; the backhaul module comprises a Focus module, an SPP module and a CBS module which are used for slicing the pictures, and a CSP module which is used for enhancing the learning performance of the whole convolutional neural network; the Neck module comprises a CBS module and a CSP module; the Head module includes a Detect layer that performs target detection on different scales of feature maps using a grid anchor-based approach.
Preferably, the yoolov 5 network model adopts a network model with the smallest model file size and the smallest depth and width of the feature map.
For further optimization, the step S10 specifically includes: firstly, collecting tea tender shoot image data by using an RGB-D camera to obtain a color image and a depth image of tea tender shoots; marking the color image by using a marking tool, performing data set enhancement operation, and expanding the number of data sets to construct a tea She Nenya image data set; finally, dividing the data set to form a training set, a test set and a verification set;
the step S20 is specifically:
s21, preprocessing the images in the training set in the step S10 and unifying the resolution of all the images in the training set; inputting the preprocessed image into a backhaul module to obtain characteristic graphs with different sizes; s22, inputting the feature maps with different sizes in the step S21 into a Neck module, and performing multi-feature fusion by adopting a bidirectional feature pyramid Network to replace an original Path Aggregation Network (PANET) in the Neck module; sequentially carrying out up-sampling and down-sampling on the feature maps, splicing the feature maps through a channel attention mechanism to generate feature maps with various sizes, and inputting the feature maps into a Detect layer of a Head module; s23, combining various loss functions to perform back propagation, and updating and adjusting the weight parameters of the gradient in the model; and S24, verifying the existing model by adopting the verification set in the step S10 to obtain an improved YoloV5 network model.
Preferably, the labeling tool is a Labelimg labeling tool.
For further optimization, the step S30 specifically includes:
s31, obtaining coordinates of a detection frame according to the result of the improved YoloV5 network model in the step S20, and generating a Region of Interest (ROI) of a color image and a corresponding depth image;
s32, obtaining corresponding mapped color image coordinates according to the mapping relation between the pixel coordinates of the depth image and the pixel coordinates of the color image and through the coordinate values, the pixel values and the recording distances of the depth image;
s33, obtaining a tea three-dimensional point cloud through coordinate fusion of the color image and the depth image, wherein the method specifically comprises the following steps:
Figure BDA0003762544240000031
in the formula (I), the compound is shown in the specification,
Figure BDA0003762544240000032
a coordinate system representing a three-dimensional point cloud;
Figure BDA0003762544240000033
a coordinate system representing the color image; d represents a depth value, obtained by a depth image; f. ofx、fyRepresents the camera focal length;
s34, the generated three-dimensional point cloud of the tea leaves comprises the tea leavesThe tender bud and the background point cloud thereof, so that the average value of the three-dimensional point cloud of the tea is obtained through calculation and is used as a distance threshold; then filtering the background point cloud larger than the distance threshold value to obtain a primarily segmented three-dimensional point cloud; adopting DBSCAN clustering algorithm, setting parameter radius Eps and minimum sample number M required to be contained in neighborhoodpGathering the preliminarily divided three-dimensional point clouds into one kind, and screening out the three-dimensional point clouds of the tea tender shoots;
s35, fitting the minimum external cuboid of the tea tender shoot at the position by adopting a Principal Component Analysis (PCA) according to the growth posture of the tea tender shoot; calculating to obtain coordinates of each vertex of the cuboid; and obtaining the coordinate of the central point of the bottom surface of the cuboid by calculating the average value of the four vertexes of the bottom surface of the cuboid, and taking the point as a picking point of the tender shoots of the tea leaves.
For further optimization, the step S35 specifically includes:
s351, screening three main directions, namely x, y and z directions, of the tea tender shoot three-dimensional point cloud by adopting a principal component analysis method, and calculating a mass center and covariance to obtain a covariance matrix; the method specifically comprises the following steps:
Figure BDA0003762544240000041
Figure BDA0003762544240000042
in the formula, PcRepresenting centroid coordinates of the three-dimensional point cloud; n represents the number of three-dimensional point clouds (i.e., the number of points); (x)i,yi,zi) Three-dimensional coordinates representing the ith point;
Figure BDA0003762544240000043
in the formula, CpA covariance matrix representing the three-dimensional point cloud;
s352, singular value decomposition is carried out on the covariance matrix to obtain an eigenvalue and an eigenvector, and the specific formula is as follows:
Figure BDA0003762544240000044
in the formula of UpRepresents a covariance matrix CpCp TA feature vector matrix of (a); dpIndicating that a diagonal non-0 value is CpCp TA diagonal matrix of the square root of the non-0 eigenvalues of (1);
Figure BDA0003762544240000051
represents a Cp TCpA feature vector matrix of (a);
the direction of the eigenvector corresponding to the maximum eigenvalue is the direction of the main axis of the cuboid;
s353, projecting the coordinate points to the direction vectors, and calculating the position coordinates P of each vertexiObtaining the maximum value and the minimum value in each direction by the inner product of the unit vector of the coordinate point, and enabling a, b and c to be the average values of the maximum value and the minimum value in x, y and z respectively to obtain the central point O and the length L of the cuboid to generate the cuboid with the most appropriate and compact tea tender shoot;
the concrete formula is as follows:
Figure BDA0003762544240000052
O＝ax+by+cz；
Figure BDA0003762544240000053
wherein, X is a unit vector of the coordinate point in the X direction; y is a unit vector of the coordinate point in the Y direction; z is a unit vector of the coordinate point in the Z direction; l isx、Ly、LzThe lengths of the cuboid in the x direction, the y direction and the z direction are respectively;
s354, judging coordinates of four minimum points in the y direction of the cuboid as coordinates of four vertexes of the bottom surface of the cuboid; and finally, obtaining the coordinate of the central point of the bottom surface of the cuboid, namely the picking point, through the average value of the coordinates of the four vertexes.
The invention has the following technical effects:
according to the method, small targets of tea tender shoots under a large visual field are inspected and positioned through a deep learning method, firstly, a detection model of the small tea tender shoots is constructed, and meanwhile, semantic expression and positioning capacity on multiple scales of an image are enhanced through an improved YooloV 5 network model, so that the method is suitable for judgment and identification of the tea tender shoots with small targets, different shapes and dense distribution space, misjudgment and misjudgment caused by difference or mutual overlapping of the tea tender shoots in the process of judging and identifying the tea tender shoots are avoided, and the accuracy of identification and judgment is improved. Simultaneously, this application is through the minimum external cuboid of fitting tealeaves burgeon, realize the accurate positioning of tealeaves picking point (use minimum external cuboid bottom surface central point as the picking point of tealeaves burgeon promptly), effectively avoid the damage that causes the tealeaves burgeon when picking, ensure simultaneously that whole tealeaves burgeon is picked to the most effective, complete, ensure the quality of tealeaves. Through the cooperation of the target identification module, the path planning module, the picking execution module and the operation control module, the shortest and optimal picking feeding path can be efficiently obtained, the picking efficiency is improved, and the picking flow is simplified, so that automatic and mechanical tea picking is realized, the use of labor force is reduced, and the tea picking time and cost are saved.
Drawings
Fig. 1 is a flow chart of a picking system for picking tea leaves according to an embodiment of the present invention.
Fig. 2 is a flowchart of a target identification module obtaining picking points according to an embodiment of the present application.
Fig. 3 is a schematic diagram of a picture labeled by a labeling tool in the embodiment of the present application.
Fig. 4 is a multi-scale feature fusion structure diagram based on a bidirectional feature pyramid network structure in the embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The embodiment is as follows:
the utility model provides a pick system is linked up to intelligence for famous and high-quality tea which characterized in that: the system comprises a target identification module, a path planning module, a picking execution module and an operation control module; the operation control module is respectively communicated and interacted with the target identification module, the path planning module and the picking execution module;
the target identification module is used for acquiring picking points of tea tender shoots, and specifically comprises the following steps:
s10, constructing a tea tender shoot image data set: firstly, collecting tea tender shoot image data by using an RGB-D camera to obtain a color image and a depth image of tea tender shoots; labeling the color image by using a labeling tool, such as a Labelimg labeling tool (as shown in FIG. 1), performing data set enhancement operation (the data set enhancement operation can adopt the conventional technical means, and those skilled in the art can understand means such as space conversion and color conversion), and expanding the number of data sets to construct a tea She Nenya image data set; and finally, dividing the data set to form a training set, a test set and a verification set.
S20, in the data set of the step S10, constructing a characteristic diagram with rich semantic information through a Bi-Directional characteristic Pyramid Network (BiFPN) and an effective Channel Attention mechanism (ECA), so as to improve a YoloV5 Network model, obtain an improved YoloV5 Network model and detect small-size tea tender shoots;
the YoloV5 network model adopts a network model with the smallest size of a model file and the smallest depth and width of a characteristic diagram, and comprises a backhaul module, a Neck module and a Head module; the backhaul module comprises a Focus module, an SPP module and a CBS module which are used for slicing the pictures, and a CSP module which is used for enhancing the learning performance of the whole convolutional neural network; the Neck module comprises a CBS module and a CSP module; the Head module comprises a Detect layer for detecting targets on feature maps of different scales by using a grid anchor-based module;
the method comprises the following steps:
s21, preprocessing the images in the training set in the step S10 and unifying the resolution of all the images in the training set; inputting the preprocessed image into a Backbone module to obtain characteristic graphs with different sizes;
s22, inputting the feature maps with different sizes in the step S21 into a Neck module, and performing multi-feature fusion by adopting a bidirectional feature pyramid Network to replace an original Path Aggregation Network (PANET) in the Neck module; sequentially carrying out up-sampling and down-sampling on the feature maps, splicing through a channel attention mechanism to generate feature maps with various sizes, and inputting the feature maps into a Detect layer of a Head module;
in a YoloV5 network model (namely in the existing YoloV5 network structure), reinforced features are used for extracting BiFPN, P5_ in is subjected to upsampling, and BiFPN _ Concat stacking is performed on the upsampled BiFPN _ Concat and P4_ in to obtain P4_ td; then, performing upsampling on the P4_ td, and performing BiFPN _ Concat stacking on the upsampled P4_ td and the P3_ in to obtain P3_ out; then, down-sampling the P3_ out, and stacking BiFPN _ Concat with the P4_ td after down-sampling to obtain P4_ out; then, the P4_ out is downsampled, and the downsampled P4_ out is stacked with the P5_ in to obtain P5_ out. The method uses efficient bidirectional cross connection to perform feature fusion, removes nodes which contribute less to the feature fusion in the PANet, adds extra connection between input and output nodes at the same level, fuses more features without adding extra cost, and enhances semantic expression and positioning capacity on multiple scales, as shown in FIG. 2.
Then, adding ECA after the 9 th layer, enabling a module to perform Global Average Pooling (Global Average Pooling) on an input feature map, changing the matrix of [ h, w, c ] into a vector of [1, c ], then calculating to obtain an adaptive one-dimensional convolution kernel size, and using the kernel _ size in one-dimensional convolution to obtain the weight of each channel in the feature map; and multiplying the normalized weight and the original input feature map one by one to generate a weighted feature map.
The attention mechanism uses a 1x1 convolution layer after the global average pooling layer, removes a full connection layer, avoids dimensionality reduction, effectively captures cross-channel interaction, and finally improves the probability of judging an object and the detection precision of a model; the concrete formula is as follows:
Figure BDA0003762544240000081
wherein C represents the channel dimension; k represents the convolution sum; y and b take the values of 2 and 1 respectively;
s23, carrying out back propagation by combining various loss functions (such as classification loss, positioning loss, execution loss and the like), and updating and adjusting the weight parameters of the gradient in the model;
and S24, verifying the existing model by adopting the verification set in the step S10 to obtain an improved YoloV5 network model.
S30, obtaining a tea three-dimensional point cloud based on the training result of the improved YooloV 5 network model in the step S20; then screening out tea tender shoot three-dimensional point clouds from the tea three-dimensional point clouds; finally fitting the minimum external cuboid of the tea tender shoots to obtain the accurate positions and picking points of the tea tender shoots; the method specifically comprises the following steps:
s31, firstly, obtaining the coordinates of a detection frame according to the result of the improved YooloV 5 network model in the step S20, and generating a Region of Interest (ROI) of a color image and a corresponding depth image;
s32, obtaining corresponding mapped color image coordinates according to the mapping relation between the pixel coordinates of the depth image and the pixel coordinates of the color image and through the coordinate values, the pixel values and the recording distance of the depth image;
s33, obtaining the tea three-dimensional point cloud through the coordinate fusion of the color image and the depth image, specifically:
Figure BDA0003762544240000091
in the formula (I), the compound is shown in the specification,
Figure BDA0003762544240000092
a coordinate system representing a three-dimensional point cloud;
Figure BDA0003762544240000093
a coordinate system representing the color image; d represents a depth value, obtained by a depth image; f. ofx、fyRepresents the camera focal length;
s34, because the generated tea three-dimensional point cloud comprises tea tender shoots and background point cloud thereof, obtaining an average value of the tea three-dimensional point cloud through calculation, and taking the average value as a distance threshold; then filtering the background point cloud larger than the distance threshold value to obtain a primarily segmented three-dimensional point cloud; adopting DBSCAN clustering algorithm, setting parameter radius Eps and minimum number M of samples required to be contained in neighborhoodpGathering the preliminarily divided three-dimensional point clouds into one kind, and screening out the three-dimensional point clouds of the tea tender shoots;
the DBSCAN clustering algorithm randomly selects a data sample in the space, and determines whether the number of the samples distributed in the neighborhood radius Eps is more than or equal to the minimum number M of the samplespA threshold number to determine if it is a core object:
if so, all the points in the neighborhood can be divided into the same cluster group, and all samples with reachable density can be found by breadth-first search and divided into the cluster group on the basis of the cluster group;
if the data sample is a non-core object, marking the data sample as a noise point for removal;
the formula is specifically as follows:
NEps(p)＝{q∈D|dist(p,q)≤Eps}；
in the formula, D represents a point cloud sample set; p and q respectively represent sample points collected by the sample set;
for any p e D, if its Eps corresponds to | NEps(p) | at leastComprising MpOne sample, then p is the core object; if q is within the Eps of p and p is the core object, then q becomes reachable by p density;
s35, fitting the minimum external cuboid of the tea tender shoot at the position by adopting a Principal Component Analysis (PCA) according to the growth posture of the tea tender shoot; calculating to obtain coordinates of each vertex of the cuboid; obtaining the coordinate of the central point of the bottom surface of the cuboid by calculating the average value of the four vertexes of the bottom surface of the cuboid, and taking the point as a picking point of the tender bud of the tea; the method comprises the following specific steps:
s351, screening out three main directions, namely x, y and z directions, of the tea tender shoot three-dimensional point cloud by adopting a main component analysis method, and calculating a mass center and covariance to obtain a covariance matrix; the method specifically comprises the following steps:
Figure BDA0003762544240000101
Figure BDA0003762544240000102
in the formula, PcRepresenting centroid coordinates of the three-dimensional point cloud; n represents the number of three-dimensional point clouds (i.e., the number of points); (x)i,yi,zi) Three-dimensional coordinates representing the ith point;
Figure BDA0003762544240000103
in the formula, CpA covariance matrix representing the three-dimensional point cloud;
s352, singular value decomposition is carried out on the covariance matrix to obtain an eigenvalue and an eigenvector, and the specific formula is as follows:
Figure BDA0003762544240000111
in the formula of UpRepresents a covariance matrix CpCp TIs characterized byA vector matrix; dpIndicating that a diagonal non-0 value is CpCp TA diagonal matrix of the square root of the non-0 eigenvalues of (1);
Figure BDA0003762544240000112
represents a Cp TCpA feature vector matrix of (a);
the direction of the eigenvector corresponding to the maximum eigenvalue is the direction of the main axis of the cuboid;
s353, projecting the coordinate points to the direction vector, and calculating the position coordinate P of each vertexiObtaining the maximum value and the minimum value in each direction by the inner product of the unit vector of the coordinate point, and enabling a, b and c to be the average values of the maximum value and the minimum value in x, y and z respectively to obtain the central point O and the length L of the cuboid to generate the cuboid with the most appropriate and compact tea tender shoot;
the concrete formula is as follows:
Figure BDA0003762544240000113
O＝ax+by+cz；
Figure BDA0003762544240000114
wherein X is a unit vector of the coordinate point in the X direction; y is a unit vector of the coordinate point in the Y direction; z is a unit vector of the coordinate point in the Z direction; l isx、Ly、LzThe lengths of the cuboid in the x direction, the y direction and the z direction are respectively;
s354, judging the coordinates of the minimum four points in the y direction of the cuboid as the coordinates of four vertexes of the bottom surface of the cuboid; and finally, obtaining the coordinate of the central point of the bottom surface of the cuboid, namely the picking point, through the average value of the coordinates of the four vertexes.
The path planning module is used for planning a traveling path of the picking execution module, and specifically comprises: the operation control module inputs the picking points obtained by the target identification module and the positions of the current picking execution modules (particularly shearing parts) into the path planning module, and the path planning module fits the three-dimensional coordinates of the picking execution modules (particularly shearing parts) with the three-dimensional coordinates of the picking points to obtain the shortest and optimal picking path;
for example: a is1Constructing a picking execution module (particularly a shearing part) picking sequence path objective function: firstly, modeling picking sequence path planning of a picking execution module (particularly a shearing part) based on Markov Decision Process (MDP); then, the operation control module controls the picking execution module (particularly the cutting part) to execute a prediction action according to the received three-dimensional information of the picking points obtained by the method in the target identification module, and then observes a new state and obtains a reward, so that the objective function maximizes the expected accumulated reward obtained by the picking execution module (particularly the cutting part); a is2Establishing a simulation environment comprising a tea plantation and a picking execution module by using simulation equipment, and training by taking physical quantities such as illumination intensity, camera orientation, tea tender shoot positions, colors and the like as parameters of the simulation environment; in the training process, the randomness of the simulation environment is gradually increased, data with continuously increased learning difficulty is acquired through the interaction collection of the robot and the environment, and the data is sampled; a is3And optimizing the objective function by adopting a Proximal strategy Optimization algorithm module (PPO) and combining a solver to obtain a picking sequence planning model of a picking execution module (particularly a shearing part).
The picking execution module comprises a motion mechanical arm and an end effector, the motion mechanical arm moves the end effector to each picking point according to a picking path planned in the path planning module, and the motion mechanical arm can adopt a multi-degree-of-freedom mechanical arm to select the degree of freedom of the mechanical arm according to the actual picking condition; and the motion mechanical arm and the end effector realize communication interaction through the operation control module. The end effector comprises a shearing part and a telescopic part, the picking of tea tender shoots is realized through the closing of the shearing part, and the collection of tea is realized through the contraction of the telescopic part and the opening of the shearing part. If necessary, a tea leaf collecting basket is arranged on the moving mechanical arm and positioned at the end effector.
The operation control module can build an Ubuntu18.04 system on the edge computer and simultaneously build a deep learning operation environment based on the Pythrch; deploying the target identification module and the path planning module into an edge computer, and coordinating and managing resource scheduling and network signal interaction of the two neural network modules; and simultaneously, a motion mechanical arm control system and an end effector driving system are deployed in the edge computer.
The specific method for picking tea in the picking system comprises the following steps:
s001, initializing the system in a tea garden, and manually controlling a motion mechanical arm to drive an end effector to move to a pre-picking tea point;
s002, the operation control module opens the target identification module and the path planning module and simultaneously opens the camera;
s003, the camera transmits the visual field to a target recognition module, and the target recognition module performs multi-target detection on tea tender shoots in the image; detecting tea tender shoots, fitting a minimum external cuboid of the tea She Nenya to obtain a bottom surface central point of the minimum external cuboid; then, the central point of the bottom surface of the minimum external cuboid is transmitted to the operation control module;
s004, the operation control module guides the center point of the bottom surface of the minimum external cuboid and the current position coordinate of the end effector into a PPO (polyphenylene oxide) model of the path planning module to obtain the shortest optimal path picked by the end effector; transmitting the path obtained by resolving to an operation control module;
s005, the operation control module transmits the path information to the picking execution module, and the picking execution module controls the end effector to reach a picking point and the end effector to realize picking and collecting actions through the motion mechanical arm; until the tea tender shoots in the current visual field range of the camera are picked up;
s006, moving the picking system to the next visual field range, and continuing picking tea tender shoots.
Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.

Claims (5)

1.一种用于名优茶的智能连贯采摘系统，其特征在于：包括目标识别模块、路径规划模块、采摘执行模块与运算控制模块；所述运算控制模块分别与目标识别模块、路径规划模块、采摘执行模块通信交互；1. An intelligent coherent picking system for famous tea, characterized in that: it comprises a target identification module, a path planning module, a picking execution module and an operation control module; the operation control module is connected with the target identification module, the path planning module, Picking execution module communication interaction; 所述目标识别模块用于获取茶叶嫩芽的采摘点；The target recognition module is used to obtain the picking point of tea buds; 所述路径规划模块用于规划采摘执行模块的行进路径，具体为：运算控制模块将目标识别模块获得的各个采摘点以及当前采摘执行模块的位置输入到路径规划模块，路径规划模块将采摘执行模块的三维坐标与各个采摘点的三维坐标进行拟合，获得最短最优的采摘路径；The path planning module is used to plan the travel path of the picking execution module, specifically: the operation control module inputs each picking point obtained by the target identification module and the position of the current picking execution module to the path planning module, and the path planning module will pick the picking execution module. The three-dimensional coordinates of each picking point are fitted with the three-dimensional coordinates to obtain the shortest and optimal picking path; 所述采摘执行模块包括运动机械臂与末端执行器，所述运动机械臂根据路径规划模块中规划的采摘路径、将末端执行器移动至各个采摘点；所述末端执行器包括剪切部与伸缩部，通过剪切部的闭合实现对茶叶嫩芽的采摘，通过伸缩部的收缩与剪切部的打开实现对茶叶的收集。The picking execution module includes a moving robot arm and an end effector, and the moving robot arm moves the end effector to each picking point according to the picking path planned in the path planning module; the end effector includes a cutting part and a telescopic The tea buds can be picked through the closing of the cutting part, and the collection of tea leaves can be realized through the contraction of the telescopic part and the opening of the cutting part. 2.根据权利要求1所述的一种用于名优茶的智能连贯采摘系统，其特征在于：所述目标识别模块获取茶叶嫩芽的采摘点的具体步骤为：2. A kind of intelligent coherent picking system for famous and high-quality tea according to claim 1, characterized in that: the specific steps for the target recognition module to obtain the picking point of tea tender buds are: S10、构建茶叶嫩芽图像数据集；S20、在步骤S10的数据集中，通过双向特征金字塔网络和通道注意力机制构造语义信息丰富的特征图，实现对YoloV5网络模型的改进、获得改进型YoloV5网络模型，实现小尺寸茶叶嫩芽的检测；S30、基于步骤S20中的改进型YoloV5网络模型的训练结果，获得茶叶三维点云；然后在茶叶三维点云中筛选出茶叶嫩芽三维点云；最后拟合茶叶嫩芽最小外接长方体，获得茶叶嫩芽的精确位置以及采摘点；S10. Construct a tea bud image data set; S20. In the data set of step S10, construct a feature map with rich semantic information through a bidirectional feature pyramid network and a channel attention mechanism to improve the YoloV5 network model and obtain an improved YoloV5 network model to realize the detection of small-sized tea buds; S30, based on the training results of the improved YoloV5 network model in step S20, obtain a three-dimensional point cloud of tea; then filter out a three-dimensional point cloud of tea buds in the three-dimensional point cloud of tea; finally Fit the smallest circumscribing cuboid of tea buds to obtain the precise location and picking point of tea buds; 所述YoloV5网络模型包括Backbone模块、Neck模块和Head模块；Backbone模块包括用于对图片进行切片操作的Focus模块、SPP模块与CBS模块，及用于增强整个卷积神经网络学习性能CSP模块；Neck模块包括CBS模块与CSP模块；Head模块包括利用基于网格anchor在不同尺度的特征图上进行目标检测的Detect层。The YoloV5 network model includes a Backbone module, a Neck module and a Head module; the Backbone module includes a Focus module, an SPP module and a CBS module for slicing pictures, and a CSP module for enhancing the learning performance of the entire convolutional neural network; Neck The module includes a CBS module and a CSP module; the Head module includes a Detect layer that uses a grid-based anchor to perform target detection on feature maps of different scales. 3.根据权利要求1或2所述的一种用于名优茶的智能连贯采摘系统，其特征在于：所述步骤S10具体为：先使用RGB-D相机采集茶叶嫩芽图像数据，获得茶叶嫩芽的彩色图像与深度图像；再采用标注工具对彩色图像进行标注，并进行数据集增强操作、扩充数据集的数量，构建出茶叶嫩芽图像数据集；最后对数据集进行划分，形成训练集、测试集与验证集；3. An intelligent coherent picking system for famous and high-quality tea according to claim 1 or 2, characterized in that: the step S10 is specifically: first use an RGB-D camera to collect image data of tea buds, and obtain tea tender buds The color image and depth image of the bud; then use the annotation tool to annotate the color image, and perform data set enhancement operations, expand the number of data sets, and construct a tea bud image data set; finally divide the data set to form a training set , test set and validation set; 所述步骤S20具体为：The step S20 is specifically: S21、对步骤S10的训练集中的图像进行预处理、统一训练集中所有图像的分辨率；再将预处理后的图像输入到Backbone模块，获得不同尺寸的特征图；S22、将步骤S21中不同尺寸的特征图输入到Neck模块中，并在Neck模块中采用双向特征金字塔网络替代原有的路径聚合网络进行多特征融合；再对特征图依次进行上采样、下采样，及通过通道注意力机制的拼接产生多种尺寸的特征图，并输入到Head模块的Detect层中；S23、结合多种损失函数进行反向传播，对模型中的梯度进行更新调节权重参数；S24、采用步骤S10中的验证集对现有模型进行验证，获得改进型YoloV5网络模型。S21, preprocessing the images in the training set of step S10, and unifying the resolution of all images in the training set; then input the preprocessed images to the Backbone module to obtain feature maps of different sizes; S22, different sizes in step S21 The feature map of the Neck module is input into the Neck module, and the two-way feature pyramid network is used in the Neck module to replace the original path aggregation network for multi-feature fusion; then the feature map is sequentially up-sampled, down-sampled, and through the channel attention mechanism. Stitching to generate feature maps of various sizes, and input them into the Detect layer of the Head module; S23, combine multiple loss functions for backpropagation, update the gradient in the model and adjust the weight parameters; S24, adopt the verification in step S10 The set is used to verify the existing model and obtain the improved YoloV5 network model. 4.根据权利要求2或3所述的一种用于名优茶的智能连贯采摘系统，其特征在于：所述步骤S30具体为：4. An intelligent coherent picking system for famous tea according to claim 2 or 3, characterized in that: said step S30 is specifically: S31、根据步骤S20的改进型YoloV5网络模型的结果，获取检测框坐标，生成彩色图像与对应深度图像的感兴趣区域；S31. According to the result of the improved YoloV5 network model in step S20, the coordinates of the detection frame are obtained, and the region of interest of the color image and the corresponding depth image is generated; S32、根据深度图像的像素坐标与彩色图像的像素坐标之间的映射关系，并通过深度图像的坐标值、像素值和记录距离，来得到对应的映射的彩色图像坐标；S32. According to the mapping relationship between the pixel coordinates of the depth image and the pixel coordinates of the color image, and through the coordinate value, pixel value and recording distance of the depth image, obtain the corresponding mapped color image coordinates; S33、通过彩色图像与深度图像的坐标融合，获得茶叶三维点云，具体为：S33. Obtain a three-dimensional point cloud of tea leaves through the coordinate fusion of the color image and the depth image, specifically:
Figure FDA0003762544230000021
Figure FDA0003762544230000021
 式中，
Figure FDA0003762544230000031
表示三维点云的坐标系；
Figure FDA0003762544230000032
表示彩色图像的坐标系；D表示深度值、通过深度图像获得；fx、fy表示相机焦距；In the formula,
Figure FDA0003762544230000031
A coordinate system representing a 3D point cloud;
Figure FDA0003762544230000032
Represents the coordinate system of the color image; D represents the depth value, obtained through the depth image; f x , f y represent the focal length of the camera; S34、由于生成的茶叶三维点云包含茶叶嫩芽及其背景点云，因此通过计算获得茶叶三维点云的平均值，并将该平均值作为距离阈值；之后将大于距离阈值的背景点云进行过滤，得到初步分割的三维点云；采用DBSCAN聚类算法，通过设置参数半径Eps和邻域中要求含有的最低样本个数Mp，将初步分割的三维点云聚为一类，并筛选出茶叶嫩芽三维点云；S34. Since the generated three-dimensional point cloud of tea leaves contains tea buds and their background point clouds, the average value of the three-dimensional point clouds of tea leaves is obtained by calculation, and the average value is used as a distance threshold; Filter to obtain the preliminary segmented 3D point cloud; use the DBSCAN clustering algorithm to cluster the preliminary segmented 3D point cloud into one class by setting the parameter radius Eps and the minimum number of samples M p required to be contained in the neighborhood, and filter out 3D point cloud of tea buds; S35、根据茶叶嫩芽的生长姿态，采用主成分分析法拟合处茶叶嫩芽的最小外接长方体；计算获得长方体各个顶点坐标；再通过求长方体底面四个顶点平均值获得长方体底面中心点的坐标，并以该点作为茶叶嫩芽的采摘点。S35. According to the growth posture of the tea buds, use the principal component analysis method to fit the minimum circumscribed cuboid of the tea buds; calculate and obtain the coordinates of each vertex of the cuboid; then obtain the coordinates of the center point of the cuboid bottom by calculating the average value of the four vertices of the cuboid bottom , and take this point as the picking point of tea buds. 5.根据权利要求4所述的一种用于名优茶的智能连贯采摘系统，其特征在于：所述步骤S35具体为：5. A kind of intelligent coherent picking system for famous tea according to claim 4, characterized in that: said step S35 is specifically: S351、采用主成分分析法筛选出茶叶嫩芽三维点云的三个主方向，即x、y、z方向，并计算质心、协方差，得到协方差矩阵；具体为：S351. Use the principal component analysis method to screen out the three main directions of the three-dimensional point cloud of tea buds, namely the x, y, and z directions, and calculate the centroid and covariance to obtain the covariance matrix; specifically:
Figure FDA0003762544230000033
Figure FDA0003762544230000033
Figure FDA0003762544230000034
Figure FDA0003762544230000034
 式中，Pc表示三维点云的质心坐标；n表示三维点云的数目(即点的数量)；(xi,yi,zi)表示第i个点的三维坐标；In the formula, P c represents the centroid coordinates of the 3D point cloud; n represents the number of 3D point clouds (that is, the number of points); ( xi , y i , z i ) represents the 3D coordinates of the i-th point;
Figure FDA0003762544230000035
Figure FDA0003762544230000035
 式中，Cp表示三维点云的协方差矩阵；In the formula, Cp represents the covariance matrix of the 3D point cloud; S352、对协方差矩阵进行奇异值分解，获得特征值与特征向量，具体公式为：S352. Perform singular value decomposition on the covariance matrix to obtain eigenvalues and eigenvectors. The specific formula is:
Figure FDA0003762544230000041
Figure FDA0003762544230000041
 式中，Up表示协方差矩阵CpCp T的特征向量矩阵；Dp表示一个对角线上的非0值是CpCp T的非0特征值的平方根的对角阵；
Figure FDA0003762544230000042
表示一个Cp TCp的特征向量矩阵；In the formula, U p represents the eigenvector matrix of the covariance matrix C p C p T ; D p represents a diagonal matrix whose non-zero value on the diagonal is the square root of the non-zero eigenvalue of C p C p T ;
Figure FDA0003762544230000042
Represents an eigenvector matrix of C p T C p ; 其中，最大特征值对应的特征向量的方向即为长方体的主轴方向；Among them, the direction of the eigenvector corresponding to the largest eigenvalue is the direction of the principal axis of the cuboid; S353、将坐标点投影到方向向量上，通过计算每个顶点位置坐标Pi与坐标点单位向量的内积获得在每个方向上的最大值与最小值，令a、b、c分别为x、y、z上最大值与最小值的平均值，获得长方体的中心点O与长度L，生成茶叶嫩芽最合适且紧凑的长方体；S353. Project the coordinate point onto the direction vector, and obtain the maximum value and minimum value in each direction by calculating the inner product of each vertex position coordinate P i and the coordinate point unit vector, let a, b, and c be x respectively , the average value of the maximum value and the minimum value on y and z to obtain the center point O and length L of the cuboid, and generate the most suitable and compact cuboid for tea buds; 具体公式为：The specific formula is:
Figure FDA0003762544230000043
Figure FDA0003762544230000043
 O＝ax+by+cz；O=ax+by+cz;
Figure FDA0003762544230000044
Figure FDA0003762544230000044
 式中，X为坐标点在x方向上的单位向量；Y为坐标点在y方向上的单位向量；Z为坐标点在z方向上的单位向量；Lx、Ly、Lz分别为长方体在x方向、y方向、z方向的长度；In the formula, X is the unit vector of the coordinate point in the x direction; Y is the unit vector of the coordinate point in the y direction; Z is the unit vector of the coordinate point in the z direction; L x , L y , L z are cuboids Length in x direction, y direction, z direction; S354、通过判断上述长方体y方向上最小的四个点的坐标，作为长方体底面的四个顶点坐标；最后通过四个顶点坐标的平均值获得长方体底面中心点的坐标、即采摘点。S354. By judging the coordinates of the four smallest points in the y direction of the cuboid as the coordinates of the four vertices of the bottom of the cuboid; finally, the coordinates of the central point of the bottom of the cuboid, that is, the picking point, are obtained by the average of the coordinates of the four vertices.
CN202210876003.6A 2022-07-25 2022-07-25 Intelligent coherent picking system for famous tea Active CN115271200B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210876003.6A CN115271200B (en) 2022-07-25 2022-07-25 Intelligent coherent picking system for famous tea

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210876003.6A CN115271200B (en) 2022-07-25 2022-07-25 Intelligent coherent picking system for famous tea

Publications (2)

Publication Number Publication Date
CN115271200A true CN115271200A (en) 2022-11-01
CN115271200B CN115271200B (en) 2023-05-30

Family

ID=83770408

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210876003.6A Active CN115271200B (en) 2022-07-25 2022-07-25 Intelligent coherent picking system for famous tea

Country Status (1)

Country Link
CN (1) CN115271200B (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116982480A (en) * 2023-08-18 2023-11-03 湖北神农草木农业开发有限公司 Automatic tea picking system and method
CN117876649A (en) * 2024-01-08 2024-04-12 华南农业大学 A method and device for identifying tea buds and determining their position and posture
CN118096891A (en) * 2024-04-24 2024-05-28 华南农业大学 A tea bud leaf pose estimation method and system based on a picking robot
CN118587288A (en) * 2024-08-06 2024-09-03 华南农业大学 A method and device for locating tea bud and leaf picking points
CN118941125A (en) * 2024-10-14 2024-11-12 宁德职业技术学院 A tea garden tea picking planning system and method based on visual recognition
CN119540731A (en) * 2025-01-17 2025-02-28 上海恒泽辅汇智能科技有限公司 Edible mushroom bud detection and collection system, method, medium and device
CN119991393A (en) * 2024-12-17 2025-05-13 湖北静香农业发展有限公司 A tea picking method with tea planting identification function

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108182374A (en) * 2016-12-08 2018-06-19 广州映博智能科技有限公司 A kind of picking point recognition methods for fruit string
WO2018157286A1 (en) * 2017-02-28 2018-09-07 深圳市大疆创新科技有限公司 Recognition method and device, and movable platform
CN111784764A (en) * 2020-05-28 2020-10-16 西南石油大学 A tea sprout identification and localization algorithm
CN111914715A (en) * 2020-07-24 2020-11-10 廊坊和易生活网络科技股份有限公司 Intelligent vehicle target real-time detection and positioning method based on bionic vision
CN113674226A (en) * 2021-07-31 2021-11-19 河海大学 A deep learning-based detection method for tea bud tip of tea picking machine
CN113807215A (en) * 2021-08-31 2021-12-17 贵州大学 Tea tender shoot grading method combining improved attention mechanism and knowledge distillation
CN114332697A (en) * 2021-12-19 2022-04-12 西安科技大学 Method, system, equipment and medium for detecting faults of multiple types of targets in power transmission line
CN216626695U (en) * 2021-12-17 2022-05-31 昭平县农业农村局 Wild tea leaf collection system
CN114731840A (en) * 2022-04-07 2022-07-12 仲恺农业工程学院 Double-mechanical-arm tea picking robot based on machine vision

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108182374A (en) * 2016-12-08 2018-06-19 广州映博智能科技有限公司 A kind of picking point recognition methods for fruit string
WO2018157286A1 (en) * 2017-02-28 2018-09-07 深圳市大疆创新科技有限公司 Recognition method and device, and movable platform
CN111784764A (en) * 2020-05-28 2020-10-16 西南石油大学 A tea sprout identification and localization algorithm
CN111914715A (en) * 2020-07-24 2020-11-10 廊坊和易生活网络科技股份有限公司 Intelligent vehicle target real-time detection and positioning method based on bionic vision
CN113674226A (en) * 2021-07-31 2021-11-19 河海大学 A deep learning-based detection method for tea bud tip of tea picking machine
CN113807215A (en) * 2021-08-31 2021-12-17 贵州大学 Tea tender shoot grading method combining improved attention mechanism and knowledge distillation
CN216626695U (en) * 2021-12-17 2022-05-31 昭平县农业农村局 Wild tea leaf collection system
CN114332697A (en) * 2021-12-19 2022-04-12 西安科技大学 Method, system, equipment and medium for detecting faults of multiple types of targets in power transmission line
CN114731840A (en) * 2022-04-07 2022-07-12 仲恺农业工程学院 Double-mechanical-arm tea picking robot based on machine vision

Non-Patent Citations (7)

* Cited by examiner, † Cited by third party
Title
原艳芳;郑相周;林卫国;: "名优茶采摘机器人路径规划", no. 03 *
吕军;夏华;方梦瑞;周礼赞;: "基于AlexNet的茶叶嫩芽状态智能识别研究", no. 02 *
王海楠;弋景刚;张秀花;: "番茄采摘机器人识别与定位技术研究进展", no. 05 *
罗陆锋: "采摘机器人视觉定位及行为控制的硬件在环虚拟实验系统设计", pages 39 *
罗陆锋;邹湘军;叶敏;杨自尚;张丛;朱娜;王成琳;: "基于双目立体视觉的葡萄采摘防碰空间包围体求解与定位", no. 08 *
罗陆锋;邹湘军;程堂灿;杨自尚;张丛;莫宇达;: "采摘机器人视觉定位及行为控制的硬件在环虚拟试验系统设计", no. 04, pages 39 *
邵佩迪;吴明晖;: "基于视觉测量的茶叶嫩芽定位方法研究", no. 09 *

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116982480A (en) * 2023-08-18 2023-11-03 湖北神农草木农业开发有限公司 Automatic tea picking system and method
CN117876649A (en) * 2024-01-08 2024-04-12 华南农业大学 A method and device for identifying tea buds and determining their position and posture
CN118096891A (en) * 2024-04-24 2024-05-28 华南农业大学 A tea bud leaf pose estimation method and system based on a picking robot
CN118096891B (en) * 2024-04-24 2024-07-30 华南农业大学 A tea bud leaf pose estimation method and system based on a picking robot
CN118587288A (en) * 2024-08-06 2024-09-03 华南农业大学 A method and device for locating tea bud and leaf picking points
CN118587288B (en) * 2024-08-06 2025-01-17 华南农业大学 Tea bud leaf picking point positioning method and device
CN118941125A (en) * 2024-10-14 2024-11-12 宁德职业技术学院 A tea garden tea picking planning system and method based on visual recognition
CN118941125B (en) * 2024-10-14 2025-02-25 宁德职业技术学院 A tea garden tea picking planning system and method based on visual recognition
CN119991393A (en) * 2024-12-17 2025-05-13 湖北静香农业发展有限公司 A tea picking method with tea planting identification function
CN119540731A (en) * 2025-01-17 2025-02-28 上海恒泽辅汇智能科技有限公司 Edible mushroom bud detection and collection system, method, medium and device

Also Published As

Publication number Publication date
CN115271200B (en) 2023-05-30

Similar Documents

Publication Publication Date Title
CN115271200B (en) Intelligent coherent picking system for famous tea
Li et al. Development and field evaluation of a robotic harvesting system for plucking high-quality tea
EP3405845B1 (en) Object-focused active three-dimensional reconstruction
CN115272791A (en) Multi-target detection positioning method for tea based on YoloV5
Zeng et al. View planning in robot active vision: A survey of systems, algorithms, and applications
CN109559320B (en) Method and system for realizing visual SLAM semantic mapping function based on hole convolution deep neural network
CN109934115B (en) Face recognition model construction method, face recognition method and electronic equipment
CN119580985B (en) Robot-assisted wound treatment method and system based on multi-mode image analysis
CN119717864B (en) Obstacle avoidance method, device and storage medium for autonomous flight of UAV power inspection based on laser radar
CN111602517A (en) A distributed visual active perception method for string fruit and its application
CN118411415B (en) Mobile robot relocalization method and device based on whale algorithm and ensemble learning
CN116138036B (en) A secondary positioning method for picking tender buds of famous and high-quality tea
CN116958823B (en) Tea tender tip identification and picking point positioning method
CN112801059A (en) Graph convolution network system and 3D object detection method based on graph convolution network system
CN115187803B (en) Positioning method for picking process of famous tea tender shoots
Dai et al. 3MSP2: Sequential picking planning for multi-fruit congregated tomato harvesting in multi-clusters environment based on multi-views
CN118675148A (en) Multi-sensor three-dimensional environment sensing algorithm in automatic driving scene
CN117944059A (en) Track planning method based on vision and radar feature fusion
CN117274313B (en) Lightweight visual SLAM system and VSLAM method under dynamic scene
CN119563455B (en) A solanaceous fruit harvesting robot and its fruit cluster observation pose estimation method and device
CN120374924A (en) Badminton intelligent recovery and integrity detection sorting method and system based on artificial intelligence
CN120032041A (en) Point cloud precise modeling method and system based on laser point cloud and deep learning algorithm
CN115439535A (en) A fast target detection and positioning method, device and unmanned aerial vehicle system
CN115526908A (en) A high-speed moving filter rod explosion bead detection and positioning method
CN119169080B (en) Intelligent robot tea picking method based on multi-stage fusion detection and posture analysis

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
OL01 Intention to license declared
OL01 Intention to license declared