CN113095279A - Intelligent visual identification method, device and system for flower amount of fruit tree and storage medium - Google Patents
Intelligent visual identification method, device and system for flower amount of fruit tree and storage medium Download PDFInfo
- Publication number
- CN113095279A CN113095279A CN202110467162.6A CN202110467162A CN113095279A CN 113095279 A CN113095279 A CN 113095279A CN 202110467162 A CN202110467162 A CN 202110467162A CN 113095279 A CN113095279 A CN 113095279A
- Authority
- CN
- China
- Prior art keywords
- flower
- fruit tree
- leaf
- tree flower
- convolution
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 235000013399 edible fruits Nutrition 0.000 title claims abstract description 167
- 230000000007 visual effect Effects 0.000 title claims abstract description 49
- 238000000034 method Methods 0.000 title claims abstract description 29
- 230000011218 segmentation Effects 0.000 claims abstract description 25
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 13
- 238000000605 extraction Methods 0.000 claims description 24
- 238000010586 diagram Methods 0.000 claims description 14
- 238000013507 mapping Methods 0.000 claims description 12
- 238000010606 normalization Methods 0.000 claims description 11
- 238000006243 chemical reaction Methods 0.000 claims description 10
- 238000012545 processing Methods 0.000 claims description 9
- 230000007704 transition Effects 0.000 claims description 9
- 230000004913 activation Effects 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000001914 filtration Methods 0.000 claims description 4
- 230000008569 process Effects 0.000 claims description 4
- 238000005096 rolling process Methods 0.000 claims description 3
- 230000017260 vegetative to reproductive phase transition of meristem Effects 0.000 abstract description 6
- 244000183278 Nephelium litchi Species 0.000 description 15
- 230000006870 function Effects 0.000 description 7
- 239000003381 stabilizer Substances 0.000 description 5
- 239000011800 void material Substances 0.000 description 4
- 230000008859 change Effects 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000000087 stabilizing effect Effects 0.000 description 2
- 235000016623 Fragaria vesca Nutrition 0.000 description 1
- 240000009088 Fragaria x ananassa Species 0.000 description 1
- 235000011363 Fragaria x ananassa Nutrition 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 239000002420 orchard Substances 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 239000013598 vector Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/30—Noise filtering
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an intelligent visual identification method, device, system and storage medium for the flower amount of a fruit tree, wherein the method comprises the following steps: acquiring a plurality of fruit tree flower images; denoising and standardizing each fruit tree flower image; extracting a fruit tree flower and leaf feature map in the processed fruit tree flower and leaf image by using a trained deep convolutional neural network; generating a fruit tree flower and leaf prediction map according to the fruit tree flower and leaf characteristic map so as to obtain a fruit tree flower and leaf segmentation map; counting the number of pixels belonging to flowers and the number of pixels belonging to leaves according to the fruit tree flower and leaf segmentation map; calculating the density of the flower according to the number of the pixels of the flower and the number of the pixels of the leaves; the density of flowers in each image of the fruit tree flower is converted to an analog quantity. The invention can provide visual support for flower thinning equipment and assist fruit growers in flowering management.
Description
Technical Field
The invention relates to an intelligent visual identification method, device and system for the flower quantity of a fruit tree and a storage medium, and belongs to the technical field of agricultural equipment.
Background
With the development of artificial intelligence, the application of machine vision technology in agriculture is more and more important, and visual support is provided for intelligent operation. The yield and quality of fruit trees are influenced by the flowering amount of the fruit trees in the flowering period, and fruit growers need to observe the flowering amount in the flowering period and conduct flower thinning according to the observed results. For large-scale orchards, the manual observation and flower thinning efficiency is low, and the labor cost is high. However, the robot vision system is mostly applied to fruit detection and picking (chinese patent application No. 201711015161.8, entitled vision system of strawberry picking robot), and lacks a vision system for fruit tree florescence management.
Disclosure of Invention
In view of the above, the present invention provides a method, an apparatus, a system and a storage medium for intelligent visual recognition of flower amount of fruit trees, which can provide visual support for flower thinning equipment and assist fruit growers in managing flowering phase.
The invention aims to provide an intelligent visual identification method for the flower quantity of a fruit tree.
The second purpose of the invention is to provide an intelligent visual identification device for the flower quantity of the fruit trees.
The third purpose of the invention is to provide an intelligent visual identification system for the flower quantity of the fruit trees.
It is a fourth object of the present invention to provide a computer-readable storage medium.
The first purpose of the invention can be achieved by adopting the following technical scheme:
an intelligent visual identification method for the flower quantity of a fruit tree, comprising the following steps:
acquiring a plurality of fruit tree flower images;
denoising and standardizing each fruit tree flower image;
extracting a fruit tree flower and leaf feature map in the processed fruit tree flower and leaf image by using a trained deep convolutional neural network;
generating a fruit tree flower and leaf prediction map according to the fruit tree flower and leaf characteristic map so as to obtain a fruit tree flower and leaf segmentation map;
counting the number of pixels belonging to flowers and the number of pixels belonging to leaves according to the fruit tree flower and leaf segmentation map;
calculating the density of the flower according to the number of the pixels of the flower and the number of the pixels of the leaves;
the density of flowers in each image of the fruit tree flower is converted to an analog quantity.
Further, the deep convolutional neural network comprises a backbone network and a pyramid structure, the backbone network comprises a transition feature extraction part and a backbone feature extraction part, and the transition feature extraction part, the backbone feature extraction part and the pyramid structure are sequentially connected;
the transition feature extraction part comprises a first convolution layer, a second convolution layer and a third convolution layer which are connected in sequence, wherein the first convolution layer, the second convolution layer and the third convolution layer are convolution layers of a 3 x 3 convolution kernel;
the main feature extraction part comprises four stages which are sequentially connected, wherein the four stages are respectively a 1 st stage, a 2 nd stage, a 3 rd stage and a 4 th stage, 6, 8, 12 and 6 convolution blocks are respectively arranged from the 1 st stage to the 4 th stage, every two continuous convolution blocks form a residual error module, and each residual error module is linearly fused with the identity mapping of the output features and the input features of the two convolution blocks and then is used as the input of a subsequent convolution block; the output characteristic quantity of the convolution blocks in the same stage is the same as the initial input of the stage, the characteristic quantity of the cross-stage is different, the convolution layer of 1 multiplied by 1 convolution kernel is used for adjusting the characteristic quantity, and each convolution is finished through the processing of an activation function and a normalization layer; in each convolution block, the feature is processed using a 3 × 3 convolution kernel, each convolution pass through an activation function and a normalization layer.
Further, a hole convolution structure is added in the convolution blocks of the 3 rd stage and the 4 th stage, and an attention module is added between every two adjacent stages.
Further, the denoising and standardizing the fruit tree flower image specifically includes:
denoising the fruit tree flower image by using median filtering;
calculating the mean value and standard deviation of RGB three channel components of the denoised consequence tree flower image;
and carrying out standardized calculation on the denoised consequence tree flower image according to the mean value and the standard deviation.
Further, the generating a fruit tree flower and leaf prediction map according to the fruit tree flower and leaf characteristic map so as to obtain a fruit tree flower and leaf segmentation map specifically comprises:
fusing the characteristic maps of the flower leaves of the fruit trees through the first rolling block to obtain a first characteristic map; wherein the first volume block has a core size of 3 × 3;
inputting the first characteristic diagram into a second volume block to obtain a second characteristic diagram; wherein the second volume block has a core size of 1 × 1;
mapping the second characteristic map into three probability prediction maps respectively representing flowers, leaves and background of the fruit tree through a full convolution layer;
solving the maximum value of the three probability prediction graphs on the characteristic dimension to obtain a fruit tree flower and leaf prediction graph;
and superposing the fruit tree flower and leaf prediction image and the fruit tree flower and leaf characteristic image according to preset weight to obtain a fruit tree flower and leaf segmentation image.
Further, the calculating the density of the flower according to the number of the flower pixels and the number of the leaf pixels specifically includes:
summing the number of the pixels of the flowers and the number of the pixels of the leaves to obtain the total number of the pixels;
the flower density is obtained by dividing the number of flower pixels by the total number of pixels.
Further, the converting the density of flowers in each fruit tree flower image into an analog quantity specifically includes:
according to the density of flowers in all the fruit tree flower images, calculating the maximum value and the minimum value as the upper limit and the lower limit of the conversion between the density of the flowers and the analog quantity;
scaling the density range of the flowers to be between 0 and 1 by using a maximum and minimum normalization method according to the upper limit and the lower limit of the conversion of the density of the flowers and the analog quantity, and enabling the analog quantity range to be between 4 and 20 milliamperes;
constructing a mapping relation between a density range and an analog quantity range of the flower by using linear scaling;
and converting the flower density in each fruit tree flower image into an analog quantity according to the mapping relation between the flower density range and the analog quantity range.
The second purpose of the invention can be achieved by adopting the following technical scheme:
an intelligent visual identification device for flower amount of fruit trees, the device comprises:
the acquiring unit is used for acquiring a plurality of fruit tree flower images;
the processing unit is used for denoising and standardizing each fruit tree flower image;
the extraction unit is used for extracting the characteristic map of the flower and leaf of the fruit tree in the processed image of the flower and the fruit tree by using the trained deep convolutional neural network;
the generating unit is used for generating a fruit tree flower and leaf prediction map according to the fruit tree flower and leaf characteristic map so as to obtain a fruit tree flower and leaf segmentation map;
the counting unit is used for counting the number of pixels belonging to flowers and the number of pixels belonging to leaves according to the fruit tree flower and leaf segmentation map;
the calculating unit is used for calculating the density of the flowers according to the number of the pixels of the flowers and the number of the pixels of the leaves;
and the conversion unit is used for converting the density of flowers in each fruit tree flower image into analog quantity.
The third purpose of the invention can be achieved by adopting the following technical scheme:
an intelligent visual identification system for the flower amount of a fruit tree comprises a camera and a processor, wherein the camera is connected with the processor, the camera has a horizontal visual angle, and the visual field of the camera comprises the whole flower thinning working range;
the camera is used for shooting the fruit tree flower image;
the processor is used for executing the intelligent visual identification method for the flower quantity of the fruit trees.
The fourth purpose of the invention can be achieved by adopting the following technical scheme:
a computer readable storage medium, storing a program, which when executed by a processor, implements the above-mentioned intelligent visual identification method for fruit tree flower amount.
Compared with the prior art, the invention has the following beneficial effects:
according to actual flower thinning equipment and a field flower thinning scene, the intelligent visual identification system for the fruit tree flower quantity is composed of a camera and a processor, can be carried on flower thinning equipment, extracts a characteristic diagram of a flower leaf of a fruit tree through a trained deep convolutional neural network, can accurately segment a flower area of the fruit tree in real time, calculates the flower density, converts the flower density into an analog quantity, transmits an analog quantity signal to the flower thinning equipment, and provides visual support for automatic flower thinning; and a plurality of stabilizing components can be arranged, so that the whole system works more stably.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the structures shown in the drawings without creative efforts.
Fig. 1 is a structural diagram of an intelligent visual identification system for fruit tree flower amount in embodiment 1 of the present invention.
Fig. 2 is a simple flowchart of an intelligent visual identification method for the flower quantity of the fruit tree in embodiment 1 of the present invention.
Fig. 3 is a detailed flowchart of the intelligent visual identification method for the flower quantity of the fruit tree in embodiment 1 of the present invention.
Fig. 4 is a feature extraction flowchart according to embodiment 1 of the present invention.
FIG. 5 is a graph showing the density calculation and analog conversion process of flowers according to example 1 of the present invention.
Fig. 6 is a block diagram of a structure of an intelligent visual identification device for fruit tree flower amount in embodiment 2 of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be described clearly and completely with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some embodiments of the present invention, but not all embodiments, and all other embodiments obtained by a person of ordinary skill in the art without creative efforts based on the embodiments of the present invention belong to the protection scope of the present invention.
Example 1:
as shown in fig. 1, the present embodiment provides an intelligent visual recognition system for the flower amount of a fruit tree, which is mounted on a flower thinning apparatus 101, and includes a camera 102 and a processor 103, the camera 102 is connected to the processor 103, the height of the camera 102 is the same as that of the flower thinning apparatus 101, the flower thinning apparatus 101 does not belong to the protection scope of the present application, and details are omitted here to show that the system of the present embodiment has mountability.
The camera 102 is a color camera, is perpendicular to the ground, has a horizontal viewing angle, has a view field including the whole thinning working range, can collect complete image information, and is mainly used for shooting images of fruit trees and flowers.
Further, in order to make the height of the camera 102 the same as the height of the flower thinning device 101, the intelligent visual identification system for the flower amount of the fruit tree of the embodiment further includes an adjustable support rod 104, and the height of the camera 102 can be adjusted through the adjustable support rod 104, so as to ensure the shooting visual field.
Further, in order to maintain the stability of the camera 102, the intelligent visual identification system for the flower amount of the fruit trees of the embodiment further comprises a stabilizer 105, the camera 102 is fixed on the upper portion of the stabilizer 105, the lower portion of the stabilizer 105 is connected with the adjustable supporting rod 104, namely the adjustable supporting rod 104 is used for adjusting the height of the stabilizer 105, and then the height of the camera 102 is adjusted, and the stabilizer 105 can reduce the shaking and vibration influence of the movement of the flower thinning equipment 101.
Further, in order to alleviate the vibration and noise influence of the flower thinning equipment 101 on the processor 103, the intelligent visual identification system for fruit tree flower amount of the embodiment further comprises a cushioning platform 106, the cushioning platform 106 is installed on the flower thinning equipment 101, the adjustable supporting rod 104 is installed on the cushioning platform 106, the processor 103 is arranged in the cushioning platform 106, and the cushioning platform 106 can ensure the normal work of the processor 103.
The processor 103 is an intavada Xavier processor, and is connected to the camera 102 through a USB data line, and mainly acquires image information captured by the camera 102, processes the image information captured by the camera, obtains a segmentation map, and calculates density and analog quantity of flowers, and transmits the segmentation map, the density and the analog quantity to the flower thinning equipment 101 for operation.
As shown in fig. 1 to fig. 3, the present embodiment provides an intelligent visual identification method for flower amount of a fruit tree, which is mainly implemented by a processor 103, and specifically includes the following steps:
s201, obtaining a plurality of fruit tree flower images.
The fruit tree flower is litchi flower, after the visual angle of the camera 101 is fixed, the visual field range can include the whole flower thinning working range, the camera 101 shoots RGB color litchi flower images in the flower thinning range, automatic shooting in a fixed time interval needs to be set as the flower thinning equipment 101 is in a moving state, the shooting interval time is determined according to the moving speed and the flower thinning speed of the flower thinning equipment 101, and then the litchi flower images are transmitted to the processor 103 through a USB data line, so that the processor 103 can obtain the litchi flower images.
S202, denoising and standardizing each fruit tree flower image.
Further, the step S202 specifically includes:
s2021, denoising the fruit tree flower image by using median filtering.
S2022, calculating the mean value and the standard deviation of RGB three channel components of the denoised consequence tree flower image.
S2023, carrying out standardized calculation on the denoised consequence tree flower image according to the mean value and the standard deviation, wherein the value range of the calculated image is 0 to 1, and the influence caused by transformation in the characteristic extraction process can be effectively reduced.
And S203, extracting the characteristic map of the flower and leaf of the fruit tree in the processed image of the fruit tree flower by using the trained deep convolutional neural network.
Corresponding to the fruit tree flower of the present embodiment being litchi flower, the fruit flower and leaf characteristic diagram of the present embodiment is litchi flower and leaf characteristic diagram; as shown in fig. 4, the deep convolutional neural network of this embodiment includes a trunk network and a pyramid structure, where the trunk network includes two parts, which are a transition feature extraction part (part 1 in the figure) and a trunk feature extraction part (part 2 in the figure), and the transition feature extraction part, the trunk feature extraction part, and the pyramid structure are connected in sequence.
The transition feature extraction part is a shallow layer network, and comprises a first convolution layer, a second convolution layer and a third convolution layer which are sequentially connected, and is mainly used for learning general features such as textures, angular points and the like, wherein the first convolution layer, the second convolution layer and the third convolution layer are convolution layers of a 3 x 3 convolution kernel in the embodiment.
The main feature extraction part is a deep network, a convolution-activation-normalization convolution block which is continuously stacked in 32 layers in depth is used as a main body and is divided into four stages which are sequentially connected, the four stages are respectively a 1 st stage, a 2 nd stage, a 3 rd stage and a 4 th stage, 6, 8, 12 and 6 convolution blocks are respectively arranged from the 1 st stage to the 4 th stage, every two continuous convolution blocks form a residual error module, specific features related to a main learning task are mainly learned, and each residual error module is linearly fused with the identity mapping of the output features and the input features of the two convolution blocks and then is used as the input of a subsequent convolution block. The output feature quantity of the convolution block in the same stage is the same as that of the initial input of the stage, the feature quantity of the cross-stage is different, the convolution layer using 1 multiplied by 1 convolution kernel adjusts the feature quantity, and each convolution is completed by being processed by an activation function and a normalization layer. In each convolution block, the convolution is to process the feature using a 3 × 3 convolution kernel, then input to the activation function, and finally undergo normalization layer processing.
Further, a hole convolution structure is added into the convolution blocks in the 3 rd stage and the 4 th stage, and the hole convolution means that 0 values of interval rules are inserted into a convolution kernel to enlarge the size of the convolution kernel, so that the receptive field of deep features is expanded, the operation amount is not increased, and meanwhile, the maximum pooling is replaced by adjusting the sliding stride of filtering operation to reduce the loss of feature information; the classical void convolution uses a large void convolution rate to realize large object feature extraction, and in the embodiment, in order to better extract litchi flower features, a small void rate convolution structure is more suitable.
Furthermore, an attention module is added between every two adjacent stages, the attention module compresses the features with the size of w × h into 1 × 1, then a learnable linear connection layer is input to obtain weight vectors of the features of different channels, each weight measures the importance degree of the input features, and the weights are broadcasted into the size of the input features and then multiplied by the original input features to play a role in screening the features.
In order to increase the richness of the learning features of the convolutional layers, dense feature connection is selectively realized according to the action of the deep and shallow convolutional layers, specifically dense feature connection is realized on deep features, the connection mode is shown as fig. 4, dense feature connection means that the input features of the next layer are the output features of all previous layers, as described above, the shallow network learning textures, corners and other general features, the deep network learning tasks have specific features, and the features of the over-shallow layer and the over-deep layer have larger difference, so that the dense connection range is adjusted to ensure the richness and consistency of feature extraction.
After the trunk features are extracted, the trunk features are processed by using a pyramid structure, and the learning capability of the network multi-scale features is enhanced; in the pyramid structure, four convolutional layers with different void ratios and filter steps are used for processing input intermediate features in parallel, then the input features and the features processed in parallel are spliced in feature dimensions, finally fusion and correction are carried out through 3 x 3 convolutional layers to obtain a multi-scale litchi mosaic feature map, 1280 litchi mosaic feature maps are obtained, and the size of the litchi mosaic feature map is 1/16 of an input image.
And S204, generating a fruit tree flower and leaf prediction map according to the fruit tree flower and leaf characteristic map, thereby obtaining a fruit tree flower and leaf segmentation map.
Corresponding to the characteristic map of the litchi flower leaf of the fruit tree flower leaf characteristic map of the embodiment, the fruit tree flower leaf prediction map of the embodiment is a litchi flower leaf prediction map, and the fruit tree flower leaf segmentation map is a litchi flower leaf segmentation map; as shown in fig. 5, this step S204 is implemented by constructing a prediction layer, where the prediction layer includes two volume blocks, where the core size of the first volume block is 3 × 3 and the core size of the second volume block is 1 × 1; the step S204 specifically includes:
s2041, fusing the characteristic diagrams of the flower and leaf of the fruit tree through the first rolling block to obtain a first characteristic diagram.
S2042, inputting the first characteristic diagram into a second convolution block to obtain a second characteristic diagram.
In steps S2041 and S2042, 1280 litchi mosaic feature maps are fused using a 3 × 3 convolution block to obtain a first feature map, and then the first feature map is input to a second convolution block to obtain 256 feature maps, i.e., a second feature map, wherein the feature size of the second feature map is unchanged.
S2043, mapping the second characteristic map into three probability prediction maps through the full convolution layer, wherein the three probability prediction maps respectively represent litchi flowers, leaves and backgrounds.
S2044, solving the maximum value of the three probability prediction maps on the characteristic dimension to obtain a fruit tree flower and leaf prediction map, wherein the fruit tree flower and leaf prediction map is provided with flower labels and leaf labels.
S2045, overlapping the fruit tree flower and leaf prediction image and the fruit tree flower and leaf characteristic image according to preset weight to obtain a fruit tree flower and leaf segmentation image.
And S205, counting the number of pixels belonging to the flower and the number of pixels belonging to the leaf according to the fruit tree flower and leaf segmentation map.
And S206, calculating the density of the flowers according to the number of the flowers and the number of the leaves.
Further, the step S206 specifically includes:
s2061, summing the number of the flower pixels and the number of the leaf pixels to obtain the total number of the pixels.
S2062, dividing the number of the flower pixels by the total number of the flower pixels to obtain the density of the flower.
And S207, converting the density of the flowers in each fruit tree flower image into an analog quantity.
Further, step S207 specifically includes:
s2071, calculating the maximum value and the minimum value according to the flower density in all the fruit tree flower images as the upper limit and the lower limit of the flower density and the analog quantity conversion.
In this embodiment, the maximum value and the minimum value may be set as the upper limit and the lower limit, respectively, but in order to ensure reliability, the upper limit may be set to 1.1 times the maximum value and the lower limit may be set to 0.
And S2072, scaling the density range of the flowers to be between 0 and 1 by using a maximum and minimum normalization method according to the upper limit and the lower limit of the conversion of the density of the flowers and the analog quantity, and enabling the analog quantity range to be between 4 and 20 milliamperes.
S2073, constructing a mapping relation between the density range and the analog quantity range of the flower by using linear scaling.
S2074, according to the mapping relation between the flower density range and the analog quantity range, the flower density in each fruit tree flower image is converted into the analog quantity.
The processor 103 of the present embodiment transmits the converted analog signal to the thinning apparatus 101 to realize automatic control of thinning.
It should be noted that although the method operations of the above-described embodiments are depicted in the drawings in a particular order, this does not require or imply that these operations must be performed in this particular order, or that all of the illustrated operations must be performed, to achieve desirable results. Rather, the depicted steps may change the order of execution. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions.
Example 2:
as shown in fig. 6, the present embodiment provides an intelligent visual identification device for the flower amount of a fruit tree, which can be applied to the processor of embodiment 1, and includes an obtaining unit 601, a processing unit 602, an extracting unit 603, a generating unit 604, a counting unit 605, a calculating unit 606, and a converting unit 607, and the specific functions of each unit are as follows:
the acquiring unit 601 is configured to acquire a plurality of images of fruit tree flowers.
The processing unit 602 is configured to perform denoising and normalization processing on each fruit tree flower image.
And an extracting unit 603, configured to extract a characteristic map of a flower and leaf of a fruit tree in the processed image of the fruit tree flower by using the trained deep convolutional neural network.
The generating unit 604 is configured to generate a fruit flower and leaf prediction map according to the fruit flower and leaf feature map, so as to obtain a fruit flower and leaf segmentation map.
And the counting unit 605 is used for counting the number of pixels belonging to the flower and the number of pixels belonging to the leaf according to the fruit tree flower and leaf segmentation map.
A calculating unit 606 for calculating the density of the flower according to the number of the flower pixels and the number of the leaf pixels.
A conversion unit 607 for converting the density of flowers in each fruit tree flower image into an analog quantity.
The specific implementation of each unit in this embodiment may refer to the intelligent visual identification method for the flower amount of the fruit tree in embodiment 1, which is not described in detail herein; it should be noted that the apparatus provided in this embodiment is only exemplified by the division of the above functional units, and in practical applications, the above functions may be allocated to different functional units as needed to complete, that is, the internal structure is divided into different functional units to complete all or part of the above described functions.
Example 3:
the present embodiment provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the method for intelligently and visually identifying the flower amount of a fruit tree in embodiment 1 is implemented as follows:
acquiring a plurality of fruit tree flower images;
denoising and standardizing each fruit tree flower image;
extracting a fruit tree flower and leaf feature map in the processed fruit tree flower and leaf image by using a trained deep convolutional neural network;
generating a fruit tree flower and leaf prediction map according to the fruit tree flower and leaf characteristic map so as to obtain a fruit tree flower and leaf segmentation map;
counting the number of pixels belonging to flowers and the number of pixels belonging to leaves according to the fruit tree flower and leaf segmentation map;
calculating the density of the flower according to the number of the pixels of the flower and the number of the pixels of the leaves;
the density of flowers in each image of the fruit tree flower is converted to an analog quantity.
It should be noted that the computer readable storage medium of the present embodiment may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
In conclusion, according to actual flower thinning equipment and a field flower thinning scene, the intelligent visual identification system for the fruit tree flower quantity is adopted by the camera and the processor, the system can be carried on flower thinning equipment, a characteristic diagram of a flower leaf of a fruit tree is extracted through a trained deep convolutional neural network, a fruit tree flower area can be accurately segmented in real time, the flower density is calculated, the flower density is converted into an analog quantity, an analog quantity signal is transmitted to the flower thinning equipment, and visual support is provided for automatic flower thinning; and a plurality of stabilizing components can be arranged, so that the whole system works more stably.
The above description is only for the preferred embodiments of the present invention, but the protection scope of the present invention is not limited thereto, and any person skilled in the art can substitute or change the technical solution and the inventive concept of the present invention within the scope of the present invention.
Claims (10)
1. An intelligent visual identification method for the flower quantity of a fruit tree is characterized by comprising the following steps:
acquiring a plurality of fruit tree flower images;
denoising and standardizing each fruit tree flower image;
extracting a fruit tree flower and leaf feature map in the processed fruit tree flower and leaf image by using a trained deep convolutional neural network;
generating a fruit tree flower and leaf prediction map according to the fruit tree flower and leaf characteristic map so as to obtain a fruit tree flower and leaf segmentation map;
counting the number of pixels belonging to flowers and the number of pixels belonging to leaves according to the fruit tree flower and leaf segmentation map;
calculating the density of the flower according to the number of the pixels of the flower and the number of the pixels of the leaves;
the density of flowers in each image of the fruit tree flower is converted to an analog quantity.
2. The intelligent visual identification method for the flower volume of the fruit tree according to claim 1, wherein the deep convolutional neural network comprises a trunk network and a pyramid structure, the trunk network comprises a transition feature extraction part and a trunk feature extraction part, and the transition feature extraction part, the trunk feature extraction part and the pyramid structure are connected in sequence;
the transition feature extraction part comprises a first convolution layer, a second convolution layer and a third convolution layer which are connected in sequence, wherein the first convolution layer, the second convolution layer and the third convolution layer are convolution layers of a 3 x 3 convolution kernel;
the main feature extraction part comprises four stages which are sequentially connected, wherein the four stages are respectively a 1 st stage, a 2 nd stage, a 3 rd stage and a 4 th stage, 6, 8, 12 and 6 convolution blocks are respectively arranged from the 1 st stage to the 4 th stage, every two continuous convolution blocks form a residual error module, and each residual error module is linearly fused with the identity mapping of the output features and the input features of the two convolution blocks and then is used as the input of a subsequent convolution block; the output characteristic quantity of the convolution blocks in the same stage is the same as the initial input of the stage, the characteristic quantity of the cross-stage is different, the convolution layer of 1 multiplied by 1 convolution kernel is used for adjusting the characteristic quantity, and each convolution is finished through the processing of an activation function and a normalization layer; in each convolution block, the feature is processed using a 3 × 3 convolution kernel, each convolution pass through an activation function and a normalization layer.
3. The intelligent visual identification method for the flower quantity of the fruit trees as claimed in claim 2, wherein a hole convolution structure is added in the convolution blocks of the 3 rd stage and the 4 th stage, and an attention module is added between every two adjacent stages.
4. The intelligent visual recognition method for fruit tree flower volume according to any one of claims 1-3, wherein the denoising and normalizing process for the fruit tree flower image specifically comprises:
denoising the fruit tree flower image by using median filtering;
calculating the mean value and standard deviation of RGB three channel components of the denoised consequence tree flower image;
and carrying out standardized calculation on the denoised consequence tree flower image according to the mean value and the standard deviation.
5. The intelligent visual recognition method for fruit tree flower quantity according to any one of claims 1-3, wherein the generating of the fruit tree flower and leaf prediction map according to the fruit tree flower and leaf feature map so as to obtain the fruit tree flower and leaf segmentation map specifically comprises:
fusing the characteristic maps of the flower leaves of the fruit trees through the first rolling block to obtain a first characteristic map; wherein the first volume block has a core size of 3 × 3;
inputting the first characteristic diagram into a second volume block to obtain a second characteristic diagram; wherein the second volume block has a core size of 1 × 1;
mapping the second characteristic map into three probability prediction maps respectively representing flowers, leaves and background of the fruit tree through a full convolution layer;
solving the maximum value of the three probability prediction graphs on the characteristic dimension to obtain a fruit tree flower and leaf prediction graph;
and superposing the fruit tree flower and leaf prediction image and the fruit tree flower and leaf characteristic image according to preset weight to obtain a fruit tree flower and leaf segmentation image.
6. The intelligent visual identification method for the flower quantity of the fruit trees according to any one of claims 1-3, wherein the calculating the flower density according to the number of the flower pixels and the number of the leaf pixels specifically comprises:
summing the number of the pixels of the flowers and the number of the pixels of the leaves to obtain the total number of the pixels;
the flower density is obtained by dividing the number of flower pixels by the total number of pixels.
7. The intelligent visual identification method for fruit tree flower volume according to any one of claims 1-3, wherein the converting the density of flowers in each fruit tree flower image into an analog volume specifically comprises:
according to the density of flowers in all the fruit tree flower images, calculating the maximum value and the minimum value as the upper limit and the lower limit of the conversion between the density of the flowers and the analog quantity;
scaling the density range of the flowers to be between 0 and 1 by using a maximum and minimum normalization method according to the upper limit and the lower limit of the conversion of the density of the flowers and the analog quantity, and enabling the analog quantity range to be between 4 and 20 milliamperes;
constructing a mapping relation between a density range and an analog quantity range of the flower by using linear scaling;
and converting the flower density in each fruit tree flower image into an analog quantity according to the mapping relation between the flower density range and the analog quantity range.
8. An intelligent visual identification device for the flower amount of a fruit tree, which is characterized by comprising:
the acquiring unit is used for acquiring a plurality of fruit tree flower images;
the processing unit is used for denoising and standardizing each fruit tree flower image;
the extraction unit is used for extracting the characteristic map of the flower and leaf of the fruit tree in the processed image of the flower and the fruit tree by using the trained deep convolutional neural network;
the generating unit is used for generating a fruit tree flower and leaf prediction map according to the fruit tree flower and leaf characteristic map so as to obtain a fruit tree flower and leaf segmentation map;
the counting unit is used for counting the number of pixels belonging to flowers and the number of pixels belonging to leaves according to the fruit tree flower and leaf segmentation map;
the calculating unit is used for calculating the density of the flowers according to the number of the pixels of the flowers and the number of the pixels of the leaves;
and the conversion unit is used for converting the density of flowers in each fruit tree flower image into analog quantity.
9. An intelligent visual identification system for the flower amount of a fruit tree is characterized by comprising a camera and a processor, wherein the camera is connected with the processor, the camera has a horizontal visual angle, and the visual field of the camera comprises the whole flower thinning working range;
the camera is used for shooting the fruit tree flower image;
the processor is used for executing the intelligent visual identification method of the flower quantity of the fruit trees according to any one of claims 1 to 7.
10. A computer-readable storage medium storing a program, wherein the program, when executed by a processor, implements the method for intelligent visual identification of fruit tree flower volume according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110467162.6A CN113095279B (en) | 2021-04-28 | 2021-04-28 | Intelligent visual recognition method, device and system for flower quantity of fruit tree and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110467162.6A CN113095279B (en) | 2021-04-28 | 2021-04-28 | Intelligent visual recognition method, device and system for flower quantity of fruit tree and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113095279A true CN113095279A (en) | 2021-07-09 |
| CN113095279B CN113095279B (en) | 2023-10-24 |
Family
ID=76681302
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110467162.6A Active CN113095279B (en) | 2021-04-28 | 2021-04-28 | Intelligent visual recognition method, device and system for flower quantity of fruit tree and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113095279B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114511849A (en) * | 2021-12-30 | 2022-05-17 | 广西慧云信息技术有限公司 | Grape thinning identification method based on graph attention network |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2948499A1 (en) * | 2016-11-16 | 2018-05-16 | The Governing Council Of The University Of Toronto | System and method for classifying and segmenting microscopy images with deep multiple instance learning |
| CN109919025A (en) * | 2019-01-30 | 2019-06-21 | 华南理工大学 | Video scene Method for text detection, system, equipment and medium based on deep learning |
| CN109978077A (en) * | 2019-04-08 | 2019-07-05 | 南京旷云科技有限公司 | Visual identity methods, devices and systems and storage medium |
| CN111523546A (en) * | 2020-04-16 | 2020-08-11 | 湖南大学 | Image semantic segmentation method, system and computer storage medium |
-
2021
- 2021-04-28 CN CN202110467162.6A patent/CN113095279B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2948499A1 (en) * | 2016-11-16 | 2018-05-16 | The Governing Council Of The University Of Toronto | System and method for classifying and segmenting microscopy images with deep multiple instance learning |
| CN109919025A (en) * | 2019-01-30 | 2019-06-21 | 华南理工大学 | Video scene Method for text detection, system, equipment and medium based on deep learning |
| CN109978077A (en) * | 2019-04-08 | 2019-07-05 | 南京旷云科技有限公司 | Visual identity methods, devices and systems and storage medium |
| CN111523546A (en) * | 2020-04-16 | 2020-08-11 | 湖南大学 | Image semantic segmentation method, system and computer storage medium |
Non-Patent Citations (2)
| Title |
|---|
| 柳长源;王琪;毕晓君;: "基于多通道多尺度卷积神经网络的单幅图像去雨方法", 电子与信息学报, no. 09, pages 224 - 231 * |
| 熊俊涛 等: "基于改进YLOL v3网络的夜间环境柑橘识别方法", 《农业机械学报》, vol. 51, no. 4, pages 199 - 207 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114511849A (en) * | 2021-12-30 | 2022-05-17 | 广西慧云信息技术有限公司 | Grape thinning identification method based on graph attention network |
| CN114511849B (en) * | 2021-12-30 | 2024-05-17 | 广西慧云信息技术有限公司 | Grape thinning identification method based on graph attention network |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113095279B (en) | 2023-10-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108710863A (en) | Unmanned plane Scene Semantics dividing method based on deep learning and system | |
| CN110222787A (en) | Multiscale target detection method, device, computer equipment and storage medium | |
| CN109886155B (en) | Single-plant rice detection and positioning method, system, equipment and medium based on deep learning | |
| CN113361373A (en) | Real-time semantic segmentation method for aerial image in agricultural scene | |
| CN112597920A (en) | Real-time object detection system based on YOLOv3 pruning network | |
| CN114140665A (en) | Dense small target detection method based on improved YOLOv5 | |
| CN114898359B (en) | Litchi plant diseases and insect pests detection method based on improvement EFFICIENTDET | |
| Peng et al. | Litchi detection in the field using an improved YOLOv3 model | |
| CN113095279B (en) | Intelligent visual recognition method, device and system for flower quantity of fruit tree and storage medium | |
| CN103793054B (en) | A kind of action identification method simulating declarative memory process | |
| CN116543165B (en) | Remote sensing image fruit tree segmentation method based on dual-channel composite depth network | |
| CN108764233A (en) | A kind of scene character recognition method based on continuous convolution activation | |
| CN113723833B (en) | Method, system, terminal equipment and storage medium for evaluating quality of forestation actual results | |
| CN116246184A (en) | Papaver intelligent identification method and system applied to unmanned aerial vehicle aerial image | |
| CN116071653A (en) | Automatic extraction method for multi-stage branch structure of tree based on natural image | |
| CN108109125A (en) | Information extracting method and device based on remote sensing images | |
| CN112082475B (en) | Living stumpage species identification method and volume measurement method | |
| CN115294467A (en) | Detection method and related device for tea diseases | |
| CN114882072A (en) | High-speed correlation filtering target tracking method based on multi-channel image feature fusion | |
| CN114694248A (en) | Hand hygiene monitoring method, system, equipment and medium based on graph neural network | |
| CN110910496B (en) | VR natural environment automatic construction method based on big data and AI | |
| CN112418112A (en) | Orchard disease and pest monitoring and early warning method and system | |
| CN112633158A (en) | Power transmission line corridor vehicle identification method, device, equipment and storage medium | |
| CN112926605B (en) | Multi-stage strawberry fruit rapid detection method in natural scene | |
| CN116416527A (en) | Remote sensing image-based power transmission line corridor object identification method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |