CN113688690A - Large-scale fruit counting method and system - Google Patents
Large-scale fruit counting method and system Download PDFInfo
- Publication number
- CN113688690A CN113688690A CN202110859589.0A CN202110859589A CN113688690A CN 113688690 A CN113688690 A CN 113688690A CN 202110859589 A CN202110859589 A CN 202110859589A CN 113688690 A CN113688690 A CN 113688690A
- Authority
- CN
- China
- Prior art keywords
- image
- fruit
- orchard
- convolution
- label
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a large-scale fruit counting method and a large-scale fruit counting system, which comprise the following steps: acquiring an orchard image of an orchard to be detected; inputting the orchard image into a fruit counting model to determine the number of target fruits in the orchard image; the fruit counting model is obtained by training a multi-column convolution neural network according to a sample image with a density image label; the density image label is generated from the label image; the marked image is obtained by marking the target fruit in the sample image. According to the large-scale fruit counting method and system provided by the invention, the orchard image is input into the fruit counting model constructed by the MCNN, so that the rapid statistics of the number of target fruits is realized, and the accurate and rapid estimation of the fruit yield is ensured.
Description
Technical Field
The invention relates to the technical field of computer image processing, in particular to a large-scale fruit counting method and system.
Background
Fruits as one of economic crops occupy a very important position in the agricultural kingdom in China. As global climate warms, the stability of fruit production can greatly impact the income level of people. In order to estimate the yield of crops, it is necessary to acquire the unit yield of regional crops and the spatial distribution information of the regional crops, on one hand, the growth condition of the crops can be timely and accurately mastered, the macro regulation and control of the crops and the safety early warning of the crops can be carried out, and on the other hand, the method has important significance on the trade circulation of the crops and the sustainable development of agriculture. Typical yield estimates are based on historical data and weather conditions, and are performed by manual counting sampling, but this method is time-consuming and labor-consuming, and the estimation results are not accurate enough, and even the distribution of fruits in the orchard is not reflected, especially the data obtained in large orchards with high spatial variability are not accurate enough.
In recent years, in order to reduce the amount of labor and improve the accuracy of results, researchers have proposed methods for predicting orchard yields through visual technology monitoring. The method comprises the steps of obtaining remote sensing image data of fruits by using a remote sensing technology, counting the fruits by using a visual technology, and constructing a fruit yield model by combining statistical data of the weight of a single fruit so as to accurately identify fruits on a tree as much as possible.
Because the position of each fruit does not need to be identified in the process of estimating the yield, only the total number needs to be obtained, the existing identification or detection model method based on deep learning has large calculation amount and has the problem of time consumption.
Disclosure of Invention
Aiming at the problems of insufficient timeliness and accuracy in the prior art, the embodiment of the invention provides a large-scale fruit counting method and system.
The invention provides a large-scale fruit counting method, which comprises the following steps: acquiring an orchard image of an orchard to be detected;
inputting the orchard image into a fruit counting model to determine the number of target fruits in the orchard image;
the fruit counting model is obtained by training a multi-column convolution neural network according to a sample image with a density image label;
the density image label is generated from a label image;
the marked image is obtained by marking a target fruit in the sample image.
According to the large-scale fruit counting method provided by the invention, the multi-column convolutional neural network is a three-column convolutional neural network and comprises a first column of convolutional units, a second column of convolutional units and a third column of convolutional units;
the convolution kernel of the convolution unit in the first row is larger than that of the convolution unit in the second row, and the convolution kernel of the convolution unit in the second row is larger than that of the convolution unit in the third row.
According to the large-scale fruit counting method provided by the invention, before the orchard image is input to the fruit counting model, the method further comprises the following steps:
acquiring a plurality of sample images; the sample image is an image including the target fruit;
marking all target fruits in any sample image to obtain a marked image corresponding to any sample image;
generating a density image label according to any one of the label images based on a label density image generation method;
combining each sample image and the density image label corresponding to each sample image to be used as a training sample, constructing a training sample set, and training the multi-column convolutional neural network by using the training sample set.
According to the large-scale fruit counting method provided by the invention, the first row of convolution units comprises 4 convolution layers which are sequentially connected, and the convolution kernel scales of the convolution layers in the first row of convolution units are sequentially 9 × 9, 7 × 7, 7 × 7 and 7 × 7;
the second row of convolution units comprises 4 convolution layers which are connected in sequence, and the convolution kernel scales of the convolution layers in the second row of convolution units are 7 × 7, 5 × 5, 5 × 5 and 5 × 5 in sequence;
the convolution units in the third row comprise 4 convolution layers which are connected in sequence, and the convolution kernel scales of the convolution layers in the convolution units in the third row are 5 × 5, 3 × 3, 3 × 3 and 3 × 3 in sequence.
According to the large-scale fruit counting method provided by the invention, based on the label density image generation method, the density image label is generated according to the marking image,
wherein x is any sample image, the continuous density function f (x) is the density image label of any sample image x, h (x) is a labeled image of any sample image x, G is a gaussian kernel, and σ is a gaussian kernel standard deviation;
n is the number of the target fruits in any sample image x, xiThe target fruit at a location is denoted as delta (x-x)i)，xiThe coordinate position of the ith target fruit center in the sample image x is obtained;
beta is a preset value, and beta is a preset value,
the average value of the Euclidean distance sum of the ith target fruit in the sample image x and k adjacent target fruits; k is the number of neighboring target fruits, i is the ith target fruit in the sample image x, j ranges from 1 to k,
the euclidean distance between the ith target fruit and the jth neighboring target fruit in the sample image x.
According to the large-scale fruit counting method provided by the invention, the orchard image of the orchard to be detected is obtained, and the method comprises the following steps: acquiring an initial image of an orchard to be tested; and carrying out size normalization processing on the initial image to obtain the orchard image.
The invention also provides a large scale fruit counting system, comprising:
the image acquisition unit is used for acquiring an orchard image of an orchard to be detected;
the counting unit is used for inputting the orchard image into a fruit counting model so as to determine the number of target fruits in the orchard image;
the fruit counting model is obtained by training a multi-column convolution neural network according to a sample image with a density image label;
the density image label is generated from a label image;
the marked image is obtained by marking a target fruit in the sample image.
According to the invention, the large-scale fruit counting system further comprises: and the normalization unit is used for carrying out size normalization processing on the initial image of the orchard to be detected so as to obtain the orchard image.
The invention also provides an electronic device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the program to realize the steps of any one of the large-scale fruit counting methods.
The invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of any of the large scale fruit counting methods described above.
According to the large-scale fruit counting method and system provided by the invention, the orchard image is input into the fruit counting model constructed by the MCNN, so that the rapid statistics of the number of target fruits is realized, and the accurate and rapid estimation of the fruit yield is ensured.
Drawings
In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
FIG. 1 is a schematic flow diagram of a large scale fruit counting method provided by the present invention;
FIG. 2 is a schematic diagram of a multi-column convolutional neural network provided by the present invention;
FIG. 3 is a schematic diagram of a model training process of the large-scale fruit counting method provided by the present invention;
FIG. 4 is a schematic diagram of the test results of the test sample provided by the present invention;
FIG. 5 is a schematic diagram of a large scale fruit counting system provided by the present invention;
fig. 6 is a schematic structural diagram of an electronic device provided in the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that in the description of the embodiments of the present invention, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element. The specific meanings of the above terms in the present invention can be understood by those skilled in the art according to specific situations.
The invention provides a basic large-scale fruit counting method, which aims at the problem that the existing recognition or detection model based on deep learning is high in accuracy but too time-consuming.
The following describes a large-scale fruit counting method and system provided by the embodiment of the invention with reference to fig. 1 to 6.
Fig. 1 is a schematic flow chart of a large-scale fruit counting method provided by the present invention, as an alternative embodiment, as shown in fig. 1, including but not limited to the following steps:
s1, acquiring an orchard image of the orchard to be detected;
s2, inputting the orchard image to a fruit counting model to determine the number of target fruits in the orchard image;
the fruit counting model is obtained by training a multi-column convolution neural network according to a sample image with a density image label;
the density image label is generated from a label image;
the marked image is obtained by marking a target fruit in the sample image.
The orchard to be tested can be partitioned according to actual requirements, image acquisition is carried out on each partition, so that the relative size of target fruits in the orchard image is convenient to identify, and counting accuracy is improved.
The orchard image of the orchard to be detected can be acquired through image acquisition equipment, and can also be a satellite image of the orchard to be detected.
As an optional embodiment, in S1, the remote sensing device carried by the unmanned aerial vehicle is used to perform image acquisition on the orchard to be tested, and an orchard image of the orchard to be tested is obtained.
The image acquisition device may be any one of a depth camera, a hyperspectral camera, a thermal infrared camera or a remote sensing camera.
Image acquisition can select in the better time quantum of illumination, or carry out the light filling to image acquisition equipment to improve the quality of orchard image gathered, and then improve the accuracy of count.
Further, in step S2, the orchard image is input to the fruit counting model, the density image corresponding to the orchard image is output, and the density image is subjected to integration processing to obtain the number of target fruits in the orchard image.
The fruit counting model is obtained by training a Multi-column Convolutional Neural Network (MCNN) according to a sample image with a density image label.
All target fruits in the sample image can be subjected to dotting marking processing through a manual marking or machine vision image processing method, so that a marked image is obtained. For better counting effect, dotting can be performed at the central position of the target fruit.
The target fruit may be apple, pear or orange, and the counting of the apples is taken as an example in the following examples of the present invention, which should not be construed as a limitation to the scope of the present invention.
And generating a density image label with apple density information according to the marking image by a label density image generation method.
According to the large-scale fruit counting method provided by the invention, the orchard image is input into the fruit counting model constructed by the MCNN, so that the rapid statistics of the number of target fruits is realized, and the accurate and rapid estimation of the fruit yield is ensured.
Fig. 2 is a schematic structural diagram of a multi-column convolutional neural network provided in the present invention, and as shown in fig. 2, the multi-column convolutional neural network is a three-column convolutional neural network, which includes a first column convolution, a second column convolution and a third column convolution;
the convolution kernel of the convolution unit in the first row is larger than that of the convolution unit in the second row, and the convolution kernel of the convolution unit in the second row is larger than that of the convolution unit in the third row.
As an alternative embodiment, the multi-column main body of the MCNN network uses three columns of convolutional neural networks, the first column of convolution is represented as L columns, and includes 4 convolutional layers (conv) connected in sequence, and the large-scale convolution kernel scales used are 9 × 9, 7 × 7, 7 × 7, and 7 × 7 in sequence; the convolution of the second row is expressed as M rows, and comprises 4 convolution layers which are connected in sequence, and the scale of the adopted mesoscale convolution kernel is 7 × 7, 5 × 5, 5 × 5 and 5 × 5 in sequence; the third column of convolutions is shown as S columns, comprising 4 convolutional layers connected in series, using small scale convolution kernels of scale 5 × 5, 3 × 3, 3 × 3, 3 × 3 in sequence, wherein the second and third convolutions in each column of convolutions employ 2 × 2 Pooling (Pooling: 2 × 2). After an Input image (Input image) is Input into the MCNN network, merging the L, M, S three-column convolutional neural networks to obtain a Merged feature map (large feature maps), and performing 1 × 1 convolution (conv: 1 × 1) to obtain a Density image (Density map) generated by the network.
According to the large-scale fruit counting method provided by the embodiment of the invention, convolution kernels of various scales are used for adapting to sizes of apples of different scales, the problem of different sizes of apples is solved, and the robustness of a fruit counting model is improved.
Based on the foregoing embodiment, as an optional embodiment, before inputting the orchard image to a fruit counting model, the method further includes:
acquiring a plurality of sample images; the sample image is an orchard image comprising the target fruit;
marking all target fruits in any sample image to obtain a marked image corresponding to any sample image;
generating a density image label according to any one of the label images based on a label density image generation method;
combining each sample image and the density image label corresponding to each sample image to be used as a training sample, constructing a training sample set, and training the multi-column convolutional neural network by using the training sample set.
The dotting mark of the target fruit in the sample image can be manually marked or marked by a visual image processing method.
As an optional embodiment, the method selects unmanned aerial vehicle remote sensing image data of an apple orchard in Yanchuan as a sample image, and the sample image contains 43 images, which are abbreviated as Ycapple. Firstly, reading an initial image of a Ycap data set as a sample image; secondly, dotting and marking each sample image of the Ycap data set at the central point of the apple to obtain a marked image corresponding to each sample image; then, converting the marked image into a density image label by using a label density image generation method of the MCNN; and taking the combination of each sample image and the density image label as a training sample, forming a training sample set by using a plurality of training samples, and randomly matching all the training samples in the training sample set according to the ratio of 8:1: the proportion of 1 is divided into a training set, a testing set and a verification set, and the fruit counting model is trained by utilizing the divided training sample set.
According to the large-scale fruit counting method provided by the embodiment of the invention, the combination of each sample image and the density image label is used as a training sample to obtain a training sample set, and the training sample set is used for training the fruit counting model. The method trains the fruit counting model based on the thought of deep learning, so that the fruit counting model learns the characteristics of the sample images corresponding to different quantity degrees, and the trained fruit counting model is favorable for counting the quantity of the target fruits in the orchard image.
As an optional embodiment, the fruit counting model is trained by using a training sample set, specifically:
for any training sample, inputting the training sample into the fruit counting model, and outputting a predicted density image corresponding to the training sample;
and performing parameter optimization processing on the trained fruit counting model according to the predicted density image and the loss of the density image label to obtain the fruit counting model.
As an optional embodiment, a fruit counting model constructed based on the MCNN network is trained on a training set, the trained fruit counting model is subjected to model performance evaluation on a test set, parameters such as learning rate and step length are adjusted, and iterative loop training is performed to optimize the mean square error and the mean absolute error of the fruit counting model on the test.
According to the large-scale fruit counting method provided by the embodiment of the invention, the trained fruit counting model is obtained by training the fruit counting model to be trained, the model has the characteristics of high efficiency and high accuracy, the density information of the target fruit on the image can be rapidly counted, and a basis is provided for estimating the fruit yield in the orchard to be measured.
Based on the above embodiment, as an optional embodiment, the first row of convolution units includes 4 convolution layers connected in sequence, and convolution kernel scales of the convolution layers in the first row of convolution units are 9 × 9, 7 × 7, 7 × 7, and 7 × 7 in sequence;
the second row of convolution units comprises 4 convolution layers which are connected in sequence, and the convolution kernel scales of the convolution layers in the second row of convolution units are 7 × 7, 5 × 5, 5 × 5 and 5 × 5 in sequence;
the convolution units in the third row comprise 4 convolution layers which are connected in sequence, and the convolution kernel scales of the convolution layers in the convolution units in the third row are 5 × 5, 3 × 3, 3 × 3 and 3 × 3 in sequence.
According to the large-scale fruit counting method provided by the embodiment of the invention, convolution kernels of various scales are used for adapting to sizes of apples of different scales, the problem of different sizes of apples is solved, and the robustness of a fruit counting model is improved.
Based on the above-described embodiment, as an alternative embodiment, a density image label is generated from the marker image based on a label density image generation method,
wherein x is any sample image, the continuous density function f (x) is the density image label of any sample image x, h (x) is a labeled image of any sample image x, G is a gaussian kernel, and σ is a gaussian kernel standard deviation;
n is the number of the target fruits in any sample image x, xiThe target fruit at a location is denoted as delta (x-x)i)，xiThe coordinate position of the ith target fruit center in the sample image x is obtained;
beta is a preset value, and beta is a preset value,
the average value of the Euclidean distance sum of the ith target fruit in the sample image x and k adjacent target fruits; k is the number of neighboring target fruits, i is the ith target fruit in the sample image x, j ranges from 1 to k,
the euclidean distance between the ith target fruit and the jth neighboring target fruit in the sample image x.
As an alternative embodiment, the expression of the labeled image h (x) of any sample image x is:
wherein N is the number of the target fruit labels in any sample image x, and delta is xiThe coordinate position of the ith target fruit center in the sample image x is obtained;
the density image label of any sample image x can be expressed as a continuous density function f (x):
wherein G is a Gaussian kernel, and sigma is a Gaussian kernel standard deviation;
the expression of the gaussian kernel standard deviation σ is:
wherein, beta is a preset value,
the average value of the sum of Euclidean distances between the ith target fruit and k adjacent target fruits in the image sample x;
wherein k is the number of neighboring target fruits, i is the ith target fruit in the image sample x, j ranges from 1 to k,
for the ith target fruit and the jth neighboring target water in the image sample xEuclidean distance of the fruit.
The preset value β can be preset or adjusted according to actual needs, and β is 0.3 in this embodiment.
According to the large-scale fruit counting method provided by the embodiment of the invention, the density image label is generated according to the label image through the label density image generation method, and the change and parameter adjustment can be carried out according to different data sets during fruit counting, so that the label density image label is used as the input of a fruit counting model, and a basis is provided for rapid apple counting.
Based on the above embodiment, as an optional embodiment, it is characterized in that, acquiring the image of the orchard to be tested includes:
acquiring an initial image of an orchard to be tested;
and carrying out size normalization processing on the initial image to obtain an image.
The size normalization processing method for the initial image may use a nearest neighbor interpolation method or a bilinear interpolation method.
According to the large-scale fruit counting method provided by the embodiment of the invention, images with different formats and sizes are converted into orchard images with standard formats and sizes through normalization processing of the initial images, and the normalized orchard images are used as the input of a fruit counting model, so that the subsequent processing is convenient, and the large-scale fruit counting method provided by the invention can adapt to the images with different sizes and formats.
FIG. 3 is a schematic diagram of a model training process of the large-scale fruit counting method provided by the present invention; as an alternative embodiment, a set of initial Data sets Data is assumed, the sample size of the Data is N, and each sample is a remote sensing image of a single apple tree; as shown in fig. 3, the initial sample image in Data is read, and dotting marking is performed on each sample image in the Data set at the central point of the apple to obtain a marked image B Data corresponding to each sample image.
And reading an annotation file in the marking image B Data, and converting the marking image into a density image by using a label density image generation method of the MCNN to obtain a density image label MB Data.
The density image file is included in the density image tag MB Data. Randomly dividing Data and MB Data into a training set Train Data, a testing set Test Data and a verification set Val Data according to the proportion of 8:1:1, wherein the training set Train Data is used for training the MCNN model, the verification set Val Data is used for evaluating the performance of the trained MCNN model, the MCNN model is subjected to parameter adjustment and optimization according to an evaluation result, and the testing set Test Data is used for testing the accuracy of the MCNN model.
Training a fruit counting model constructed based on an MCNN network on a training set Train Data, performing model performance evaluation on the trained fruit counting model on a Test set Test Data, adjusting parameters such as learning rate and step length, and performing iterative loop training until the mean square error and the average absolute error of the fruit counting model on the Test set Test Data are optimal.
Fig. 4 is a schematic diagram of a test result of a test sample provided by the present invention, as shown in fig. 4, the abscissa is the actual number of apples, and the ordinate is the predicted number of apples; the data set selected by the invention is unmanned aerial vehicle remote sensing image data of an apple orchard in Yanchuan, and the data set comprises 43 images, which are abbreviated as Ycap. This example was examined with this data set. The quantitative indexes adopted by the invention are Mean Absolute Error (MAE) and Mean Squared Error (MSE). The result shows that the result of the Ycap data set test by the method provided by the invention is shown in Table 1 and FIG. 4, the obtained mean square error is 5.20, the average absolute error is 27.02, and the method has practical value. The density image label and the predicted density image of the test specimen are also highly coincident.
TABLE 1
According to the large-scale fruit counting method provided by the embodiment of the invention, the orchard image is input into the fruit counting model constructed by the MCNN, so that the rapid statistics of the number of target fruits is realized, and the accurate and rapid estimation of the fruit yield is ensured.
Fig. 5 is a schematic structural diagram of a large-scale fruit counting system provided by the present invention, as an alternative embodiment, as shown in fig. 5, mainly including but not limited to the following units:
an image obtaining unit 501, configured to obtain an orchard image of an orchard to be tested;
a counting unit 502, configured to input the orchard image to a fruit counting model to determine the number of target fruits in the orchard image;
the fruit counting model is obtained by training a multi-column convolution neural network according to a sample image with a density image label;
the density image label is generated from a label image;
the marked image is obtained by marking a target fruit in the sample image.
The large-scale fruit counting system provided by the invention further comprises: and the normalization unit is used for carrying out size normalization processing on the initial image of the orchard to be detected so as to obtain the orchard image.
In one embodiment, the size normalization processing is firstly performed on the initial image of the orchard to be tested through the normalization unit to obtain the orchard image. The image obtaining unit 501 obtains an orchard image of an orchard to be tested; the counting unit 502 inputs the orchard image to a fruit counting model to determine the number of target fruits in the orchard image; the fruit counting model is obtained by training a multi-column convolution neural network according to a sample image with a density image label; the density image label is generated from a label image; the marked image is obtained by marking a target fruit in the sample image.
The orchard to be tested can be partitioned according to actual requirements, image acquisition is carried out on each partition, so that the relative size of target fruits in the orchard image is convenient to identify, and counting accuracy is improved.
The orchard image of the orchard to be detected can be acquired through image acquisition equipment, and can also be a satellite image of the orchard to be detected.
As an optional embodiment, the image acquisition unit 501 acquires an image of the orchard to be tested by using a remote sensing device carried by the unmanned aerial vehicle to acquire an image of the orchard to be tested.
The image acquisition device may be any one of a depth camera, a hyperspectral camera, a thermal infrared camera or a remote sensing camera.
Image acquisition can select in the better time quantum of illumination, or carry out the light filling to image acquisition equipment to improve the quality of the image of gathering, and then improve the accuracy of count.
Further, the counting unit 502 inputs the orchard image into the fruit counting model, outputs a density image corresponding to the orchard image, and performs integration processing on the density image to obtain the number of target fruits in the orchard image.
The fruit counting model is obtained by training MCNN according to a sample image with a density image label.
All target fruits in the sample image can be subjected to dotting marking processing through a manual marking or machine vision image processing method, so that a marked image is obtained. For better counting effect, dotting can be performed at the central position of the target fruit.
The target fruit may be apple, pear or orange, and the counting of the apples is taken as an example in the following examples of the present invention, which should not be construed as a limitation to the scope of the present invention.
And generating a density image label with apple density information according to the marking image by a label density image generation method.
It should be noted that, in specific implementation, the large-scale fruit counting system provided in the embodiment of the present invention can be implemented based on the large-scale fruit counting method in any of the above embodiments, and details of this embodiment are not described herein.
According to the large-scale fruit counting system provided by the invention, the orchard image is input into the fruit counting model constructed by the MCNN, so that the rapid statistics of the number of target fruits is realized, and the accurate and rapid estimation of the fruit yield is ensured.
Fig. 6 is a schematic structural diagram of an electronic device provided in the present invention, and as shown in fig. 6, the electronic device may include: a processor (processor)610, a communication Interface (Communications Interface)620, a memory (memory)630 and a communication bus 640, wherein the processor 610, the communication Interface 620 and the memory 630 communicate with each other via the communication bus 640. The processor 610 may invoke logic instructions in the memory 630 to perform a large-scale fruit counting method comprising: acquiring an orchard image of an orchard to be detected; inputting the orchard image into a fruit counting model to determine the number of target fruits in the orchard image; the fruit counting model is obtained by training a multi-column convolution neural network according to a sample image with a density image label; the density image label is generated from the label image; the marked image is obtained by marking the target fruit in the sample image.
In addition, the logic instructions in the memory 630 may be implemented in software functional units and stored in a computer readable storage medium when the logic instructions are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
In another aspect, the present invention also provides a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions which, when executed by a computer, enable the computer to perform the large-scale fruit counting method provided by the above methods, the method comprising: acquiring an orchard image of an orchard to be detected; inputting the orchard image into a fruit counting model to determine the number of target fruits in the orchard image; the fruit counting model is obtained by training a multi-column convolution neural network according to a sample image with a density image label; the density image label is generated from the label image; the marked image is obtained by marking the target fruit in the sample image.
In yet another aspect, the present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program, which when executed by a processor is implemented to perform the large-scale fruit counting method provided in the above embodiments, the method comprising: acquiring an orchard image of an orchard to be detected; inputting the orchard image into a fruit counting model to determine the number of target fruits in the orchard image; the fruit counting model is obtained by training a multi-column convolution neural network according to a sample image with a density image label; the density image label is generated from the label image; the marked image is obtained by marking the target fruit in the sample image.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and the parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods of the various embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A large scale fruit counting method, comprising:
acquiring an orchard image of an orchard to be detected;
inputting the orchard image into a fruit counting model to determine the number of target fruits in the orchard image;
the fruit counting model is obtained by training a multi-column convolution neural network according to a sample image with a density image label;
the density image label is generated from a label image;
the marked image is obtained by marking a target fruit in the sample image.
2. The large-scale fruit counting method according to claim 1, wherein the plurality of columns of convolutional neural networks are three columns of convolutional neural networks, and comprise a first column of convolutional units, a second column of convolutional units and a third column of convolutional units;
the convolution kernel of the convolution unit in the first row is larger than that of the convolution unit in the second row, and the convolution kernel of the convolution unit in the second row is larger than that of the convolution unit in the third row.
3. The large-scale fruit counting method according to claim 1, further comprising, before inputting the orchard image to a fruit counting model:
acquiring a plurality of sample images; the sample image is an image including the target fruit;
marking all target fruits in any sample image to obtain a marked image corresponding to any sample image;
generating a density image label according to any one of the label images based on a label density image generation method;
combining each sample image and the density image label corresponding to each sample image to be used as a training sample, constructing a training sample set, and training the multi-column convolutional neural network by using the training sample set.
4. The large-scale fruit counting method according to claim 2, wherein the first row of convolution units comprises 4 convolution layers connected in sequence, and the convolution kernel scales of the convolution layers in the first row of convolution units are 9 × 9, 7 × 7, 7 × 7 and 7 × 7 in sequence;
the second row of convolution units comprises 4 convolution layers which are connected in sequence, and the convolution kernel scales of the convolution layers in the second row of convolution units are 7 × 7, 5 × 5, 5 × 5 and 5 × 5 in sequence;
the convolution units in the third row comprise 4 convolution layers which are connected in sequence, and the convolution kernel scales of the convolution layers in the convolution units in the third row are 5 × 5, 3 × 3, 3 × 3 and 3 × 3 in sequence.
5. The large-scale fruit counting method according to claim 3, wherein a density image label is generated from the marker image based on a label density image generation method,
wherein x is any sample image, the continuous density function f (x) is the density image label of any sample image x, h (x) is a labeled image of any sample image x, G is a gaussian kernel, and σ is a gaussian kernel standard deviation;
n is the number of the target fruits in any sample image x, xiThe target fruit at a location is denoted as delta (x-x)i)，xiThe coordinate position of the ith target fruit center in the sample image x is obtained;
beta is a preset value, and beta is a preset value,
the average value of the Euclidean distance sum of the ith target fruit in the sample image x and k adjacent target fruits; k is the number of neighboring target fruits, i is the ith target fruit in the sample image x, j ranges from 1 to k,
the euclidean distance between the ith target fruit and the jth neighboring target fruit in the sample image x.
6. The large-scale fruit counting method according to claim 1, wherein obtaining an orchard image of an orchard to be tested comprises:
acquiring an initial image of an orchard to be tested;
and carrying out size normalization processing on the initial image to obtain the orchard image.
7. A large scale fruit counting system, comprising:
the image acquisition unit is used for acquiring an orchard image of an orchard to be detected;
the counting unit is used for inputting the orchard image into a fruit counting model so as to determine the number of target fruits in the orchard image;
the fruit counting model is obtained by training a multi-column convolution neural network according to a sample image with a density image label;
the density image label is generated from a label image;
the marked image is obtained by marking a target fruit in the sample image.
8. The large scale fruit counting system according to claim 7, further comprising:
and the normalization unit is used for carrying out size normalization processing on the initial image of the orchard to be detected so as to obtain the orchard image.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor when executing the computer program implements the large scale fruit counting method steps according to any one of claims 1 to 6.
10. A non-transitory computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements the large scale fruit counting method steps according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110859589.0A CN113688690A (en) | 2021-07-28 | 2021-07-28 | Large-scale fruit counting method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110859589.0A CN113688690A (en) | 2021-07-28 | 2021-07-28 | Large-scale fruit counting method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113688690A true CN113688690A (en) | 2021-11-23 |
Family
ID=78578128
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110859589.0A Pending CN113688690A (en) | 2021-07-28 | 2021-07-28 | Large-scale fruit counting method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113688690A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114511636A (en) * | 2022-04-20 | 2022-05-17 | 科大天工智能装备技术（天津）有限公司 | Fruit counting method and system based on double-filtering attention module |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112861666A (en) * | 2021-01-26 | 2021-05-28 | 华南农业大学 | Chicken flock counting method based on deep learning and application |
-
2021
- 2021-07-28 CN CN202110859589.0A patent/CN113688690A/en active Pending
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112861666A (en) * | 2021-01-26 | 2021-05-28 | 华南农业大学 | Chicken flock counting method based on deep learning and application |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114511636A (en) * | 2022-04-20 | 2022-05-17 | 科大天工智能装备技术（天津）有限公司 | Fruit counting method and system based on double-filtering attention module |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Apolo-Apolo et al. | Deep learning techniques for estimation of the yield and size of citrus fruits using a UAV | |
| Li et al. | Automatic organ-level point cloud segmentation of maize shoots by integrating high-throughput data acquisition and deep learning | |
| Zheng et al. | Remote sensing and machine learning in crop phenotyping and management, with an emphasis on applications in strawberry farming | |
| CN109141248B (en) | Pig weight measuring and calculating method and system based on image | |
| Halme et al. | Utility of hyperspectral compared to multispectral remote sensing data in estimating forest biomass and structure variables in Finnish boreal forest | |
| CN111898503B (en) | Crop identification method and system based on cloud coverage remote sensing image and deep learning | |
| Jin et al. | High-throughput measurements of stem characteristics to estimate ear density and above-ground biomass | |
| CN110378909A (en) | Single wooden dividing method towards laser point cloud based on Faster R-CNN | |
| CN105224960A (en) | Based on the corn seed classification hyperspectral imagery model of cognition update method of clustering algorithm | |
| Chen et al. | Predicting individual apple tree yield using UAV multi-source remote sensing data and ensemble learning | |
| CN113008742B (en) | Method and system for detecting deposition amount of fog drops | |
| CN111241939A (en) | Rice yield estimation method based on unmanned aerial vehicle digital image | |
| CN113269825B (en) | Forest breast diameter value extraction method based on foundation laser radar technology | |
| CN114444378A (en) | Short-term power prediction method for regional wind power cluster | |
| CN113223040B (en) | Banana estimated yield method and device based on remote sensing, electronic equipment and storage medium | |
| Sun et al. | Wheat head counting in the wild by an augmented feature pyramid networks-based convolutional neural network | |
| CN106383088A (en) | A seed purity rapid nondestructive testing method based on a multispectral imaging technique | |
| CN111291686B (en) | Extraction method and system for crop root-fruit phenotype parameters and root-fruit phenotype discrimination method and system | |
| CN115880487A (en) | Forest laser point cloud branch and leaf separation method based on deep learning method | |
| Cheng et al. | Maize seedling and weed detection based on MobileNetv3-YOLOv4 | |
| CN113688690A (en) | Large-scale fruit counting method and system | |
| CN111783538A (en) | Remote sensing estimation method and device for wheat biomass, electronic equipment and storage medium | |
| Suarez et al. | Learning image vegetation index through a conditional generative adversarial network | |
| Sumnall et al. | Effect of varied unmanned aerial vehicle laser scanning pulse density on accurately quantifying forest structure | |
| CN116863341B (en) | Crop classification and identification method and system based on time sequence satellite remote sensing image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20220124 Address after: 1107, block a, Nongke building, No. 11, Shuguang garden middle road, Haidian District, Beijing, 100097 Applicant after: Information technology research center of Beijing Academy of agricultural and Forestry Sciences Applicant after: Nongxin Technology (Beijing) Co., Ltd Address before: 100097 room 301-a315, 03 / F, 11 Shuguang Huayuan Middle Road, Haidian District, Beijing Applicant before: NONGXIN TECHNOLOGY (BEIJING) CO.,LTD. |