CN113409298A - Banana plant growth evaluation system based on Kinect V2 sensor - Google Patents
Banana plant growth evaluation system based on Kinect V2 sensor Download PDFInfo
- Publication number
- CN113409298A CN113409298A CN202110773975.8A CN202110773975A CN113409298A CN 113409298 A CN113409298 A CN 113409298A CN 202110773975 A CN202110773975 A CN 202110773975A CN 113409298 A CN113409298 A CN 113409298A
- Authority
- CN
- China
- Prior art keywords
- data
- image
- sensor
- growth
- kinect
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 240000000905 Nymphoides indica Species 0.000 title claims abstract description 57
- 235000017590 Nymphoides indica Nutrition 0.000 title claims abstract description 57
- 238000011156 evaluation Methods 0.000 title claims abstract description 19
- 230000008635 plant growth Effects 0.000 title claims abstract description 18
- 230000012010 growth Effects 0.000 claims abstract description 81
- 238000007405 data analysis Methods 0.000 claims abstract description 59
- 238000001914 filtration Methods 0.000 claims abstract description 40
- 238000007781 pre-processing Methods 0.000 claims abstract description 9
- 238000004458 analytical method Methods 0.000 claims abstract description 8
- 241000234295 Musa Species 0.000 claims description 34
- 235000018290 Musa x paradisiaca Nutrition 0.000 claims description 33
- 238000004891 communication Methods 0.000 claims description 30
- 238000000034 method Methods 0.000 claims description 23
- 230000033001 locomotion Effects 0.000 claims description 19
- 238000013500 data storage Methods 0.000 claims description 18
- 230000009467 reduction Effects 0.000 claims description 14
- 230000003993 interaction Effects 0.000 claims description 12
- 238000012545 processing Methods 0.000 claims description 12
- 230000011218 segmentation Effects 0.000 claims description 11
- 238000006243 chemical reaction Methods 0.000 claims description 9
- 230000002146 bilateral effect Effects 0.000 claims description 6
- 230000003750 conditioning effect Effects 0.000 claims description 6
- 238000003702 image correction Methods 0.000 claims description 6
- 238000013136 deep learning model Methods 0.000 claims description 3
- 238000004146 energy storage Methods 0.000 claims description 3
- 230000001502 supplementing effect Effects 0.000 claims description 3
- 238000012549 training Methods 0.000 claims description 3
- 238000000605 extraction Methods 0.000 abstract description 14
- 238000005070 sampling Methods 0.000 abstract description 7
- 238000001514 detection method Methods 0.000 abstract 1
- 238000005259 measurement Methods 0.000 description 33
- 241000196324 Embryophyta Species 0.000 description 12
- 239000002420 orchard Substances 0.000 description 9
- 238000010586 diagram Methods 0.000 description 7
- 230000000694 effects Effects 0.000 description 4
- 239000000284 extract Substances 0.000 description 3
- 230000000670 limiting effect Effects 0.000 description 3
- 238000004519 manufacturing process Methods 0.000 description 3
- 230000002829 reductive effect Effects 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 239000002689 soil Substances 0.000 description 3
- 238000012937 correction Methods 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 230000002411 adverse Effects 0.000 description 1
- 235000021015 bananas Nutrition 0.000 description 1
- 230000000052 comparative effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 239000003337 fertilizer Substances 0.000 description 1
- 230000005078 fruit development Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000002401 inhibitory effect Effects 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 238000000691 measurement method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 235000015097 nutrients Nutrition 0.000 description 1
- 235000016709 nutrition Nutrition 0.000 description 1
- 210000000056 organ Anatomy 0.000 description 1
- 238000002203 pretreatment Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000002035 prolonged effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30181—Earth observation
- G06T2207/30188—Vegetation; Agriculture
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Quality & Reliability (AREA)
- Geometry (AREA)
- Investigating Or Analysing Materials By Optical Means (AREA)
Abstract
The invention discloses a banana plant growth evaluation system based on a Kinect V2 sensor, which comprises a data acquisition system, a data analysis system, a big data analysis system and a mobile terminal system, wherein the data acquisition system is used for acquiring a banana plant growth state; the invention generates color point cloud data by acquiring the depth image and the color image of the banana plant and converting coordinates. The point cloud preprocessing is realized through the ROI extraction of the depth image, the voxel down sampling and the classified noise filtering, and the automatic extraction of characteristic parameters is completed through a corresponding parameter extraction algorithm, so that the detection efficiency and the automation degree are improved; the provided system is low in cost and meets market demands, characteristic parameters such as pseudostem width and stem height of banana plants are effectively extracted through a built-in program in the system, growth analysis and prediction are carried out on the banana plants, a solution is provided according to the prediction, and the practicability and the popularization of the system are effectively improved.
Description
Technical Field
The invention relates to the field of intelligent identification, in particular to a banana plant growth evaluation system based on a Kinect V2 sensor.
Background
The pseudostem of banana is a key organ for providing support and nutrient delivery, and the stem width and stem height of the pseudostem directly influence the biological yield, economic yield, nutritional quality, safety and the like of banana, and are important parameters for predicting the growth vigor of banana. The 2 key parameters are accurately detected, so that the growth and development dynamics and the production characteristics of bananas can be identified in time, important data are provided for the management of water and fertilizers, the high-yield rule is summarized, and a classification standard is provided for orchard management. In production and scientific research, the growth characteristics of plants are of great significance in growth monitoring, morphology evaluation, propagation selection, seedling evaluation and the like.
The non-contact nondestructive measurement based on machine vision mainly collects image information of a measurement object through a sensor device with an imaging function, and the information can effectively express corresponding characteristics of the measurement object, such as color, shape, structure and texture, and has huge research potential and application prospect. A set of system is urgently needed to realize the real-time observation and growth prediction of the banana growth vigor, and the system is low in cost and meets the market demand.
Disclosure of Invention
In order to solve the problems of the prior art, the invention provides a banana plant growth evaluation system based on a Kinect V2 sensor, which is characterized by comprising a data acquisition system, a data analysis system, a big data analysis system and a mobile terminal system;
the data acquisition system is used for acquiring image data of banana plants and wirelessly transmitting the image data to the data analysis system;
the data analysis system is used for carrying out analysis processing according to the image data to obtain the growth condition of the banana plants;
the big data analysis system is used for carrying out big data analysis according to the growth situation, carrying out growth prediction on the banana plants to obtain growth prediction information and providing a solution according to the growth prediction information;
the mobile terminal system is used for displaying the growth situation, the growth prediction information and the solution.
Preferably, the data acquisition system comprises:
the Kinect V2 sensor is used for acquiring image data of the banana pseudostem, wherein the image data comprises an original depth image and a color image;
the sensor carrying device is connected with the Kinect V2 sensor and used for carrying the Kinect V2 sensor and carrying out moving image acquisition on the banana pseudostem;
the first data storage module is arranged on the sensor carrying device, is connected with the Kinect V2 sensor and is used for storing the original depth image and the color image into different storage spaces respectively;
the Beidou positioning module is connected with the sensor carrying device and used for providing position information of banana plants;
the control module is respectively connected with the Kinect V2 sensor, the sensor carrying device, the Beidou positioning module and the data storage module and used for providing a control instruction for the data acquisition system;
and the first wireless communication module is connected with the control module and is used for data interaction between the data acquisition system and the data analysis system.
Preferably, the sensor-equipped device includes,
the power supply unit is used for supplying power to the data acquisition system;
the motion unit is used for providing movement capacity for the data acquisition system, wherein the movement capacity comprises horizontal movement, vertical movement and steering movement;
the carrying unit is respectively connected with the Kinect V2 sensor, the control module, the first wireless communication module, the Beidou positioning module, the first data storage module, the moving unit and the power supply unit and used for fixing the data acquisition system.
Preferably, the power supply unit comprises a solar unit, a storage battery unit and a power supply conditioning unit;
the solar unit is used for supplementing electric energy for the power supply unit;
the storage battery unit is an energy storage unit;
the power supply conditioning unit is used as an electric energy conversion unit and is used for providing stable and reliable standard voltage output for the data acquisition system.
Preferably, the data analysis system comprises,
the second wireless communication module is used for the data analysis system to respectively perform data interaction with the data acquisition system and the big data analysis system;
the second data storage module is used for storing image data, growth conditions and position information;
the data preprocessing module is used for converting the image data into color original point cloud data, and performing background segmentation, space topological relation establishment and noise reduction filtering processing on the color original point cloud data to obtain initial point cloud data;
the first data analysis module is used for extracting characteristic parameters according to the initial point cloud data to obtain a characteristic data set, training the characteristic data set through a deep learning model and verifying the characteristic data set through manually collected data to construct a banana growth analysis model to obtain a growth situation;
and the first image display unit is used for displaying the growth situation, the position information and the image data.
Preferably, the data pre-processing module comprises,
an image conversion unit for converting the image data into color original point cloud data;
the background segmentation unit is used for segmenting a complex background of the color original point cloud data to obtain a first image;
the image correction unit is used for carrying out image correction on the first image through the space topological relation gap and the normal vector to obtain a second image;
and the image filtering and denoising unit is used for carrying out filtering and denoising processing on the second image to obtain initial point cloud data.
Preferably, the method for filtering and denoising the second image by the image filtering unit comprises one or more of point cloud reduction, statistical filtering, radius filtering and bilateral filtering.
Preferably, the big data analyzing system includes,
the third wireless communication module is used for the big data analysis system to respectively perform data interaction with the data analysis system and the mobile terminal system;
the third data storage module is used for storing position information, growth conditions, growth prediction information and solutions;
and the second data analysis module is used for acquiring growth vigor prediction information by comparing the difference between the growth vigor situation and the historical growth vigor situation according to the historical growth vigor situation of the banana plants and providing a solution based on the position information according to the growth vigor prediction information, wherein the second data analysis module acquires the environment information of the area where the banana plants are located according to the position information, performs environment prediction and acquires the solution by combining the growth vigor prediction information.
Preferably, the mobile terminal system includes,
the fourth wireless communication module is used for the mobile end system to perform data interaction with the data analysis system and the big data analysis system respectively;
a second image display unit for displaying position information, growth situation, growth prediction information, solution, image data;
and the fourth data storage module is used for storing the position information, the growth situation, the growth prediction information and the solution.
Preferably, the first wireless communication module, the second wireless communication module, the third wireless communication module and the fourth wireless communication module are 5G communication modules;
the first image display unit is also used for displaying the growth prediction information.
The invention discloses the following technical effects:
the system provided by the invention has low manufacturing cost, meets the market demand, effectively extracts the characteristic parameters of the banana plants such as the width of the pseudostem, the height of the pseudostem and the like through a built-in program in the system, analyzes and predicts the growth vigor of the banana plants, provides a solution according to the prediction, and effectively increases the practicability and the popularization of the system.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
FIG. 1 is a system architecture diagram according to the present invention;
FIG. 2 is a block diagram of a data acquisition system according to the present invention;
FIG. 3 is a diagram of a data analysis system architecture according to the present invention;
FIG. 4 is a big data analysis system architecture diagram according to the present invention;
fig. 5 is a diagram illustrating an architecture of a mobile end system according to the present invention;
FIG. 6 is a diagram of a sensor-carrying device according to the present invention;
FIG. 7 is a block diagram of a data preprocessing module according to the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As shown in fig. 1-7, the invention provides a banana plant growth evaluation system based on a Kinect V2 sensor, which is characterized in that the banana plant growth evaluation system comprises a data acquisition system, a data analysis system, a big data analysis system and a mobile terminal system;
the data acquisition system is used for acquiring image data of banana plants and wirelessly transmitting the image data to the data analysis system;
the data analysis system is used for carrying out analysis processing according to the image data to obtain the growth condition of the banana plants;
the big data analysis system is used for carrying out big data analysis according to the growth situation, carrying out growth prediction on the banana plants to obtain growth prediction information and providing a solution according to the growth prediction information;
the mobile terminal system is used for displaying the growth situation, the growth prediction information and the solution.
Further, the data acquisition system comprises:
the Kinect V2 sensor is used for acquiring image data of the banana pseudostem, wherein the image data comprises an original depth image and a color image;
the sensor carrying device is connected with the Kinect V2 sensor and used for carrying the Kinect V2 sensor and carrying out moving image acquisition on the banana pseudostem;
the first data storage module is arranged on the sensor carrying device, is connected with the Kinect V2 sensor and is used for storing the original depth image and the color image into different storage spaces respectively;
the Beidou positioning module is connected with the sensor carrying device and used for providing position information of banana plants;
the control module is respectively connected with the Kinect V2 sensor, the sensor carrying device, the Beidou positioning module and the data storage module and used for providing a control instruction for the data acquisition system;
and the first wireless communication module is connected with the control module and is used for data interaction between the data acquisition system and the data analysis system.
Further, a sensor-equipped device includes,
the power supply unit is used for supplying power to the data acquisition system;
the motion unit is used for providing movement capacity for the data acquisition system, wherein the movement capacity comprises horizontal movement, vertical movement and steering movement;
the carrying unit is respectively connected with the Kinect V2 sensor, the control module, the first wireless communication module, the Beidou positioning module, the first data storage module, the moving unit and the power supply unit and used for fixing the data acquisition system.
Further, the power supply unit comprises a solar unit, a storage battery unit and a power supply conditioning unit;
the solar unit is used for supplementing electric energy for the power supply unit;
the storage battery unit is an energy storage unit;
the power supply conditioning unit is used as an electric energy conversion unit and is used for providing stable and reliable standard voltage output for the data acquisition system.
Further, the data analysis system includes,
the second wireless communication module is used for the data analysis system to respectively perform data interaction with the data acquisition system and the big data analysis system;
the second data storage module is used for storing image data, growth conditions and position information;
the data preprocessing module is used for converting the image data into color original point cloud data, and performing background segmentation, space topological relation establishment and noise reduction filtering processing on the color original point cloud data to obtain initial point cloud data;
the first data analysis module is used for extracting characteristic parameters according to the initial point cloud data to obtain a characteristic data set, training the characteristic data set through a deep learning model and verifying the characteristic data set through manually collected data to construct a banana growth analysis model to obtain a growth situation;
and the first image display unit is used for displaying the growth situation, the position information and the image data.
Further, the data pre-processing module comprises,
an image conversion unit for converting the image data into color original point cloud data;
the background segmentation unit is used for segmenting a complex background of the color original point cloud data to obtain a first image;
the image correction unit is used for carrying out image correction on the first image through the space topological relation gap and the normal vector to obtain a second image;
and the image filtering and denoising unit is used for carrying out filtering and denoising processing on the second image to obtain initial point cloud data.
Further, the method for filtering and denoising the second image by the image filtering unit comprises one or more of point cloud reduction, statistical filtering, radius filtering and bilateral filtering.
Further, the big data analyzing system includes,
the third wireless communication module is used for the big data analysis system to respectively perform data interaction with the data analysis system and the mobile terminal system;
the third data storage module is used for storing position information, growth conditions, growth prediction information and solutions;
and the second data analysis module is used for acquiring growth vigor prediction information by comparing the difference between the growth vigor situation and the historical growth vigor situation according to the historical growth vigor situation of the banana plants and providing a solution based on the position information according to the growth vigor prediction information, wherein the second data analysis module acquires the environment information of the area where the banana plants are located according to the position information, performs environment prediction and acquires the solution by combining the growth vigor prediction information.
Further, the mobile terminal system includes,
the fourth wireless communication module is used for the mobile end system to perform data interaction with the data analysis system and the big data analysis system respectively;
a second image display unit for displaying position information, growth situation, growth prediction information, solution, image data;
and the fourth data storage module is used for storing the position information, the growth situation, the growth prediction information and the solution.
Further, the first wireless communication module, the second wireless communication module, the third wireless communication module and the fourth wireless communication module are 5G communication modules;
the first image display unit is also used for displaying the growth prediction information.
Example 1: the original image acquisition place of the banana is located in a banana plantation in New City of subtropical agriculture department in Guangxi province, the province occupies 1800 mu of land, and the banana is in the fruit development period during the experiment. The equipment comprises a Kinect V2 sensor, a Del Precision7530 mobile workstation (Intel-i9CPU, 32GB high-speed memory, NVIDIAQuadrop2000 video card). The requirement for data acquisition is to acquire color and depth images with as high precision as possible by means of the Kinect V2 sensor. The accuracy of data acquisition of Kinect V2 is limited by distance. The farther the Kinect V2 is from the plant to be measured, the lower the measurement accuracy. Summarizing the influences of a plurality of factors such as environmental factors, tested plants, experimental requirements, external limiting conditions, and Kinect V2 limiting factors, a proper data acquisition visual angle and acquisition distance need to be determined so as to balance the factors and obtain high-precision data flow information.
The main parameters detected by the invention are the stem width and stem height of the banana pseudostem. The pseudostem of the banana plant can be approximately regarded as a cylinder with thick lower part and thin upper part after the banana plant in the orchard is observed. The angle of view of the sensor collecting the original image and the distance of the plant to be measured influence the accuracy of the measurement. If the data acquisition is carried out at an oblique forward viewing angle. As the orchard is too extensive in planting, the non-measured banana plants and the auxiliary supports can shield the measured plants, so that the interference measurement is realized. And the interference on the measurement of the sensor perpendicular to the tested plant is reduced. Therefore, the oblique forward-looking angle position is not favorable for banana plant data acquisition. The comprehensive analysis is carried out by combining the environmental factors of the orchard, the Kinect V2 sensor is erected perpendicular to the banana plants, and the data of the banana plants are collected from the view angle of the main view.
Besides selecting the position of Kinect V2 shooting, data acquisition is limited by factors such as orchard environment, visual angle of acquisition equipment, precision and size of an acquisition object. Therefore, how to select the measuring distance of the Kinect V2 is also a problem to be solved. The effective measurement distance of the Kinect V2 is 0.5-4.5 m, and the data obtained at 0.5m is the highest in accuracy in principle. However, limited by the plant planting spacing and row spacing in the orchard and the Kinect V2 field angle limitations. The working distance cannot be too close.
Aiming at the width of the banana pseudostem, the width of the banana pseudostem is between 10cm and 40cm, if the distance between the banana pseudostem and a tested plant is too far during collection, the relative error is too large, and the absolute error needs to be controlled to be not more than 2 cm. Therefore, the distance between the banana pseudostem width measuring device and the measured plant is not easy to exceed 1m when the banana pseudostem width measuring device is used for measuring the banana pseudostem width. The pseudostem width is detected by collecting data at measuring distances of 0.5m, 0.7m, 0.9m and 1.0m, and the measured results are shown in table 1.
TABLE 1
Aiming at the height of the pseudostem of the banana, the height of the pseudostem at the middle and later growth stages of the banana is generally between 2m and 4 m. Limited by the field range of Kinect V2, if the distance is too close to the tested plant, the complete banana cauloid can not be collected; and considering that the actual height of the pseudostem is higher, the increase of the measurement error caused by increasing the measurement distance is acceptable, and the measurement requirement can be met by keeping the absolute error not more than 10cm in principle. Therefore, 3 different distances with measuring distances of 3m, 4m and 5m are selected for comparative experiments.
The data acquired using Kinect V2 are the original depth image and color image. It is therefore necessary to preprocess the raw data in order to efficiently extract the corresponding feature parameters. The chapter mainly researches how to obtain the point cloud data of the banana plants of a single plant, extracts the banana pseudostem point cloud through the background segmentation, the point cloud reduction and the point cloud noise reduction, and prepares for extracting the subsequent key characteristic parameters.
The invention provides a point cloud pretreatment process and a point cloud pretreatment method. The method comprises 6 parts in total, namely, original data conversion, complex background segmentation, topological relation establishment, point cloud normal vector estimation, point cloud reduction and point cloud noise classification filtering. The depth image and the color image are converted into a corresponding color point cloud by a correspondence between a pixel coordinate system and a space coordinate system in an original data conversion section. In the complex background segmentation part, the complex background can be removed more quickly by determining the depth image ROI extraction method through comparing point cloud condition filtering and the depth image ROI extraction method. In the point cloud reduction part, two methods of point cloud boundary extraction and voxel down-sampling are compared. Although the point cloud boundary extraction can effectively reduce the number of the point clouds and is not easy to lose the characteristic information of the point clouds, the processing speed is too slow, and the real-time performance of measurement is influenced; the voxel down-sampling method meets the measurement requirements in reducing the density of cloud points of points and the operation speed. In the point cloud noise filtering part, classified noise reduction is performed according to the characteristics of noise in the point cloud information collected by the Kinect V2, and compared with a single noise reduction method, the point cloud noise can be removed in a more targeted manner, and the noise reduction effect is more obvious.
The invention provides a corresponding characteristic parameter extraction method for point cloud data after being preprocessed. Aiming at the stem width of the banana pseudostem, a method of evaluating from top to bottom, filtering by a threshold condition, then taking an average value, and finally finishing measurement by angle correction is provided. Direct measurement and RANSAC soil fitting measurement are provided for the height of the cauloid of the banana. And comparing the stem width and stem height of the banana pseudostem extracted at different distances with the manual measurement value, and calculating the corresponding average absolute error, average relative error and correlation so as to determine the optimal measurement distance. And the cause of the measurement error is analyzed, and a corresponding improvement method is provided.
The invention provides a corresponding method for extracting the width and the height of the stem of the banana pseudostem aiming at the characteristics of the banana pseudostem and the characteristics of an orchard environment, and characteristic parameters are extracted at different distances and compared with manual measurement values. Experimental results show that the corresponding characteristic parameter extraction method has good measurement precision and correlation, and the measurement can be carried out in real time in about 100-300 ms with short measurement time.
The invention provides a method for extracting banana plant characteristic parameters based on Kinect V2, aiming at the requirement of an orchard on quick extraction of banana plant characteristic parameters. The method comprises the steps of obtaining a main view of banana plants by using a Kinect, and extracting point cloud information of the banana plants of a single plant by means of complex background segmentation, point cloud down-sampling, classified noise filtering and the like. And finally, extracting two key characteristic parameters of the pseudostem width and the stem height by a corresponding characteristic extraction algorithm. And the following conclusions are reached:
a method for collecting image data of banana plants in an orchard environment is designed and realized by utilizing Kinect V2, a depth image and a color image are converted into high-precision color point cloud data through a space coordinate conversion relation, and the acquired 3D point cloud information meets the requirement of subsequent processing.
And aiming at the acquired original point cloud information, selecting a depth image ROI algorithm as a method for removing the complex background by comparing the advantages and disadvantages of a point cloud condition filtering algorithm and a depth image ROI extraction algorithm. And (3) selecting an ROI from the original point cloud obtained by the Kinect V2 in the depth image, converting the ROI into the point cloud, and setting a threshold value on a point cloud z axis for filtering. Invalid regions, invalid backgrounds and small region noises in the point cloud are filtered, and the processing time is short and is about 20 ms.
Because the characteristic parameters extracted by the method are pseudostem width and stem height, excessively dense point clouds can adversely affect the measurement speed. Therefore, it is desirable to reduce the number of point clouds to increase the measurement speed. According to the invention, the voxel down-sampling method is obtained by comparing the point cloud boundary extraction algorithm and the voxel down-sampling algorithm, so that good effects are obtained in the aspects of reducing the number of point cloud points and improving the measurement speed. Meanwhile, the proper down-sampling proportion is selected, so that the measurement precision is improved, and the measurement error is reduced. The number of point cloud points sampled by voxels is reduced by about 70%, and the measurement time is prolonged by about 150 ms.
And aiming at the difference of the point cloud noise in size, a method for classifying and reducing the noise according to the noise size is provided. And removing large-size noise by adopting a mode of combining statistical filtering and radius filtering. The method has a certain inhibiting effect on small-size noise while removing large-size noise. The traditional bilateral filtering is improved aiming at small-size noise. The characteristic weight retention factor of the traditional bilateral filtering is replaced by the included angle of the normal vector of the point cloud, so that the bilateral filtering can be more sensitive to small-size noise, the phenomenon of point cloud characteristic information loss caused by over-noise reduction can not occur, and the small-size noise can be effectively smoothed. The experimental result shows that compared with a single filtering method, the point cloud filtering performed by adopting the noise classification method has a better filtering effect and can more effectively inhibit noise.
Extracting characteristic parameters based on banana plants. Two characteristic parameters of the pseudostem width and stem height of the banana plants are respectively extracted. According to the characteristics of the banana pseudostem, a point cloud segmentation measurement method is provided, invalid measurement results are removed through setting a threshold condition, and angle correction is carried out on the valid measurement values to obtain final measurement results. Aiming at the stem height measurement of the pseudostem, a method for fitting the soil surface by using an RANSAC algorithm to calculate the average height of the soil surface is provided. Experimental results show that the extraction of characteristic parameters of banana plants by using Kinect V2 has certain feasibility. The measuring method provided by the invention has the characteristics of real-time property, high efficiency, accuracy and universality, and provides corresponding reference for measuring the characteristic parameters of similar banana plants.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus once an item is defined in one figure, it need not be further defined and explained in subsequent figures, and moreover, the terms "first", "second", "third", etc. are used merely to distinguish one description from another and are not to be construed as indicating or implying relative importance.
Finally, it should be noted that: the above-mentioned embodiments are only specific embodiments of the present invention, which are used for illustrating the technical solutions of the present invention and not for limiting the same, and the protection scope of the present invention is not limited thereto, although the present invention is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive the technical solutions described in the foregoing embodiments or equivalent substitutes for some technical features within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the present invention in its spirit and scope. Are intended to be covered by the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. The banana plant growth evaluation system based on the Kinect V2 sensor is characterized by comprising a data acquisition system, a data analysis system, a big data analysis system and a mobile terminal system;
the data acquisition system is used for acquiring image data of banana plants and wirelessly transmitting the image data to the data analysis system;
the data analysis system is used for carrying out analysis processing according to the image data to obtain the growth condition of the banana plants;
the big data analysis system is used for carrying out big data analysis according to the growth situation, carrying out growth prediction on the banana plants to obtain growth prediction information and providing a solution according to the growth prediction information;
the mobile terminal system is used for displaying the growth situation, the growth prediction information and the solution.
2. The Kinect V2 sensor-based banana plant growth evaluation system according to claim 1,
the data acquisition system includes:
a Kinect V2 sensor for acquiring the image data of a banana pseudostem, wherein the image data comprises an original depth image and a color image;
the sensor carrying device is connected with the Kinect V2 sensor and used for carrying the Kinect V2 sensor and carrying out moving image acquisition on the banana pseudostem;
a first data storage module, provided on the sensor mounting device, connected to the Kinect V2 sensor, for storing the original depth image and the color image in different storage spaces, respectively;
the Beidou positioning module is connected with the sensor carrying device and used for providing position information of the banana plants;
the control module is respectively connected with the Kinect V2 sensor, the sensor carrying device, the Beidou positioning module and the data storage module and is used for providing a control instruction for the data acquisition system;
and the first wireless communication module is connected with the control module and is used for data interaction between the data acquisition system and the data analysis system.
3. The Kinect V2 sensor-based banana plant growth evaluation system according to claim 2,
the sensor-equipped device includes a sensor-equipped unit,
the power supply unit is used for supplying power to the data acquisition system;
the motion unit is used for providing movement capacity for the data acquisition system, wherein the movement capacity comprises horizontal movement, vertical movement and steering movement;
and the carrying unit is respectively connected with the Kinect V2 sensor, the control module, the first wireless communication module, the Beidou positioning module, the first data storage module, the movement unit and the power supply unit and is used for fixing the data acquisition system.
4. The Kinect V2 sensor-based banana plant growth evaluation system according to claim 3,
the power supply unit comprises a solar unit, a storage battery unit and a power supply conditioning unit;
the solar unit is used for supplementing electric energy for the power supply unit;
the storage battery unit is an energy storage unit;
the power supply conditioning unit is used as an electric energy conversion unit and is used for providing stable and reliable standard voltage output for the data acquisition system.
5. The Kinect V2 sensor-based banana plant growth evaluation system according to claim 2,
the data analysis system comprises a data analysis module and a data analysis module,
the second wireless communication module is used for the data analysis system to respectively perform data interaction with the data acquisition system and the big data analysis system;
the second data storage module is used for storing the image data, the growth situation and the position information;
the data preprocessing module is used for converting the image data into color original point cloud data, and performing background segmentation, space topological relation establishment and noise reduction filtering processing on the color original point cloud data to obtain initial point cloud data;
the first data analysis module is used for extracting characteristic parameters according to the initial point cloud data to obtain a characteristic data set, training the characteristic data set through a deep learning model and verifying the characteristic data set through manually collected data to construct a banana growth condition analysis model to obtain the growth condition;
and the first image display unit is used for displaying the growth situation, the position information and the image data.
6. The Kinect V2 sensor-based banana plant growth evaluation system according to claim 5,
the data pre-processing module comprises a data pre-processing module,
an image conversion unit for converting the image data into the color original point cloud data;
the background segmentation unit is used for segmenting the complex background of the color original point cloud data to obtain a first image;
the image correction unit is used for carrying out image correction on the first image through the space topological relation gap and the normal vector to obtain a second image;
and the image filtering and denoising unit is used for carrying out filtering and denoising processing on the second image to obtain the initial point cloud data.
7. The Kinect V2 sensor-based banana plant growth evaluation system according to claim 6,
the method for filtering and denoising the second image by the image filtering unit comprises one or more of point cloud reduction, statistical filtering, radius filtering and bilateral filtering.
8. The Kinect V2 sensor-based banana plant growth evaluation system according to claim 5,
the big data analysis system comprises a big data analysis system,
the third wireless communication module is used for the big data analysis system to respectively perform data interaction with the data analysis system and the mobile terminal system;
a third data storage module, configured to store the location information, the growth situation, the growth prediction information, and the solution;
and the second data analysis module is used for obtaining the growth condition prediction information by comparing the difference between the growth condition and the historical growth condition according to the historical growth condition of the banana plants, and providing the solution based on the position information according to the growth condition prediction information, wherein the second data analysis module is used for collecting the environment information of the area where the banana plants are located according to the position information, performing environment prediction and obtaining the solution by combining the growth condition prediction information.
9. The Kinect V2 sensor-based banana plant growth evaluation system according to claim 8,
the system at the mobile terminal comprises a mobile terminal,
the fourth wireless communication module is used for the mobile terminal system to respectively perform data interaction with the data analysis system and the big data analysis system;
a second image display unit for displaying the position information, the growth situation, the growth prediction information, the solution, and the image data;
a fourth data storage module, configured to store the location information, the growth situation, the growth prediction information, and the solution.
10. The Kinect V2 sensor-based banana plant growth evaluation system according to claim 9,
the first wireless communication module, the second wireless communication module, the third wireless communication module and the fourth wireless communication module are 5G communication modules;
the first image display unit is further configured to display the growth prediction information.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110773975.8A CN113409298A (en) | 2021-07-08 | 2021-07-08 | Banana plant growth evaluation system based on Kinect V2 sensor |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110773975.8A CN113409298A (en) | 2021-07-08 | 2021-07-08 | Banana plant growth evaluation system based on Kinect V2 sensor |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113409298A true CN113409298A (en) | 2021-09-17 |
Family
ID=77685673
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110773975.8A Pending CN113409298A (en) | 2021-07-08 | 2021-07-08 | Banana plant growth evaluation system based on Kinect V2 sensor |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113409298A (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107807540A (en) * | 2017-10-16 | 2018-03-16 | 吉林农业大学 | A kind of agriculture intelligent platform based on image procossing |
| CN109115776A (en) * | 2018-08-27 | 2019-01-01 | 江苏大学 | A kind of Plug seedling growing way non-destructive monitoring method and device based on color and depth information |
| CN109874584A (en) * | 2019-03-19 | 2019-06-14 | 广州辰轩农业科技有限公司 | A kind of fruit tree growing way monitoring system based on deep learning convolutional neural networks |
| CN110532936A (en) * | 2019-08-26 | 2019-12-03 | 李清华 | A kind of method and system identifying field crop growing way monitoring image Green plant |
| CN210639476U (en) * | 2019-11-04 | 2020-05-29 | 南京信息工程大学 | Wild animal monitoring system |
| CN111460903A (en) * | 2020-03-05 | 2020-07-28 | 浙江省农业科学院 | System and method for monitoring growth of field broccoli based on deep learning |
| CN112435239A (en) * | 2020-11-25 | 2021-03-02 | 南京农业大学 | Scindapsus aureus leaf shape parameter estimation method based on MRE-PointNet and self-encoder model |
| CN112509029A (en) * | 2020-11-30 | 2021-03-16 | 广西慧云信息技术有限公司 | Method for remotely analyzing growth vigor of grapes |
| US20210158041A1 (en) * | 2017-08-25 | 2021-05-27 | The Board Of Trustees Of The University Of Illinois | Apparatus and method for agricultural data collection and agricultural operations |
-
2021
- 2021-07-08 CN CN202110773975.8A patent/CN113409298A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20210158041A1 (en) * | 2017-08-25 | 2021-05-27 | The Board Of Trustees Of The University Of Illinois | Apparatus and method for agricultural data collection and agricultural operations |
| CN107807540A (en) * | 2017-10-16 | 2018-03-16 | 吉林农业大学 | A kind of agriculture intelligent platform based on image procossing |
| CN109115776A (en) * | 2018-08-27 | 2019-01-01 | 江苏大学 | A kind of Plug seedling growing way non-destructive monitoring method and device based on color and depth information |
| CN109874584A (en) * | 2019-03-19 | 2019-06-14 | 广州辰轩农业科技有限公司 | A kind of fruit tree growing way monitoring system based on deep learning convolutional neural networks |
| CN110532936A (en) * | 2019-08-26 | 2019-12-03 | 李清华 | A kind of method and system identifying field crop growing way monitoring image Green plant |
| CN210639476U (en) * | 2019-11-04 | 2020-05-29 | 南京信息工程大学 | Wild animal monitoring system |
| CN111460903A (en) * | 2020-03-05 | 2020-07-28 | 浙江省农业科学院 | System and method for monitoring growth of field broccoli based on deep learning |
| CN112435239A (en) * | 2020-11-25 | 2021-03-02 | 南京农业大学 | Scindapsus aureus leaf shape parameter estimation method based on MRE-PointNet and self-encoder model |
| CN112509029A (en) * | 2020-11-30 | 2021-03-16 | 广西慧云信息技术有限公司 | Method for remotely analyzing growth vigor of grapes |
Non-Patent Citations (5)
| Title |
|---|
| YANG HU 等: "Automatic Non-Destructive Growth Measurement of Leafy Vegetables Based on Kinect", 《SENSORS》 * |
| 仇瑞承等: "基于RGB-D相机的玉米茎粗测量方法", 《农业工程学报》 * |
| 夏春华等: "基于TOF深度传感的植物三维点云数据获取与去噪方法", 《农业工程学报》 * |
| 江晓庆等: "基于Kinect的农作物长势深度图像实时获取算法", 《广东农业科学》 * |
| 王金志: "基于KinectV2的香蕉植株特征检测技术的研究", 《中国优秀博硕士学位论文全文数据库(硕士)农业科技辑》 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2014054042A1 (en) | Device and method for detecting plantation rows | |
| Zhu et al. | A calculation method of phenotypic traits based on three-dimensional reconstruction of tomato canopy | |
| CN106067169A (en) | Water Stress state automatic monitoring method and system | |
| CN108805924A (en) | A kind of lily picking independent positioning method and system | |
| CN113920190B (en) | Ginkgo flower spike orientation method and system | |
| CN112200163B (en) | Underwater benthos detection method and system | |
| CN112434569B (en) | Unmanned aerial vehicle thermal imaging system | |
| CN115187803B (en) | Positioning method for picking process of famous tea tender shoots | |
| CN115239625B (en) | Top cover welding spot cloud defect detection method, device, equipment and storage medium | |
| CN102855485A (en) | Automatic wheat earing detection method | |
| CN114004814A (en) | Coal gangue identification method and system based on deep learning and gray scale third moment analysis | |
| CN112906719A (en) | Standing tree factor measuring method based on consumption-level depth camera | |
| CN116977960A (en) | Rice seedling row detection method based on example segmentation | |
| CN115854895A (en) | Non-contact stumpage breast diameter measurement method based on target stumpage form | |
| He et al. | A calculation method of phenotypic traits of soybean pods based on image processing technology | |
| CN113723314A (en) | Sugarcane stem node identification method based on YOLOv3 algorithm | |
| Zhu et al. | Research on deep learning individual tree segmentation method coupling RetinaNet and point cloud clustering | |
| CN113409298A (en) | Banana plant growth evaluation system based on Kinect V2 sensor | |
| CN116897668A (en) | Electric-drive crop sowing and fertilizing control method and system | |
| CN116740337A (en) | Safflower picking point identification positioning method and safflower picking system | |
| CN116739739A (en) | Loan amount evaluation method and device, electronic equipment and storage medium | |
| Wang et al. | Banana pseudostem width detection based on kinect V2 depth sensor | |
| CN114898100A (en) | Point cloud data extraction method, device, system, equipment and storage medium | |
| CN113421301B (en) | Method and system for positioning central area of field crop | |
| CN112082475B (en) | Living stumpage species identification method and volume measurement method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210917 |
|
| RJ01 | Rejection of invention patent application after publication |