KR20240083254A - Automatic corn growth measurement method using 3D depth camera - Google Patents

Abstract

The present invention includes the steps of (a) the data collection module 100 acquiring 3D image RGB data using data captured by a 3D depth camera on a corn colony; (b) The artificial intelligence model creation module 200 uses the 3D image RGB data acquired in the data collection module 100 as input data, and the measured data measured to correspond to the data, such as corn plant height, canopy layer, and leaf area, And generating an artificial intelligence model for automatically measuring corn growth by learning learning data with the amount of weeds as output data; And (c) inputting data captured by a 3D depth camera on a corn colony to the generated artificial intelligence model, outputting the plant height, canopy layer, leaf area, and amount of weeds of corn.

Description

Automatic corn growth measurement method using 3D depth camera}

The present invention relates to a method for automatically measuring corn growth, and to a method for automatically measuring corn growth using a 3D depth camera.

In measuring the growth of corn, changes in grass height, leaf area, and canopy layer are important indicators in predicting corn production, but existing methods for measuring this are methods that contact or are destructive to the crop and also require a lot of labor. There is a problem.

Measurement methods using drone filming and 2D images can measure the area occupied by a corn colony, but cannot measure the plant height of the crop. Conversely, the side camera used to measure the plant height of the crop measures the area occupied by the corn colony in the field area. There is a limitation that cannot be measured.

Technologies are being developed to overcome the problems of conventional corn growth measurement methods.

U.S. Patent Publication No. 2022-0138925 relates to a system and method for measuring the optical yield of crops grown in the open field, and estimates the spatial area of pixels occupied by one or more target plants and the pixels of harvestable plants from the acquired image data. Discloses a system and method that can identify crops adjacent to weeds and measure the yield of crops including corn as a target crop.

Disclosed is the ability to differentiate or classify harvestable plant pixels from background pixels including weed pixels.

US Patent No. 10891482 discloses an on-site diagnostic method, device and system for predicting growth stage 1 and crop yield in the plant area. In this prior art, a method for predicting the yield of an unspecified crop is disclosed, but since it targets an unspecified number of crops, a measurement method specialized for corn cannot be provided.

KR 10-2017-0114065 A US 2022-0138925 A1 US 2021-0133443 A1 US 10891482 B2

The present invention was created to solve the above problems.

In order to solve crop destructive measurement problems such as direct contact that occurred when measuring indicators related to corn growth, we aim to provide a method to measure corn growth-related indicators non-contactly without measuring through direct contact.

One embodiment of the present invention to solve the above problems includes: (a) the data collection module 100 acquiring 3D image RGB data using data captured by a 3D depth camera on a corn colony; (b) The artificial intelligence model creation module 200 uses the 3D image RGB data acquired in the data collection module 100 as input data, and the measured data measured to correspond to the data, such as corn plant height, canopy layer, and leaf area, And generating an artificial intelligence model for automatically measuring corn growth by learning learning data with the amount of weeds as output data; and (c) inputting data captured by a 3D depth camera on a corn colony to the generated artificial intelligence model, outputting the plant height, canopy layer, leaf area, and amount of weeds of corn.

In addition, in step (b), the preprocessing module 300 receives data captured by the 3D depth camera on a corn colony from the data collection module 100, and the noise removal unit of the preprocessing module 300 ( Obtaining 3D image RGB data by filtering the input data with the ExGR index in 320 and correcting the coordinates of the input data with the RANSAC algorithm in the coordinate correction unit 330 of the pre-processing module 300. It is desirable to include ;.

In addition, in step (a), it is preferable to include (a1) the data collection module 100 collecting data captured by a 3D depth camera on a corn colony at a height of 3 m.

Another embodiment of the present invention provides a computer-readable recording medium on which a computer program for executing the method according to the above embodiment is recorded on a computer.

Another embodiment of the present invention provides a computer program stored in a recording medium for executing the method according to the above embodiment on a computer.

By solving the above problems, the present invention achieves the following effects.

First, it is possible to measure corn growth without using destructive methods such as direct measurement. Indicators related to corn growth, such as grass height, canopy layer, and leaf area, which previously had to be measured manually and manually, can be measured automatically and non-contactly.

Second, it is possible to measure and provide corn growth indicators that were difficult or impossible to measure using existing 2D photographic data alone.

Third, non-contact measurement values can be provided with high accuracy as analysis results such as daily growth, canopy evaluation, and leaf area changes.

1 is a diagram for explaining a system for performing a method according to the present invention.
Figure 2 is a diagram for explaining the data processing process of points extracted from a 3D depth camera using the method according to the present invention.
Figure 3 is a diagram for explaining the processing process of RGB image data extracted from a 3D depth camera using the method according to the present invention.
Figure 4 is a diagram for explaining a data file extracted from a 3D depth camera.
Figure 5 is a diagram for explaining a method of evaluating the canopy layer of a corn colony by the method according to the present invention.
6 to 7 are diagrams for explaining image data filtered by a noise removal unit for the method according to the present invention.
Figure 8 is a graph of the leaf area of a corn colony measured by the method according to the present invention.
Figure 9 is a growth rate analysis graph calculated by the method according to the present invention.
10 to 11 are diagrams for verification of an artificial intelligence model according to the present invention.

The above objectives, features and other advantages of the present invention will become more apparent by describing the preferred embodiments of the present invention in detail with reference to the accompanying drawings. In this process, the thickness of lines or sizes of components shown in the drawings may be exaggerated for clarity and convenience of explanation. In addition, the terms described below are terms defined in consideration of functions in the present invention, and may vary depending on the intention or custom of the user or operator. Therefore, definitions of these terms should be described based on the content throughout this specification.

Additionally, the described embodiments are provided as examples for explanation of the invention and do not limit the technical scope of the invention.

Hereinafter, a method for automatically measuring corn growth according to a preferred embodiment of the present invention will be described in detail with reference to the attached drawings.

Description of 3D Depth Camera

The 3D depth camera, which is an essential component of the present invention, will first be described.

In order to build a model for carrying out the method according to the present invention, the corn colony will be photographed using a 3D depth camera.

A 3D depth camera can be any camera that captures 3D videos and images using infrared rays. 3D video and image data captured by a 3D depth camera can be extracted into multiple data files including RGB image data, as shown in FIG. 4.

The 3D depth camera may preferably use Microsoft Corporation's Azure kinect, but is not limited as long as it is suitable for photographing the corn colony described in the present invention.

When photographing a corn colony using a 3D depth camera, measurements can preferably be made at a horizontal height of about 3 m from the corn colony. Through this, learning data for model creation is obtained. When using the created model, the plant height, canopy layer, and amount of weeds can be analyzed using only images captured by a 3D depth camera. Details will be described later.

Description of the system

Referring to Figure 1, a system for performing the corn growth measurement method according to the present invention will be described.

The system of the present invention for performing the method according to the present invention is a system that uses data from a 3D depth camera and includes a data collection module 100, an artificial intelligence model creation module 200, and a data preprocessing module 300.

The data collection module 100 is connected to a 3D depth camera and can collect data from the 3D depth camera, and includes a measurement data collection unit 101 that collects measurement data, and 3D image data that collects data from the 3D depth camera. It includes a unit 110 and an RGB data unit 120.

The measurement data collection unit 101 collects automatically or manually entered data, and will not be limited as long as it is data for corn growth. Examples include corn plant height, leaf area, and canopy layer. There are no particular restrictions on the automatic input method, and as a manual input method, for example, handwriting may be used.

Image data from the 3D depth camera that is not extracted in the 3D image data unit 110 is collected.

The RGB data unit 120 extracts and collects RGB data from image data of the 3D depth camera.

The artificial intelligence model creation module 200 can generate an artificial intelligence model built by learning using the data of the data collection module 100 and the data of the data pre-processing module 300 as learning data.

The artificial intelligence model used in the artificial intelligence model creation module 200 may preferably use the U-net model and the Adam function. U-net is an artificial intelligence model used for image segmentation and analysis and has shown excellent performance even with training data. In fact, in the present invention, the model is used to distinguish corn and weeds in images taken in open fields and to analyze pixel information. When used, the performance was found to be almost the same as the directly measured results.

The data preprocessing module 300 may preprocess the data so that the data of the data collection module 100 can be used as learning data by the artificial intelligence model creation module 200.

The data pre-processing module 300 includes a data analysis unit 310, a noise removal unit 320, a coordinate correction unit 330, and a labeling unit 340.

The data analysis unit 310 can analyze 3D images and image data from a 3D depth camera and provide a library for analysis.

The data analysis unit 310 may use Python, Pandas, matplotlib, opencv2, etc., but is not limited thereto. For example, Pandas is used for text file-based 3D coordinate data analysis, matplotlib is used for 3D image visualization and graphing, opencv2 is used for RGB image preprocessing, and Python is used for image analysis and artificial intelligence model development. can be used for

Due to the nature of the 3D depth camera using reflected infrared rays, noise may occur in the output data. The noise removal unit 320 filters the 3D image data using the ExGR index to remove the resulting noise. ( 6 and 7)

The coordinate correction unit 330 is intended to match the coordinate values of data measured by a 3D depth camera and the actual measured value, and can correct the coordinates using the RANSAC (Random Sample consensus) algorithm.

The labeling unit 340 may label the image data of the 3D depth camera to provide it as learning data for the artificial intelligence model creation module 200.

When such preprocessing is completed by the data preprocessing module 300, the data from the 3D depth camera collected by the data collection module 100 will be converted into 3D image RGB data.

Description of method

The method for measuring corn growth will be described in detail with further reference to FIGS. 2 and 3.

First, the data collection module 100 acquires 3D image RGB data using data captured by a 3D depth camera on a corn colony.

At this time, the data collection module 100 preferably collects data captured by a 3D depth camera on a corn colony from a height of 3 m.

Next, the artificial intelligence model creation module 200 uses the 3D image RGB data acquired from the data collection module 100 as input data, and the measured data measured to correspond to the input data are corn plant height, canopy layer, leaf area, and weeds. An artificial intelligence model for automatically measuring corn growth is created by learning learning data that uses quantity as output data.

At this time, the data acquired by the artificial intelligence model creation module 200 is data preprocessed by the preprocessing module 300.

Specifically, the preprocessing module 300 receives data captured by a 3D depth camera on a corn colony from the data collection module 100, and converts the data input from the noise removal unit 320 of the preprocessing module 300 into an ExGR index. By filtering and correcting the coordinates of the data input from the coordinate correction unit 330 of the preprocessing module 300 using the RANSAC algorithm, 3D image RGB data will be obtained.

Finally, when data captured by a 3D depth camera on a corn colony is input into the created artificial intelligence model, the corn plant height, canopy layer, leaf area, and amount of weeds are output.

Below, we will explain the case of using measured values to evaluate the growth of corn using this method.

Explain how to measure the plant height of corn.

To evaluate the printed corn plant height measurement values, you can further use programs such as cloudcompare that visualizes 3D data. The constructed artificial intelligence model will select the highest point for each head of corn and store the coordinate values, and the average value of the stored coordinate values will be measured. According to this method, in order to accurately measure corn plant height using image data from a 3D depth camera, only the point data of the corn colony is extracted using the ExGR index, and then the points in the top 1% and 10% of the Z-axis coordinates from all points of the corn colony are extracted. Since the average value of the coordinates can be measured as the average plant height of the corn colony, although it is a measurement value that was not actually measured, it will have a result similar to the actually measured data value (see Figure 2).

Describe how to evaluate the canopy of a corn colony. From the image data of the 3D depth camera, the points of Can be evaluated (see Figure 5)

Explain how to measure the area occupied by a corn colony. After processing the image data captured by the 3D depth camera using the data preprocessing module 300, the area (coverage) occupied by the corn colony in the field soil can be measured by dividing the number of pixels representing corn by the total number of image pixels.

Explain how to measure the area occupied by weeds in a colony. When image data taken from a 3D depth camera is input into a pre-trained artificial intelligence model, the area occupied by weeds in the colony is automatically measured and provided. Figure 6 shows an example, and here shows the result of calculating the amount of weeds by excluding the leaf area of corn calculated in the artificial intelligence model from the total area.

Describe the method for evaluating leaf area of corn colonies. By the noise removal unit 320, the values of red, blue, and green can be extracted from the image RGB data, and the area occupied by the corn colony can be extracted from the image RGB data using the ExGR index. In another embodiment, the ratio of the total number of pixels of the image RGB data and the number of pixels of the image from which only corn is extracted may be expressed as a percentage. At this time, the bitwise function provided by opencv2 of the python language can be used, but it is not limited to this. Measurement results provided as an image as shown in FIG. 7 can also be provided as a time series graph as shown in FIG. 8.

In addition, a growth rate analysis graph can also be provided as shown in Figure 9. Using the values measured according to the present invention, the results were measured by subtracting the corn pasture height during the day from the corn pasture height at night on each day, and the corn pasture height at night minus the corn pasture height during the day of the next day. It will be provided using .

In addition, of course, the method according to the present invention can be provided in the form of a computer-readable recording medium and a computer program on which a computer program for executing the method according to the present invention is recorded on a computer.

verification

With further reference to Figures 10 and 11, the effect of measuring corn growth using the method according to the present invention was verified.

First, it will be described with reference to FIG. 10.

Figure 10 is a graph comparing and analyzing the actual measured value of the target corn and the measured value using the present invention.

A model was created using 3D video and image data of corn colonies captured every 30 minutes from June 8 to July 26, 2022. The measurement data used as a control group to evaluate the generated model were data from 5:00 AM to 7:00 AM and data from 5:00 PM to 7:00 PM on the above data collection day.

As a control, the plant height of corn measured at the top 10% point and 1% point was visually checked in each state, then the highest point was selected and measured directly, and the value was compared with the result from the method according to the present invention. analyzed. As a result of the analysis, it was found that the results according to the present invention and the directly measured values of the control group showed a high correlation (1% R2=0.9959, 10% R2=0.9974).

In other words, there was a high correlation between the corn plant height value derived by the method according to the present invention and the directly measured corn plant height value, and the accuracy of the method according to the present invention was verified to be high.

Next, the artificial intelligence model verification results are explained using Figure 11.

A total of 234 images were used, and data in which the RGB value of the part of each image excluding the corn pixel area was set to 0 was used for learning. The size of the input image used for analysis was 512 * 512.

The optimizer of the U-net model used in the learning process was Adam, and the number of learning times (epoch) was 50. The learning results showed overall high accuracy of over 99% and low loss of less than 0.02%.

Above, the present invention has been described with reference to the embodiments shown in the drawings so that those skilled in the art can easily understand and reproduce the present invention. However, these are merely illustrative examples, and various modifications and equivalent alternatives can be made by those skilled in the art from the embodiments of the present invention. It will be appreciated that embodiments are possible. Therefore, the scope of protection of the present invention should be determined by the scope of the patent claims.

100: data acquisition module
101: Measurement data unit
110: 3D image data unit
120: RGB data unit
200: Artificial intelligence model creation module
300: Data preprocessing module
310: Data analysis department
320: noise removal unit
330: Coordinate correction unit
340: Labeling unit

Claims (5)

(a) the data collection module 100 acquiring 3D image RGB data using data captured by a 3D depth camera on a corn colony;
(b) The artificial intelligence model creation module 200 uses the 3D image RGB data acquired in the data collection module 100 as input data, and the measured data measured to correspond to the data, such as corn plant height, canopy layer, and leaf area, And generating an artificial intelligence model for automatically measuring corn growth by learning learning data with the amount of weeds as output data; and
(c) inputting data captured by a 3D depth camera on a corn colony to the generated artificial intelligence model, outputting the plant height, canopy layer, leaf area, and amount of weeds of the corn;
method.
According to claim 1,
In step (b),
The pre-processing module 300 receives data captured by the 3D depth camera on a corn colony from the data collection module 100, and converts the input data into the ExGR index from the noise removal unit 320 of the pre-processing module 300. Filter by,
Comprising: acquiring 3D image RGB data by correcting the coordinates of the input data using the RANSAC algorithm in the coordinate correction unit 330 of the pre-processing module 300,
method.
According to claim 1,
In step (a),
(a1) the data collection module 100 collects data captured by a 3D depth camera on a corn colony at a height of 3 m;
method.
A computer program for executing the method according to any one of claims 1 to 3 is recorded on a computer,
A recording medium that can be read by a computer.
Stored on a recording medium for executing the method according to any one of claims 1 to 3 on a computer,
computer program.

Images