KR20200084638A - Method, apparatus and computer program for crop monitoring - Google Patents

Abstract

The present invention relates to a method for monitoring crops. The method comprises: a step a of generating a growth model of crops by using a large amount of growth data including growth environment information and growth progress information of the crops in a service server; a step b of generating an image by sensing at least one of the temperature, air pressure, humidity, wind speed, and amount of sunlight in a predetermined area at a predetermined period and photographing the crops in the area in an unmanned aerial vehicle; and a step c of estimating a harvest probability of the crops by applying the growth data of the crops to the growth model in the service server.

Description

Agricultural crop monitoring method, device and computer program {METHOD, APPARATUS AND COMPUTER PROGRAM FOR CROP MONITORING}

The present invention relates to a method for monitoring crops. More specifically, the present invention collects data related to the growth of a crop in an unmanned aerial vehicle, and uses a machine learning framework modeling the growth of the crop to check for abnormal signs related to crop growth and to predict a crop condition of the crop. will be.

Recently, various methods of using drones have been devised with the development of unmanned aerial vehicles, that is, drone technology. In the agricultural field, sowing using drones, spraying pesticides, etc. has reached the practical stage, and the thermal efficiency sensor or multi-spectrum camera is mounted on the drone to determine the dry area of the farmland and intensively supply the water to improve relational efficiency. The method of raising is also being tested.

On the other hand, in agriculture, crop monitoring is very important for risk management. For example, orchards can improve the quality and quantity of fruit trees by early treatment of abnormal fruit trees through crop monitoring, and solve the problem of evidence even when claiming insurance due to wrong crop growth. In order to increase the reliability of monitoring, it is appropriate to conduct a full-scale survey of each crop, but monitoring a large area of farmland through a full-scale survey method is expensive, and thus, sampling is conducted through sampling.

Korean Patent Publication 10-1536095 (2015.7.6)

An object of the present invention is to provide a method for monitoring crops using a drone.

Method for monitoring a crop according to an embodiment of the present invention, a service server, a step of generating a growth model of the crop using a large amount of growth data including growth environment information and growth progress information of the crop; In an unmanned aerial vehicle, a step b is generated at a predetermined cycle, sensing at least one of air temperature, air pressure, humidity, wind speed, and sunshine in a predetermined area, and photographing crops in the area to generate an image; And a step c of estimating a harvest probability of the crop by applying the growth data of the crop to the growth model in the service server.

According to the present invention, data related to the growth of crops is collected in a drone, and the growth of crops can be modeled in a service server running a machine learning framework. Through this, the abnormal signs related to the growth of the crops are checked, and the crop yield can be predicted with high reliability.

On the other hand, according to the present invention, since the crop growth image is photographed at a preset cycle in a drone, and data on environmental conditions of the crop is collected, the effect of using the crop growth image and weather information data to compensate for crop damage caused by abnormal weather There is.

1 is a view for explaining the configuration of a drone for collecting a growing image of a crop, sensing weather information and location information according to an embodiment of the present invention
Figure 2 is a flow chart for explaining a method for monitoring crops according to an embodiment of the present invention

The present invention is not limited to the description of the embodiments described below, and it is apparent that various modifications may be made without departing from the technical spirit of the present invention. In describing the embodiments, descriptions of technical contents that are widely known in the technical field to which the present invention pertains and which are not directly related to the technical subject matter of the present invention will be omitted.

Meanwhile, in the accompanying drawings, the same components are represented by the same reference numerals. In the accompanying drawings, some components may be exaggerated, omitted, or schematically illustrated. This is to clearly explain the gist of the present invention by omitting unnecessary descriptions not related to the gist of the present invention.

1 is a view for explaining the configuration of a drone for collecting a growing image of a crop and sensing weather information and location information according to an embodiment of the present invention.

As illustrated in FIG. 1, a drone for collecting growth data according to an embodiment of the present invention includes a drone body 10, a frame 20 connecting the main body and a propeller, a propeller 30, and a gimbal connecting the main body and a camera. 40, a camera 50, a protective guard 60 may be included.

The drone body 10 may include an acceleration sensor, a gyro sensor, a GPS, a temperature sensor, an air pressure sensor, a humidity sensor, a sensor module including an altitude sensor and a wind speed sensor, a control unit, a memory, a communication module, a battery, and the like.

The sensor module may collect data for drone flight and hover control. The sensor module collects information such as the position, speed, direction, and obstacle of the drone, and the control unit can control the horizontal and heading direction of the drone using the collected information.

Furthermore, the sensor module according to the embodiment of the present invention may perform a function of collecting growth data of a crop. For example, the sensor module can sense weather information such as temperature, air pressure, humidity, wind speed, and sunlight, and can sense location information such as altitude, GPS coordinates, and acceleration.

The control unit may perform a function of controlling the operation of the drone. Further, the drone according to the embodiment of the present invention may collect fruit growth image data while flying through a narrow tunnel made of fruit trees in an orchard. In this case, the controller may control the drone to hover below 50 centimeters in radius while maintaining an altitude of 2 meters or less on the ground.

The control unit according to an embodiment of the present invention may perform a function of combining clock information with image data captured by the camera module 50 and weather information and location information generated by the sensor module and storing it in a memory. This is for synchronizing sensor data and image data based on a creation time, and stored image data, weather information data, and location information data may be transmitted to a service server according to an embodiment of the present invention through a communication module.

The memory may store image data captured by the camera module 50 and weather information and location information data generated by the sensor module.

As shown in Figure 1, the frame 20 of the drone according to the embodiment of the present invention is formed in the right and left opposing relative to the main body 10, it can perform the function of connecting the main body and the propeller. The propeller 30 includes a motor and is formed at both ends of the frame as illustrated in FIG. 1 to perform a function of taking off and landing or moving the drone.

The gimbal 40 connects the main body and the camera, and may perform a function of controlling the camera module not to shake even if the drone main body is shaken in order to collect stable image data.

The camera module 50 may perform a function of photographing a growing image of a crop. Furthermore, the camera module 50 may be provided with a shake correction sensor to secure a clear image, and may perform omnidirectional shooting. Furthermore, the camera module may generate a visible light, an infrared image, a hyperspectral image, or a multispectral image depending on the implementation. To this end, the camera module may include a light source for acquiring an image of a specific wavelength.

In particular, the drone according to the embodiment of the present invention is characterized in that a camera module is formed on the top of the body in order to collect a growing image of a crop, especially a fruit. For example, the growth image of a fruit is typically a flower and a fruit. When the camera module is located at the bottom of the main body, the flower and the fruit are obscured by the leaves of the tree, so the growth image cannot be properly photographed.

Therefore, the drone according to the embodiment of the present invention is characterized in that the gimbal 40 and the camera module 50 are located above the main body, and the protection guard 60 is located below the main body.

However, it should be noted that the drone of FIG. 1 is only an example, and the scope of the present invention is not limited to the shape of the drone illustrated in FIG. 1. That is, the drone according to the embodiment of the present invention can be variously modified.

A drone according to another embodiment of the present invention may form a gimbal and a camera module at the bottom of the main body to collect growth data of crops such as beans and barley, and in this case, a protective guard may be formed at the top or other direction of the main body. Can.

Further, the drone according to another embodiment of the present invention may include a configuration capable of selectively or simultaneously mounting the gimbal for shooting on the top or bottom of the main body. According to the above embodiment, the drone is equipped with camera modules on the upper and lower sides of the main body, so that while moving through the fruit tree tunnel, the flower eye and the fruit growth image can be photographed while simultaneously photographing the falling fruit image.

2 is a flow chart for explaining a method for monitoring a crop according to an embodiment of the present invention.

Although the example of FIG. 2 is shown for a method of monitoring fruit growth in an orchard, it should be noted that the present invention is not limited thereto. That is, according to an embodiment of the present invention, the growth of crops can be monitored in agricultural fields such as orchards, rice fields, fields, and mountains. In the present specification, crops may be understood as meaning encompassing grain crops such as fruit crops such as peppers, root vegetables such as radish, fruit trees such as apples, flowers such as roses, sesame seeds, and field crops such as sesame seeds and rice.

A crop monitoring system according to an embodiment of the present invention includes a drone for collecting growth data of a crop, a service server for modeling the growth of the crop, checking the abnormality by applying the collected data to the growth model, and generating a monitoring report. It may include a user smartphone receiving the alarm information or monitoring report from the service server.

In Dean 210 to 225, the drone may collect data related to fruit growth while flying through the fruit tree tunnel of the orchard at a predetermined cycle.

More specifically, the drone may collect temperature, air pressure, humidity, wind speed, and sunshine information using the mounted sensor module. Furthermore, the drone can collect location information such as altitude, GPS coordinates, and acceleration information. In particular, a drone according to an embodiment of the present invention may collect fruit growth images such as flowers and fruits using a mounted camera module.

Although not illustrated in FIG. 1, the drone may add time or clock information to the collected weather information, location information, and image data. This is for synchronizing weather information, location information, and image information collected based on a time axis.

The drone can then transmit the collected weather, location, and image data to the service server. (Steps 230 and 235) For example, the drone may transmit growth data collected through a short range wireless communication module to a user's smartphone, and may transmit growth data from a user's smartphone to a service server through a Wi-Fi and LTE communication module. . On the other hand, if the drone is equipped with a communication module such as Wi-Fi, LTE, the collected data can be transmitted to a service server at a preset cycle.

Meanwhile, the service server may collect a large amount of crop growth data and generate a database for it. (Steps 240, 245)

The crop growth data may include crop growth image data such as flowers and fruits collected from multiple orchards over the years. The crop growth image may be a set of images for a series of processes, from the time the flower eyes occur to the flowering of the flower, the fertilization of the fruit, the removal of the fruit, the removal of the fruit (fruit picking), the growth of the fruit, the fruit, and the harvest of the fruit.

Furthermore, the crop growth data includes data on a change value of the growth image, time information at which the growth image was collected, location information at which the growth image was collected, temperature, air pressure, humidity of the time and place at which the growth image was collected, It may include weather information such as wind speed and sunshine, and data on the amount of change in the weather information.

For example, when the service server collects data photographed from under a tree at a predetermined period for a certain period in a specific orchard, the growth region, such as a flower or fruit, is identified from the image data, and is not sensitive to distortion and less likely to change. It is possible to correct and synchronize the scale and RGB values of the image data over time based on arbitrary feature points that can be used, for example, the feature points of branches where branches are split from the main body.

Furthermore, the service server calculates feature descriptors for information such as brightness, color, sharpness, gradient, scale, pattern value, change value, and distribution value of the growth region from the synchronized image data, thereby obtaining an image vector for the growth region. Can be created. The image vector may include, for example, information about a size of a growth region at a specific time point, a range of a growth region, color, and distribution. Furthermore, the service server may process the image vector to include change information of the growth region image that changes over time.

Furthermore, the service server may include time information at which the growth image was collected at a predetermined period in a specific orchard at a predetermined period, location information at which the growth image was collected, temperature, air pressure, humidity, wind speed of the time and place at which the growth image was collected, Weather information such as sunshine, and data on the amount of change in the weather information may be collected. At this time, the service server may aggregate growth image data and data on the time, location, and weather information based on a time axis to be integrated into a growth information vector.

Thereafter, the service server may model a large amount of growth information vector in the machine learning framework to model fruit growth information according to the time from the harvest of the fruit to the next harvest. (Step 250)

Thereafter, the service server may estimate growth probability, growth rate, and growth quartile of the orchard by applying growth information at an arbitrary time point to the machine learning framework.

For example, the service server may receive growth data including weather information, location information, and image data collected at any time from a specific orchard. (Step 235)

The image included in the growth data may be data captured by a drone for collecting growth information according to an embodiment of the present invention under a tree at a specific date and time. Furthermore, the weather information and the location information may be temperature, air pressure, humidity, wind speed, sunshine, and GPS data sensed by the drone while photographing the image.

When the growth data is received, the service server identifies a growth area such as a flower or fruit from the image data, corrects the scale and RGB values of the image data based on a preset characteristic point to a preset range, and the brightness and color of the growth area The image vector for the growth region may be generated by calculating feature descriptors for information such as, sharpness, gradient, scale, pattern value, change value, and distribution value. (Step 255)

Furthermore, the service server may process the image vector to include change information of the growth region image that changes over time. For example, the service server may calculate change information of the growth region of the image data previously received and the growth region of the image data received this time, and may reflect the change information in the image vector for the growth region.

Furthermore, the service server may generate a growth data set by aggregating time information, location information, and weather information on which the image data is collected from the growth region image vector. (Step 260)

Thereafter, the service server may apply the growth data set to the fruit growth model generated in step 250. The fruit growth model can estimate the probability of harvesting fruit from each growth region, the fruit growth rate and growth quartile of each growth region based on information included in the corresponding growth data set.

Furthermore, the service server can search for abnormal signs of fruit growth from the growth data set. (Step 270) For example, the service server may detect an abnormal symptom when the amount of change in a growth area such as a fruit or a flower is too large or small in an arbitrary area. For example, the service server may detect that the growth is sluggish when the growth area change amount is less than the expected value, estimate the cause of the growth sluggish by referring to the log of the growth data, and notify the user to this. (Step 275)

Furthermore, the service server can search for anomalies of the growth environment from the growth data set. For example, if the temperature, air pressure, humidity, wind speed, and the amount of sunlight collected during a specific period are different from the expected values of the period, the service server may hold it as an abnormal sign. For example, if the amount of sunshine during a certain period is too low or too high for the average amount of sunshine for the period, if the humidity is too low or too high for the average period for a certain period, it can be detected as an abnormality and notify the user. . (Step 275)

Meanwhile, a user who manages an orchard can install an application that provides an orchard monitoring function according to an embodiment of the present invention to a terminal such as his or her smartphone, tablet pc, laptop, or desktop computer. (Step 160)

The application may perform a function of receiving data related to fruit growth collected by the drone by connecting to a drone collecting data related to fruit growth of an orchard in a predetermined cycle and short-range wireless communication. (Step 230)

For example, the user can check the image file of the drone photographed from under the fruit tree on a daily basis through the application. Furthermore, the user can check the growth environment information of the fruit through the data sensed by the drone, for example, daily temperature, air pressure, humidity, wind speed, and sunshine of the orchard on a daily basis.

Meanwhile, the user terminal may receive a monitoring report on fruit growth of the orchard from the service server and display it. (Step 280)

For example, the service server provides information on the growth quantity, such as the number of flower eyes, the number of fertilized fruit, the number of narrowed fruits, and the number of matured fruits in fruit tree units, and image data displayed by marking the growth amount in fruit tree units. You can receive it from and display it. The user may submit it to an insurance company later and use it as proof to receive compensation.

As another example, the user terminal may receive and display information on the number of fruits having a high probability of harvest in units of fruit trees, the growth rate of fruits, growth quartiles, and the like from the service server. The user will be able to make a farming plan using this information.

As another example, the user terminal may receive a notification from the service server about abnormal signs of fruit growth. For example, if the fruit growth of a particular tree in the orchard is sluggish, you may be notified that the tree needs care. Or, you may get a notch that the orchard is abnormal in the growth environment, such as when the orchard is dry or humid compared to the average year, when the amount of sunlight is insufficient or too high, or when the temperature is too high or low. The user will be able to make a farming plan using this information.

The embodiments of the present invention published in the present specification and drawings merely describe specific examples of the present invention and provide specific examples to help understanding of the present invention, and are not intended to limit the scope of the present invention. It is obvious to those of ordinary skill in the art to which the present invention pertains that other modifications based on the technical spirit of the present invention can be implemented in addition to the embodiments disclosed herein.

Claims (4)

In the service server, how to monitor the crop,
A step of generating a growth model of the crop using a large amount of growth data including growth environment information and growth progress information of the crop;
Step b, in the unmanned aerial vehicle, at a predetermined cycle, sensing at least one of air temperature, air pressure, humidity, wind speed, and sunshine in a predetermined area, and photographing crops in the area to generate an image;
A step c of estimating a harvest probability of the crop by applying the growth data of the crop to the growth model in the service server; And
And extracting an arbitrary feature point that is not sensitive to distortion and less likely to change, and normalizing the image in a preset form based on the feature point, in a cropped image. The method of claim 1, wherein step c,
Retrieving a flower or fruit object from a normalized image, specifying the flower or fruit object as a growth region, and generating an image vector for the growth region; And
And aggregating data sensing at least one of the air temperature, air pressure, humidity, wind speed, and sunshine, and the image vector to generate growth data of the crop. The method of claim 2, wherein step c,
Reflecting the change information of the first growth data generated based on the data collected in the first cycle and the second growth data generated based on the data collected in the second cycle to the second growth data. Crop monitoring method comprising a. The method of claim 3, wherein step c,
And estimating a probability that a fruit can be harvested from each growth region, a fruit growth rate, and a growth quartile of each growth region based on the information included in the growth data.

Images