JP2018029568A - Wilting condition prediction system and wilting condition prediction method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To predict with high accuracy the wilting condition of a plant according to the cultivation environment without requiring complicated procedures.SOLUTION: A watering control system 10 comprises: a camera 3 which acquires images related to the posture of a target plant S; an environmental sensor 7 which outputs measured data related to the surrounding environment of the target plant S; an image feature amount calculation unit 17 which calculates a numerical value vector which is a feature amount related to the wilting condition of the target plant S using mechanical learning on the basis of the images acquired by the camera 3; and a learning machine 21 which derives a predicted value which is a numerical value indicating the wilting condition of the target plant using the mechanical learning on the basis of the numerical vector calculated by the image feature amount calculation unit 17 and the measured data output by the environmental sensor 7.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a wilting degree prediction system and a wilting degree prediction method for predicting the wilting degree of a plant.

  In recent years, a mechanism for automatically cultivating plants with a certain quality without depending on the skill of the grower has been studied. For example, in Patent Document 1 below, the predicted sugar content at the time of ripening of the cultivated fruit is obtained from the component information of the cultivated fruit, and solution control during the growth is automatically performed so that the predicted sugar content is close to the target value. A solution control device is disclosed. Specifically, in this solution control apparatus, the dry matter rate and starch content are calculated based on the near-infrared spectrum of the fruit obtained using a halogen lamp and an NIR spectroscopic analyzer, and the dry matter rate and starch content are calculated. From the above, the soluble solid content of the fruit at the time of ripening is calculated using a predetermined calculation model. This soluble solid content corresponds to the sugar content of the fruit at ripening.

JP 2008-54573 A

  In the solution control apparatus described in Patent Document 1 described above, the sugar content is predicted as a value indicating the growth state of the future fruit, based on the near-infrared spectrum obtained for each fruit. Handling such as installation of near-infrared spectrometers and input of measurement values is complicated. In addition, the method for predicting the value indicating the future plant growth state based on the near-infrared spectrum of fruits assumes only predictions based on simple calculation formulas, and grows according to differences and changes in the cultivation environment. The state prediction accuracy is not sufficient.

  The present invention has been made in view of the above problems, and a wilting condition prediction system capable of predicting a wilting condition of a plant according to a cultivation environment with high accuracy without requiring complicated handling and An object of the present invention is to provide a method for predicting the degree of wilting.

  In order to solve the above problems, a wilting degree prediction system according to an aspect of the present invention includes an imaging unit that acquires an image related to the appearance of a target object that is a plant, and an environmental sensor that outputs measurement data related to the surrounding environment of the target object. An image feature amount calculation unit that calculates a numerical sequence that is a feature amount related to the degree of wilting of the object using machine learning based on an image acquired from the imaging unit, and a numerical sequence calculated by the image feature amount calculation unit And a predicted value deriving unit for deriving a predicted value of a numerical value representing the degree of deflation of the object using machine learning based on the measurement data output from the environmental sensor.

  Alternatively, in the wilting degree prediction method according to another aspect of the present invention, an image relating to the appearance of an object that is a plant is obtained using an imaging unit, and measurement data relating to the surrounding environment of the object is obtained using an environment sensor. Based on the image acquired from the image capturing unit by the image feature amount calculation unit, the numerical value sequence that is a feature amount related to the condition of the wrinkle of the object is calculated using machine learning, and the image feature is calculated by the prediction value deriving unit. Based on the numerical sequence calculated by the quantity calculation unit and the measurement data acquired by the environmental sensor, a predicted value of the numerical value representing the degree of deflation of the object is derived using machine learning.

  According to the deflation degree prediction system or the deflation degree prediction method of the above form, a numerical sequence that is a feature amount is calculated by machine learning based on the appearance image of the target acquired by the imaging unit, and the numerical sequence and the environment By machine learning using the measurement data acquired by the sensor, a numerical prediction value representing the degree of deflation of the object is derived. Thereby, the system which estimates the deflation degree of a target object by simple operation using the simple system containing an imaging part and an environmental sensor is realizable. Moreover, since the predicted value is derived using the measurement data acquired by the environmental sensor together with the image of the appearance of the target object, it is possible to accurately predict the degree of plant wilting according to the cultivation environment.

  Here, the predicted value deriving unit may derive a numerical predicted value related to the diameter of the stem of the object as the predicted value. In this case, the degree of wilt can be appropriately evaluated based on the predicted value of the numerical value related to the diameter of the stem of the object.

  Moreover, it is good also as providing the sugar content prediction part which estimates the sugar content of a target object based on a predicted value. In this case, it is possible to appropriately evaluate the future sugar content of the object based on the predicted value of the numerical value relating to the degree of wilting of the object.

  The environment sensor may output data relating to temperature and humidity as measurement data, and the predicted value deriving unit may derive a predicted value based on data relating to temperature and humidity. If it carries out like this, the wilting condition of the plant according to cultivation environment can be estimated with higher precision by deriving a predicted value using data about temperature and humidity.

  The environment sensor may output brightness-related data as measurement data, and the predicted value deriving unit may derive a predicted value based on the brightness-related data. In this case, by deriving a predicted value using data relating to brightness, it is possible to predict the wilting state of the plant according to the cultivation environment with higher accuracy.

  According to the present invention, it is possible to accurately predict the degree of plant wilting according to the cultivation environment without requiring complicated handling.

It is a figure which shows schematic structure of the irrigation control system concerning embodiment. It is a figure which shows the hardware constitutions of the computer which comprises the data processor of FIG. It is a block diagram which shows the function structure of the data processor 1 of FIG. It is a graph which shows the image of the data of the transition of the predicted value of the difference stem diameter DSD output by the learning device of FIG. It is a flowchart which shows the process sequence of the irrigation control by the data processor 1 of FIG.

  Hereinafter, a preferred embodiment of a deflation degree prediction system according to the present invention will be described in detail with reference to the drawings. In the description of the drawings, the same or corresponding parts are denoted by the same reference numerals, and redundant description is omitted.

  First, the function and structure of the irrigation control system 10 which concerns on one Embodiment of the deflation degree prediction system of this invention are demonstrated using FIGS. 1-3. An irrigation control system 10 shown in FIG. 1 is a system that predicts the degree of wiping of an object S that is a plant such as a tomato by a numerical value, and controls irrigation on the object based on the numerical value. In the irrigation control system 10, the degree of deflation of the object S is predicted by machine learning.

  The machine learning used in the present embodiment is a process of generating a pattern function by learning training data that is a set of known values and predicting an unknown value using the pattern function. In the present embodiment, training data that is a numerical string is used, and a value at a future time point is predicted using a pattern function obtained from the training data. The numerical sequence is a series of numerical values obtained by various observation values of the phenomenon related to the object S, and is a series of numerical values obtained by observing the phenomenon based on a certain rule. Examples of machine learning include convolutional neural network (CNN), recurrent neural network (RNN), artificial neural network (ANN), support vector machine (SVM), or support vector regression (SVR) corresponding to the SVM for regression, Decision tree learning, association rule learning, Bayesian network, and the like can be mentioned, but the irrigation control system 10 may use other algorithms.

  The target predicted by the irrigation control system 10 includes the diameter of the stem of the object S as the deflation degree corresponding to the water stress of the object S, but is not limited to this. For example, the irrigation control system 10 may predict other numerical values indicating the degree of wilt or water stress, such as the inclination of the stem, the degree of leaf spread, and the color of the leaves.

  As shown in FIG. 1, the irrigation control system 10 is a camera as an image capturing unit that periodically (for example, every 10 minutes) acquires images of a grass shape (appearance) of an object S to be cultivated at a predetermined cycle. 3, a stem diameter sensor 5 that measures the diameter of the stem of the object S, an environment sensor 7 that periodically outputs measurement data related to the surrounding environment of the object S at predetermined intervals (for example, every 10 minutes), An irrigation control device 9 that controls the timing of irrigation or the amount of irrigation to an object by an external control signal, and these cameras 3, stem diameter sensor 5, environment sensor 7, and irrigation control device 9 wirelessly or by wire communication The data processing apparatus 1 is connected via a network N. The camera 3 is installed toward the object S so that an image of the entire appearance of the object S can be acquired. The camera 3 only needs to be installed at a position where it is easy to detect a change in the wrinkle of the object S, and may be a position where an image of only the upper part of the object S can be acquired. You may install in the direction which can image from a direction (horizontal direction), and may install in the direction which can image the target object S from the perpendicular direction (upward direction or downward direction). The stem diameter sensor 5 is attached to the stem of the object S, and repeatedly measures and outputs stem diameter measurement data at a predetermined timing. As such a stem diameter sensor 5, for example, a laser line sensor including a projector and a light receiver is used. However, the stem diameter sensor 5 is not limited to a specific configuration as long as the stem diameter can be measured. Here, the measurement data output from the stalk diameter sensor 5 is used as training data representing an actual measurement value by machine learning in the data processing device 1, but a pattern function has already been generated and held in the data processing device 1. If it is, the stem diameter sensor 5 may be removed or may not be included in the irrigation control system 10. The environment sensor 7 is a sensor device that is installed in the cultivation environment of the object S and can measure a measurement value related to the surrounding environment of the object. For this environmental sensor 7, for example, a sensor device capable of measuring temperature, relative humidity, solar radiation (brightness), photosynthetic effective photon flux density (PPFD), etc. is selected, but some of these can be measured. It may be a simple sensor device, or may be a sensor device capable of measuring measurement values related to other environments. Moreover, it is not limited to installing the environmental sensor 7 in one place, A plurality may be installed in a plurality of places under cultivation environment.

  The irrigation control system 10 is configured such that the image acquired by the camera 3, the measurement data acquired by the stem diameter sensor 5 and the environment sensor 7 can be acquired by the data processing device 1 via the communication network N. The irrigation timing or the amount of irrigation by the irrigation control device 9 is configured to be controllable by a control signal sent from the data processing device 1 via the communication network N. In addition, it replaces with the irrigation control apparatus 9, and the other apparatus which controls the cultivation conditions of the target object S may be provided. For example, an air conditioning control device that controls the temperature and humidity of the cultivation environment may be provided, or a device that controls the supply timing or supply amount of nutrients may be provided.

  When the data processing device 1 includes one or more computers and includes a plurality of computers, each functional element of the data processing device 1 described later is realized by distributed processing. The type of individual computer is not limited. For example, a stationary or portable personal computer (PC) may be used, a workstation may be used, or a portable terminal such as a high-functional portable telephone (smart phone), a portable telephone, or a personal digital assistant (PDA). May be used. Alternatively, the data processing apparatus 1 may be constructed by combining various types of computers. When a plurality of computers are used, these computers are connected via a communication network such as the Internet or an intranet.

  A general hardware configuration of each computer 100 in the data processing apparatus 1 is shown in FIG. A computer 100 includes a CPU (processor) 101 that is an arithmetic device that executes an operating system, application programs, and the like, a main storage unit 102 that includes a ROM and a RAM, and an auxiliary storage unit that includes a hard disk, a flash memory, and the like. 103, a communication control unit 104 including a network card or a wireless communication module, an input device 105 such as a keyboard and a mouse, and an output device 106 such as a display and a printer. Of course, the hardware modules to be mounted differ depending on the type of the computer 100. For example, stationary PCs and workstations often include a keyboard, a mouse, and a monitor as input devices and output devices, but in a smartphone, a touch panel often functions as an input device and an output device.

  Each functional element of the data processing apparatus 1 to be described later reads predetermined software on the CPU 101 or the main storage unit 102, and operates the communication control unit 104, the input device 105, the output device 106, and the like under the control of the CPU 101. This is realized by reading and writing data in the main storage unit 102 or the auxiliary storage unit 103. Data and a database necessary for processing are stored in the main storage unit 102 or the auxiliary storage unit 103.

  As shown in FIG. 3, the data processing apparatus 1 includes, as functional components, an actual measurement value acquisition unit 11, an image acquisition unit 13, a measurement data acquisition unit 15, an image feature quantity calculation unit 17, an environmental feature quantity calculation unit 19, A learning device (predicted value deriving unit) 21, a sugar content predicting unit 23, and an irrigation control unit 25 are provided.

  The actual measurement value acquisition unit 11 acquires measurement data related to the stem diameter output by the stem diameter sensor 5. Then, the actual measurement value acquisition unit 11 converts the measurement data into data of actual measurement values for machine learning in the image feature amount calculation unit 17 and the learning device 21, and converts the actual measurement value data into the image feature amount calculation unit 17 and Delivered to the learning device 21. For example, when the numerical value (objective variable) to be predicted by machine learning in the image feature quantity calculation unit 17 and the learning device 21 is a differential stem diameter (DSD), data of the differential stem diameter is used as the data of the actual measurement value. When the numerical value (objective variable) to be predicted by machine learning is a relative stem diameter (RSD: Relative Stem Diameter), it is converted into relative stem diameter data as measured value data. The differential stem diameter DSD is a value calculated by subtracting the current stem diameter value from the maximum stem diameter value observed in the past. The relative stem diameter RSD is a value calculated by dividing the current stem diameter value by the maximum stem diameter value observed in the past. Both values DSD and RSD are used as evaluation values representing the degree of wilting of the object S.

  The image acquisition unit 13 receives images continuously acquired by the camera 3 as image data. The image acquisition unit 13 delivers the received image data to the image feature amount calculation unit 17.

  The measurement data acquisition unit 15 receives measurement data regarding the surrounding environment continuously acquired by the environment sensor 7 as time-series environment measurement data. The measurement data acquisition unit 15 delivers the received time-series environmental measurement data to the environmental feature quantity calculation unit 19.

  The image feature amount calculation unit 17 calculates a numerical vector (numerical string) that is a feature amount related to the deflation of the object S using machine learning based on the image data. For example, the image feature amount calculation unit 17 uses the image data and the data of the actual measurement value, and uses the difference stem diameter DSD in the future (for example, after a predetermined time from the acquisition time of each image data) as a target variable. Perform machine learning. An example of machine learning is a convolutional neural network (CNN) which is a known deep learning technique. CNN is a technique that can mechanically extract feature values from image data by repeating convolution and pooling. As an example in the case of using CNN, the image feature quantity calculation unit 17 repeats the process in the convolution layer and the pooling layer several times (for example, 5 times), and then performs the process in the output layer through the process in the output layer. The predicted value of the differential stem diameter DSD, which is a variable, is calculated. This predicted value represents the future deflation of the object S. At that time, the image feature amount calculation unit 17 can update the pattern function such as a filter parameter used in the convolution layer by advancing machine learning based on the calculated predicted value and measured value data. Here, after the pattern function is once constructed, the image feature amount calculation unit 17 may stop updating the pattern function using the data of the actual measurement value. When the pattern function is constructed from the beginning, The function of updating the pattern function may not be included. Further, the image feature quantity calculation unit 17 uses the output of the last stage of the hidden layer of the CNN targeted for the image data in the constructed state as a numeric vector (for example, a 256-column numeric vector) that is a feature quantity. Output to the learning device 21.

  The environmental feature value calculation unit 19 calculates a feature value related to the degree of wilting of the object in time series based on time series environment measurement data. For example, the environmental feature quantity calculator 19 calculates two types of saturations VDP and HD from the temperature and relative humidity included in the environmental measurement data. Saturation is an index indicating the amount of water vapor that can be contained in a certain space. Moreover, the environmental feature-value calculation part 19 calculates the value of a leaf area index LAI (Leaf Area Index) from the difference, when there exists the brightness of the upper part and inside of a plant community as environmental measurement data. The leaf area index is an index indicating the amount of leaves of a plant community. Then, the environmental feature value calculation unit 19 outputs the environmental measurement data to which the calculated feature value is added to the learning device 21 in time series.

  The learning device 21 uses the numerical vector calculated by the image feature quantity calculation unit 17 and the time-series environmental measurement data output from the environmental feature quantity calculation unit 19 as a numerical value representing the degree of deflation of the object S. Then, a predicted value of the difference stem diameter DSD is derived. For the derivation of the predicted value, a method of SW-SVR (Sliding Window-based Support Vector Regression) which is an autonomous adaptation type learning device by the present inventors (see Japanese Patent Laid-Open No. 2006-099738), or a known RNN Can be used. SW-SVR is a technique specialized in predicting micro-meteorological data whose characteristics change in a complicated manner with time. An example of using the SW-SVR method will be described. The learning device 21 executes machine learning by the SW-SVR method based on a numerical vector obtained by combining a numerical vector and time-series environmental measurement data. Thus, the transition of the predicted value of the differential stem diameter DSD, which is the objective variable, is calculated. At that time, the learning device 21 updates the pattern function used in the SW-SVR method by executing machine learning based on the measured value data obtained from the measured value acquisition unit 11. Here, after the pattern function is once constructed, the learning device 21 may stop updating the pattern function using the measured value data. If the pattern function has been constructed from the beginning, the pattern function may be stopped. The update function may not be included. Then, the learning device 21 outputs, to the sugar content prediction unit 23, the transition data of the predicted value of the differential stem diameter DSD by machine learning in a state where the pattern function is constructed.

  FIG. 4 shows an image of transition data of the predicted value of the differential stem diameter DSD output by the learning device 21. In this manner, the learning device 21 predicts a change in the stem diameter of the target object that increases or decreases due to a change in the cultivation environment, and enables evaluation of the future deflation of the target object S based on the transition data of the stem diameter.

  The sugar content prediction unit 23 predicts the sugar content of the object S based on the transition data regarding the stem diameter of the object S output from the learning device 21. For example, the sugar content prediction unit 23 calculates a wilting frequency evaluation value representing the frequency of wilting by evaluating the number of minimum values of the transition data of the difference stem diameter DSD within a predetermined period. In addition, the sugar content prediction unit 23 calculates a wilting strength evaluation value representing the degree of wilting by evaluating the integrated value of the transition data of the difference stem diameter DSD within a predetermined period. These wilting frequency evaluation values and wilting strength evaluation values are highly correlated with the sugar content of the future object S. Using the property, the sugar content prediction unit 23 calculates the sugar content of the target object at a predetermined future time point as a predicted value based on the wilting frequency evaluation value and the wilting strength evaluation value. Furthermore, the sugar content prediction unit 23 delivers the predicted value of the calculated sugar content to the irrigation control unit 25.

  The irrigation control unit 25, based on the predicted value of the sugar content of the object S calculated by the sugar content prediction unit 23, the timing of irrigation to the object S or the amount of irrigation so that the sugar content of the object S approaches the target value A control signal for controlling the signal is generated. And the irrigation control part 25 transmits the produced | generated control signal toward the irrigation control apparatus 9. FIG. Thereby, the sugar content in the harvest time can be controlled to a constant value by controlling the water stress of the object S using irrigation control.

  Hereinafter, with reference to FIG. 5, the procedure of the irrigation control by the data processing device 1 described above will be described, and the procedure of the wilting condition prediction method according to the present embodiment will be described in detail. FIG. 5 is a flowchart showing a processing procedure of irrigation control by the data processing device 1.

  First, when the irrigation control process by the data processing device 1 is started in response to an instruction input by the user, the actual value data is acquired by the actual value acquisition unit 11, the image data is acquired by the image acquisition unit 13, and the measurement is performed. Acquisition of time-series environmental measurement data by the data acquisition unit 15 is started (step S01). In addition, when the construction of the pattern function in the image feature quantity calculation unit 17 and the learning device 21 has already been completed, the process of acquiring the data of the actual measurement values may be omitted.

  Thereafter, machine learning using the image data is executed by the image feature quantity calculation unit 17, thereby generating a numeric vector as a feature quantity representing the degree of deflation of the object S, and the generated numeric vector is used as a learning device. 21 (step S02). At the same time, the environmental feature quantity calculation unit 19 calculates two types of feature quantities such as saturations VDP and HD based on the environmental measurement data, and the environmental measurement data to which these feature quantities are added is learned in time series. 21 (step S03).

  In response to this, machine learning using numerical vectors and time-series environmental measurement data is executed by the learning device 21, so that transition data of predicted values related to the stem diameter of the object S is generated. It is output to the sugar content prediction unit 23 (step S04). Next, the sugar content prediction unit 23 predicts the sugar content of the object based on the transition data (step S05). Finally, the irrigation control unit 25 generates a control signal for controlling irrigation to the object S based on the predicted sugar content, and transmits the control signal to the irrigation control device 9 (step S06). ). The irrigation control by the data processing apparatus 1 as described above may be activated each time according to a user's instruction input, or may be automatically activated at a predetermined timing (for example, periodically).

  According to the irrigation control system 10 described above, based on the grass-like image of the object S acquired by the camera 3, a numerical vector, which is a feature amount, is calculated by machine learning. By machine learning using the acquired measurement data, a predicted value of a numerical value related to the stem diameter representing the degree of deflation of the object S is derived. Thereby, the system which estimates the deflation degree of the target object S by simple operation using the simple system containing the camera 3 and the environment sensor 7 is realizable. In addition, since the predicted value is derived using the measurement data acquired by the environmental sensor 7 together with the image of the grass shape of the object S, it is possible to predict the degree of plant wilting according to the cultivation environment with high accuracy. .

  Here, since the predicted value of the numerical value regarding the diameter of the stalk of a target object is derived | led-out in the irrigation control system 10, the degree of deflation of the target object S can be evaluated appropriately.

  Furthermore, according to the irrigation control system 10, the sugar content of the object S is also predicted based on the predicted value related to the stem diameter. Thereby, the future sugar content of the target object S can be evaluated appropriately.

  Moreover, in the irrigation control system 10, the predicted value regarding the stem diameter of the target object S is derived | led-out using the environmental measurement data regarding temperature, humidity, and brightness. Thereby, the wilting degree of the plant according to the cultivation environment can be predicted with higher accuracy.

  In addition, this invention is not limited to the aspect of embodiment mentioned above.

  For example, the measurement data acquisition unit 15 acquires time-series environment measurement data, and the learning device 21 derives a predicted value indicating the degree of deflation of the object S using the time-series environment measurement data. This is not a limitation. That is, the measurement data acquisition unit 15 acquires the environmental measurement data together with time data indicating the time when the data was acquired, and the learning device 21 uses the environmental measurement data including the time data to check the degree of deflation of the object S. The predicted value shown may be derived.

  DESCRIPTION OF SYMBOLS 1 ... Data processing device, 3 ... Camera, 5 ... Stem diameter sensor, 7 ... Environmental sensor, 10 ... Irrigation control system (deflection condition prediction system), 17 ... Image feature-value calculation part, 21 ... Learner (predicted value derivation part) ), 23 ... sugar content prediction unit, S ... object.

Claims (6)

An imaging unit that acquires an image related to the appearance of an object that is a plant;
An environmental sensor that outputs measurement data relating to the surrounding environment of the object;
Based on the image acquired from the imaging unit, an image feature amount calculation unit that calculates a numerical sequence that is a feature amount related to the degree of wilting of the object using machine learning;
Prediction for deriving a predicted value of a numerical value representing the degree of deflation of the object using machine learning based on the numerical value sequence calculated by the image feature amount calculation unit and the measurement data output from the environmental sensor A value derivation unit;
A twilight condition prediction system comprising: The predicted value derivation unit derives a numerical predicted value related to the diameter of the stem of the object as the predicted value.
The wilting degree prediction system according to claim 1. A sugar content prediction unit for predicting the sugar content of the object based on the predicted value;
The withering degree prediction system according to claim 1 or 2. The environmental sensor outputs data on temperature and humidity as the measurement data,
The predicted value deriving unit derives the predicted value based on data on temperature and humidity.
The wilt condition prediction system according to any one of claims 1 to 3. The environmental sensor outputs data relating to brightness as the measurement data,
The predicted value deriving unit derives the predicted value based on data on brightness;
The wilt condition prediction system according to any one of claims 1 to 4. Using the imaging unit, obtain an image related to the appearance of the object that is a plant,
Using environmental sensors, obtain measurement data related to the surrounding environment of the object,
Based on the image acquired from the image capturing unit, the image feature amount calculating unit calculates a numerical sequence that is a feature amount related to the degree of wilting of the object using machine learning,
Based on the numerical value sequence calculated by the image feature value calculating unit and the measurement data acquired by the environmental sensor by a predicted value deriving unit, a numerical value representing the degree of deflation of the object using machine learning. To derive the predicted value,
A method for predicting the degree of wilting.

Images