JP7473951B2 - Information processing device, information processing system, information processing method, and computer program - Google Patents

Description

The technology disclosed in this specification relates to an information processing device that estimates the growth rate of a plant under cultivation.

In recent years, various technologies have been proposed to improve the efficiency and optimization of plant cultivation (leafy vegetables, fruit trees, ornamental plants, etc.). For example, a technology is known in which, in fruit cultivation carried out by dedicated farmers, the size of the fruit and the integrated value of the leaf area are calculated from images of the fruit trees, a growth curve is determined that approximates the growth state of the fruit based on the integrated value of the fruit size and leaf area, the growth curve is used to evaluate the growth state of the cultivated fruit, and the harvest date of the fruit is predicted based on the evaluation results (see, for example, Patent Document 1).

JP 2019-187259 A

The above conventional technology has the problem that the process for evaluating the growth status of cultivated plants is complicated.

This specification discloses a technology that can solve the above-mentioned problems.

The technology disclosed in this specification can be realized, for example, in the following forms:

(1) The information processing device disclosed in this specification includes a target data acquisition unit that acquires target image data, which is image data generated by photographing a target plant that is a specific type of plant being cultivated, and a growth rate estimation unit that estimates a growth rate, which is a ratio of the growth state of the target plant represented by the image data to the growth state of the target plant at the time of harvesting, using image data including color information of the target image data and a model created by a predetermined machine learning. In this way, the information processing device acquires target image data generated by photographing a target plant being cultivated, and can estimate the growth rate of the target plant using image data including color information of the target image data and a model created by machine learning. Therefore, according to the information processing device, the growth rate of the target plant can be estimated simply by acquiring the target image data, and plant cultivation can be made more efficient and optimized. Furthermore, according to the information processing device, the growth rate of the target plant can be estimated simply by acquiring the target image data, and therefore the process for estimating the growth rate of the target plant can be prevented from becoming complicated while suppressing a decrease in the accuracy of the estimation of the growth rate.

(2) The information processing device may further include a harvest date and time prediction unit that predicts a harvest date and time of the target plant based on the growth rate estimated for the target image data. According to this information processing device, it is possible to predict a harvest date and time of the target plant simply by acquiring the target image data, and it is possible to effectively realize efficiency and optimization of plant cultivation.

(3) In the above information processing device, the target data acquisition unit may acquire a plurality of sets of target image data generated by photographing the target plant at a plurality of dates and times, the growth rate estimation unit may estimate the growth rate for the plurality of sets of target image data, and the harvest date and time prediction unit may predict the harvest date and time of the target plant for one set of target image data based on the growth rate estimated for the one set of target image data and the growth rates estimated for each set of target image data photographed before the photographing date and time of the one set of target image data. According to this information processing device, the harvest date and time of the target plant can be predicted with high accuracy, and the efficiency and optimization of plant cultivation can be more effectively achieved.

(4) The information processing device may further include a training data acquisition unit that acquires training data that associates multiple training image data, which is image data generated by photographing the specific type of plant at multiple dates and times within the period from planting to harvesting, with growth rate data that indicates the growth rate of the plant represented by the training image data, and a model acquisition unit that acquires the model by the machine learning using the training data. According to this information processing device, it is possible to acquire a model for estimating the growth rate of a plant without using another device, and to estimate the growth rate of a target plant using the model.

(5) In the above information processing device, the growth rate data may be data that indicates the ratio of the time elapsed from planting to the date and time of capture of each of the training image data to the time required from planting to harvesting of the specific type of plant as the growth rate. According to this information processing device, the growth rate of a plant can be easily and uniformly specified, and the efficiency and optimization of plant cultivation can be more effectively achieved.

(6) The information processing system disclosed in this specification is an information processing system including the information processing device and a terminal device capable of communicating with the information processing device via a communication network, and the information processing device further includes an information transmission unit that transmits information indicating the estimated growth rate of the target plant to the terminal device via the communication network. According to this information processing system, the information transmission unit of the information processing device transmits information indicating the estimated growth rate of the target plant to the terminal device, so that a farm worker holding the terminal device can grasp the growth rate of the target plant even if he or she is in a location other than the field, and the efficiency and optimization of plant cultivation can be more effectively achieved.

(7) The above information processing system may further include a photographing device that is installed in the field where the target plant is cultivated and is capable of communicating with the information processing device via a communication network, and the target data acquisition unit may be configured to acquire the target image data generated by the photographing device photographing the target plant via the communication network. According to this information processing system, the photographer (e.g., a farm worker) can acquire the target image data without going to the field, and the acquired target image data can be used to estimate the growth rate of the target plant, thereby making it possible to extremely effectively realize efficiency and optimization of plant cultivation.

The technology disclosed in this specification can be realized in various forms, such as an information processing device, an information processing system, an information processing method, a computer program that realizes these methods, a non-transitory recording medium on which the computer program is recorded, etc.

FIG. 1 is an explanatory diagram showing a schematic configuration of a plant cultivation management system 10 according to an embodiment of the present invention. A block diagram showing a schematic configuration of a plant cultivation management server 100. A flowchart showing the contents of a growth rate estimation model creation process in this embodiment. FIG. 1 is an explanatory diagram conceptually illustrating an example of training data TD. FIG. 1 is an explanatory diagram conceptually illustrating an example of a growth rate estimation model MO. A flowchart showing the contents of a plant cultivation support process according to the present embodiment. FIG. 2 is an explanatory diagram showing an example of a screen displayed on the display unit 210 of the terminal device 200 during the plant cultivation support process in the present embodiment. FIG. 1 is an explanatory diagram showing an example of a method for predicting a harvest date and time Te based on a growth rate G in this embodiment. FIG. 2 is an explanatory diagram showing an example of a screen displayed on the display unit 210 of the terminal device 200 during the plant cultivation support process in the present embodiment. An explanatory diagram showing the cultivation results of the plants used in this embodiment. FIG. 1 is an explanatory diagram showing image data used in this embodiment. FIG. 1 is an explanatory diagram showing the transition of the loss function for each epoch in this embodiment. FIG. 13 is an explanatory diagram showing the prediction accuracy of the growth rate G in this embodiment. FIG. 13 is an explanatory diagram showing the prediction accuracy of the growth rate G in this embodiment. FIG. 13 is an explanatory diagram showing the prediction accuracy of the growth rate G in this embodiment.

A. Embodiments:
A-1. Configuration of the plant cultivation management system 10:
FIG. 1 is an explanatory diagram showing a schematic configuration of a plant cultivation management system 10 in this embodiment. The plant cultivation management system 10 is a system for managing the cultivation of plants in a field, and more specifically, it is a system that uses image data of the plant under cultivation to grasp the growth state of the plant and the predicted harvest date and time, thereby improving the efficiency and optimization of plant cultivation. In this embodiment, the field is a semi-open type plant factory PF. Here, the semi-open type plant factory PF is a field surrounded by a material that does not block sunlight, such as a vinyl greenhouse. However, the plant cultivation management system 10 can also be applied to other types of fields (for example, other types of plant factories such as closed type and open type, fields for outdoor cultivation, etc.). Note that only one plant factory PF is shown in FIG. 1, but the plant cultivation management system 10 can also be applied to the management of plant cultivation in each of multiple plant factories PF. In addition, the plant to be cultivated may be any type of plant, such as leafy vegetables (for example, spinach), fruit trees (for example, tomatoes), and ornamental plants. The plant cultivation management system 10 corresponds to an information processing system in the claims.

The plant cultivation management system 10 comprises a plant cultivation management server 100, a terminal device 200 used by a farm worker P1, and a photography device 300 installed in the plant factory PF. Each device constituting the plant cultivation management system 10 is connected to each other so as to be able to communicate with each other via a communication network NET.

The photographing device 300 is installed in the plant factory PF and is a device that photographs the plants cultivated in the plant factory PF to generate image data (e.g., RGB still image data) of the plants. In this embodiment, the photographing device 300 photographs a preset range (including the cultivated plants) in the plant factory PF periodically (e.g., every hour). The photographing device 300 also has a communication interface (not shown), and transmits image data generated by photographing to the plant cultivation management server 100 via the communication network NET, for example, each time photographing is performed. Note that multiple photographing devices 300 that photograph different ranges as subjects may be installed in one plant factory PF. In addition, when there are multiple plant factories PFs that are managed by the plant cultivation management system 10, a photographing device 300 is installed in each plant factory PF.

The terminal device 200 is, for example, a smartphone or a tablet terminal. The terminal device 200 has a display unit 210 configured, for example, by a liquid crystal display or the like (see Figures 7 and 9). In addition, an application program for plant cultivation management is installed in the terminal device 200. As will be described in more detail below, a user of the terminal device 200 (such as farm worker P1) can use the application program to grasp the growth status of plants being cultivated in the plant factory PF and the predicted harvest date and time.

The plant cultivation management server 100 is a server device for managing the cultivation of plants in the plant factory PF. FIG. 2 is a block diagram showing a schematic configuration of the plant cultivation management server 100. The plant cultivation management server 100 includes a control unit 110, a memory unit 120, a display unit 130, an operation input unit 140, and an interface unit 150. These units are connected to each other via a bus 190 so as to be able to communicate with each other. The plant cultivation management server 100 corresponds to an information processing device in the claims.

The display unit 130 of the plant cultivation management server 100 is, for example, a liquid crystal display, and displays various images and information. The operation input unit 140 is, for example, a keyboard, mouse, buttons, microphone, and the like, and accepts operations and instructions from the administrator. The display unit 130 may be equipped with a touch panel so as to function as the operation input unit 140. The interface unit 150 is, for example, a LAN interface, a USB interface, and the like, and communicates with other devices via wired or wireless connection.

The storage unit 120 of the plant cultivation management server 100 is composed of, for example, a ROM, a RAM, a hard disk drive (HDD), etc., and is used to store various programs and data, and as a work area when executing various programs, and as a temporary storage area for data. For example, the storage unit 120 stores a plant cultivation management program CP, which is a computer program for executing the growth rate estimation model creation process and the plant cultivation support process described below. The plant cultivation management program CP is provided in a state stored in a computer-readable recording medium (not shown), such as a CD-ROM, DVD-ROM, or USB memory, and is stored in the storage unit 120 by installing it in the plant cultivation management server 100.

In addition, the storage unit 120 of the plant cultivation management server 100 stores training data TD, a growth rate estimation model MO, target image data I(o), estimated growth rate data GD, and predicted harvest date and time data PD during the growth rate estimation model creation process and plant cultivation support process described below. The contents of this information will be explained in conjunction with the growth rate estimation model creation process and plant cultivation support process described below.

The control unit 110 of the plant cultivation management server 100 is configured, for example, by a CPU, and controls the operation of the plant cultivation management server 100 by executing a computer program read from the storage unit 120. For example, the control unit 110 functions as a plant cultivation management unit 111 that executes a growth rate estimation model creation process and a plant cultivation support process, which will be described later, by reading and executing a plant cultivation management program CP from the storage unit 120. The plant cultivation management unit 111 includes a training data acquisition unit 112, a model acquisition unit 114, a target data acquisition unit 115, a growth rate estimation unit 116, a harvest date and time prediction unit 117, and an information transmission unit 118. The functions of each of these units will be described in conjunction with the growth rate estimation model creation process and the plant cultivation support process, which will be described later.

A-2. Growth rate estimation model creation process:
Next, a growth rate estimation model creation process executed by the plant cultivation management server 100 of this embodiment will be described. Fig. 3 is a flowchart showing the contents of the growth rate estimation model creation process in this embodiment. The growth rate estimation model creation process is a process for creating a growth rate estimation model MO for estimating the growth rate G of a plant represented by image data generated by photographing a specific type of plant (e.g., spinach) using a predetermined machine learning. The growth rate estimation model creation process is started in response to an administrator operating the operation input unit 140 of the plant cultivation management server 100 to input a start instruction.

Here, the growth rate G of a plant means the ratio of the growth state of the plant at the present time to the growth state of the plant at the (expected) time of harvest. In this embodiment, the growth rate G of a plant at a certain date and time Ti (for example, the date and time of photographing the plant) is expressed as the ratio of the elapsed time from the planting of the plant to the date and time Ti to the total time required for cultivating the plant (the time required from planting (the start of cultivation) to harvest (the end of cultivation)). In other words, the growth rate G is expressed by the following formula (1).
G = (Ti - Ts) / (Te - Ts) ... (1)
however,
Ts: planting date and time Te: harvest date and time of the plant Ti: date and time for which the growth rate G is to be determined

In the growth rate estimation model creation process, first, the training data acquisition unit 112 (FIG. 2) of the plant cultivation management server 100 acquires training data TD (S110). The training data TD is data in which multiple training image data I(t) generated by photographing a specific type of plant (e.g., spinach) cultivated in the plant factory PF at multiple dates and times during the period from planting to harvesting are associated with growth rate data indicating the growth rate G of the plant represented by each training image data I(t) (i.e., the growth rate G at the date and time of photography). The acquired training data TD is stored in the memory unit 120 of the plant cultivation management server 100.

Figure 4 is an explanatory diagram conceptually showing an example of training data TD. As described above, the photographing device 300 installed in the plant factory PF takes photographs periodically (for example, every hour), and transmits image data generated by photographing each time to the plant cultivation management server 100 via the communication network NET. In this embodiment, the photographing device 300 also takes photographs and transmits image data before planting and after harvesting the plants. The training data acquisition unit 112 of the plant cultivation management server 100 acquires the image data transmitted from the photographing device 300 via the interface unit 150 and stores it in the memory unit 120. The training data acquisition unit 112 may perform a predetermined process, such as trimming, on the image data transmitted from the photographing device 300 before storing it in the memory unit 120.

The training data acquisition unit 112 selects a series of multiple image data (i.e., image data for one crop) generated by photographing a specific type of plant at multiple dates and times within the period from the planting date and time Ts to the harvest date and time Te from among the multiple image data stored in the memory unit 120. In the following, the image data photographed and generated at the date and time Ti is represented as image data Ii. The image data photographed at the planting date and time Ts is image data Is, and the image data photographed at the harvest date and time Te is image data Ie. The training data acquisition unit 112 identifies the image data Is photographed at the planting date and time Ts and the image data Ie photographed at the harvest date and time Te from among the multiple image data stored in the memory unit 120 by a known image recognition process. In addition, the training data acquisition unit 112 selects image data I whose photographing date and time is within the period from the planting date and time Ts to the harvest date and time Te from among the multiple image data stored in the memory unit 120. However, this embodiment is targeted at a semi-open plant factory PF where cultivation is performed using sunlight, and image data I taken outside of relatively bright hours (e.g., 7:00 to 17:00) is excluded because there is a risk that the plants will not be clearly visible in the image data I taken during such hours. In this way, the training data acquisition unit 112 selects multiple image data Ii (Is, I1, I2, ..., Ie) taken and generated at multiple dates and times Ti (Ts, T1, T2, ..., Te) during the period from planting date and time Ts to harvest date and time Te as training image data I(t).

The training data acquisition unit 112 also calculates the growth rate Gi (Gs, G1, G2, ..., Ge) as a label for each of the selected multiple image data Ii (i.e., image data for one crop) according to the above formula (1), and creates training data TD as labeled data by associating the calculated growth rate Gi with the image data Ii. For example, as shown in FIG. 4, for image data Im taken at a certain date and time Tm, if the elapsed time from planting date and time Ts to the date and time Tm is 20% of the total time required for cultivation (the time from planting date and time Ts to harvest date and time Te), the growth rate Gm is calculated to be "0.2". Therefore, data in which the value of 0.2 is associated with the image data Im as the growth rate Gm is created as the training data TD. The same applies to the other image data Ii. Note that the growth rate Gs at the planting date and time Ts is 0 (zero), and the growth rate Ge at the harvest date and time Te is 1.0.

The training data acquisition unit 112 may acquire training data TD for one crop of a specific type of plant, but it is preferable to acquire training data TD for multiple crops of a specific type of plant in order to improve the accuracy of the growth rate estimation model MO. Furthermore, when multiple image capture devices 300 are installed in one plant factory PF, the training data acquisition unit 112 may acquire training data TD based on image data acquired from the multiple image capture devices 300. Furthermore, the training data acquisition unit 112 may acquire training data TD based on image data acquired from the image capture devices 300 installed in each of the multiple plant factories PF, as long as the type of plant being cultivated is the above-mentioned specific type.

Next, the model acquisition unit 114 (FIG. 2) of the plant cultivation management server 100 creates a growth rate estimation model MO by a predetermined machine learning using the training data TD (S120). FIG. 5 is an explanatory diagram conceptually showing an example of the growth rate estimation model MO. As shown in FIG. 5, the growth rate estimation model MO is a model for estimating the growth rate G of a plant represented by image data I (i.e., the growth rate G at the shooting date and time) from image data including color information (e.g., RGB values) of the image data I generated by shooting a specific type of plant. The model acquisition unit 114 creates the growth rate estimation model MO by performing a known machine learning using at least one of the root mean square error (RMSE), mean absolute error (MAE), and coefficient of determination (R ² ) as an evaluation index, using the training image data I(t) included in the training data TD as input data and the growth rate G associated with each training image data I(t) as an objective variable. The created growth rate estimation model MO is stored in the storage unit 120 of the plant cultivation management server 100. In addition, various known machine learning algorithms such as multiple regression analysis, Random Forest, XGBoost, and convolutional neural network (CNN) can be used to create the growth rate estimation model MO. The growth rate estimation model MO corresponds to the model in the claims. The above steps complete the growth rate estimation model creation process for creating the growth rate estimation model MO.

A-3. Plant cultivation support treatment:
Next, a plant cultivation support process executed by the plant cultivation management server 100 of this embodiment will be described. Fig. 6 is a flowchart showing the contents of the plant cultivation support process in this embodiment. Figs. 7 and 9 are explanatory diagrams showing an example of a screen displayed on the display unit 210 of the terminal device 200 during the plant cultivation support process in this embodiment, and Fig. 8 is an explanatory diagram showing an example of a method for predicting a harvest date and time Te based on a growth rate G in this embodiment.

The plant cultivation support process is a process for supporting the cultivation of a specific type of plant (target plant) by the farm worker P1. More specifically, the plant cultivation support process is a process for estimating the growth rate G of the target plant represented by the target image data I(o) (i.e., the growth rate G at the shooting date and time of the target image data I(o)) using image data including color information of the target image data I(o), which is image data I generated by photographing the target plant under cultivation, and the above-mentioned growth rate estimation model MO, predicting the harvest date and time Te of the target plant based on the estimated growth rate G, and displaying information indicating the growth rate G and the harvest date and time Te on the terminal device 200 of the farm worker P1. The plant cultivation support process is started in response to input of a start instruction by the farm worker P1 operating the terminal device 200 or the administrator of the plant cultivation management server 100 operating the operation input unit 140.

In the plant cultivation support process, first, the target data acquisition unit 115 (FIG. 2) of the plant cultivation management server 100 acquires the target image data I(o), which is image data I generated by photographing the target plant under cultivation (S210). As described above, the photographing device 300 installed in the plant factory PF takes photographs periodically (for example, every hour), and transmits the image data I generated by photographing each time to the plant cultivation management server 100 via the communication network NET. The target data acquisition unit 115 of the plant cultivation management server 100 acquires the image data I transmitted from the photographing device 300 via the interface unit 150 and stores it in the storage unit 120 as the target image data I(o). Note that the target data acquisition unit 115 may perform a predetermined process (for example, trimming) on the image data I transmitted from the photographing device 300 so that it is in a suitable format for input to the growth rate estimation model MO, and then store it in the storage unit 120 as the target image data I(o).

Next, the growth rate estimation unit 116 (FIG. 2) of the plant cultivation management server 100 estimates the growth rate G of the target plant represented by the target image data I(o) (i.e., the growth rate G at the shooting date and time of the target image data I(o)) using the target image data I(o) acquired in the immediately preceding S210 and the above-mentioned growth rate estimation model MO (S220). As shown in FIG. 5, the growth rate estimation unit 116 inputs image data including color information of the target image data I(o) into the growth rate estimation model MO, and estimates the growth rate G by acquiring the growth rate G output from the growth rate estimation model MO. For example, in the example shown in FIG. 5, the growth rate G of the target plant is estimated to be "0.5". The growth rate estimation unit 116 generates estimated growth rate data GD, which is information indicating the estimated growth rate G, and stores it in the memory unit 120 of the plant cultivation management server 100.

Next, the plant cultivation management unit 111 (FIG. 2) of the plant cultivation management server 100 judges whether the number N of the target image data I(o) for which the growth rate G has been estimated is equal to or greater than a predetermined threshold Th (e.g., 30) (S230). In the judgment of S230, if it is judged that the number N of the target image data I(o) for which the growth rate G has been estimated is less than the threshold Th (S230: NO), the prediction of the harvest date and time Te (described later) is not executed because it is difficult to formulate. In this case, the information transmission unit 118 (FIG. 2) of the plant cultivation management server 100 transmits the target image data I(o) acquired in the immediately preceding S210 and the estimated growth rate data GD indicating the growth rate G estimated in the immediately preceding S220 to the terminal device 200 of the farmer P1 via the interface unit 150 and the communication network NET (S250). As a result, as shown in FIG. 7, the display unit 210 of the terminal device 200 displays a current photo (more precisely, the time when the target image data I(o) was captured) and an estimated growth rate G of the target plant (spinach in the example of FIG. 7) being cultivated in the plant factory PF. This allows the farm worker P1 to ascertain the current growth rate G of the target plant without having to go to the plant factory PF. Note that, as described above, if it is determined in the determination of S230 that the number N of target image data I(o) for which the growth rate G has been estimated is less than the threshold value Th, the prediction of the harvest date and time Te (S240) has not been performed, and therefore the symbol "-" indicating unpredictability is displayed in the column for the predicted harvest date and time Te.

After the process of S250, the plant cultivation management unit 111 (FIG. 2) of the plant cultivation management server 100 judges whether the cultivation has ended (whether the plant has been harvested) (S270). If it is judged in the judgment of S270 that the cultivation has not yet ended (S270: NO), the process from S210 onwards is repeatedly executed.

The process from S210 onwards is repeatedly executed, and if it is determined in S230 that the number N of the target image data I(o) for which the growth rate G has been estimated is equal to or greater than the threshold value Th (S230: YES), the harvest date and time prediction unit 117 (FIG. 2) of the plant cultivation management server 100 predicts the harvest date and time Te of the target plant based on the growth rate G estimated for each target image data I(o) (S240). As shown in FIG. 8, in this embodiment, the growth rate Gi is used as the objective variable, and the elapsed time ΔTi (=Ti-Ts) from the planting date and time Ts is used as the explanatory variable to set up a regression equation by simple regression analysis, and the elapsed time ΔTi at which the growth rate Gi becomes 1.0 in the regression equation is calculated backward to predict the harvest date and time Te. For example, in FIG. 8, at the time when the estimation of the growth rate G for the target image data I(o) taken at the date and time Tk is completed, (k+1) points (ΔTi, Gi) showing the data for which the growth rate G has been estimated so far are plotted. The harvest date and time prediction unit 117 uses these (k+1) pieces of data to derive the simple regression equation RL: Gi = a x ΔTi + b. The harvest date and time prediction unit 117 then calculates the value of ΔTi (= ΔTe) at which Gi = 1.0 in this simple regression equation RL, and calculates the date and time (= Ts + ΔTe) at which this elapsed time ΔTe has elapsed since the planting date and time Ts as the predicted harvest date and time Te. The harvest date and time prediction unit 117 generates predicted harvest date and time data PD, which is information indicating the predicted harvest date and time Te, and stores it in the memory unit 120 of the plant cultivation management server 100.

When the harvest date and time Te is predicted (S240), the information transmission unit 118 (FIG. 2) of the plant cultivation management server 100 transmits the target image data I(o) acquired in the immediately preceding S210, the estimated growth rate data GD indicating the growth rate G estimated in the immediately preceding S220, and the predicted harvest date and time data PD indicating the harvest date and time Te predicted in the immediately preceding S240 to the terminal device 200 of the farmer P1 via the interface unit 150 and the communication network NET (S260). As a result, as shown in FIG. 9, the display unit 210 of the terminal device 200 displays a photo of the target plant (spinach in the example of FIG. 9) at the current time (more precisely, at the time of taking the target image data I(o)), the estimated growth rate G, and the predicted harvest date and time Te for the target plant (spinach in the example of FIG. 9) being cultivated in the plant factory PF. This allows the farmer P1 to grasp the current growth rate G of the target plant and the predicted harvest date and time Te without going to the plant factory PF.

After the process of S260, the plant cultivation management unit 111 (FIG. 2) of the plant cultivation management server 100 judges whether cultivation has ended (whether the plant has been harvested) (S270). If it is judged in the judgment of S270 that cultivation has not yet ended (S270: NO), the process from S210 onwards is repeatedly executed. The above-mentioned process is repeatedly executed, and if it is judged in the judgment of S270 that cultivation has ended (S270: YES), the plant cultivation support process is terminated.

A-4. Advantages of this embodiment:
As described above, the plant cultivation management server 100 of this embodiment includes a target data acquisition unit 115 and a growth rate estimation unit 116. The target data acquisition unit 115 acquires target image data I(o), which is image data generated by photographing a target plant, which is a specific type of plant being cultivated. The growth rate estimation unit 116 estimates the growth rate G of the target objects using image data including color information of the target image data I(o) and a growth rate estimation model MO created by a predetermined machine learning. Here, the growth rate G is the ratio of the growth state of the target plant represented by the target image data I(o) to the growth state of the target plant at the time of harvest.

In this way, the plant cultivation management server 100 of this embodiment can acquire target image data I(o) generated by photographing the target plant under cultivation, and estimate the growth rate G of the target plant using image data including color information of the target image data I(o) and the growth rate estimation model MO. Therefore, according to the plant cultivation management server 100 of this embodiment, the growth rate G of the target plant can be estimated simply by acquiring the target image data I(o), and the efficiency and optimization of plant cultivation can be realized. In addition, according to the plant cultivation management server 100 of this embodiment, the growth rate G of the target plant can be estimated simply by acquiring the target image data I(o), so that the accuracy of the estimation of the growth rate G can be suppressed from decreasing, while the process for estimating the growth rate G of the target plant can be suppressed from becoming complicated.

The plant cultivation management server 100 of this embodiment further includes a harvest date and time prediction unit 117. The harvest date and time prediction unit 117 predicts the harvest date and time Te of the target plant based on the growth rate G estimated for the target image data I(o). Therefore, according to the plant cultivation management server 100 of this embodiment, the harvest date and time Te of the target plant can be predicted simply by acquiring the target image data I(o), and the efficiency and optimization of plant cultivation can be effectively achieved.

In the plant cultivation management server 100 of this embodiment, the target data acquisition unit 115 acquires multiple target image data I(o) generated by photographing the target plant at multiple dates and times, and the growth rate estimation unit 116 estimates the growth rate G for the multiple target image data I(o). The harvest date and time prediction unit 117 predicts the harvest date and time Te of the target plant for one target image data I(o) based on the growth rate G estimated for the one target image data I(o) and the growth rate G estimated for each target image data I(o) photographed before the photographing date and time of the one target image data I(o). Therefore, according to the plant cultivation management server 100 of this embodiment, the harvest date and time Te of the target plant can be predicted with high accuracy, and the efficiency and optimization of plant cultivation can be more effectively realized.

The plant cultivation management server 100 of this embodiment further includes a training data acquisition unit 112 that acquires training data TD that associates multiple training image data I(t), which is image data generated by photographing a specific type of plant at multiple dates and times within the period from planting to harvest, with growth rate data indicating the growth rate G of the plant represented by the training image data I(t), and a model acquisition unit 114 that acquires a growth rate estimation model MO by machine learning using the training data TD. Therefore, according to the plant cultivation management server 100 of this embodiment, it is possible to acquire the growth rate estimation model MO without using any other device, and to estimate the growth rate G using the growth rate estimation model MO.

In addition, in the plant cultivation management server 100 of this embodiment, the growth rate data included in the training data TD is data that indicates, as a growth rate G, the ratio of the elapsed time from planting to the date and time of capture of each training image data I(t) to the time required from planting to harvesting of a specific type of plant. Therefore, according to the plant cultivation management server 100 of this embodiment, the growth rate G of a plant can be easily and uniformly determined, and the efficiency and optimization of plant cultivation can be more effectively achieved.

The plant cultivation management system 10 of this embodiment also includes a plant cultivation management server 100 and a terminal device 200 capable of communicating with the plant cultivation management server 100 via a communication network NET. The plant cultivation management server 100 also includes an information transmission unit 118 that transmits estimated growth rate data GD indicating the estimated growth rate G of the target plant to the terminal device 200 via the communication network NET. In this way, in the plant cultivation management system 10 of this embodiment, the information transmission unit 118 of the plant cultivation management server 100 transmits the estimated growth rate data GD indicating the estimated growth rate G of the target plant to the terminal device 200, so that the farm worker P1 holding the terminal device 200 can grasp the growth rate G of the target plant even if he or she is at a location other than the plant factory PF, and the efficiency and optimization of plant cultivation can be more effectively achieved.

The plant cultivation management system 10 of this embodiment further includes a photographing device 300 that is installed in a field (e.g., a plant factory PF) where the target plant is cultivated and can communicate with the plant cultivation management system 10 via a communication network NET. The target data acquisition unit 115 of the plant cultivation management system 10 acquires target image data I(o), which is image data I generated by the photographing device 300 photographing the target plant, via the communication network NET. Therefore, in the plant cultivation management system 10 of this embodiment, the photographer (e.g., farm worker P1) can acquire the target image data I(o) without going to the field, and the growth rate G of the target plant can be estimated using the acquired target image data I(o), thereby making it possible to achieve extremely effective efficiency and optimization of plant cultivation.

A-5. Example:
An example performed to verify the accuracy of the estimation of the growth rate G by the plant cultivation management server 100 of the present embodiment described above will be described below. FIG. 10 is an explanatory diagram showing the cultivation results of the plant used in this example. In this example, data from six spinach crops were used in a semi-open plant factory PF (vinyl house). FIG. 10 shows the planting date and time Ts and the harvest date and time Te for each crop. A photographing device 300 was installed at the top of the plant factory PF, photographed every hour, and image data I obtained by photographing was used. However, only image data I photographed between 7:00 and 17:00 was used, and image data I photographed during other times was excluded. In addition, there were times when image data I could not be obtained due to a system error. In addition, a region of 224×224 pixels was cut out by trimming for each image data I.

In this embodiment, a model based on a convolutional neural network (CNN) was used as an estimation model for the growth rate G. More specifically, as transfer learning using ImageNet, a large-scale trained network, the output layer of ImageNet was partially changed and retrained, and a model based on a CNN (VGG16) with 13 convolutional layers and 2 fully connected layers was used. The parameters during learning were as follows.
Loss function: Root mean square error (RMSE)
Activation function: ReLU
Optimization function: Adam
Learning rate: 0.001 (if the loss does not decrease for three consecutive epochs, the learning rate is reduced to 1/10.)
Batch size: 32
Number of epochs: 50
Condition for ending learning: If the loss does not decrease for five consecutive epochs, learning ends without waiting for the maximum number of epochs, 50.

Figure 11 is an explanatory diagram showing the image data used in this example. As shown in Figure 11, in this example, of the image data for all six works (1,064 images), the image data for the sixth work (220 images) was used as test image data. In addition, the image data for the remaining six works (844 images) was shuffled, and 80% of the image data (675 images) was used as training image data, and the remaining 20% (169 images) was used as verification image data.

Figure 12 is an explanatory diagram showing the progress of the loss function for each epoch in this embodiment. As shown in Figure 12, in this embodiment, the value of the loss function for the verification image data is decreasing, so it can be said that learning is progressing.

13 to 15 are explanatory diagrams showing the prediction accuracy of the growth rate G in this embodiment. In FIG. 13, the values of the root mean square error (RMSE), the mean absolute error (MAE), and the coefficient of determination (R ² ) are shown for each of the training image data and the test image data. In addition, in FIG. 14, the relationship between the true value and the estimated value of the growth rate G for the training image data is shown, and in FIG. 15, the relationship between the true value and the estimated value of the growth rate G for the test image data is shown. As shown in FIG. 14 and FIG. 15, in this embodiment, each point showing the relationship between the true value and the estimated value of the growth rate G is approximately on a straight line, and it can be said that the growth rate G can be estimated approximately correctly. In addition, as shown in FIG. 13, the coefficient of determination (R ² ) for the test image data exceeds 0.95. In addition, the mean absolute error (MAE) is 0.047, so the error is within 4.7% on average. Because the number of cultivation days for the test image data in this study was 21 days, the 4.7% error is less than one day when converted to days, which indicates that sufficient estimation accuracy has been achieved. When predictions were made using images taken immediately before harvest, the coefficient of determination ( ^R2 ) was approximately 0.85 to 0.97, which also indicates that sufficient estimation accuracy has been achieved.

B. Variations:
The technology disclosed in this specification is not limited to the above-described embodiments, and can be modified in various forms without departing from the spirit of the invention. For example, the following modifications are also possible.

The configuration of the plant cultivation management system 10 in the above embodiment is merely an example and can be modified in various ways. For example, in the above embodiment, the plant cultivation management system 10 is equipped with the photographing device 300, but the plant cultivation management system 10 does not have to be equipped with the photographing device 300. Even if the plant cultivation management system 10 does not have the photographing device 300, for example, the farm worker P1 uses a photographing device (e.g., a camera mounted on the terminal device 200) to photograph a plant cultivated in the plant factory PF to generate image data, and transmits the image data to the plant cultivation management server 100, so that the plant cultivation management server 100 can obtain image data of the plant.

In addition, in the above embodiment, the photographing device 300 transmits image data generated by photographing to the plant cultivation management server 100 each time photographing is performed, but the image data may be transmitted to the plant cultivation management server 100 at other times (for example, when a request is received from the plant cultivation management server 100).

In the above embodiment, the plant cultivation management server 100 executes a growth rate estimation model creation process to create the growth rate estimation model MO, but the growth rate estimation model creation process may be executed by a device other than the plant cultivation management server 100, and the model acquisition unit 114 of the plant cultivation management server 100 may acquire the growth rate estimation model MO from the other device. Alternatively, the plant cultivation management server 100 may transmit the target image data I(o) to the other device, and the plant cultivation management server 100 may acquire the growth rate G estimated by the other device using the growth rate estimation model MO.

The contents of the plant cultivation support process in the above embodiment are merely examples and can be modified in various ways. For example, in the plant cultivation support process in the above embodiment, the estimation of the growth rate G (S220) and the prediction of the harvest date and time Te (S240) are performed, but it is also possible to perform only the estimation of the growth rate G and not the prediction of the harvest date and time Te.

In addition, in the above embodiment, a semi-open type plant factory PF is used as an example of a farm field, but the technology disclosed in this specification can be similarly applied to other types of farm fields (e.g., other types of plant factories such as closed and open types, fields for outdoor cultivation, etc.).

In addition, in each of the above embodiments, some of the configurations realized by hardware may be replaced by software, and conversely, some of the configurations realized by software may be replaced by hardware.

10: Plant cultivation management system 100: Plant cultivation management server 110: Control unit 111: Plant cultivation management unit 112: Training data acquisition unit 114: Model acquisition unit 115: Target data acquisition unit 116: Growth rate estimation unit 117: Harvest date and time prediction unit 118: Information transmission unit 120: Memory unit 130: Display unit 140: Operation input unit 150: Interface unit 190: Bus 200: Terminal device 210: Display unit 300: Photographing device CP: Plant cultivation management program GD: Estimated growth rate data MO: Growth rate estimation model NET: Communication network P1: Farm worker PD: Estimated harvest date and time data PF: Plant factory TD: Training data

Claims (7)

An information processing device,
a target data acquisition unit that acquires target image data, the target image data being image data generated by photographing a target plant that is a specific type of plant being cultivated;
a growth rate estimation unit that estimates a growth rate, which is a ratio of a growth state of the target plant represented by image data to a growth state of the target plant at the time of harvest, using image data including color information of the target image data and a model created by a predetermined machine learning method;
a harvest date prediction unit that predicts a harvest date and time of the target plant based on the growth rate estimated for the target image data;
Equipped with
the target data acquisition unit acquires a plurality of pieces of target image data generated by photographing the target plant at a plurality of dates and times;
The growth rate estimation unit estimates the growth rate for a plurality of the target image data,
the harvest date and time prediction unit predicts a harvest date and time of the target plant for one of the target image data based on the growth rate estimated for the one of the target image data and the growth rate estimated for each of the target image data photographed before the photographing date and time of the one of the target image data;
Information processing device. The information processing device according to claim 1 , further comprising:
a training data acquisition unit that acquires training data in which a plurality of training image data, which is image data generated by photographing the specific type of plant at a plurality of dates and times within a period from planting to harvesting, is associated with growth rate data indicating the growth rate of the plant represented by the training image data;
a model acquisition unit that acquires the model by the machine learning using the training data;
Equipped with
Information processing device. 3. The information processing device according to claim 2 ,
The growth rate data is data showing a ratio of an elapsed time from planting to a photographing date and time of each of the training image data to a time required from planting to harvesting of the specific type of plant, as the growth rate.
Information processing device. An information processing device according to any one of claims 1 to 3 ;
a terminal device capable of communicating with the information processing device via a communication network;
An information processing system comprising:
The information processing device further includes an information transmitting unit that transmits information indicating the estimated growth rate of the target plant to the terminal device via the communication network.
Information processing system. The information processing system according to claim 4 , further comprising:
a photographing device that is installed in a field where the target plant is cultivated and is capable of communicating with the information processing device via a communication network;
The target data acquisition unit acquires, via the communication network, the target image data generated by the photographing device photographing the target plant.
Information processing system. 1. An information processing method, comprising:
acquiring target image data, the target image data being image data generated by photographing a target plant, the target plant being a specific type of plant being cultivated;
A step of estimating a growth rate, which is a ratio of the growth state of the target plant represented by the image data to the growth state of the target plant at the time of harvest, using image data including color information of the target image data and a model created by a predetermined machine learning method;
predicting a harvest date and time of the target plant based on the growth rate estimated for the target image data;
Equipped with
The step of acquiring the target image data is a step of acquiring a plurality of target image data generated by photographing the target plant at a plurality of dates and times,
the step of estimating the growth rate is a step of estimating the growth rate for a plurality of the target image data,
The step of predicting the harvest date and time is a step of predicting the harvest date and time of the target plant for one of the target image data based on the growth rate estimated for the one of the target image data and the growth rate estimated for each of the target image data photographed before the photographing date and time of the one of the target image data.
Information processing methods. A computer program comprising:
On the computer,
acquiring target image data, the target image data being image data generated by photographing a target plant, the target plant being a specific type of plant being cultivated;
A process of estimating a growth rate, which is a ratio of the growth state of the target plant represented by the image data to the growth state of the target plant at the time of harvest, using image data including color information of the target image data and a model created by a predetermined machine learning method;
A process of predicting a harvest date and time of the target plant based on the growth rate estimated for the target image data;
Run the command ,
The process of acquiring the target image data is a process of acquiring a plurality of target image data generated by photographing the target plant at a plurality of dates and times,
the process of estimating the growth rate is a process of estimating the growth rate for a plurality of the target image data,
The process of predicting the harvest date and time is a process of predicting the harvest date and time of the target plant for one of the target image data based on the growth rate estimated for the one of the target image data and the growth rate estimated for each of the target image data photographed before the photographing date and time of the one of the target image data.
Computer program.

Images