JP6505904B1 - Judgment system, judgment device, judgment method and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To predict more appropriate harvest timing of agricultural products. SOLUTION: An environmental information sensor is installed in a cultivation place where a specific agricultural product is grown, and acquires environmental information indicating an environment in the cultivation place, and an imaging which is installed in the cultivation place and picks up an image of the agricultural product Based on the device, the environmental information, and the image information which is information obtained from the image of the agricultural product, the determination to determine the remaining integrated value which is the integrated value of the predetermined environmental values until the appropriate harvest of the agricultural product And a device. [Selected figure] Figure 1

Description

  The present invention relates to a technique for estimating the growth state of agricultural products.

  Fruit and vegetables such as tomato have a complicated growth process. Therefore, it is difficult to accurately predict the growth state. Such problems are not limited to fruits and vegetables, but are common to agricultural products in general.

  As techniques for solving such problems, techniques shown in Patent Documents 1 to 3 have been proposed. Patent Document 1 proposes a system for suppressing variation in crop quality and end. Patent Document 2 proposes a technique for estimating the end with high accuracy by using a satellite image or the like. Patent Document 3 proposes a technique for predicting the amount of produce that meets the shipping standard based on the amount of produce delivered from the producer.

JP, 2016-049102, A JP, 2015-000049, A JP, 2016-118895, A

However, due to factors such as changes in weather immediately before the harvest time, the appropriate harvest time often goes back and forth. Therefore, it was difficult to predict an appropriate harvest date for agricultural products.
In view of the above-mentioned circumstances, the present invention aims to provide a technology that makes it possible to predict more appropriate harvest timing of agricultural products.

  One aspect of the present invention is installed in a cultivation place where a specific agricultural product is grown, an environmental information sensor for acquiring environmental information indicating an environment in the cultivation place, and installed in the cultivation place, the image of the agricultural product Based on the imaging device to be imaged, the environmental information, and the image information which is information obtained from the image of the agricultural product, a remaining integrated value which is an integrated value of predetermined environmental values up to the appropriate harvest of the agricultural product It is a determination system provided with the determination apparatus which determines.

  According to one aspect of the present invention, up to an appropriate harvest of the agricultural product based on environmental information indicating an environment in a cultivation place where a specific agricultural product is grown and image information which is information obtained from an image of the agricultural product. And a determination unit that determines a remaining integration value that is an integration value of predetermined environmental values of

  One embodiment of the present invention is the above-described determination apparatus, wherein the determination unit is further configured to include real environment information, real image information, and real image information acquired before harvest for agricultural products actually harvested at appropriate timing. The residual integration value is determined by using the result of the learning process performed based on the actual environment information or the actual value of the residual integration value from the acquisition of the actual image information to the harvest timing.

  In one aspect of the present invention, an environmental information sensor installed in a cultivation site where a specific agricultural product is grown acquires environmental information indicating an environment in the cultivation site, and an imaging device installed in the cultivation site An integrated value of predetermined environmental values until an appropriate harvest of the agricultural product based on the step of imaging the agricultural product, the environmental information, and the image information which is information obtained from the agricultural product image And determining the remaining integrated value.

  According to one aspect of the present invention, up to an appropriate harvest of the agricultural product based on environmental information indicating an environment in a cultivation place where a specific agricultural product is grown and image information which is information obtained from an image of the agricultural product. A determination step of determining a remaining integrated value which is an integrated value of predetermined environmental values of

  One embodiment of the present invention is a computer program for causing a computer to function as the above determination system.

  One embodiment of the present invention is a computer program for causing a computer to function as the above-described determination device.

  The present invention makes it possible to predict more appropriate harvest timing of agricultural products.

It is a schematic block diagram which shows the system configuration | structure of the determination system 200 of this invention. FIG. 8 is a schematic block diagram showing an example of a functional configuration of a determination device 60. It is a figure which shows the concept of a residual integration value. It is a sequence chart which shows the specific example of the flow of the determination process of the determination system 200. FIG.

Hereinafter, specific configuration examples of the present invention will be described with reference to the drawings.
FIG. 1 is a schematic block diagram showing a system configuration of a determination system 200 of the present invention. The determination system 200 is a system that determines the appropriate harvest timing of the agricultural product to be determined.

  First, the network configuration of the determination system 200 will be described. The determination system 200 includes a plurality of environment information sensors 10, a plurality of imaging devices 20, a control device 30, a log data storage device 40, a terminal device 50, and a determination device 60. The plurality of environmental information sensors 10 and the plurality of imaging devices 20 are communicably connected to the control device 30, respectively. Communication performed by the plurality of environment information sensors 10 and the plurality of imaging devices 20 and the control device 30 may be realized by wireless communication or may be realized by wired communication. The control device 30 and the log data storage device 40 are communicably connected. Control device 30, terminal device 50 and determination device 60 are communicably connected via network 80. The network 80 is configured using a communication network such as, for example, the Internet or a public network. The determination device 60 is communicably connected to the weather forecast information providing device 70 via the network 80.

Next, each device related to the determination system 200 will be described.
The plurality of environmental information sensors 10 and the plurality of imaging devices 20 are installed in the cultivation site 100 to be determined. In the cultivation site 100, one or more types of specific agricultural products are cultivated. For example, in the cultivation site 100, fruit vegetables such as tomato and eggplant are grown. In this case, the determination device 60 of the determination system 200 determines an appropriate harvest timing of fruits and vegetables such as tomato and eggplant grown in the cultivation place 100.

  The environmental information sensor 10 acquires specific information (hereinafter referred to as “environmental information”) related to the environment of the cultivation place 100. The environmental information sensor 10 acquires, for example, any one or more of the temperature, humidity, precipitation, sunshine duration, air volume, and the like in the cultivation site 100. As the environment information sensor 10, one type of sensor may be installed in the cultivation place 100, or a plurality of types of sensors may be installed in the cultivation place 100. The environmental information sensor 10 may be fixedly installed at a predetermined position or may be movably installed. Each environmental information sensor 10 transmits data indicating the acquired environmental information to the control device 30. Data indicating the environmental information transmitted from the environmental information sensor 10 may be transmitted directly to the control device 30 by wireless communication or wire communication, or may be transmitted to the control device 30 via a relay device (not shown). Good.

  The imaging device 20 photographs the agricultural products to be judged grown in the cultivation place 100. The imaging device 20 may be configured to capture an agricultural product with a predetermined imaging parameter (angle of view, magnification, etc.) in a predetermined imaging direction from a predetermined imaging position. The imaging device 20 may be fixedly installed at a predetermined position and a predetermined direction, may be installed at a predetermined position so as to be able to change the imaging direction, and may be installed movably. . For example, the imaging device 20 may be configured to move in the cultivation plant 100 by being fixed to a moving body that moves along a predetermined guide, or may be grown by being fixed to an aircraft such as a drone. It may be configured to move within station 100. The imaging device 20 transmits data of an image (hereinafter, referred to as “captured image”) obtained by shooting to the control device 30. The imaging device 20 may associate the identification information of the own device with the data of the captured image and transmit it. Data of a photographed image transmitted from the imaging device 20 may be transmitted directly to the control device 30 by wireless communication or wire communication, or may be transmitted to the control device 30 via a relay device (not shown).

  The control device 30 is configured using an information processing device such as a personal computer or a programmable logic controller (PLC). The control device 30 receives data of environmental information from the environmental information sensor 10. The control device 30 records the received data of environmental information in the log data storage device 40 in association with date and time information. The date and time information to be recorded may be date and time information when environmental information data was acquired by the environmental information sensor 10, and may be date and time information when environmental information data was received by the control device 30, or log data The date and time information recorded in the storage device 40 may be used. The control device 30 receives data of a photographed image from the imaging device 20. The control device 30 records the received data of the photographed image in the log data storage device 40 in association with the date and time information. The date and time information to be recorded may be date and time information on which captured image data was acquired in the imaging device 20, date and time information on captured image data received in the control device 30, or log data storage The date and time information recorded in the device 40 may be used.

  The log data storage device 40 is configured using a storage device such as a magnetic hard disk drive or a semiconductor storage device. The log data storage device 40 stores a log of data obtained by the environment information sensor 10 and the imaging device 20 installed in the cultivation place 100.

  The terminal device 50 is a device having an information processing function such as a personal computer, a server, a smartphone, a tablet, a smart speaker, and a television receiver. The terminal device 50 is used, for example, by a user such as a manager or manager of the cultivation site 100. The terminal device 50 receives the information on the crop harvest time from the determination device 60 via the network 80. The terminal device 50 outputs the received information to the user in a predetermined manner. For example, the terminal device 50 may output the information on the harvest time of the agricultural product by displaying the image on the display, or may output the information on the harvest time of the agricultural product by voice.

  The determination device 60 is configured using an information processing device such as a personal computer or a server device. FIG. 2 is a schematic block diagram showing an example of a functional configuration of the determination device 60. As shown in FIG. The determination device 60 includes a communication unit 61, a log storage unit 62, a learning information storage unit 63, a learning result storage unit 64, and a control unit 65.

  The communication unit 61 is configured using a communication interface. The communication unit 61 performs data communication with other devices via the network 80. For example, the communication unit 61 receives data from the control device 30. For example, the communication unit 61 transmits data indicating the determination result to the terminal device 50.

  The log storage unit 62 is configured using a storage device such as a magnetic hard disk drive or a semiconductor storage device. The log storage unit 62 stores a log of environmental information and a captured image. The log storage unit 62 stores, for example, a record in which date and time information and environment information are associated with each other. The log storage unit 62 further stores, for example, a record in which date and time information and a photographed image are associated with each other.

  The learning information storage unit 63 is configured using a storage device such as a magnetic hard disk drive or a semiconductor storage device. The learning information storage unit 63 stores learning information. The learning information is data used when the learning unit 652 of the control unit 65 performs the learning process. The information for learning includes, for example, evaluation information indicating whether or not the timing of harvesting was appropriate for agricultural products that were already harvested in the cultivation site 100, and captured images of individual agricultural products captured before harvesting (hereinafter referred to as “ The actual image information which is information obtained from the “actual individual image”, the actual environment information indicating the environment information acquired before harvesting, and the residual integration at the timing when the data of the actual individual image or the actual environment information is acquired The information indicating the actual value of the value may be the correlated information. The actual image information may be, for example, information indicating the color of the agricultural product appearing in the actual individual image, or may be information indicating the size of the agricultural product on the actual individual image.

  The information for learning includes, for example, real image information obtained from real individual images taken before harvest and real environment information obtained before harvest for agricultural products that have already been harvested at appropriate times in the cultivation site 100, and It may be information associated with information indicating the actual value of the remaining integrated value at the timing when the data of the actual individual image or the actual environment information is acquired. The residual integrated value indicates an integrated value of specific environmental information values from that day to the optimum harvest timing. The learning information may further be associated with weather forecast information at the time when each real environment information or a real individual image is acquired. The weather forecast information is information indicating the weather forecast of the area where the cultivation place 100 is included. The weather forecast information is information indicating forecasted values such as weather, temperature, humidity, barometric pressure, and the like after the target day.

  FIG. 3 is a diagram showing the concept of the residual integration value. In the example of FIG. 3, the environmental information represented by the residual integration value is a temperature value. When the temperature of the day (for example, the maximum temperature, the average temperature, the minimum temperature, etc.) is integrated every day, a value integrated from a certain judgment reference date to the optimum harvest timing is called a residual integration value. In the example of FIG. 3, the residual integration value is 0 degree on the day of harvest which is the optimal timing. The fact that the residual integrated value is 0 degree indicates that it is the optimal timing for harvesting. If the remaining integrated value is 40 degrees, it indicates that the timing when the temperature is 40 degrees when the temperature is integrated every day from that day is the optimum harvest timing. Therefore, when the remaining integrated value is 40 degrees and the temperature of the day is 40 degrees, it indicates that the next day is the optimal harvest timing. If the remaining integrated value is 40 degrees and the temperatures on the current day and the next day (after one day) are 20 degrees, it means that two days later is the optimum harvest timing. FIG. 3 shows such an example. Even if the residual integrated value is 40 degrees, if the temperature is 10 degrees every day from that day, it indicates that four days after is the optimum harvest timing. In addition, based on the value of a residual integration value, you may change the timing of a harvest to a more suitable thing suitably. For example, the harvest timing may be changed depending on circumstances such as whether the product is sold immediately after harvesting or is transported to a remote location and then sold.

  The correspondence between each residual integrated value and the number of days until harvest shown in FIG. 3 is only one specific example. Even if it is the same residual integrated value, the number of days until the optimal harvest changes depending on what the temperature after that day is.

  It returns to the explanation of FIG. The learning information stored in the learning information storage unit 63 is given in advance to the determination device 60 by a predetermined means. For example, learning information may be given by recording learning information recorded in a storage medium such as a USB memory or a CD-ROM in the learning information storage unit 63. For example, the learning information may be provided to the learning information storage unit 63 of the determination device 60 by transmitting the learning information to the determination device 60 from another information processing device. For example, with respect to log information stored in the log storage unit 62, evaluation information indicating whether or not the harvest timing is appropriate for the user, and an actual individual image extracted from the captured image recorded as the log information The learning information is generated by adding the actual environment information calculated based on the actually harvested timing or the actual value of the remaining integrated value corresponding to the actual individual image, and the learning information is generated, and the learning information storage unit 63 may be given.

  The learning result storage unit 64 is configured using a storage device such as a magnetic hard disk drive or a semiconductor storage device. The learning result storage unit 64 stores information indicating the learning result obtained by the learning process of the learning unit 652 of the control unit 65. The learning result stored by the learning result storage unit 64 is used by the determination unit 654.

  The control unit 65 is configured using a processor such as a CPU (Central Processing Unit) connected by a bus and a memory. The control unit 65 functions as a log control unit 651, a learning unit 652, a detection unit 653, a determination unit 654, and an information providing unit 655 when the processor executes a program.

  Note that all or part of the log control unit 651, the learning unit 652, the detection unit 653, the determination unit 654, and the information providing unit 655 realized by the processor executing a program may be an application specific integrated circuit (ASIC) or a PLD. It may be realized using hardware such as (Programmable Logic Device) or FPGA (Field Programmable Gate Array). The program may be recorded on a computer readable recording medium. Examples of the computer-readable recording medium include portable media such as flexible disks, magneto-optical disks, ROMs, CD-ROMs, semiconductor storage devices (for example, SSD: Solid State Drive), hard disks incorporated in computer systems, and semiconductor storage A storage device such as a device. The program may be transmitted via a telecommunication line.

  The log control unit 651 receives each environment information and data of a photographed image from the control device 30. The log control unit 651 records the received data as a log in the log storage unit 62 in association with date and time information.

  The learning unit 652 performs machine learning using the learning information stored in the learning information storage unit 63. The learning unit 652 acquires a learning result for obtaining the value of the remaining integrated value up to the optimum harvest among the plurality of values included in the learning information. For the learning unit 652, a machine learning technique for generating or updating a classifier by, for example, supervised learning is applied. Specific examples of machine learning techniques include SVM (Support Vector Machine) and DL (Deep Learning).

  The detection unit 653 detects an image of an individual of the agricultural product to be processed from the photographed image. For example, when the agricultural product to be processed is a tomato and ten tomatoes under cultivation are shown in the photographed image, the detection unit 653 detects each one of the ten tomatoes. The detection unit 653 extracts an image of each detected tomato (hereinafter, referred to as an “individual image”). The detection unit 653 determines the position of the agricultural product in the extracted individual image in the cultivation site 100 based on the identification information of the imaging device associated with the captured image. The detection unit 653 outputs the extracted individual images one by one to the determination unit 654 as an image to be processed.

  The determination unit 654 acquires current environment information from the log control unit 651. In addition, the determination unit 654 acquires, from the detection unit 653, an individual image in which each agricultural product to be subjected to the determination process is captured. The determination unit 654 acquires image information from the individual image. The image information may be, for example, information indicating the color of the agricultural product appearing in the individual image, or may be information indicating the size of the agricultural product on the individual image. The determination unit 654 determines the remaining integrated value of the agricultural products captured in the individual image based on the acquired environment information and image information and the learning result stored in the learning result storage unit 64. The determination unit 654 may use weather forecast information in the determination process in addition to the environment information and the image information. In this case, the learning process needs to be performed to perform the determination process based on the weather forecast information. The weather forecast information may be acquired from the weather forecast information provision device 70 via the network 80, for example. The weather forecast information providing device 70 is a server installed by, for example, the Japan Meteorological Agency or a specific corporation or individual, and is a device that provides weather forecast information.

  The information provision unit 655 generates provision information including information indicating the determination result by the determination unit 654. For example, the provided information includes the individual image subjected to the determination process, the residual integrated value indicating the result of the determination process performed by the determination unit 654 on the individual image, and the cultivation site 100 of the produce photographed in the individual image. The information which shows a place may be included.

  FIG. 4 is a sequence chart showing a specific example of the flow of the determination process of the determination system 200. Each environmental information sensor 10 acquires environmental information at a predetermined timing and transmits it to the control device 30 (step S101). Each imaging device 20 performs imaging at a predetermined timing, and transmits a captured image to the control device 30 (step S102).

  The control device 30 records the environmental information received from the environmental information sensor 10 and the photographed image received from the imaging device 20 in the log data storage device 40 (step S103). Then, the control device 30 transmits the environment information received from the environment information sensor 10 and the photographed image received from the imaging device 20 to the determination device 60 (step S104).

  The log control unit 651 of the determination device 60 receives the environmental information and the photographed image from the control device 30. The log control unit 651 associates the received environment information and the photographed image with the time information, and records them as a log in the log storage unit 62 (step S105). The detecting unit 653 detects an agricultural product appearing as a subject in the newly acquired captured image (step S106). The detection unit 653 applies a label to the detected image (individual image) of each agricultural product (step S107). The determination unit 654 is configured to determine the agricultural product to be determined based on the received environmental information, the image information of the agricultural product to be determined, and the learning result stored in the learning information storage unit 63. The remaining integrated value is determined (step S108). The information provision unit 655 generates provision information including the result of the determination process (step S109). The information providing unit 655 transmits the provided information to the terminal device 50 associated with the cultivation place 100 that is the target of the determination process (step S110). The terminal device 50 outputs the received provision information to the user.

  According to the determination system 200 configured as described above, the remaining integrated value is determined as information indicating the appropriate timing of harvesting for the agricultural products grown in the cultivation place 100. The value of the residual integration value itself is not easily affected by changes in weather after the timing of determination. That is, although the date which is the timing suitable for harvesting changes according to the actual air temperature etc. after the timing to which the determination was performed, the remaining integrated value itself at the timing to which the determination was performed is uniquely determined. By providing the user with such residual integrated values as information indicating the appropriate timing of harvesting, it is possible to more accurately determine the timing of harvesting that is more appropriate for the user.

  Such a determination system 200 is particularly effective for agricultural products of a type in which the integrated value of environmental information from a predetermined event (for example, flowering) in agricultural products to the optimal harvest timing is constant. Such agricultural products include, for example, fruits and vegetables such as tomatoes. Although FIG. 3 mentioned above demonstrated air temperature as a specific example as a remaining integration value, a remaining integration value does not necessarily need to be an integration value regarding air temperature. For example, it may be an integrated value of sunshine hours.

  In the determination system 200, the remaining integrated value is determined based not only on the environment information but also on the image information of the individual image. Therefore, it is possible to determine the remaining integrated value more accurately than when the remaining integrated value is determined based on only the environmental information.

(Modification)
Any means may be adopted as means for causing environmental information to reach the determination device 60 from each environmental information sensor 10. In the above description, environmental information is transmitted from each environmental information sensor 10 to the determination device 60 via the control device 30. For example, transmission from each environmental information sensor 10 to the determination device 60 without the intervention of the control device 30 It may be done. Also, for example, any one or more of the environmental information sensors 10 collect environmental information from other environmental information sensors 10, and collectively transmit one or more environmental information to the control device 30 or the determination device 60. It is also good. The same applies to the imaging device 20.

  The determination device 60 does not necessarily have to be configured as a single device. For example, the determination device 60 may be configured using a plurality of information processing devices. The plurality of information processing apparatuses constituting the determination apparatus 60 may be communicably connected via a communication path such as the network 80, and may be configured as a system such as a cluster machine or a cloud.

  The determination device 60 may be configured as a device or system that does not include the learning unit 652. In this case, the processing performed by the learning unit 652 described above may be executed by another device or system as a learning device. In this case, the determination apparatus including the determination unit 654 acquires data of the learning result from the learning apparatus and performs determination processing.

  A storage battery or a charger may be connected to the environmental information sensor 10. By this configuration, it is possible to reduce the restriction on the installation location of the environmental information sensor 10. For example, the environmental information sensor 10 can be installed at a place where electricity does not pass. In addition, a device that generates power by natural energy may be connected to the environment information sensor 10. For example, a solar panel or a generator such as a wind power generator may be connected. The environmental information sensor 10 may operate based on the power generated by the device that generates such power.

  The determination unit 654 and the learning result storage unit 64 may be included in the control device 30. With this configuration, it is possible to more quickly determine the residual integrated value of the agricultural products in each individual image.

  The learning unit 652 may perform the learning process for each of a plurality of sets of learning information. For example, the learning unit 652 may perform the learning process on the basis of each learning information prepared in advance for each type and variety of agricultural products. In this case, the learning result storage unit 64 stores the learning result obtained for each type or each type. The determination unit 654 selects the optimal learning result to be used according to the type and cultivar of agricultural products appearing in the individual image to be processed, and uses the selected optimal learning result to calculate the remaining integration. Determine the value. With this configuration, it is possible to determine the residual integrated value with higher accuracy. The plurality of sets of learning information are not limited to the above-described patterns (types and types). For example, a plurality of sets of learning information may be used depending on differences in cultivation methods (house hydroponic, house soil cultivation, alley, etc.), seasons (spring, summer, autumn, and winter).

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes design and the like within the scope of the present invention.

100: cultivation plant, 200: determination system, 10: environmental information sensor, 20: imaging device, 30: control device, 40: log data storage device, 50: terminal device, 60: determination device, 61: communication unit, 62 ... Log storage unit 63 Learning information storage unit 64 Learning result storage unit 65 Control unit 651 Log control unit 652 Learning unit 653 Detecting unit 654 Determining unit 655 Information providing unit

Claims (7)

An environmental information sensor installed in a cultivation site where a specific agricultural product is grown, and acquiring environmental information indicating an environment in the cultivation site,
An imaging device installed in the cultivation site and imaging an image of the agricultural product;
A determination device that determines a remaining integrated value that is an integrated value of predetermined environmental values up to an appropriate harvest of the agricultural product based on the environment information and image information that is information obtained from an image of the agricultural product;
Judgment system provided with   Accumulation of a predetermined environmental value up to an appropriate harvest of the agricultural product based on environmental information indicating an environment in a cultivation site where a specific agricultural product is grown and image information which is information obtained from an image of the agricultural product A determination apparatus comprising: a determination unit that determines a remaining integration value that is a value.   Furthermore, the determination unit may be configured to obtain the actual environment information, the actual image information, and the actual environment information or the actual image information acquired before harvesting the agricultural products actually harvested at the appropriate timing. The determination apparatus according to claim 2, wherein the remaining integrated value is determined by using a result of learning processing performed based on the actual value of the remaining integrated value up to the timing of. An environmental information sensor installed in a cultivation site where a specific agricultural product is grown, acquiring environmental information indicating an environment in the cultivation site;
An imaging device installed in the cultivation site imaging an image of the agricultural product;
Determining a remaining integrated value, which is an integrated value of predetermined environmental values up to an appropriate harvest of the agricultural product, based on the environmental information and image information which is information obtained from the image of the agricultural product Judgment method to have.   Accumulation of a predetermined environmental value up to an appropriate harvest of the agricultural product based on environmental information indicating an environment in a cultivation site where a specific agricultural product is grown and image information which is information obtained from an image of the agricultural product A determination step of determining a remaining integrated value which is a value.   A computer program for causing a computer to function as the determination system according to claim 1.   A computer program for causing a computer to function as the determination device according to claim 2 or 3.

Images