JP2023135998A - Information processor, information processing method, and program - Google Patents

Abstract

To provide information for deciding measures to be taken in order to stabilize yield in fruit cultivation.SOLUTION: An image acquisition part 30 acquires an image in which a subject includes a plant that bears fruits as an input image. A fruits number estimation part 31 analyzes the input image, and acquires the number of fruits being an estimated value of the number of fruits that a plant bears. A leaves number information acquisition part 32 analyzes the input image, and acquires the leaves number information being information regarding the number of leaves of a plant. A leaves-fruits ratio estimation part 33 estimates the leaves-fruits ratio of a plant by using the fruits number information and the leaves number information. An output part 34 outputs the leaves-fruits ratio.SELECTED DRAWING: Figure 2

Description

The present invention relates to an information processing device, an information processing method, and a program.

In recent years, technology using computer vision has been increasingly introduced into the agricultural field. For example, Non-Patent Document 1 proposes an automatic cucumber harvesting robot technology, and also discloses a technology for detecting cucumbers in a greenhouse by analyzing images.

E.J. VAN HENTEN, J. HEMMING, B.A.J. VAN TUIJL, J.G. KORNET, J. MEULEMAN, J. BONTSEMA AND E.A. VAN OS, “An Autonomous Robot for Harvesting Cucumbers in Greenhouses,” in Autonomous Robots 13, 2002, pp. 241-- 258.

In general, it is known that fruit trees tend to have repeated years in which they produce a lot of fruit (also called "top years") and years in which they do not produce much fruit (also called "back years"). ing. This is because the year after a fruit tree produces too many fruits, it tends to be a bad year. When carrying out fruit cultivation as a business, there is a desire to suppress the repetition of the front and back years of fruit trees and stabilize the yield.

In order to suppress the repetition of the front year and the back year in fruit trees, one idea is to thin the fruit in the front year to prevent overbearing, but it is difficult to grasp information such as the number of fruits for each fruit tree. It's complicated.

The present invention has been made in view of these points, and an object of the present invention is to provide information for determining measures to be taken in order to stabilize yields in fruit tree cultivation.

A first aspect of the present invention is an information processing device. This device includes an image acquisition unit that acquires as an input image an image that includes a plant that is bearing fruit as an input image, and a fruit that is an estimated value of the number of fruits that the plant is bearing by analyzing the input image. a fruit number estimation unit that acquires the number of fruits; a leaf number information acquisition unit that analyzes the input image to acquire leaf number information that is information about the number of leaves of the plant; and a leaf number information acquisition unit that uses the fruit number and the leaf number information. and a leaf-fruit ratio estimating section that estimates the leaf-fruit ratio of the plant, and an output section that outputs the leaf-fruit ratio.

The image acquisition unit may acquire as the input image an infrared image captured during a time period from sunset to sunrise, and the fruit number estimation unit may input an infrared image that includes plants as subjects. The number of fruits may be acquired by inputting the infrared image acquired by the image acquisition unit to a machine learning model that has been trained to output the number of fruits that the subject is growing when

The image acquisition unit may acquire two or more input images of the plant from two or more different directions, and the fruit number estimation unit may acquire the number of fruits imaged in each of the two or more input images. The number of fruits in the image, which is the number of , may be specified, and the number of fruits may be specified based on a predetermined relational expression between the number of fruits and the number of fruits in the image that is 2 or more.

The leaf number information acquisition unit includes an area setting unit that sets a predetermined area including the plant in the input image, and calculates a porosity, which is a ratio of the area other than the plant to the area, as the leaf number information. A porosity calculation unit may be provided.

The leaf-fruit ratio estimating unit may specify the leaf-fruit ratio based on a predetermined relational expression regarding the leaf-fruit ratio and information regarding a ratio between the void ratio and the number of fruits.

The leaf number information acquisition unit inputs the input image to a machine learning model that is trained to output the number of leaves of the subject when an image including a plant bearing fruit as the subject is input. The number of leaves obtained may be acquired as the leaf number information, and the leaf-fruit ratio estimating unit may specify the leaf-fruit ratio based on the number of leaves and the number of fruits.

The information processing device includes an identifier acquisition unit that acquires a plant identifier for identifying a plant included in the input image, a plant identifier, and a reference leaf-fruit ratio that is set in advance for the plant identified by the plant identifier. a database management unit that acquires a reference leaf-fruit ratio associated with the plant identifier acquired by the identifier acquisition unit by referring to a leaf-fruit ratio database stored in association with the leaf-fruit ratio; a fruit-thinning number calculation unit that calculates a fruit-thinning number, which is the number of fruits that should be thinned in order for the leaf-fruit ratio of the plant included in the input image to become the reference leaf-fruit ratio, based on the ratio and the reference leaf-fruit ratio; , and the output unit may further output the number of fruits to be harvested.

The information processing device further includes a fruit set number calculation unit that calculates a fruit set number, which is the number of fruits that the plant bears after fruit thinning, by subtracting the fruit set number from the fruit number estimated by the fruit number estimation unit. The output unit may further output the number of fruit set.

A second aspect of the present invention is an information processing method. In this method, the processor obtains, as an input image, an image including a plant producing fruits as an input image, and analyzes the input image to estimate the number of fruits produced by the plant. acquiring the number of fruits; analyzing the input image to acquire leaf number information that is information about the number of leaves of the plant; and determining the number of leaves and fruits of the plant using the number of fruits and the leaf number information. A step of estimating a ratio and a step of outputting the leaf-fruit ratio are executed.

The third aspect of the present invention is a program. This program provides a computer with a function to acquire an image containing a plant producing fruit as an input image, and an estimate value of the number of fruits produced by the plant by analyzing the input image. a function of acquiring the number of fruits; a function of analyzing the input image to acquire leaf number information that is information about the number of leaves of the plant; and a function of acquiring the number of leaves of the plant using the number of fruits and the leaf number information. A function of estimating the ratio and a function of outputting the leaf-fruit ratio are realized.

In order to provide this program or to update a part of the program, a computer-readable recording medium recording this program may be provided, and this program may be transmitted over a communication line.

Note that arbitrary combinations of the above components and expressions of the present invention converted between methods, apparatuses, systems, computer programs, data structures, recording media, etc. are also effective as aspects of the present invention.

According to the present invention, it is possible to provide information for determining measures to be taken in order to stabilize yield in fruit tree cultivation.

FIG. 2 is a diagram for explaining an overview of processing of the information processing device according to the embodiment. FIG. 1 is a diagram schematically showing a functional configuration of an information processing device according to an embodiment. It is a figure which shows typically the relationship between the number of fruits and the number of fruits in an image of 2 or more. FIG. 3 is a diagram schematically showing an internal structure of a leaf number information acquisition unit according to an embodiment. FIG. 2 is a diagram schematically showing the relationship between porosity and the number of leaves in one tree as a whole. FIG. 2 is a diagram schematically showing the relationship between the leaf-fruit ratio and the ratio of porosity to fruit number. FIG. 2 is a diagram schematically showing a data structure of a leaf/fruit ratio database according to an embodiment. FIG. 3 is a diagram schematically showing an example of information output by an output unit according to an embodiment. 3 is a flowchart for explaining the flow of information processing executed by the information processing apparatus according to the embodiment.

<Overview of embodiment>
An outline of an embodiment of the present invention will be described. An information processing device according to an embodiment of the present invention estimates the leaf-fruit ratio of a plant by analyzing an image that includes the plant as a subject. Here, the "leaf-fruit ratio" is the number of leaves per fruit in a fruit tree that bears fruit. It has been empirically known that the appropriate leaf-fruit ratio for various types of fruit trees is approximately 40 to 70 apples. By optimizing the leaf-fruit ratio of fruit trees, it is expected that yields can be stabilized in fruit tree cultivation.

However, it is very time-consuming for producers to count the number of fruits that fruit trees bear and the number of leaves on fruit trees, so it is not practical to determine the number of fruits set based on the leaf-fruit ratio in the field of fruit cultivation. This is a difficult problem. Since the information processing device according to the embodiment of the present invention calculates the leaf-fruit ratio from images of fruit trees that bear fruit, the user can easily decide what measures to take to stabilize the yield in fruit tree cultivation. I can do it.

FIG. 1 is a diagram for explaining an overview of processing of an information processing device 1 according to an embodiment. Below, an overview of the processing of the information processing device 1 will be explained in the order of (1), (2), (3), (4), and (5). , (3), (4) and (5).

(1) The information processing device 1 obtains, as an input image, an image that includes a fruit-bearing plant as a subject. FIG. 1 shows an example of an input image in which a yuzu tree is the subject.
(2) The information processing device 1 estimates the number of fruits that the plant is producing by analyzing the input image. Specifically, when the information processing device 1 receives an image that includes a plant as a subject, it calculates the number of fruits using a machine learning model trained to output the number of fruits that the subject is growing. Estimate. Hereinafter, the number of fruits estimated by the information processing device 1 may be referred to as "number of fruits."

(3) The information processing device 1 acquires leaf number information regarding the number of leaves, which is the number of leaves grown on a plant, by analyzing the input image. FIG. 1 shows an example in which the information processing device 1 calculates "porosity" as leaf number information. Here, "porosity" is the ratio of areas other than plants to a predetermined area containing plants. In the example shown in FIG. 1, the porosity is determined as the percentage of the background within a circle whose diameter is the tree crown height (the broken line circle in FIG. 1). The porosity decreases as the number of leaves of a plant increases. Note that the predetermined area for calculating the porosity is not limited to the above, and may have other shapes such as a circle circumscribing the plant or a rectangular area circumscribing the plant.

(4) The information processing device 1 estimates the leaf-fruit ratio of the plant based on the estimated number of fruits and information on the number of leaves. Specifically, the information processing device 1 estimates the leaf-fruit ratio based on a predetermined relational expression regarding the leaf-fruit ratio and information regarding the ratio between leaf number information and fruit number.
(5) The information processing device 1 outputs the estimated leaf-fruit ratio. Thereby, the user of the information processing device 1 can obtain an estimated value of the leaf-fruit ratio of the plant by simply preparing an image of the plant. As a result, the user can calculate the number of fruits that should set in order to make the leaf-fruit ratio of the plant appropriate.

<Functional configuration of information processing device 1 according to embodiment>
FIG. 2 is a diagram schematically showing the functional configuration of the information processing device 1 according to the embodiment. The information processing device 1 includes a storage section 2 and a control section 3. In FIG. 2, arrows indicate main data flows, and there may be data flows that are not shown in FIG. In FIG. 2, each functional block shows the configuration of a functional unit, not the configuration of a hardware (device) unit. Therefore, the functional blocks shown in FIG. 2 may be implemented within a single device, or may be implemented separately within multiple devices. Data may be exchanged between functional blocks via any means such as a data bus, a network, or a portable storage medium.

The storage unit 2 includes a ROM (Read Only Memory) that stores the BIOS (Basic Input Output System) of the computer that implements the information processing device 1, a RAM (Random Access Memory) that serves as a work area of the information processing device 1, and an OS (OS). HDD (Hard Disk Drive) and SSD (Solid Disk Drive) that store various information that is referred to when executing the application program, such as the application program and the leaf-fruit ratio database 20 that stores the appropriate leaf-fruit ratio for each plant. A large-capacity storage device such as a state drive.

The control unit 3 is a processor such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit) of the information processing device 1, and executes a program stored in the storage unit 2 to perform image acquisition unit 30 and fruit number estimation. It functions as a section 31 , a leaf number information acquisition section 32 , a leaf/fruit ratio estimation section 33 , an output section 34 , an identifier acquisition section 35 , a database management section 36 , a fruit thinning number calculation section 37 , and a fruit set number calculation section 38 .

Note that FIG. 2 shows an example in which the information processing device 1 is configured as a single device. However, the information processing device 1 may be realized by calculation resources such as a plurality of processors and memories, such as a cloud computing system, for example. In this case, each unit constituting the control unit 3 is realized by executing a program by at least one of the plurality of different processors.

The image acquisition unit 30 acquires, as an input image, an image that includes a fruit-bearing plant as a subject. Specifically, the image acquisition unit 30 may acquire an image captured by an external imaging device by wire or wirelessly, or the information processing device 1 may acquire an image captured by an external imaging device, or the information processing device 1 may acquire an image captured by an external imaging device. ) If a known solid-state imaging device such as an image sensor is provided, the image may be obtained by directly imaging the subject.

The fruit number estimation unit 31 analyzes the input image acquired by the image acquisition unit 30 and acquires the number of fruits, which is an estimated value of the number of fruits produced by the plant. Further, the leaf number information acquisition unit 32 analyzes the input image acquired by the image acquisition unit 30 and acquires leaf number information that is information regarding the number of leaves of the plant. Details of the acquisition of the number of fruits by the fruit number estimation unit 31 and the acquisition of leaf number information by the leaf number information acquisition unit 32 will be described later, but both the fruit number estimation unit 31 and the leaf number information acquisition unit 32 By analyzing the information, the number of fruits and information on the number of leaves are obtained respectively.

The leaf-fruit ratio estimation unit 33 estimates the leaf-fruit ratio of the plant in the input image using the number of fruits acquired by the fruit number estimation unit 31 and the leaf number information acquired by the leaf number information acquisition unit 32. The output unit 34 outputs the leaf-fruit ratio estimated by the leaf-fruit ratio estimation unit 33 by displaying it on a display unit (not shown) included in the information processing device 1 or by transmitting it to a predetermined destination. Thereby, the information processing device 1 can provide the leaf-fruit ratio, which is useful information for determining measures to be taken in order to stabilize the yield in fruit tree cultivation.

(Estimation of number of fruits)
The process of estimating the number of fruits by the fruit number estimation unit 31 will be described in more detail.

When growing fruit trees, some types of plants are suitable for growing in greenhouses, while others are suitable for growing outdoors. For example, citrus plants such as mandarin oranges and yuzu are generally grown outdoors. When capturing an image of a plant grown outdoors, an image that is naturally influenced by the outdoor environment will be generated. For example, if you take an image of a plant outdoors during the daytime on a sunny day, an image with strong contrast will be generated. In such an image, it may become difficult to detect the fruit in the image due to the effect of contrast.

Therefore, when acquiring an input image using visible light as a light source, the image acquisition unit 30 preferably acquires an image captured on a cloudy day. Further, it is further preferable that the image acquisition unit 30 acquires an infrared image captured during night time as an input image. The night time period is a time period when there is little or no sunlight, such as the time period between sunset and sunrise.

In order to capture an infrared image during night time, it is necessary to irradiate a subject with infrared light and capture the reflected light with an infrared camera. Therefore, by adjusting the amount of infrared light and the distance from the infrared light source to the subject, it is possible to reduce the amount of infrared light irradiated onto subjects other than the plants targeted for leaf-fruit ratio estimation. For example, it is possible to suppress other plants or distant scenery existing in the background of plants from appearing in the image. Thereby, the fruit number estimating section 31 and the leaf number information acquisition section 32 can improve the accuracy of estimating the number of fruits and the leaf number information, respectively.

The inventor of the present application learned using known machine learning methods such as CNN (Convolutional Neural Network) using an infrared image that includes plants as a subject and the number of fruits in the infrared image as training data. By doing this, they built a machine learning model that inputs an infrared image and outputs the number of fruits contained in that image. The fruit number estimating unit 31 inputs the infrared rays acquired by the image acquiring unit 30 into a machine learning model that has been trained to output the number of fruits that the subject is producing when an infrared image containing a plant as the subject is input. Obtain the number of fruits by inputting an image. Thereby, the fruit number estimating unit 31 can easily and accurately acquire the number of fruits shown in the infrared image.

Here, if a plant is imaged from one direction, there may be fruits that are blocked by the plant itself and are not imaged. Therefore, the image acquisition unit 30 may acquire two or more input images of plants captured from two or more different directions. In this case, the fruit number estimating unit 31 first specifies the number of fruits in each image, which is the number of fruits imaged in each of the two or more input images acquired by the image acquiring unit 30. Subsequently, the image acquisition unit 30 specifies the number of fruits based on a predetermined relational expression regarding the number of fruits and the number of fruits in the image that is 2 or more.

FIG. 3 is a diagram schematically showing the relationship between the number of fruits and the number of fruits in an image that is two or more. Specifically, the horizontal axis of the graph shown in Figure 3 shows the average number of fruits per image in eight images obtained by imaging plants from eight different directions, and the vertical axis shows the number of fruits that were the subject. Shows the actual measured value of fruit for one whole tree. Further, the X mark in FIG. 3 is data obtained through experiments by the inventor of the present application, and the straight line is a regression line explaining the experimental data. When the horizontal axis in FIG. 3 is x and the vertical axis is y, the regression line can be expressed by a relational expression of the form y=αx+β, where α and β are real values. In this way, the fruit number estimating unit 31 can improve the accuracy of estimating the number of fruits by using a plurality of input images obtained by imaging plants from two or more different directions.

(leaf number information)
Next, the leaf number information acquisition process by the leaf number information acquisition unit 32 will be described in more detail.

As mentioned above, the leaf-fruit ratio is defined as the number of leaves per fruit for a fruit tree that bears fruit. Therefore, in order to directly determine the leaf-fruit ratio, it is necessary to obtain the number of leaves attached to a fruit tree. However, there are many leaves on fruit trees, and some leaves are not imaged because they are obscured by the trunk or other leaves. For this reason, the process of directly counting the number of leaves from an image that includes plants as subjects is a complicated process. Therefore, the leaf number information acquisition unit 32 according to the embodiment calculates the above-mentioned porosity instead of directly counting the number of leaves from the input image.

FIG. 4 is a diagram schematically showing the internal structure of the leaf number information acquisition unit 32 according to the embodiment. As shown in FIG. 4, the leaf number information acquisition section 32 includes a region setting section 320 and a porosity calculation section 321.

The area setting unit 320 sets a predetermined area that includes plants in the input image acquired by the image acquisition unit 30. Specifically, the area setting unit 320 sets in the input image a circular area whose diameter is the height of the tree crown and which includes the tree crown. The porosity calculation unit 321 calculates the porosity, which is the ratio of areas other than plants to the area set by the area setting unit 320, and uses it as leaf number information. Generally, it is considered that the greater the number of leaves of a plant that exists in the area set by the area setting unit 320, the greater the ratio of the number of leaves shown in that area.

FIG. 5 is a diagram schematically showing the relationship between porosity and the number of leaves in one tree as a whole. Specifically, the horizontal axis of the graph shown in FIG. 5 shows the porosity, and the vertical axis shows the number of leaves in one tree as a whole. Further, the black circles in FIG. 5 are data obtained through experiments by the inventor of the present application, and the straight lines are regression lines that explain the experimental data. FIG. 5 shows that the smaller the porosity, the larger the number of leaves per tree as a whole tends to be. That is, FIG. 5 shows that the porosity can be said to be information that reflects the number of leaves of a plant. The area setting unit 320 can improve the accuracy of the calculated information by calculating the porosity as the leaf number information instead of the number of leaves of the plant.

FIG. 6 is a diagram schematically showing the relationship between the leaf-fruit ratio and the ratio of porosity to fruit number. Specifically, the horizontal axis of the graph shown in FIG. 6 shows the value obtained by dividing the porosity by the number of fruits in the image, and the vertical axis shows the actually measured value of the leaf-fruit ratio. Furthermore, the white pentagons in FIG. 6 are data obtained through experiments by the inventor of the present application, and the straight lines are regression lines that explain the experimental data. Similar to the regression line shown in FIG. 3, the regression line in FIG. 6 can also be expressed by a relational expression of the form y=γx+δ (γ and δ are real numbers).

The relational expressions determined for the leaf-fruit ratio and the information regarding the ratio between the porosity and the number of fruits shown in FIG. 6 are determined in advance through experiments and stored in the storage unit 2. The leaf-fruit ratio estimating unit 33 specifies the leaf-fruit ratio based on a relational expression defined for the leaf-fruit ratio and information regarding the ratio between the porosity and the number of fruits. Thereby, the information processing device 1 can acquire the leaf-fruit ratio of a plant from an input image that includes the plant as a subject.

(Information output by the output unit 34)
As mentioned above, by optimizing the leaf-fruit ratio of fruit trees, it can be expected to stabilize the yield in fruit tree cultivation. Therefore, it would be convenient for those working in fruit tree cultivation sites to be able to grasp at a glance how many fruits should be picked from each fruit tree to achieve an appropriate leaf-fruit ratio.

Therefore, the information processing device 1 according to the embodiment includes an identifier acquisition unit 35 that acquires a plant identifier for identifying a plant included in the input image acquired by the image acquisition unit 30. Here, the "plant identifier" is an identifier uniquely assigned to each plant in order to identify each plant included in the input image.

The identifier acquisition unit 35 may acquire a plant identifier input by a user of the information processing device 1 via a user interface (not shown) of the information processing device 1, or may acquire information indicating a plant identifier in the input image (for example, a plant If the information contains encoded information (information encoding the identifier), the plant identifier may be acquired based on that information.

FIG. 7 is a diagram schematically showing the data structure of the leaf/fruit ratio database 20 according to the embodiment. As shown in FIG. 7, the leaf/fruit ratio database 20 stores plant identifiers in association with reference leaf/fruit ratios set in advance for plants identified by the plant identifiers. Here, the "standard leaf-fruit ratio" is a leaf-fruit ratio that is considered appropriate for each plant, and is determined in advance through testing. This experiment is found, for example, by changing the leaf-fruit ratio over several years. For those engaged in fruit cultivation, it is desirable to increase the yield of fruits (in other words, reduce the leaf-fruit ratio), but on the other hand, if the leaf-fruit ratio is too small, there will be differences between the front year and the back year. may occur repeatedly. Therefore, by lowering the leaf-fruit ratio each year, we determine the leaf-fruit ratio that will maintain a stable state while increasing the yield, and determine the appropriate leaf-fruit ratio.

In FIG. 7, the type of plant identified by the plant identifier PID0001 is yuzu, and the standard leaf-fruit ratio is 80 to 120. Further, the type of plant identified by the plant identifier PID0002 is Satsuma mandarin orange, and the standard leaf-fruit ratio is 25 to 30. The same applies below. Although not shown in FIG. 7, the leaf-fruit ratio database 20 includes, in addition to plant types and standard leaf-fruit ratios, identifiers for identifying managers who cultivate plants, and information indicating the locations of plants. , other information such as information regarding the soil of the land where the plant exists may be stored in association with the plant identifier.

The database management unit 36 refers to the leaf-fruit ratio database and acquires the reference leaf-fruit ratio associated with the plant identifier acquired by the identifier acquisition unit 35. The number calculation unit 37 calculates the number of plants included in the input image acquired by the image acquisition unit 30 based on the leaf and fruit ratio estimated by the leaf and fruit ratio estimation unit 33 and the reference leaf and fruit ratio acquired by the database management unit 36. The number of fruits to be thinned, which is the number of fruits that should be thinned so that the leaf-fruit ratio becomes the standard leaf-fruit ratio, is calculated. The output unit 34 further outputs the number of fruits calculated by the number calculation unit 37 of fruits. As a result, a user working at a fruit tree cultivation site can grasp at a glance the number of fruits to be harvested for each tree.

By optimizing the leaf-fruit ratio, it is expected that the repetition of the front and back years can be suppressed and the yield of fruit can be stabilized. If this is the case, it will be easier for those engaged in fruit cultivation to formulate a business plan if they can know how much harvest they can expect if the harvest amount is stabilized.

Therefore, the information processing device 1 calculates the number of fruits set, which is the number of fruits that the plant produces after fruit thinning, by subtracting the number of fruits calculated by the fruit thinning number calculating section 37 from the number of fruits estimated by the fruit number estimating section 31. A fruit set number calculation unit 38 is also provided. The output unit 34 further outputs the number of fruit set calculated by the number of fruit set calculation unit 38.

FIG. 8 is a diagram schematically showing an example of information output by the output unit 34 according to the embodiment, and is a diagram showing an example of a screen displayed on the display unit of the information processing device 1 by the output unit 34. As shown in FIG. 8, the output unit 34 creates a screen that includes a map of farmland where plants are cultivated, thumbnails of input images, and plant identifiers of plants included in the input images. The screen created by the output unit 34 includes the type of plant identified by the plant identifier (yuzu in the example of FIG. 8), an administrator ID (Identifier) for identifying the administrator of the plant, the administrator's name, It also includes the name of the orchard where the plant is grown.

As shown in FIG. 8, the output unit 34 outputs the number of fruits estimated by the fruit number estimation unit 31, the porosity obtained by the leaf number information acquisition unit 32, and the leaf-fruit ratio estimated by the leaf-fruit ratio estimation unit 33. is outputting. The output unit 34 further includes the reference leaf/fruit ratio read out from the leaf/fruit ratio database 20 by the database management unit 36, the number of fruit thinning calculated by the fruit thinning number calculation unit 37, and the fruit set after fruit thinning calculated by the fruit set number calculation unit 38. Both numbers are output. The output unit 34 also outputs a graph showing changes in the leaf-fruit ratio of the plant identified by the plant identifier. In this way, by creating a screen in which the output unit 34 displays various types of information in a list, a user working at a fruit tree cultivation site can see at a glance how much fruit should be picked from each fruit tree to achieve an appropriate leaf-fruit ratio. and can be grasped.

<Processing flow of information method executed by information processing device 1>
FIG. 9 is a flowchart for explaining the flow of information processing executed by the information processing device 1 according to the embodiment. The processing in this flowchart starts, for example, when the information processing device 1 is started up.

The image acquisition unit 30 acquires, as an input image, an image that includes a fruit-bearing plant as a subject (S2). The fruit number estimation unit 31 analyzes the input image acquired by the image acquisition unit 30 and acquires the number of fruits, which is an estimated value of the number of fruits that the plant is producing (S4). Specifically, when the fruit number estimating unit 31 inputs an infrared image containing a plant as a subject, the image acquiring unit 30 uses a machine learning model trained to output the number of fruits that the subject is growing. Obtain the number of fruits by inputting the infrared image obtained by .

The leaf number information acquisition unit 32 analyzes the input image and acquires leaf number information, which is information regarding the number of leaves of the plant (S6). Step 6 of acquiring leaf number information includes step S60 of setting a predetermined area and step S62 of calculating porosity. Specifically, first, the region setting unit 320 sets a predetermined region including plants in the input image (S60). Subsequently, the porosity calculation unit 321 calculates the porosity, which is the ratio of areas other than plants to the area set by the area setting unit 320, as leaf number information (S62).

The leaf-fruit ratio estimating unit 33 estimates the leaf-fruit ratio of the plant using the number of fruits and the information on the number of leaves (S8). Specifically, the leaf-fruit ratio estimating unit 33 specifies the leaf-fruit ratio based on a predetermined relational expression regarding the leaf-fruit ratio and information regarding the ratio between the porosity and the number of fruits.

The output unit 34 outputs the leaf-fruit ratio estimated by the leaf-fruit ratio estimation unit 33 (S10). When the output unit 34 outputs the leaf-fruit ratio, the processing in this flowchart ends.

<Effects achieved by the information processing device 1 according to the embodiment>
As explained above, according to the information processing device 1 according to the embodiment, it is possible to provide information for determining measures to be taken in order to stabilize the yield in fruit tree cultivation.

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes can be made within the scope of the gist. be. For example, all or part of the device can be functionally or physically distributed and integrated into arbitrary units. In addition, new embodiments created by arbitrary combinations of multiple embodiments are also included in the embodiments of the present invention. The effects of the new embodiment resulting from the combination have the effects of the original embodiment.

<First modification example>
In the above, a case has been described in which the leaf number information acquisition unit 32 acquires the porosity as the leaf number information. Alternatively, the leaf number information acquisition unit 32 may analyze the input image and directly acquire the leaf number as the leaf number information.

In this case, the storage unit 2 stores a machine learning model that has been trained to output the number of leaves in an image when an image including a fruit-bearing plant as an object is input. This machine learning model can be generated using a known machine learning method such as CNN using an image including a plant as a subject and the number of leaves of the plant determined by actual measurements as training data. The leaf number information acquisition unit 32 acquires the leaf number obtained by inputting the input image into the machine learning model read from the storage unit 2 as leaf number information.

The leaf-fruit ratio estimating unit 33 calculates leaves according to the definition of the leaf-fruit ratio, based on the number of leaves acquired by the leaf number information acquisition unit 32 according to the first modification and the number of fruits acquired by the fruit number estimation unit 31. Identify the fruit ratio. Although the porosity may change due to the movement of plant leaves due to the influence of wind, etc., the leaf number information acquisition unit 32 directly estimates the number of leaves from the input image, thereby reproducing the leaf-fruit ratio estimation. It can be expected to improve sexual quality.

<Second modification example>
In the above, a case has been described in which the leaf-fruit ratio estimation section 33 estimates the leaf-fruit ratio based on the number of fruits acquired by the fruit number estimation section 31 and the leaf number information acquired by the leaf number information acquisition section 32. Alternatively, the leaf-fruit ratio estimating unit 33 may analyze the input image and directly estimate the leaf-fruit ratio.

In this case, the storage unit 2 stores in advance a machine learning model trained to output the leaf-fruit ratio of a fruit-bearing plant when an image including the plant as a subject is input. This machine learning model can be generated using a known machine learning method such as CNN, using an image including a plant as a subject and the leaf-fruit ratio of the plant determined by actual measurements as training data. Since the process of estimating the number of fruits and the information on the number of leaves can be omitted, the leaf-fruit ratio estimation unit 33 according to the second modification can simplify the process of estimating the leaf-fruit ratio.

<Third modification example>
Above, the case where only one plant is included as a subject in the input image acquired by the image acquisition unit 30 has been mainly described. However, depending on the input image, other plants near the plant to be analyzed may be reflected. In such a case, one plant to be analyzed may be cut out from the input image as pre-processing.

For this reason, the information processing device 1 according to the third modification includes a cutout section (not shown). As described above, in an input image obtained by capturing an image with an infrared camera while irradiating a plant with infrared light, the brightness value of the plant to be analyzed tends to be significantly different from that of other subjects. For example, when an infrared camera forms a captured image such that the brightness value increases as the amount of infrared light increases, the pixel values of pixels forming the plant to be analyzed in the input image increase.

Therefore, the cutout section binarizes the input image using a predetermined threshold value, and integrates the pixel values of each pixel constituting the input image in each of the row and column directions. From the data obtained by integrating the pixel values constituting the input image in the row direction, it is possible to determine the area in which pixels with large values are distributed in the vertical direction of the input image. Similarly, from the data obtained by integrating the pixel values constituting the input image in the column direction, it is possible to determine the area in which pixels with large values are distributed in the horizontal direction of the input image. Thereby, the cutting unit can identify and cut out a region where pixels with large values are distributed in the input image (that is, a region where a plant to be analyzed exists where a lot of infrared light is reflected).

1 Information processing device 2 Storage unit 20 Leaf-fruit ratio database 3 Control unit 30 Image acquisition unit 31 Fruit number estimation unit 32 Leaf number information acquisition unit 320 Region setting unit 321 Porosity calculation unit 33 Leaf-fruit ratio estimation unit 34 Output unit 35 Identifier Acquisition unit 36 Database management unit 37 Thinning number calculation unit 38 Fruit set number calculation unit

Claims (10)

an image acquisition unit that acquires, as an input image, an image that includes a fruit-bearing plant as a subject;
a fruit number estimating unit that analyzes the input image to obtain a fruit number that is an estimated value of the number of fruits that the plant is producing;
a leaf number information acquisition unit that analyzes the input image to acquire leaf number information that is information about the number of leaves of the plant;
a leaf-fruit ratio estimation unit that estimates a leaf-fruit ratio of the plant using the fruit number and the leaf number information;
an output unit that outputs the leaf-fruit ratio;
An information processing device comprising: The image acquisition unit acquires an infrared image captured in a time period from sunset to sunrise as the input image,
The fruit number estimating unit applies the infrared rays acquired by the image acquiring unit to a machine learning model that is trained to output the number of fruits that the subject is producing when an infrared image containing a plant as a subject is input. obtaining the number of fruits by inputting an image;
The information processing device according to claim 1. The image acquisition unit acquires two or more input images of the plant from two or more different directions,
The fruit number estimating unit specifies the number of fruits in each image, which is the number of fruits captured in each of the two or more input images, and calculates a predetermined number of fruits with respect to the number of fruits and the number of fruits in the two or more images. determining the number of fruits based on a relational expression;
The information processing device according to claim 1 or 2. The leaf number information acquisition unit includes:
an area setting unit that sets a predetermined area including the plant in the input image;
a porosity calculation unit that calculates a porosity, which is a ratio of the area other than the plant to the area, as the leaf number information;
The information processing device according to any one of claims 1 to 3. The leaf-fruit ratio estimating unit specifies the leaf-fruit ratio based on a predetermined relational expression for the leaf-fruit ratio and information regarding the ratio of the void ratio to the number of fruits.
The information processing device according to claim 4. The leaf number information acquisition unit inputs the input image to a machine learning model that is trained to output the number of leaves of the subject when an image including a plant bearing fruit as the subject is input. obtain the number of leaves obtained as the number of leaves information,
The leaf-fruit ratio estimating unit specifies the leaf-fruit ratio based on the number of leaves and the number of fruits.
The information processing device according to any one of claims 1 to 3. an identifier acquisition unit that acquires a plant identifier for identifying a plant included in the input image;
A standard associated with the plant identifier acquired by the identifier acquisition unit by referring to a leaf/fruit ratio database that stores the plant identifier in association with a standard leaf/fruit ratio set in advance for the plant identified by the plant identifier. a database management unit that obtains leaf-fruit ratio;
Based on the leaf-fruit ratio estimated by the leaf-fruit ratio estimating unit and the reference leaf-fruit ratio, determine the number of fruits that should be removed in order for the leaf-fruit ratio of the plant included in the input image to reach the reference leaf-fruit ratio. further comprising a fruit picking number calculation unit that calculates a certain number of fruit picking,
The output unit further outputs the number of fruits to be thinned.
The information processing device according to any one of claims 1 to 6. further comprising a fruit set number calculation unit that calculates a fruit set number, which is the number of fruits that the plant bears after fruit thinning, by subtracting the fruit number from the fruit number estimated by the fruit number estimation unit;
The output unit further outputs the number of fruits set.
The information processing device according to claim 7. The processor
obtaining an image including a plant bearing fruit as an input image;
analyzing the input image to obtain a number of fruits that is an estimate of the number of fruits that the plant is producing;
analyzing the input image to obtain leaf number information that is information about the number of leaves of the plant;
estimating the leaf-fruit ratio of the plant using the number of fruits and the number of leaves information;
outputting the leaf-fruit ratio;
An information processing method that performs. to the computer,
A function that obtains an image that includes a fruit-bearing plant as an input image,
a function of analyzing the input image to obtain a number of fruits that is an estimated value of the number of fruits that the plant is producing;
A function of analyzing the input image to obtain leaf number information that is information about the number of leaves of the plant;
A function of estimating the leaf-fruit ratio of the plant using the number of fruits and the number of leaves information;
a function of outputting the leaf-fruit ratio;
A program that makes this possible.

Images