JP2013005724A - Display control device, display control system, display control method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a display control device, display control system, display control method, and program which facilitate management of the growth condition of plants.SOLUTION: The display control device (70) includes a display control part (73) for displaying information formed from information on the growth of the plant and stored in a memory area in association with identification information identifying a target plant to be displayed, in association with an image representing the target to be displayed.

Description

  The present invention relates to a display control device, a display control system, a display control method, and a program.

A plant cultivation system in a plant cultivation plant (plant factory) is attracting attention as being capable of reducing the load caused by cultivation work and stabilizing the supply of plants (for example, see Patent Document 1 and Non-Patent Document 1). .
In the technique of Patent Document 1, it is possible to determine the similarity between information based on a plant image obtained by a camera arranged at a fixed point and information (standard data) indicating characteristics of a standard plant. It is shown.
Further, the technique of Non-Patent Document 1 shows that the ratio of the plant image to occupy the screen is detected from the plant image obtained by a camera arranged at a fixed point.

Japanese Utility Model Publication No. 5-17505

Hiroshige Nishina, “Progress of“ Regional Base Type Intelligent Solar Plant Factory ””, Science Council of Japan, Public Symposium “Intelligent Solar Plant Plant”, July 3, 2009.

By the way, in the cultivation of plants, the cultivation method depending on the experience and intuition of the grower was the mainstream.
Moreover, according to the technique of patent document 1, it is only shown that the similarity of the image of the plant obtained with the camera arrange | positioned at the fixed point and standard data is determined, and the determination result is displayed. Further, according to Non-Patent Document 1, it is only shown that the ratio of the plant image obtained by the camera arranged at a fixed point occupying the screen is detected.
In short, in the cultivation of plants, there is a problem that it is difficult to manage the growth status of plants from the cultivation methods that depend on the experience and intuition of producers and the cultivation methods that have been disclosed so far. is there.
The present invention has been made in view of such circumstances, and provides a growth degree detection device, a plant cultivation system, a plant cultivation plant, a growth degree detection method, and a program that can make it easier to manage the growth status of plants. For the purpose.

  In order to solve the above-described problem, the present invention relates to display information generated from information relating to the growth of the plant stored in the storage area in association with identification information for identifying the plant to be displayed. A display control apparatus comprising a display control unit that displays an image in association with an image.

  In addition, the present invention displays display information generated from information relating to the growth of the plant stored in the storage area in association with identification information for identifying the display target plant in association with the display target image. A display control system including a display control unit.

  Further, according to the present invention, the display control unit displays the display information generated from the information regarding the growth of the plant stored in the storage area in association with the identification information for identifying the plant to be displayed. A display control method characterized by including a process of displaying in association with

  In addition, the present invention provides a computer provided in the display control device, and displays display information generated from information related to plant growth stored in a storage area in association with identification information for identifying a plant to be displayed. This is a program for causing a display control unit to function in association with a target image.

As described above, according to the present invention, it is possible to make it easier to manage the growth status of plants.

It is a block diagram which shows the structure of the plant cultivation plant by this embodiment of this invention. It is a figure which shows the mobile detection apparatus 40 in this embodiment. It is a block diagram which shows the structure of the plant cultivation system in this embodiment. It is a figure which shows control of the harvest time of the plant in this embodiment. It is a figure which shows the positional relationship of the flower and fruit in this embodiment. It is a figure explaining the detection of the actual maturity in this embodiment. It is a figure which shows the state of the leaf in this embodiment. It is a figure which shows the growth degree detection apparatus 70 in this embodiment. It is a block diagram which shows the growth degree detection apparatus 70 in this embodiment. It is a figure which shows the mode of the plant which changes according to growth. It is a figure which shows arrangement | positioning of the feature point of the detection target shown in FIG. It is a figure which shows the process which makes a feature point correspond according to the conditions shown in FIG. It is a figure which shows arrangement | positioning of the feature point of the detection target shown in FIG. 10, and shows a process when one feature point cannot be detected by growth. It is a figure which shows the process in the case of estimating the movement range of a feature point based on the conditions shown in FIG. 13 based on a different feature point. It is a figure which shows an example of arrangement | positioning and the feature-value of the feature point of the detection target shown in FIG. It is a figure shown about detection of the growth degree by the 1st detection method. It is a figure shown about detection of the growth degree by the 2nd detection method. It is a figure shown about the detection of the growth degree by the 3rd detection method. It is a figure which shows the process in the case of detecting the growth degree based on the change of a color. It is a figure which shows the change of each actual hue according to progress of time. It is a figure which shows the produced | generated synthesized image. It is a block diagram which shows the information provision system containing the growth degree detection apparatus 70 in this embodiment. It is a figure which shows the example of the screen displayed when selecting the position to refer. It is a figure which shows the example (the 1) of the screen which displays the state of the plant along a channel | path. It is a figure which shows the example (the 2) of the screen which displays the state of the plant which faces the channel | path. It is a figure which shows the example (the 3) of the screen which displays the state of the plant which faces the channel | path. It is a figure which shows the example of the screen which displays the evaluation information about the selected real.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a diagram showing an outline of a plant cultivation plant according to this embodiment. FIG. 1A shows a cross-sectional view thereof, and FIG. 1B shows a plan view thereof.
The plant cultivation plant 4 of the present invention is a facility that includes the plant cultivation system 1 (FIG. 3) and that grows plants.
As shown in FIG. 1, the plant cultivation plant 4 illuminates a vegetation floor 6 for plant cultivation, a plant enclosure 8 for separating the vegetation floor 6 from the outside air space 7, and a plant 9 vegetated on the vegetation floor 6. Is provided. The plant cultivation plant atmosphere adjusting device 2 adjusts the atmosphere of the inner space 13 of the plant enclosure 8. The vegetation floor 6 is a soil floor in the illustrated example, but may be a hydroponic floor for hydroponics that includes a liquid fertilizer circulation device (not shown).

  The plant enclosure 8 is for keeping the vegetation floor 6 and the outside air space 7 in an atmosphere suitable for the plant to be cultivated. The lamp 10 is installed inside the plant enclosure 8 in the illustrated example.

  Furthermore, the plant cultivation plant 4 includes an air conditioner 12 for supplying air adjusted to a predetermined condition inside the plant enclosure 8. Normally, the air conditioner 12 takes air from the inner space 13 through the suction port 16, adjusts it, and returns it from the discharge port 18 to the inner space 13.

  In the plant cultivation plant atmosphere adjusting device 2, the lamp 10 is surrounded by a lamp enclosure 20. At least a part of the wall of the lamp enclosure 20 is transparent. Connected to the lamp enclosure 20 is one conduit 22 that selectively connects the inside of the lamp enclosure 20 to the inner space 13 and the outside air space 7. The lamp enclosure 20 is connected to another duct 24 that selectively conducts the inside of the lamp enclosure 20 to the inner space 13 and the outside air space 7. That is, one pipe line 22 branches off at the base of the bifurcated pipe 26, and one of the bifurcated pipes 26 communicates with the inner space 13 and the other communicates with the outside air space 7. The other duct 24 branches off at the base of the bifurcated pipe 28, and one of the bifurcated pipes 28 communicates with the inner space 13 and the other communicates with the outside air space 7. Two-way switching valve portions 30 and 32 are provided at the roots of the bifurcated tubes 26 and 28, respectively.

  When the temperature of the outside air space 7 is higher than the set temperature of the inner space 13, both the one pipe line 22 and the other pipe line 24 are set by the two-way switching valve portions 30 and 32 so as to communicate with the outside air space 7. When the temperature of the outside air space 7 is lower than the set temperature of the inner space 13, both the one pipe line 22 and the other pipe line 24 are set by the two-way switching valve portions 30 and 32 so as to communicate with the inner space 13.

  With this selective setting, when the temperature of the outside air space 7 is higher than the set temperature of the inner space 13, air is taken in from the outside air space 7, and the air passes through the one pipe line 22, the lamp enclosure 20, and the other pipe line 24. The lamp 10 is cooled by this air because it proceeds in the direction of the arrow A through this order and is discharged again into the outside air space 7. Further, the air heated by the heat of the lamp 10 does not flow into the inner space 13. Therefore, an increase in the cooling load of the air conditioner 12 due to the heat of the lamp 10 is prevented, and the air conditioning load is reduced.

Further, by selective setting, when the temperature of the outside air space 7 is lower than the set temperature of the inner space 13, air is taken in from the inner space 13, and the air is one conduit 22, the lamp enclosure 20, and another conduit 24. Since the lamp 10 is cooled by this air and the air warmed by the heat of the lamp 10 returns to the inner space 13. . Therefore, the heating load of the air conditioner 12 can be reduced by the heat of the lamp 10.
Note that one duct 22 is provided with a blowing means 36 such as a fan for blowing air in the direction of arrow A or arrow B.
Moreover, as shown in FIG.1 (b), the inner space 13 of the plant enclosure 8 is divided | segmented by the movable partition 38, and is divided into inner space 13-1 and 13-2. Each of the inner spaces 13-1 and 13-2 can be set to different environmental conditions.
In the passage between the vegetation floor 6 where the plant 9 is cultivated, the mobile detection device 40 is self-propelled to detect the state of the plant 9.

[Management of plant cultivation status]
First, in the plant cultivation plant 4, the case where the state of the plant is detected while the mobile detection device 40 moves is illustrated.

With reference to FIG. 2, the mobile detection apparatus 40 which detects the state of the plant in the plant cultivation plant 4 is shown.
FIG. 2 is a diagram showing the mobile detection device 40 in the present embodiment.
The mobile detection device 40 uses the mobile vehicle body 41 as a carriage. The mobile vehicle body 41 includes wheels 41W serving as moving means, and moves by power supplied from a drive unit (not shown). On the upper surface of the movable vehicle body 41, a bracket 42B is provided that is supported rotatably about a vertical axis. The bracket 42B is provided with an arm portion 42, and the bracket 42B supports the arm portion 42 so as to roll freely in the front-rear direction. As shown in FIG. 2, the mobile detection device 40 includes a plurality of arm portions 42 (42-1, 42-2, 42-3, and 42-4) that are connected in series, and the arm portion 42. An imaging unit 300 is provided at the tip opposite to the movable vehicle body 41.

The mobile detection device 40 includes a power supply unit (not shown) that allows the arm unit 42, the imaging unit 300, and the control unit (not shown) of the mobile detection device 40 to function and to be able to run on its own.
The control unit of the mobile detection device 40 controls the movement of the mobile vehicle main body 41, drives the arm unit 42 as required, drives to determine the position and direction of the imaging unit 300, and causes the imaging unit 300 to function. Control, communication control for communicating detection processing, processing results, and the like based on captured image information are performed.
The mobile detection device 40 detects the state of the plant 9 (FIG. 1) cultivated in the plant cultivation plant 4 (FIG. 1) while moving.
The mobile detection device 40 may be a pedestal provided in place of the mobile vehicle body 41.

Here, with reference to FIG. 3, the outline | summary of the plant cultivation system which controls cultivation of a plant, ie, agricultural crops, is demonstrated.
FIG. 3 is a block diagram showing the configuration of the plant cultivation system in the present embodiment.
The plant cultivation system 1 controls the harvesting time by controlling environmental conditions such as the growing environment in the process from harvesting and shipping the plant to be cultivated, thereby increasing the yield at the scheduled harvesting time. it can.
In order to increase the harvest amount at the scheduled harvest time, there are direct control for managing plant growth and indirect control in each process from germination to harvest.
Direct control includes processing the plant itself by thinning, pruning, or the like.
The indirect control includes setting various environmental conditions.
The plant cultivation system 1 controls the plant and the environment in which the plant is cultivated as an object to be controlled, and optimizes the growing state of the plant, the harvesting time, and the like so as to increase the harvest amount at the scheduled harvesting time.

In the plant cultivation system 1 shown in this embodiment, control of each cultivation process is facilitated by modeling the plant cultivation environment as a control target.
In that control, the plant itself cannot be completely controlled by a direct method, and even in an indoor environment, it is difficult to ensure the uniformity of the environment so that there is no variation, It becomes a factor which makes control difficult. In addition, the degree of growth depends on individual differences between plants, and even within the same individual, variation occurs in the actual harvest time, which makes control difficult.
In this embodiment, the control target is modeled, the factors that make the control difficult are regarded as disturbances, and the model is optimized according to the growth situation, thereby making it easy to control the amount of harvest at the scheduled harvest time To.
A configuration example of such a plant cultivation system will be described.

  The plant cultivation system 1 shown in FIG. 3 includes, for example, a control target unit 100, a determination unit 200, an imaging unit 300, an imaging position moving unit 400, a control unit 500, and a growth degree detection device 70 (display control device). Prepare.

The control target unit 100 includes a plant 9 that is a management target and an environment driving unit 120 that controls the growing environment of the plant 9.
The plant 9 can determine the growth status based on the state of leaves, stems, flowers / fruits, and roots after germination. The growth status can be determined in relation to the elapsed time from the reference time (eg, germination).
For example, in the determination based on the leaf state, the number of leaves, the size of the leaf, the projected area of the overgrown leaf, the wilting condition (angle, aspect ratio), and the like can be given. In the judgment based on the state of the stem, the height, the condition of the branch, the thickness, and the like can be given. In the determination based on the state of the flower / fruit, the number, density, arrangement, fruit maturity, etc. In the determination based on the state of the root, the length, the extent of spread, and the like can be given.

The environment driving unit 120 controls the growing environment where the plant is cultivated according to the amount of environment control supplied from the control unit 500.
The environment driving unit 120 includes a light control unit 121, an air conditioning facility 122, a carbon dioxide processing unit 123, and a water adjustment facility 124.

  The light control unit 121 (light control means) includes a light emitting unit or a light shielding unit, and controls the light amount or wavelength of light given to the plant 9. The general plant 9 performs photosynthesis by the amount of light depending on the appropriate light intensity and irradiation time. In such a plant 9, the growth of the plant 9 is promoted based on the photosynthesis. In addition, the light control unit 121 controls the direction in which the plant 9 grows by controlling the light irradiation direction by using the circadian property of the plant 9. Moreover, unlike natural light such as the sun, the light control unit 121 promotes specific growth by limiting and giving a specific wavelength. For example, light emitting units that can limit the emission wavelength include light emitting diodes and high pressure neon lamps. When the wavelength is limited by the light shielding portion, a wavelength selective filter is used.

The air conditioner 122 controls the ambient temperature, humidity, and air volume where the plant 9 is placed.
Growth can be promoted without damaging the growth of the plant 9 by providing a sufficient volume of air while controlling the air to an appropriate ambient temperature, humidity, and air volume.
The carbon dioxide processing unit 123 controls the carbon dioxide concentration in the air managed by the air conditioning equipment 122. Photosynthesis can be promoted by maintaining an appropriate amount of carbon dioxide. The carbon dioxide processing unit 123 does not simply generate carbon dioxide, but may reuse carbon dioxide generated by other equipment.

The water conditioning facility 124 controls at least one of the amount of water supplied to the plant 9, the water temperature, and the nutrient concentration. Especially in the case of hydroponics, water temperature and nutrient concentration are important management items.
Thus, in the plant cultivation system 1, by providing the environment driving unit 120, while shielding the change of the outdoor environment, and using the form of building the cultivation environment under the artificial environment and the change of the outdoor environment, The present invention can be applied to any management support type cultivation environment in which an appropriate environment is supplementarily constructed. Which form is adopted is appropriately selected according to conditions such as the type of plant to be cultivated, the cultivation period, the growth process, and the environment in which the facility is placed.
For example, it is known that there are plants whose growth is promoted by continuous irradiation of artificial light for a predetermined period from germination. In addition, by selecting the wavelength to be irradiated, it is possible to control the growth amount and promote the target growth.

The determination unit 200 determines the growth status of the plant 9 based on the image information captured by the imaging unit 300 and the growth model of the plant 9.
The determination unit 200 includes a detection unit 210, a growth model unit 220, a situation storage unit 230, and a growth situation determination unit 240.
The detection unit 210 detects the growth status of the plant 9 from the imaged image information of the plant 9, and stores the image information and the growth status in the status storage unit 230.
Moreover, the detection unit 210 detects the position of the feature point of the plant 9 included in the image information from the image information of the captured plant 9 and the position information where the image is captured, and images the detected position information. The time information is stored in association with the image information stored in the status storage unit 230.
The training model unit 220 includes a plurality of training models. The plurality of breeding models set in the breeding model unit 220 is, for example, a breeding situation model 221 (first breeding model) based on an elapsed time that associates an elapsed time from a reference time (such as germination) with the breeding situation of the plant 9. An object to be detected using an actual arrangement model 222 (second breeding model) with respect to the position of the flower and a pattern based on the size, shape or color, in which the actual position when the flower has grown is estimated Are extracted, and a harvest time determination model 223 (third breeding model) for determining the harvest time is included.

The situation storage unit 230 stores the image information of the plant 9, the growing situation, the position information of the feature points of the plant 9 included in the image information, and the captured time information in association with each other.
The growth status determination unit 240 determines the growth status based on the growth model of the plant 9 using at least one of the stored image information and the growth status.
The breeding situation determination unit 240 includes a breeding situation model 221 (first breeding model) based on elapsed time, an actual arrangement model 222 (second breeding model) with respect to the position of the flower, and a harvest time judgment included in the breeding model unit 220. The growth status of the plant 9 is determined based on at least one of the models 223 (third growth model).

The imaging unit 300 images the plant 9 (and the situation around the plant 9), and generates image information. The imaging unit 300 includes an imaging device equipped with a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) sensor, and a light emitting device for auxiliary light for imaging.
The imaging unit 300 is mounted on the distal end portion of the arm unit 42 included in the movable detection device 40 (FIG. 2) included in the imaging position moving unit 400. The imaging position is changed according to the movement of the tip of the arm part 42, and the relative position with respect to the plant 9 as the subject changes. In addition, the imaging unit 300 is mounted on a swivel provided at the tip of the arm unit 42, and can be swung in an arbitrary direction by controlling the swivel. By combining these controls, the imaging unit 300 can image the plant 9 from an arbitrary position and an arbitrary direction.

The imaging position moving unit 400 (imaging position moving unit) moves the position captured by the imaging unit 300 to the instructed position so that the plant 9 can be imaged from the instructed direction.
The imaging position moving unit 400 (imaging position moving means) includes a movable detection device 40 (FIG. 2) that moves an imaging position. 42 (robot arm) is provided.
An optical system that guides light from the subject to the imaging unit 300 is provided at the tip of the arm unit 42, and the imaging position moving unit 400 rotates the optical axis of the optical system in a designated direction.

  The growth degree detection device 70 requests the control unit 500 to provide various types of information. The growth degree detection device 70 detects the degree of growth of the target plant 9 based on various information acquired from the control unit 500 in response to the request. The growth degree detection device 70 is carried by a user who moves in the plant cultivation plant 4 and detects the degree of growth of the plant 9 in the plant cultivation plant 4.

The control unit 500 (control unit) determines the growth status based on the growth model of the plant 9 based on the image information of the plant 9 imaged from the position moved by the imaging position moving unit 400 (imaging position moving unit). According to the result, the growing environment of the plant 9 is controlled.
The control unit 500 includes a growth model generation unit 510, an environment control unit 520, a position control unit 530, a production plan design unit 540, a harvest plan design unit 550, a pruning plan design unit 570, and a communication processing unit 590.
The training model generation unit 510 generates a training model based on the image information, and updates the training model held in the training model unit 220.
The growth model generation unit 510 is a first model generation unit that determines the first growth model according to the growth status generated from the image information, and a second model that detects the position of the flower at the time of flowering from the image information and generates a second growth model. The model generation unit includes a third model generation unit that generates a third breeding model for determining the harvest time from the image information.
Further, when generating the growth model, the growth model generation unit 510 may generate the growth model based on the information about the growth status of each plant 9 detected by the growth degree detection device 70 or image information. .

The environment control unit 520 generates an environment control amount that controls the growing environment according to the determined growing situation. For example, the environment control unit 520 controls the light amount or wavelength of light given to the plant 9 by the light control unit 121 when the light given to the plant 9 is controlled according to the determined growing situation. The environment control unit 520 controls the carbon dioxide concentration at which the plant 9 is placed by the carbon dioxide processing unit 123 when the carbon dioxide concentration to be given to the plant 9 is controlled according to the determined growing situation. When the environment control unit 520 controls the air to be given to the plant 9 according to the determined growth status, the air conditioning facility 122 uses at least one of the temperature, humidity, and air volume of the air supplied to the plant 9. Control one. When the environment control unit 520 controls the water to be supplied to the plant 9 according to the determined breeding situation, the water control facility 124 supplies at least one of the amount of water, the water temperature, and the nutrient concentration supplied to the plant 9. Control one.
The position control unit 530 controls the imaging position moving unit 400 to instruct the position and direction in which the plant 9 is imaged. The position control unit 530 controls the imaging position moving unit 400 to change the position and imaging direction of the imaging unit 300 and capture image information according to the purpose.

The production plan design unit 540 designs a production plan based on the captured image information.
The harvest plan design unit 550 designs a harvest plan based on the third breeding model.
The arrangement plan design unit 560 determines the position of the plant 9 in the plant cultivation plant 4 based on the positional relationship with other plants 9 located around the target plant 9 according to the first growth model of the target plant 9. Design a deployment plan.
The pruning plan design unit 570 (first pruning plan design unit) designs a pruning plan for pruning the plant 9 according to the first breeding model. Further, the pruning plan design unit 570 (second pruning plan design unit) designs a pruning plan for pruning the flowers (fruits) of the plant 9 according to the second growth model.
The communication processing unit 590 performs communication processing with the growth degree detection device 70. The communication processing unit 590 accepts a request and various information from the growth degree detection device 70 to the control unit 500, and transmits various information corresponding to the request to the growth degree detection device 70.
The communication processing unit 590 receives information or image information about the growth status of each plant 9 detected by the growth degree detection device 70.
Moreover, the various information which the communication process part 590 transmits is the arrangement | positioning information of the plant 9, a 1st breeding model, etc., for example.

(Control of plant harvest time)
With reference to FIG. 4, control of the harvest time of the plant in this embodiment is demonstrated.
FIG. 4 is a diagram illustrating the control of the plant harvest time in the present embodiment.
In the graph shown in FIG. 4, the horizontal axis indicates the number of days (hours) elapsed from germination, the vertical axis indicates the degree of plant growth, and has a so-called growth curve shape. As shown in this graph, the degree of growth can be defined as a function according to the number of days elapsed. The standard.

The graph S4a is a breeding model showing a growth curve that is used as a reference for the management of the cultivation status of the plant 9. The harvest time can be set based on this graph S4a.
The graph S4b shows a process of growing later than the growth model shown in the graph S4a without managing the plant growth status. When growing as shown in the graph S4b, the harvesting time is delayed from the plan. Therefore, in the present plant cultivation system 1, the delay of the breeding situation detected in the process of breeding is controlled so as to promote the breeding situation by controlling the surrounding environment.
When promoting the growing situation, the processing content is selected according to the type of the plant 9 to be controlled and the time when the countermeasure is to be implemented. For example, as a breakdown of the measures, there are methods such as increasing the amount of light irradiation, increasing the ambient temperature and water temperature, and increasing the required nutrient amount.

  On the other hand, also in the case of delaying the breeding status, the processing content is selected according to the type of the plant 9 to be controlled and the timing of implementing the countermeasure. For example, as a breakdown of the measures, there are methods such as reducing the amount of light irradiation, reducing the ambient temperature and water temperature, and reducing the amount of necessary nutrients.

In addition, the detection of the breeding status by the detection unit 210 can be performed by detecting the status of each individual and making a comprehensive determination based on the detected results. In the comprehensive determination, various statistical processing techniques such as averaging the detected detection results and detecting variations can be applied.
For example, it is also possible to divide the whole into a plurality of sections and collectively determine the results detected from the plants 9 included in the sections. By dividing the partition in this way, it is possible to detect variations due to the arranged positions.

It is also possible to set different environments by arranging different types of plants for each section.
The detection of the growth status of the plant 9 may be detected based on one determination criterion or based on a plurality of variables detected based on a plurality of determination criteria.

(Determination of plant growth status)
With reference to FIGS. 5 to 7, the determination of plant growth status in this embodiment will be described.
In the determination of the growth status of the plant 9 performed by the determination unit 200, a plurality of determination methods can be selected according to the detection target.
The detection target can be selected from leaves, stems, flowers / fruits, and roots.
The detection unit 210 detects the state of the selected detection target in accordance with the determination items shown below.
The breeding situation determination unit 240 judges the breeding situation based on the breeding model of the plant 9 using at least one of the stored image information and the breeding situation for the detected state of the detection target. The training status determination unit 240 selects a training model to be used as a criterion for determination from a plurality of training models provided in the training model unit 220 according to the determination item.

In the determination based on the leaf state in the determination unit 200, the number of leaves, the size of the leaf, the projected area of the overgrown leaf, the wilting condition (angle, aspect ratio of one leaf), and the like are listed as determination items. .
The number of leaves is detected by counting the number of leaves existing within a predetermined range or the number of leaves attached to a stem (specified branch).
The size of one leaf is the size of the specified leaf, the average of the size of the leaf existing within a predetermined range, or the size of the leaf attached to the stem (specified branch) It is detected by calculating the average of. The size of the leaves can be the length in the longitudinal direction (vertical), the length in the direction orthogonal to the longitudinal direction (horizontal), and the area.
The projected area of the overgrown leaves detects the area projected from the side surface of the plant 9 onto the vertical plane.
The degree of leaf wilting can easily detect stress such as lack of moisture. In the wilted state, the leaf tip faces downward, and therefore, it can be detected from the angle formed by the horizontal plane and the leaf surface. Alternatively, the aspect ratio of one leaf may be calculated. By wilting, there is no leaf tension, and the apparent width becomes narrower. Therefore, the aspect ratio of a single leaf changes depending on the condition of wilting.

In the determination based on the state of the stem in the determination unit 200, height, branch tension, thickness, and the like are listed as determination items.
The height detects the height of the tip of the stem (or the leaf at the tip). The height of the imaging unit 300 can be detected from the height of the imaging unit 300 by matching the height of the imaging unit 300 with the height of the stem.
The degree of branch tension detects the number of branches or the length of the branches.
The thickness of the stem detects the thickness of the stem having a reference height.

In the determination based on the state of the flower / fruit in the determination unit 200, the number, density, arrangement, actual maturity, and the like are listed as determination items.
FIG. 5 is a diagram showing the positional relationship between flowers and fruits.
FIGS. 5 (a) and 5 (c) show the flowering state, and FIGS. 5 (b) and 5 (d) show the fruiting state.
Since the actual position depends on the position of the flower, the actual number and position can be specified by specifying the number and position of the flowers. Flowers f1 and f2 shown in the flowering period change as fruits F1 and F2 in the mature period.
The number of flowers (fruits) is detected by counting the quantity existing within a predetermined range or the number of flowers (fruits) attached to the stem (specified branch).
The density of the flower (fruit) can be calculated by dividing the quantity of the detected flower (fruit) by the unit capacity. Or, simply, the number of flowers (fruits) existing in a predetermined range of the captured image is counted.
The arrangement of flowers (fruits) is detected by using a three-dimensional measurement technique. Moreover, the actual position of the fruiting period can be estimated from the detected position of the flower. The actual estimated position can be calculated based on the three-dimensional position of the flower by assuming the actual size and weight and using the strength and length of the branch from which the fruit hangs as calculation conditions.
From this calculated estimated position, it is possible to estimate interference with an adjacent actual object. That is, it can be estimated that interference occurs when the relative distance is closer than the actual size.

Actual maturity may be detected based on actual color changes.
FIG. 6 is a diagram for explaining detection of actual maturity.
As shown in FIG. 6A, the present invention can be applied when the color of the ripe fruit F3 is different from the color of the fruit F4 that is not fully ripe. The degree of maturity can be detected by detecting the change in color of the actual fruit F4 that is not fully ripe to the color C2 of the fruit F3 that is ripe.
As shown in FIG. 6B, a part of the actual color C2 (for example, the lower side) is discolored, and the maturity can be detected by detecting the discolored range (Z2).
The actual maturity level may change higher as the maturity level increases. Actually, a predetermined amount of light (measurement auxiliary light) is irradiated to detect the amount of transmitted light. For example, the imaging unit 300 detects the transmitted light by irradiating measurement auxiliary light from a position close to the actual lower part (or side surface). At that time, the positional relationship is adjusted so that direct light from the auxiliary light source does not enter the angle of view of the imaging unit 300.

  In the determination based on the state of the root in the determination unit 200, the length, the extent of spread, and the like are listed as determination items. If it is hydroponics, the length of a root may be detectable. The length of the root is detected from the position of the strain. The extent of root detection detects the area where the root has spread from the position of the strain. Although both the length and extent of roots are underwater, they are difficult to detect directly, but it is possible to indirectly detect shadows by roots by irradiating auxiliary light.

In addition to the direct detection method for detecting the plant 9, the detection unit 210 shown above indirectly detects information that cannot be directly detected by creating a three-dimensional model based on the directly detected result. May be.
FIG. 7 is a diagram showing the state of leaves.
FIG. 7 (a) shows an image of a bird's-eye view of four stems with leaves.
The image information captured from the side surface of the plant 9 can acquire the image information as shown in this figure, but it is difficult to detect a planar state. Therefore, four leaves are extracted as three-dimensional information.
FIG. 7B shows the result of projecting on the horizontal plane based on the result of using the four leaves shown in FIG. 7A as the three-dimensional information. By adopting such an order conversion method, it is possible to create a planar projection view of leaves included in an arbitrary height range even when it is difficult to image from the upper part of the plant 9. A broken-line circle indicates a circle passing near the tip of the leaf.

FIG. 7C shows a state where the four leaves shown in FIG. The state where the tip of each leaf hangs down toward the ground is shown. The four leaves in this state are extracted as three-dimensional information.
FIG. 7 (d) shows the result of projection on the horizontal plane based on the result of using the four leaves shown in FIG. 7 (c) as three-dimensional information in the same manner as FIG. 7 (b). Since the four leaves are in a wilted state, the circle passing near the tip of the leaf is smaller than the circle passing near the tip of the leaf shown in FIG.
By detecting the diameter of the circle passing through the vicinity of the tip of the leaf indicated by the projected leaf image, it is possible to detect the degree of leaf growth, the condition of wilting, and the like.
Moreover, it can be applied to detecting the state of a plant flower such as “chrysanthemum” where a flower blooms at the tip of the stem.

(Generation of training model)
Next, the growth model generated by the growth model generation unit 510 in the control unit 500 will be described.
In the above-mentioned “(Plant Growing Status Judgment)”, the detection of the plant 9 growing status by the detection condition suitable for the purpose according to the judgment item is shown. It shows about the production | generation of the breeding model used as a reference | standard (determination item) for determining the detected breeding condition.
First, in order to determine the growth status of the plant 9 according to the elapsed time from the reference time (such as germination), the process of associating the elapsed time from the reference time (such as germination) with the growth status of the plant 9 A training situation model by time (first training model) is required.
Various items can be selected for the first breeding model. For example, in the determination based on the leaf state, the number of leaves, the size of the leaf, the projected area of the overgrown leaf, the wilting condition (angle, aspect ratio), and the like can be given. In the judgment based on the state of the stem, the height, the condition of the branch, the thickness, and the like can be given. In the determination based on the state of the flower / fruit, the number, density, arrangement, fruit maturity, etc. In the determination based on the state of the root, the length, the extent of spread, and the like can be given.
In the selected item, for example, modeling can be performed by using a numerical value based on a growth curve corresponding to an elapsed time from a reference time (such as germination) as a reference value.
This first breeding model can be used not only as a reference for growth of an individual but also when determining interference with an adjacent individual.

Further, an actual arrangement model (second breeding model) with respect to the position of the flower, which estimates the actual position when the flower has grown, is required.
In this second breeding model, the number, density, arrangement, and the like are raised in the determination based on the state of the flower / fruit.
In the selected item, the model can be modeled by using the estimated position of the actual position with respect to the position of the flower, the position with respect to the feature point, and the actual size as reference values.
This second breeding model can be used not only as a reference for individual actual growth, but also when determining adjacent real interference based on the size and isolation distance of each adjacent real.

  In addition, a harvest time determination model (third breeding model) is required that uses the pattern based on size, shape, or color to extract the characteristics of the target to be detected and determine the harvest time.

That is, the breeding model generation unit 510 generates a reference value by modeling a numerical value corresponding to the item selected as the determination item, and sets it as the initial value of the growth model unit 220.
The growth model generation unit 510 generates and updates the generation of the growth model based on the image information. If there is a large discrepancy between the generated breeding model and the actual state, an unreasonable control amount may be given. In that case, the growth model generation unit 510 determines and corrects the growth model based on a predetermined threshold value, and updates the value held in the growth model unit. The update of the breeding model may be changed by a growth process determined in stages.

  In the determination unit 200, a value detected by determining (calculating) the growth status determination unit 240 based on the value detected by the detection unit 210 and the value of each growth model held in the growth model unit 220. And the difference with the value of each breeding model held in breeding model part 220 is calculated, and the breeding situation is judged.

  The breeding model generation unit 510 may calculate the value of the breeding model by statistically processing the detected value detected in the past, the value of the model set in the past, and the like.

(Control of the surrounding environment)
The environment control unit 520 generates an environment control amount that controls the growing environment according to the determined growing situation. For example, the environment control unit 520 controls the light amount or wavelength of light given to the plant 9 by the light control unit 121 when the light given to the plant 9 is controlled according to the determined growing situation. The environment control unit 520 controls the carbon dioxide concentration at which the plant 9 is placed by the carbon dioxide processing unit 123 when the carbon dioxide concentration to be given to the plant 9 is controlled according to the determined growing situation. When the environment control unit 520 controls the air to be given to the plant 9 according to the determined growth status, the air conditioning facility 122 uses at least one of the temperature, humidity, and air volume of the air supplied to the plant 9. Control one. When the environmental control unit 520 controls the water to be supplied to the plant 9 according to the determined breeding situation, the water adjustment facility 124 supplies at least one of the amount of water supplied to the plant 9, the water temperature, and the nutrient concentration. Control.
For example, in order to promote the growing situation, the environment control unit 520 controls the light control unit 121 to increase the illuminance of light given to the plant 9, extend the irradiation time, and increase the light amount. Moreover, the raising condition can be promoted by increasing the ambient temperature of the plant 9 by the air conditioner 122. The control of such conditions varies depending on the type of plant, the growing process, etc., and the control content suitable for each is determined.
Control of the surrounding environment is repeatedly performed at a relatively short period by the environment control unit 520 at the time of cultivation so that the damage to the plant 9 can be reduced and the cultivation can be performed in an appropriate environment by detecting the abnormal value early. To do.

(Control of imaging position)
The position control unit 530 controls the imaging position moving unit 400 to instruct the position and direction in which the plant 9 is imaged. The position control unit 530 controls the imaging position moving unit 400 in order to change the position and imaging direction of the imaging unit 300 and capture image information according to the purpose.
Since the imaging position moving unit 400 includes the movable detection device 40 including the multi-indirect type arm unit 42, the position of the imaging unit 300 can be set flexibly.
Even if the fruit of the plant 9 becomes a shadow of a leaf or other fruit, the imaging unit 300 can be moved to a position that is not affected by them. Thereby, even if the fruit of the plant 9 to be photographed is hidden behind the leaves or the shadows of other fruits, the fruit of the plant 9 to be photographed can be reliably imaged by the imaging unit 300 without being hindered by them. it can.
In addition, although the imaging part 300 shall be arrange | positioned at the front-end | tip part of the arm part 40, you may be provided in the arm near the front-end | tip of the arm part 40, or the pan head provided in the same arm. Moreover, although it was set as the front-end | tip of the arm part 40, you may incorporate in the arm part 40. FIG.

(Design of production plan)
The production plan design unit 540 designs a production plan based on the captured image information. By designing a production plan based on the image information, it is possible to efficiently cultivate the plant 9 by effectively using the facilities of the plant cultivation system 1 as well as determining whether or not to harvest. This production plan can also be used to adjust the shipment amount at the target time.

(Design of harvest plan)
The harvest plan design unit 550 designs a harvest plan based on the third breeding model.
By designing based on the third breeding model, it is possible to increase the accuracy of determination as to whether or not harvesting is possible. By performing this determination, for example, the actual harvest time can be optimized, and the quality can be made uniform.
Image information for creating a harvest plan can also be earned when approaching each plant 9 for other processing. Alternatively, if the harvesting order is formulated by a harvesting plan designed in the past, the detection frequency can be controlled according to the harvesting order. That is, it is possible to increase the detection frequency for the fruit whose harvest time is determined to be delayed by the harvest plan and to increase the detection frequency for the fruit whose harvest time is close, so that the detection efficiency can be increased.
It can also be used as a determination condition for raising or postponing the harvest time. Thereby, the quantity according to the required shipment quantity can be harvested.

(Design of layout plan)
The arrangement plan design unit 560 designs an arrangement plan of the plants 9 located around according to the first breeding model. By designing based on the first breeding model, the size of the individual plant 9 after growth can be calculated.
By formulating this arrangement plan, it is possible to efficiently arrange plants while preventing interference with adjacent plants or facilities in the growth process. This arrangement plan is a criterion for determining an appropriate arrangement or the like when so-called thinning is performed or when different types of plants do not match each other.

(Design of pruning plan)
The pruning plan design unit 570 designs a pruning plan for pruning the plant 9 according to the first breeding model.
Thereby, the case where it interferes with the adjacent plant 9 or its own branch can be estimated, and the branch to be pruned and its position can be selected appropriately.
Moreover, the pruning plan design part 570 designs the pruning plan which prunes the flower (fruit) of the plant 9 according to a 2nd growth model.
Thereby, the case where it interferes with an own branch and a fruit can be estimated by calculating the position which grows in the plant 9, the branch and the fruit to be pruned can be selected, and the pruning position can be appropriately selected. .

(Control of plant cultivation system)
In the plant cultivation system 1 shown in the present embodiment, the imaging unit 300 images the plant 9 and generates image information.
The imaging position moving unit 400 (imaging position moving unit) includes an (articulated) arm unit 42 (robot arm) that moves the position captured by the imaging unit 300, so that the plant 9 can be moved from any position and direction. The situation can be measured, and the harvesting time can be controlled by controlling the environmental conditions based on the measured image information.

Such a plant cultivation system 1 makes it easy to increase the scale.
Moreover, in the plant cultivation system 1, productivity can be improved and the quality improvement can be contributed by determining correctly the growth condition (growth degree) of the plant 9. FIG.

[Growth detection device]
Then, with reference to FIG. 8, it shows about the growth degree detection apparatus which detects the growth degree of the plant in the plant cultivation system 1. FIG.
In the following description, the degree of plant growth is an index indicating the progress of growth in the growth process. In order to detect the degree of growth of the plant 9 in the growth process of the plant 9 (FIG. 1) that is the detection target of the growth degree detection device, a plurality of stages for classifying according to the growth of the plant 9 are provided.
The growth degree detection device 70 shown in this figure will be described by taking as an example a case where the growth degree detection device is configured as a portable terminal device carried and used by a user.
FIG. 8 is a diagram showing a growth degree detection device 70 in the present embodiment.
The growth degree detection device 70 shown in this figure is configured as a tablet-type device.
The growth degree detection device 70 includes an imaging unit 71 that images a subject and a display unit 74 that displays the captured image, and is configured in an integrated housing. The degree-of-growth detection device 70 displays an image captured by the imaging unit 71 directed toward the subject on the display unit 74. Moreover, the growth degree detection apparatus 70 acquires the image information of the plant 9, and detects the growth degree of the plant 9 according to the stage provided according to the growth of the plant 9 based on the acquired image information.

FIG. 9 is a block diagram showing the growth degree detection device 70 in the present embodiment.
The same components as those in FIGS. 1 to 3 and FIG.

The growth degree detection device 70 includes an imaging unit 71, a growth state detection unit 72, a display control processing unit 73, a display unit 74, an operation input detection unit 75, an acceleration detection unit 76, and a control unit 79.
The imaging unit 71 includes a camera unit 711, an imaging processing unit 712, a storage unit 713, and an imaging position acquisition unit 714.
The camera unit 711 converts a plant obtained as a detection target as a main subject into an image signal from an image obtained by imaging the subject.

The imaging processing unit 712 controls the camera unit 711 to perform imaging.
Further, the imaging processing unit 712 obtains image information based on the image signal converted by the camera unit 711. The imaging processing unit 712 obtains subject position information for specifying the position of the photographed subject in correspondence with the image information. The imaging processing unit 712 uses the identification information corresponding to the detection target range set by the detection target range setting unit 723 as subject position information of the image information. Alternatively, the imaging processing unit 712 may generate position information corresponding to the feature points extracted from the captured image information, and the generated position information may be the subject position information.
In addition, the imaging processing unit 712 stores the imaging position information obtained by the imaging position detection unit 714, the subject position information obtained by the imaging processing unit 712, and the imaging time information in association with the captured image information. Stored in the unit 713.

  The imaging position detection unit 714 obtains imaging position information that specifies the imaging position from the image information indicated by the image signal captured by the camera unit 711. A reference point for obtaining the imaging position information is provided in the vicinity of the plant 9 in advance, and the imaging position detection unit 714 calculates the imaging position information based on the reference point included in the captured image. Alternatively, the imaging position detection unit 714 may acquire position information obtained by a GPS system and use the acquired position information as imaging position information.

  The storage unit 713 associates the imaging position information obtained by the imaging position detection unit 714, the subject position information obtained by the imaging processing unit 712, and the imaging time information with the image information captured by the camera unit 711. Remember. Various types of information stored in the storage unit 713 are stored as the latest information. The subject position information may be identification information of the corresponding plant 9.

Further, the imaging unit 71 may synthesize a panoramic image or a three-dimensional composite image centered on the captured imaging position based on the image information obtained by the camera unit 711 by the imaging processing unit 712. When synthesizing a panoramic image or a three-dimensional synthesized image, the imaging processing unit 712 captures image information based on the synthesized panoramic image or three-dimensional synthesized image, imaging time information, imaging position information, and subject position information. The image information may be stored in the storage unit 713 in association with each other. Thus, by combining the panoramic image and the three-dimensional composite image from the image information obtained by capturing the inside of the plant cultivation plant 4, the image information of the panoramic image and the three-dimensional composite image corresponding to each captured image position is obtained. (3-dimensional composite image information) can be obtained.
In addition, it is desirable that the imaging unit 71 has a camera shake correction function for correcting camera shake that acts on the imaging unit 71.

The breeding status detection unit 72 detects the degree of growth of the plant 9 from the state of the plant 9 included in the detection target range set in accordance with the stage of growth.
The breeding status detection unit 72 detects the degree of growth of the plant 9 based on the image information of the plant 9 imaged by the imaging unit 71, and sets the information as the information indicating the growth status (growth level) of the plant 9.
The breeding status detection unit 72 includes a growth degree detection unit 721, a growth stage determination unit 722, a detection target range setting unit 723, an information acquisition unit 725, and a past information storage unit 726.

The information acquisition unit 725 acquires the image information of the plant 9 captured at predetermined time intervals from the imaging unit 71.
For example, the information acquisition unit 725 includes image information based on the image of the plant 9 captured by the camera unit 711, imaging time information indicating the time when the image was captured, and imaging position information that identifies the captured position. The subject position information for specifying the position of the photographed subject is acquired from the imaging unit 71 including the camera unit 711. The information acquisition unit 725 stores the acquired image information, imaging time information, imaging position information, and subject position information in the past information storage unit 726 in association with each other.
The image information acquired by the information acquisition unit 725 is either image information captured by the camera unit 711 or image information stored in the storage unit 713 of the imaging unit 71. Further, the image information stored in the storage unit 713 of the imaging unit 71 is image information based on 3D composite image information indicating a panoramic image or a 3D image synthesized based on the image information captured by the imaging unit 71. It may be included.
The past information storage unit 726 stores the image information acquired by the information acquisition unit 725, the imaging time information, the imaging position information, and the subject position information in association with each other. The past information storage unit 726 may store the combined panoramic image acquired by the information acquisition unit 725 and image information based on 3D composite image information indicating a 3D image.

The growth degree detection unit 721 detects the degree of growth of the plant 9 based on the image information of the plant 9 captured at a predetermined time interval.
First, the growth degree detection unit 721 refers to image information stored in the past information storage unit 726 in a predetermined specific detection target range based on the subject position information and the imaging time information, and is determined in advance. The image information of the plant 9 acquired at a predetermined time interval is obtained. The growth degree detection unit 721 obtains difference information of the detection target indicated by the image information from the image information of the plant 9 captured at the predetermined time interval. The growth degree detection unit 721 detects the growth of the plant 9 based on the difference information of the detection target. This difference information to be detected includes difference information regarding lengths such as branch length, stem thickness, root length, and fruit size, and difference information regarding hues such as leaves, stems, and fruits. .
Further, the growth degree detection unit 721 detects to what extent the actual growth amount has reached the growth amount expected to reach a predetermined period, and indicates the degree as the growth degree. For example, the growth degree detection unit 721 may use the ratio of the actual growth amount to the growth amount expected to reach a predetermined period as the growth degree.
In the growth of the plant 9, the growth degree for each specific part in the plant 9 does not progress uniformly because the degree of growth is different. As described above, in the plant 9, a part is appropriately selected, and the growth degree detection unit 721 can detect a degree of growth suitable for each time by detecting the degree of growth of the part.

The degree of growth of the plant 9 is detected at a predetermined time interval that is predetermined according to the stage of growth for each specific detection target range that is defined.
For example, the degree of growth of the plant 9 is detected based on two images taken at a predetermined time interval.
When the growth process of the plant 9 is divided into a plurality of stages, the amount of change in the degree of growth of the plant 9 is different in each stage and each part. Therefore, the time interval required to detect the degree of growth of the plant 9 is determined according to each stage and each part of the growth process. Thereby, the growth degree detection unit 721 can reduce the number of detections without performing detection more than necessary, and can efficiently collect necessary information.
When the change amount of the growth degree is large, the time interval is set to be relatively short so that the growth degree can be detected, and when the change amount of the growth degree is small, the time interval is set to be relatively long. Thus, even when the time interval is determined according to the growth stage and the number of detections is reduced, the degree of growth can be detected.

  The growth stage determination unit 722 determines the stage of growth of the plant 9 based on the degree of growth calculated from the difference information of each part detected by the growth degree detection unit 721.

The detection target range setting unit 723 sets the detection target range of the plant 9 according to the determined stage of growth. For example, the detection target range setting unit 723 may set, as the detection target range, a range in which a portion of the plant 9 that is likely to ripen is included in the harvest stage in which the fruit of the plant 9 is harvested.
By the way, it is known that the easiness of ripeness varies depending on the plant 9. In the case of cocoons, bunches near the tip of the branch tend to ripen faster. Further, if the same tress is used, the upper part of the tress, that is, the portion closer to the branch tends to ripen faster. Therefore, the detection target range setting unit 723 sets a weight for setting a detection target range to be preferentially selected for each detection target range. In this weighting, a range including a portion that is easily ripened is set to be preferentially set as a detection target range. The detection target range setting unit 723 can efficiently detect the ripe fruit by setting the detection target range according to this weighting.

In addition, the detection target range setting unit 723 determines each detection target range so as to correspond to the actual harvesting of the detection target range, or to include the fruits that can be harvested at the same time.
Further, the detection target range setting unit 723 obtains the identification information of the harvested fruit with reference to the identification information for identifying the fruit, and from among the fruit remaining after harvesting the fruit (that is, the unharvested fruit The weighting is defined so that a range including a portion that is easy to mature is given priority as a detection target range. As a result, even when fruits are harvested in stages, ripe fruits can be detected efficiently.

Alternatively, the detection target range setting unit 723 may set, as the detection target range, a range including a portion that is easy to ripen in the fruit in the harvesting stage in which the fruit of the plant 9 is harvested.
It is known that the easiness of ripening differs even in the same fruit of the plant 9. In the case of apples, the upper half tends to ripen faster than the lower half when attached to the branch. Therefore, the detection target range setting unit 723 sets a weight for setting a detection target range to be preferentially selected for each detection target range. In this weighting, a range in which a portion of the plant 9 that is easy to ripen is included is set to be preferentially set as a detection target range. The detection target range setting unit 723 can efficiently detect actual ripeness by setting the detection target range according to this weighting. Or you may set fixedly the range in which the part which is easy to ripen in the fruit of the plant 9 is contained as the detection target range of the fruit.

The detection target range setting unit 723 selects a desired detection target range from the detection target range candidates determined according to the stage of growth.
For example, when the stage of growth is determined such as germination, flowering, pollination, and maturity (harvest), a desired detection target can be selected from the detection target ranges according to each growth stage. Select a range. As a detection target range candidate, a range set so as to be able to image the detection target shown in the above-mentioned “(Plant growth status determination)” can be set as a detection target range candidate.
As a specific example, a detection target range is set for leaves or stems at the stage from germination to flowering, and a detection target range for flowers at the stage from flowering to pollination. In the stage from the time of pollination to the time of harvesting, the detection target range is set for the fruit. In setting the detection target range, a desired detection target range is determined according to the type of target plant.

The display control processing unit 73 generates an image to be displayed on the display unit 74 based on the image information captured by the imaging unit 71 and the image information stored in the past information storage unit 726.
The display control processing unit 73 includes a display control unit 731 and a display information generation unit 732.
The display control unit 731 corresponds the display information generated from the information related to the growth of the plant stored in the past information storage unit 726 in association with the identification information for identifying the display target plant to the display target image. To display.
The display control unit 731 is image information of images captured at the first time interval, and is a display target image stored in the past information storage unit 726 in association with time-series information based on the order of imaging. Is read from the past information storage unit 726 and displayed according to the second time interval according to the time-series information. Thereby, the display control unit 731 can perform display similar to fast-forward playback in a so-called recording apparatus.
Further, the display control unit 731 displays the display information corresponding to the display target image in association with the display target image that is sequentially read from the past information storage unit 726 according to the time series information and displayed. Even when the display information corresponding to the display target image changes with the passage of time, the display control unit 731 can display the display information corresponding to the display target images that are sequentially read and displayed.
The display control unit 731 may display the time information in association with the image of the image information associated with the time information.

The display information generation unit 732 generates display information related to the display target plant as display information associated with the display target plant based on the identification information for identifying the display target plant.
In addition, the display information generation unit 732 generates display information based on the estimation information obtained by estimating the stage of plant growth.

In the display control processing unit 73 configured as described above, the display control unit 731 includes the position of the first image based on the image information of the plant 9 newly captured by the imaging unit 71 and stored in the past information storage unit 726. Then, it generates an image for determining the position of the second image based on the image information stored in the past information storage unit 726 that is image information captured in the past.
The display control unit 731 generates a composite image that is an image based on the first image and the second image, and causes the display unit 74 to display the composite image.

In the display control processing unit 73, the display information generation unit 732 changes the size of the display information to be generated and the area for displaying the display information according to the acceleration detected by the acceleration detection unit 76. Alternatively, the display information generation unit 732 may interrupt the display of display information according to the magnitude of acceleration detected by the acceleration detection unit 76.
Detailed processing by the display control processing unit 73 will be described later.

The display unit 74 displays the image generated by the display control processing unit 73.
The operation input detection unit 75 acquires operation information for causing the growth degree detection device 70 to function by detecting a user operation, and notifies the control unit 79 of the operation information. The operation input detection unit 75 may be configured as a touch panel provided on the display surface of the display unit 74.
The acceleration detection unit 76 includes an acceleration sensor that detects a displacement of a position or an angle of the growth degree detection device 70 as an acceleration. For example, by using the acceleration detection unit 76 as a 6-axis acceleration sensor, information for detecting the inclination and movement of the growth degree detection device 70 can be obtained based on the acceleration detected by the acceleration detection unit 76. In short, the acceleration detection unit 76 detects the displacement or angular displacement of the imaging unit 71 and the display unit 74 included in the growth degree detection device 70 as acceleration.

The control unit 79 controls each unit constituting the growth degree detection device 70 based on the condition detected by the operation input detection unit 75.
The control unit 79 supplies the detected acceleration to the imaging unit 71, the growth status detection unit 72, and the display control processing unit 73. Based on the acceleration detected by the acceleration detector 76, the control unit 79 calculates the displacement or angular displacement of the growth degree detection device 70 and the amount of change thereof, and detects the inclination and movement of the growth degree detection device 70. To do. The control unit 79 calculates the imaging direction from the information on the inclination and movement of the growth degree detection device 70, and supplies the imaging direction to the imaging unit 71 and the growth state detection unit 72.

Then, the process which detects the growth degree of the detection target by the growth degree detection apparatus 70 is demonstrated in order.
(About the position detection method for detecting the degree of plant growth)
With reference to the drawings, detection of the degree of growth of a plant that is a detection target in the present embodiment will be described.
FIG. 10 is a diagram showing a state of a plant that changes according to growth.
When the detection target is the plant 9, the growth direction of the detection target changes according to the growth process.
For example, as shown in the figure, when a plant grows, the branch grows, and when the fruit matures, the branch becomes a branch due to the weight of the fruit and changes flexibly according to the stress. Even if a predetermined part of the plant 9 is defined as a feature point, the position of the feature point changes according to the growth of the plant 9 and is not fixed at a specific position.
Furthermore, there may be a case where the leaves are overgrown and the position (feature point) of the mark selected in the past is hidden.
As described above, in order to detect the growth in the plant 9, the position selected as the mark (position of the feature point) changes according to the growth process, so that it is difficult to detect the growth of the plant that changes in the long term. Is a factor.

As a general position detection method, there is a detection method for detecting the position of a detection target from image information obtained by capturing the detection target whose position and shape do not change, such as a landscape or a building. Such a detection target does not change the relative positional relationship of the feature points in the detection target. Therefore, it is possible to detect a position where an image is captured using a detection method on condition that the relative positional relationship between feature points of a detection target does not change, and to detect a difference between two image positions. The corrected image can be corrected.
However, the general position detection method is based on the condition that the relative positional relationship between the feature points of the detection target does not change, and is therefore applied when detecting a long-term change in growth of the plant 9. Is difficult.
Thus, even if the detection method on condition that the relative positional relationship of the feature points in the detection target does not change is applied as it is, the growth of the plant whose feature points change in the long term can be detected. I can't.
In order to continue detecting the position of the feature point of the plant 9 for a long period of time, a method for detecting the position corresponding to the growth of the plant 9 that changes with the growth is required.

(Principle of plant growth detection method in this embodiment)
Therefore, the growth degree detection device 70 according to the present embodiment detects plant growth on the assumption that the relative positions of feature points within the detection target change.
In the process of detecting the growth of the plant by the growth degree detection device 70, a process of detecting the imaging position and a process of detecting a change in the growth of the plant are performed.
First, a process for detecting the imaging position by the growth degree detection device 70 will be described. In the process of detecting the imaging position, for example, three reference points are provided around the plant 9, and the growth degree detection device 70 detects the position of the reference point. Since the position of such a reference point does not change according to the growth of the plant 9, the above-described “detection method on condition that the relative positional relationship of the feature points in the detection target does not change” or the like is applied. be able to. Thus, by detecting the imaging position in advance before detecting the change in plant growth, the detection accuracy of the change in plant growth can be increased. Alternatively, the imaging position may be comprehensively detected including position information obtained from GPS (Global Positioning System) in addition to the imaging position information detected from the image information.

Next, processing for detecting changes in plant growth by the growth degree detection device 70 will be described.
Since the plant 9 grows, the detection target may move to a different position. In the process of detecting the growth change of the plant by the growth degree detection device 70, even if the position of the imaging unit 71 is fixed and the plant 9 can be imaged, the plant 9 grows and the detection target is at a different position. Due to the movement, there may be a case where the image sticks out of the imageable region. Therefore, it is necessary to take an image so that the detection target does not protrude from the imageable region.
Further, in order to determine the degree of growth from the obtained image, the growth degree detection device 70 compares the image taken before the growth with the image taken after the growth. In order to compare the two images, it is desirable to capture the image after growth so that the image is similar to the image captured before the growth. In order to photograph the detection target in this way, a detection target range is set so that the detection target moving to a different position according to the growth can be photographed.
This detection target range indicates an area that is set so that the growth of the plant can be estimated and an image including the imaging target can be captured.

In addition, since the image pickup unit 71 is moved, after the position of the image pickup unit 71 is detected as described above, the image pickup direction is adjusted so that a predetermined image pickup direction can be picked up, or according to the image pickup direction. It is necessary to correct the image information.
Therefore, the growth degree detection device 70 displays a direction suitable for photographing so that the direction from the growth degree detection device 70 to the detection target (plant 9) falls within a predetermined range. For example, the growth degree detection device 70 superimposes and displays an image captured in the past and an image captured this time on the same screen so as to select a capturing position so that the detection target is within a suitable position in the screen. To.

In order to adjust the imaging direction, first, the growth degree detection device 70 detects the position of the feature point determined as the detection target.
As a feature point to be determined as a detection target, a branch branch point, a position where a leaf / flower / fruit is attached to the branch, and a position selected from the position of the branch / leaf tip are determined.

With reference to FIGS. 11 to 15, detection of the position of the feature point determined as the detection target by the growth degree detection device 70 will be described.
FIG. 11 is a diagram showing the arrangement of the feature points to be detected shown in FIG.
Based on the feature points shown in this figure, the principle of processing for detecting the growth of plants will be described in order.
In the following description, a case where a branch point of a branch is a feature point will be described as an example.
In order to detect the growth of the plant 9 in the long term, the degree-of-growth detection device 70 acquires and compares images at that time. In order to detect the change, the degree of growth is detected by detecting the position of the feature point (for example, N2 (k), N3 (k), N41 (k), N42 (k)) of the plant 9 from time to time. Is detected.
The feature points shown in this figure are determined as branch points of the branches of the plant 9 as points (N2 (k), N3 (k), N41 (k), N42 (k)).

Therefore, in the present embodiment, photographing according to the growth process of the plant 9 is enabled by the following detection method.
The position of the camera unit 711 is detected based on three or more feature points.
First, in order to simplify the explanation, a camera unit 711 is arranged so that three feature points (N3 (k), N41 (k), N42 (k)) can be captured as one image, and the camera in the initial state The angle of view and shooting direction of the unit 711 are determined.

Of the three feature points, one point (for example, N3 (k)) is determined as a representative point, and the other two points (N41 (k), N42 (k)) are positioned based on the relative position based on the representative point. Indicates.
The position of the representative point in the reference coordinate system and the relative position of the other two points based on the representative point are different positions (N3 (k + 1), N41 (k + 1), N42 (k + 1)) depending on the growth of the plant 9. Moving.
For example, as shown in this figure, when two feature points (N3 (k) and N41 (k)) are defined along the branch, the two feature points ( N3 (k + 1) and N41 (k + 1)) are widened.
Further, when the feature point is defined at the actual position, the position of the feature point defined at the actual position is lowered due to the influence of the actual weight as the actual size is increased. In addition, the position of the feature point (N41 (k + 1)) defined on the branch where the fruit arrived is also lowered. Even if the position of the feature point moves in this way, it is possible to take a picture without protruding from the image by estimating the position of the feature point that moves according to the growth.

A specific example of the process performed by the growth degree detection device 70 that detects the feature points as described above will be described.
A typical plant can be defined as a tree structure.
For example, the position of the feature point is defined using the position of the plant 9 stock as a reference point. From the reference point, each feature point is reached by sequentially tracing the branched branches (stems). Therefore, a branch structure of branches (stems) is defined as a tree structure with nodes and branches (stems) between nodes as branches.
Thus, by defining the plant 9 as a tree structure, the feature points are defined as nodes, the positional relationship between the feature points becomes clear, and the feature points can be easily detected. The spacing between the nodes, indicated as the length of each branch, increases with plant growth.
In addition, each branch is defined as a vector, and each vector from the reference point to the representative point is integrated to obtain the position of the representative point. Similarly, with respect to the representative point, the other two points are also defined as vectors, and the respective positions of the other two points are obtained.

Each vector changes according to the growth, but the topology of the tree structure does not change except when a branch is lost or a sprout is generated. Therefore, by estimating and integrating the amount of change of each vector, the position of the representative point that moves due to growth can be estimated. Similarly, it is possible to estimate the position of each of the other two points by estimating the vector change for the other two points with reference to the representative point.
The change of each said vector is computable according to the arithmetic expression (Formula (1)) defined based on the above-mentioned 1st breeding model.

↑ x _n (k + 1) = g (↑ x _n (k), k) = ↑ x _n (k) + ↑ Δx _n (k) (1)

In the above formula (1), “↑ x _n (•)” indicates “vector x _n”, which indicates a vector corresponding to the branch indicated by the subscript n. The subscript n indicates an identifier for uniquely identifying the vector, and k indicates a variable indicating the detection time.
Further, "↑ Δx n _(·)" represents "vector [Delta] x _n", determined corresponding to the "vector x _n" is a vector indicating an amount of change "vector x _n".
By defining “vector Δx _n ” as a vector in this way, it is possible to define not only the magnitude change but also the change direction. This “vector Δx _n ” can be modeled by defining it based on the first growth model described above.

  The growth degree detection device 70 can set the detection target range according to the position of the movement destination of each feature point estimated as described above, and can perform imaging adapted to the movement of the position according to the growth. Become.

With reference to the drawings, a process for photographing with matching feature points will be described.
FIG. 12 is a diagram illustrating a process of capturing images with matching feature points according to the conditions illustrated in FIG.
When detecting the degree of growth of the plant 9, the detection target range setting unit 723 estimates a region to be photographed by the imaging unit 71 as a detection target range.
Here, as shown in the figure, a representative feature point (N3 (k)) is displayed as a reference point, and N3 (k) is moved to the reference point N3 (k). The detection target range is displayed so as to overlap the movement destinations of (k + 1). The user determines the imaging direction so that the feature point N3 (k + 1) in the image to be photographed now matches N3 (k + 1) indicated in the displayed detection target range.
In this way, the user sets the position of the detection target range so that the feature point N3 (k + 1) in the image to be captured is matched with N3 (k) that is the reference point.
Thereby, when the imaging unit 71 performs shooting, the detection target range setting unit 723 estimates the position where the feature point moves, and displays the estimated position on the image. Can be easily taken.

Further, even when the reference feature point is changed, the above-described growth model is referred to according to the period from the time point detected in the past to the time point next detected and the degree of growth of the plant 9. By doing so, the growth degree detection device 70 can estimate the range in which the feature point at that time moves.
As described above, even when the position of the feature point moves, the growth degree detection device 70 estimates the range in which the feature point moves based on the growth model, and uses the estimated range in which the feature point moves as a reference. By determining the imaging range, it becomes easy to continuously detect the detection target.

In the above description, the case where all of the three feature points can be continuously detected is shown as an example. However, when any of the three points (nodes) cannot be detected, In addition, since the position information of each node in the tree structure is provided, it is possible to estimate the position of the node that has become undetectable.
FIG. 13 is a diagram showing the arrangement of the feature points to be detected shown in FIG. 10 and the processing when one feature point cannot be detected due to growth.
As shown in this figure, when the growth degree detection device 70 cannot detect the feature point of N3 (k + 1) due to growth, the growth degree detection device 70 uses the detected feature point (node N3 (k + 1)) as a reference to It becomes impossible to estimate the movement possibility of (N41 (k + 1) and N42 (k + 1)).

Therefore, the growth degree detection device 70 estimates the movement range of the feature points based on the different feature points.
FIG. 14 is a diagram illustrating processing when the moving range of the feature point is estimated based on different feature points based on the conditions illustrated in FIG. 13.
The growth degree detection device 70 selects nodes (for example, the node N2 (k)) that can be detected in order in a direction away from the detection target range based on the feature points (node N3 (k)) detected before the growth. To do. Based on the position information of the newly selected (node N2 (k)), the position of (node N3 (k + 1)) that has become undetectable is estimated. In this estimation process, the position of the node (N3 (k + 1)) is estimated based on the amount of change in the interval between (node N2 (k)) and (node N3 (k)), and the estimated node (N3 ′ (k + 1) )). The position of the estimation node (N3 ′ (k + 1)) can be calculated by the same calculation as the above-described calculation formula (Formula (1)). The amount of change from (node N3 (k)) to the estimated node (N3 ′ (k + 1)) is defined as ΔN3 ′ (k).

Then, the growth degree detection device 70 estimates the movement range of the nodes (N41 (k + 1) and N42 (k + 1)) with reference to the estimated node (N3 ′ (k + 1)). The node (N41 ′ (k + 1)) indicating the movement range of the node (N41 (k + 1)) estimates the growth amount and direction based on the positional relationship between the node (N3 (k)) and the node (N41 (k)). It can be estimated by synthesizing the change amount ΔN3 (k + 1) that changes from the (node N3 (k)) defined above to the estimated node (N3 ′ (k + 1)).
As described above, the growth degree detection device 70 includes three or more feature points (nodes) in the detection target range, and even if a specific feature point (node) cannot be detected, the feature points (nodes) can be detected over a long period of time. ) Can be estimated, and the degree of growth can be continuously detected.
In this way, the detection target range is set so as to include the movement range of each point selected as the feature point.

In the above description, the three feature points have been described. However, one feature point and a sphere (circle) centered on the feature point are defined, and the radius of the sphere (circle) is characterized. It can be an amount.
FIG. 15 is a diagram illustrating an example of the arrangement of feature points to be detected and feature amounts shown in FIG.
As shown in this figure, in the case of a detection target of a sphere like a fruit, by defining a sphere (circle) centering on the feature point and setting the radius of the sphere (circle) as a feature amount, It is easier to define the range of change due to growth.
The degree-of-growth detection device 70 includes a case where there is a change due to growth by setting a range that includes three or more feature points and includes a change due to growth of the detection target defined as a sphere as a detection target range. In addition, the detection target range can be set so as to include the detection target.

(Selection of target part of detection target range)
In the above description, the case where the detection target range is set using the branching point of the branch as a feature point is illustrated, but the growth degree detection device 70 is shown as the determination target in the above-mentioned “(Plant breeding status determination)”. The target location of the detection target range can be selected from each item and the following target items.
The target item determined based on the image which expanded and imaged the plant 9 is shown.
The target location of the detection target range of the plant 9 is determined so that the state of the leaf vein obtained by enlarging the plant 9 or the state of the vellus hair generated on the stem and the root hair generated on the root can be detected.

  For example, it is known that when the balance of nutrients supplied when cultivating the plant 9 is lost, the state of the leaves changes. When the state of the plant 9 deteriorates, the color of the leaves changes or deforms. Even if the leaf is deformed, the configuration of the leaf vein does not change, only the shape indicated by the leaf vein changes. Therefore, the growth degree detection device 70 determines a detection target range so that the state of the veins can be detected, and detects the change. Thereby, the growth degree detection device 70 can detect the shape of the leaf from the configuration of the leaf vein, and even if a part of the leaf is missing, the growth of the leaf from the state of the leaf vein of a part of the leaf The degree can be detected.

  For example, it is known that sugar content can be increased by restricting hydration when cultivating tomatoes. When hydration is restricted, hair growth grows on the stem. The degree-of-growth detection device 70 determines a detection target range so as to detect the vellus hair of the stem portion, and detects a change in the vellus hair of the stem portion. Thereby, as a result of restricting hydration, it can be determined whether or not it has grown to a state where the sugar content can be increased, and it is possible to detect the degree of growth at this stage.

  Moreover, the growth degree detection apparatus 70 can detect the state of the plant 9 from the initial stage in hydroponics by selecting the root hair generated in the root as a detection target range for detecting the state of the plant 9. It becomes possible.

(State detection when the soil part is the detection target range)
The degree-of-growth detection device 70 detects so as to include the soil in which the plant 9 is cultivated in addition to each item that directly detects the plant selected as the selection target in the above-mentioned “(Selection of target portion of detection target range)”. Select the target part of the target range.
By determining the detection target range of the plant 9 so as to include the soil in which the plant 9 included in the detection target range of the plant 9 is cultivated, the state of the plant 9 can be detected indirectly. In particular, it is possible to obtain effective information when estimating the state after germination from before germination.
For example, the present invention can be applied to a case where a process for restricting hydration is performed as in the above-described example of tomato. Depending on the type of plant 9, control such as increasing the sugar content may be performed by controlling the amount of water given after germination. In such a case, the growth degree detection device 70 can indirectly detect the state by setting the detection target range of the plant 9 so as to include the soil where the plant 9 is cultivated.
As a specific detection method, for example, the degree-of-growth detection device 70 can detect the soil portion according to a predetermined relational expression based on the change in luminance of the soil portion and the spatial frequency in the image signal based on the captured image. Detect dryness.
When the soil portion has a large amount of moisture and is in a sticky state, the light reflection coefficient of the soil portion is low, so that the change in luminance in the image signal is small and the spatial frequency is low. On the other hand, when the soil part is in a dry state with a small amount of water, the soil becomes hardened, and there is a part where the reflection coefficient of light becomes high. The spatial frequency is distributed over a wide range.
As described above, the growth degree detection device 70 can detect the dry state of the soil based on the amount of change in luminance in the image signal and the spatial frequency distribution.
The growth degree detection device 70 determines in advance the relationship between the dry state of the soil and the degree of growth of the plant 9 in the dry state, so that the growth degree detection unit 721 determines the degree of growth of the plant 9 from the dry state of the soil. Can be obtained.

(Detecting the degree of growth)
The detection of the growth degree will be described with reference to FIGS.
In the detection of the degree of growth by the growth stage determination unit 722, a detection method suitable for the purpose can be selected from the following detection methods.
The first detection method of the degree of growth by the growth stage determination unit 722 is a method of detecting the degree of growth of the plant 9 based on the ratio of the detection target of the plant 9 in the image of the detection target range of the plant 9.
The second method for detecting the degree of growth by the growth stage determination unit 722 is a method for detecting the degree of growth of the plant 9 based on the ratio of the feature color in the image of the detection target range of the plant 9.
The third method of detecting the degree of growth by the growth stage determination unit 722 is to determine whether or not the color information indicated by the image of the detection target range of the plant 9 is included in a pre-registered color space region. This is a method for detecting the degree of growth of the plant 9 based on the result.

Hereinafter, the first to third detection methods will be described in order with specific display examples.
First, in the first detection method, the degree of growth of the plant 9 is detected based on the ratio of the detection target of the plant 9 in the image of the detection target range of the plant 9.
In the first detection method, it is possible to detect a change in the relative size of the detection target based on the detection target range. The distance to the detection target can be kept constant, and the detection target range indicated by the images taken at the same angle of view can be maintained at a constant size. The degree of growth of the plant 9 can be detected from the ratio of the detection target of the plant 9 in the image.

FIG. 16 is a diagram illustrating the detection of the degree of growth by the first detection method.
In this figure, an image display area P151 and operation input detection areas P156, P157, P158, and P159 are shown.
In the image display area P151, the detection target shown in FIG. 10 is modeled and displayed.
The detection target ranges displayed individually when the detection target is photographed are collectively shown as P154a, P154b, P154c, and P154d. In actual display, any detection target range corresponding to the purpose is selected and displayed.
The detection target range indicated as P154a indicates a range including three feature points selected at time k and a detection target flower associated with the feature points.
The detection target range indicated as P154b indicates a range including the three feature points selected at time k and the detection target associated with the feature points.

The detection target range indicated as P154c indicates the estimated range at time (k + 1). The detection target range P154c is a range obtained by combining a range in which the three selected feature points are estimated to move by growth and a range including a size when the detection target associated with the feature points has grown. Indicates the combined range.
The detection target range P154c is set at the stage of time k by the breeding status detection unit 72, and is estimated from a range that grows and changes until time (k + 1) at the stage of time k. Based on the detection target range P154c, the breeding state detection unit 72 detects the degree of growth of the plant 9 by determining the progress of growth.
The detection target range indicated as P154d indicates the estimated range at time (k + 1). The detection target range P154d indicates a range including the size when the detection target fruit associated with the three selected feature points grows. The detection target range P154d is set so that the actual value is included in the range. The breeding state detection unit 72 detects the degree of growth of the plant 9 from the ratio of the detection target of the plant 9 occupying the detection target range P154d in which a certain size is maintained.
In this way, the detection target range is provided according to the stage of growth, and is selected by the growth status detection unit 72 according to the detection target condition.

Of the operation screens shown in the figure, the operation input detection area P156 indicates an area for detecting an operation for switching whether or not to display the detection target range P154c. The operation input detection unit 75 supplies a control signal for selecting display of the detection target range P154c to the control unit 79 by detecting a user operation in the operation input detection region P156.
The operation input detection area P157 indicates an area for detecting an operation for switching whether or not to display an explanation display. The operation input detection unit 75 supplies a control signal for switching whether to display the explanation display to the control unit 79 by detecting a user operation in the operation input detection region P157.
The operation input detection area P158 indicates an area for detecting an operation instructing the selected actual harvest. The operation input detection unit 75 supplies a control signal instructing the selected actual harvest to the control unit 79 by detecting a user operation in the operation input detection region P158.
The operation input detection area P159 indicates an area for detecting an operation for returning to the display before the display of the image display area P151 is selected. The operation input detection unit 75 supplies a control signal to the control unit 79 for returning to the display before the display of the image display region P151 is selected by detecting a user operation in the operation input detection region P159.

  In addition, although the case where the magnitude | size of the detection target range was made constant above was shown, the growth degree detection apparatus 70 can also widen a detection target range according to the growth of the plant 9. Even if the detection target range is widened according to the growth of the plant 9, the growth degree detection device 70 indicates that the rate of change in the size of the detection target (for example, actual) included in the range is within the detection target range. If it becomes larger than the rate of change in size, the degree of growth of the plant 9 can be detected relatively.

Next, the second detection method will be described.
In the second detection method, the degree of growth of the plant 9 is detected based on the ratio of the feature color in the image of the detection target range of the plant 9.
According to this second detection method, there is a case where a part of the fruit is hidden by a shadow of a leaf around the fruit or a bag actually covered, and the whole fruit cannot be imaged. Also, the degree of growth of the plant 9 can be detected. When the degree of growth of the plant 9 is detected, the ratio of the characteristic color changes in the image in the detection target range due to the change in the color of a part of the captured real image. The degree-of-growth detection device 70 detects the change in the proportion of the characteristic color in the image in the detection target range, and detects the degree of growth of the plant 9. For example, the proportion of the feature color is determined based on a predetermined threshold, and the degree of growth of the plant 9 is detected based on the determination result.

FIG. 17 is a diagram illustrating detection of the degree of growth by the second detection method.
This figure shows an image display area P161 and operation input detection areas P166, P167, P168, and P169.
In the image display area P161, a part of the detection target shown in FIG. 10 is displayed. Further, detection target ranges P154d and P154e displayed when the detection target is photographed are shown.
As the detection target range indicated as P154e, a part of the range estimated as the detection target range P154d at time (k + 1) is selected.
The detection target range P154e indicates a range selected as a portion where an actual color change can be detected without being a shadow of a bag that is actually applied. In the detection target range P154e thus selected, it is detected that the ratio of the actual characteristic color changes, and the degree of growth of the plant 9 is detected.
Of the operation screens shown in the figure, the operation input detection area P166 indicates whether or not the selected fruit is "around" according to the result determined based on the image of the detection target range P154e. The area | region which detects operation which switches whether to display the information display P163 is shown. The operation input detection unit 75 detects the user's operation in the operation input detection area P166, and according to the result determined based on the image of the detection target range P154e, whether the selected fruit is "around" A control signal for controlling whether or not to display the information display P163 indicating whether or not is supplied to the control unit 79.
The operation input detection areas P167, P168, and P169 are the same as the operation input detection areas P157, P158, and P159, respectively.

Next, the third detection method will be described.
In the third detection method, the degree of growth of the plant 9 is determined based on the result of determining whether the color information indicated by the image of the detection target range of the plant 9 is included in the area of the color space registered in advance. To detect.
According to the third detection method, detection is performed based on a change in color. Depending on the state of the plant 9, the color of the leaf of the plant 9 may change.
For example, a reference leaf or fruit color is determined, and the growth degree detection device 70 can detect a color change from the difference from the determined color.
The detection of the change from the reference color is executed by determining whether or not it is included in a color space area registered in advance. Alternatively, the detection of the change from the reference color is performed by dividing the color space of the changing range into a plurality of regions and determining which color space is included based on the area ratio of each divided region. It is executed by determining.

FIG. 18 is a diagram illustrating the detection of the degree of growth by the third detection method.
In this figure, an image display area P171 and operation input detection areas P176, P177, P178, and P179 are shown.
In the image display area P171, the detection target shown in FIG. 10 is modeled and displayed.
Detection target ranges P173 and P174 displayed when the detection target is photographed are shown. The detection target range indicated as P173 indicates a range selected as an area for detecting real (bunch) P172. The detection target range indicated as P174 is a part of the range selected as the detection target range P173, and indicates a range set as an easily ripened part in the actual P172.
The degree-of-growth detection apparatus 70 detects that the part that is actually easy to ripen has matured in the detection target range P174 and detects the degree of growth of the plant 9.
Of the operation screens shown in the figure, the operation input detection area P176 indicates whether or not the selected fruit is “about eaten” according to the result determined based on the image of the detection target range P174. The area | region which detects operation which switches whether to display the information display P173 is shown. The operation input detection unit 75 detects the user's operation in the operation input detection area P156, and according to the result determined based on the image of the detection target range P174, whether the selected fruit is "around" A control signal for controlling whether or not to display the information display P173 indicating whether or not is supplied to the control unit 79.
The operation input detection areas P177, P178, and P179 are the same as the operation input detection areas P157, P158, and P159, respectively.

A specific determination will be described with reference to FIGS. 19 to 20 by taking an example of harvesting straw.
FIG. 19 is a diagram illustrating a process when the degree of growth is detected based on a change in color.
FIG. 19A is a diagram illustrating a color space.
In this figure, it shows a range obtained by dividing the color space through which until turned purple from green to a plurality of regions as a region R _G to R _P.
A representative color corresponding to each divided area in the color space is determined, and a value (from the point P1) indicating the representative color determined from each range of the divided area by a projection function based on the position information of the color space. Project to Pn).
FIG. 19B is a diagram showing the relationship between actual growth and color change.
In the graph shown in this figure, the vertical axis indicates the hue from green to purple, and the horizontal axis indicates the size of each individual fruit. The following description is an example in which individual actual sizes are also detected.
In the immature stage, the size of each fruit is also small and green.
As growth progresses, individual fruits gradually grow. On the way, the color of each individual fruit changes to purple from the stage where the amount of change in individual fruit size decreases. However, even when the fruit arrives in the same bunch, there are individual differences in the timing when each individual color changes. Therefore, when the representative detection target range is determined, the detection target range setting unit 723 selects an easily matured portion with little individual difference and sets it as the detection target range.

Next, a procedure for detecting the degree of growth from changes in hue over time in individual fruits that have arrived in the same tuft will be described.
FIG. 20 is a diagram illustrating changes in individual actual hues as time elapses.
In the graph shown in this figure, the vertical axis indicates the hue from green to purple, and the horizontal axis indicates the passage of time. The size of the circle indicates the actual number classified into the hue.

By dividing the color space as described above, the growth degree detection unit 721 can detect a change in hue in stages.
The stage at time t0 is in an immature stage, and all fruits are green. The size of the circle indicates the number of actual hues classified into the hue.
As time advances from time t1 to time tn, the frequency distribution classified into each hue from green to purple decreases on the green side and increases on the purple side.
For example, when individually harvesting ripe fruits, the growth degree detection unit 721 may set a threshold value for identifying a purple region in the above classification, and select the fruits classified into the purple region. Because it is possible, it becomes possible to harvest.
Further, for example, when harvesting at the stage of ripening in units of bunches, the growth degree detection unit 721 sets a threshold value for identifying a purple region in the above classification, and the actual value classified into the purple region Since the bunches whose frequency distribution has reached a predetermined ratio can be selected, harvesting becomes possible. For example, in the case shown in this figure, the stage indicated as time t4 can be determined by using the actual ratio indicating the representative color indicated by P3 and P4 as a determination criterion.

Furthermore, as a method of detecting the growth level by the growth stage determination unit 722, there is a method of detecting under a condition different from the conditions shown in the first to third detection methods.
It is known that the living body changes with a period of about one day, and the periodic change is called “circadian rhythm”.
By irradiating the plant 9 with the light of the illumination 10 (FIG. 1) that changes one day in a cycle, the “circadian rhythm” can be synchronized with the cycle in which the light of the illumination 10 is changed.
In the plant 9, the planting plant 4 is controlled so as to generate a “circadian rhythm” synchronized with the change of the illumination light, and the breeding status detection unit 72 determines the amount of change in the “circadian rhythm”. The plant 9 is detected from the captured image. In short, the breeding status detection unit 72 determines the detection target range of the plant 9 so as to include a portion where the change amount in the “circadian rhythm” appears largely, and detects the change amount in the “circadian rhythm” of the plant 9. To do.
For example, since the sunflower flower has sunflower characteristics, when it is cultivated in a natural light environment, the direction of the flower changes periodically. However, as the seed matures, the amount of change in flower orientation decreases. Thus, the growth degree of the plant 9 can be detected by utilizing the fact that the amount of change in the “circadian rhythm” changes according to the growth.
In the description of the sunflower flower as an example, the “circadian rhythm” is controlled by natural light. However, the irradiation direction of artificial light (light from the illumination 10) may be controlled. Further, the “circadian rhythm” of the plant 9 may be controlled by controlling other environmental conditions such as temperature and water temperature, not limited to the control of the light irradiation direction.

(Change of detection target range)
When a plurality of detection targets are detected in the image of the detection target range of the plant 9, the growth stage determination unit 722 changes the detection target range according to the distribution of the size of the detected detection target image. For example, there is a case where a plurality of fruits to be detected are collected within a predetermined range.
Since the distance from the imaging position to the detection target and the imaging direction are not appropriate, a detection target that cannot be detected at a desired size may be included. For example, in the case of a spear, when a nearby tuft is detected, even a far tuft located in the background enters the same detection target range.
In such a case, the detection target range setting unit 723 changes the detection target range so that the size of the detection target image has a desired distribution. For example, the detection target range setting unit 723 can set a new detection target range from the image of the imaging range in which the distance to the detection target and the imaging direction are changed by changing the imaging position and the imaging direction.
Alternatively, there are cases where other detection objects are shown together in the background image of the detection object. In such a case, the imaging unit 71 does not change the imaging position with respect to the detection target and controls the optical system of the imaging unit 71 while maintaining the same distance and angle with the detection target to control the depth of field. Can be used to optically remove unwanted objects. For example, the depth of field is made shallow so that the desired detection target is in focus, and the other detection target images that were captured together as the background image of the detection target are blurred. By setting in this way, the detection target range setting unit 723 can set a detection target range including a desired detection target.

The growth stage determination unit 722 changes the detection target range according to the number of detected detection targets when a plurality of detection targets are detected in the image of the detection target range of the plant 9.
When a specific detection target is imaged, detection target candidates other than the specific detection target may be included in the same image. In such a case, the detection target range setting unit 723 divides the captured image area so that detection target candidates other than the specific detection target are not included, and performs detection processing for each divided image area. do. Thereby, the detection target range setting unit 723 can fit a specific detection target in the divided image area.

(Synthetic image generation method)
The display control unit 731 generates a transmission image that transmits at least one of the two images to be compared, and generates a composite image that is superimposed on the other image that is not transmitted.
FIG. 21 is a diagram showing the generated composite image.
For example, as shown in FIG. 21, an image captured in the past is made transparent as it is (indicated by a solid line in the figure), and a transparent image (in the figure is transmitted) through which the image to be photographed is transmitted. (Shown with a dotted line).
Therefore, a composite image is generated by superimposing a transparent image (an image to be captured this time) that has been transmitted on an image that has not been transmitted (past image).

Further, FIG. 21 shows a detection target range (dotted line) in the other image (past image) that is not transmitted, and a detection target range (solid line) in the transmitted image that is transmitted (image to be captured this time). And a detection target range (two-dot chain line) in which the growth range is estimated.
FIG. 21 also shows a display showing feature points to be matched when the positions of two images are overlapped. In accordance with the display indicating the feature points to be matched, the user adjusts the imaging direction so that the two images overlap.
Thereby, it is possible to easily compare an image captured in the past with an image captured this time.
Furthermore, a display of the degree of growth detected based on the image captured in the past and the image captured this time and a display of the stage in the growth process are shown on the screen.
This makes it possible to confirm the degree of growth detected based on the image picked up this time and the stage in the growth process while corresponding to the display of the image.

  Note that FIG. 21 shows the date and time of imaging in the past in correspondence with the detection target range (dotted line) in the other image (past image) that is not transmitted. Corresponding to the detection target range (solid line) in the image to be taken this time, the date and time taken as the current image are shown. By displaying the date and time, it is possible to easily obtain a period during which two images to be compared are captured.

Alternatively, in order to generate such a composite image, the display control unit 731 generates a contour image obtained by extracting the contour of at least one of the first image and the second image, and the other image A composite image superimposed on the image is generated.
Thereby, it is possible to easily compare an image captured in the past with an image captured this time.

(Growth degree detection process procedure)
Then, an example of the procedure of the growth degree detection process in this embodiment is shown.

<Procedure 0: Initialization information setting process>
Prior to the growth degree detection process, the following initialization process is performed in the plant cultivation system 1.
In the following description, it is assumed that the first growth model generated in advance by the growth model generation unit 510 is stored in the growth model unit 220.
It is assumed that a feature point in the plant 9 to be detected is determined, and position information of an initial position of the determined feature point is acquired.
In the following description of the procedure, the growth degree detection device 70 indicates a case where the image is taken from the same position, and description of information regarding the position where the image is taken is omitted.

<Procedure 1: Acquisition process of reference image information>
Based on this initialization information, the imaging unit 71 of the growth degree detection device 70 detects image information indicating the state of the plant 9 in the plant cultivation plant 4. The growth degree detection unit 721 includes a camera unit 711 that includes image information based on the image of the plant 9 captured by the camera unit 711, imaging time information indicating the time at which the image information was captured, and subject position information. Obtained from the imaging unit 71. The growth degree detection unit 721 stores the acquired image information, imaging time information, and subject position information in the past information storage unit 726 in association with each other.

<Procedure 2: Acquisition processing of detected image>
Here, the degree-of-growth detection unit 721 refers to the image information, the imaging time information, and the subject position information stored in the past information storage unit 726, and acquires the plant 9 that has been acquired at a predetermined time interval. Get image information.
For example, the display control processing unit 73 generates a detection target range as shown in FIG.
The detection target range is set so that the plant 9 can be easily captured and compared with past images even when the plant 9 grows and the detection target of the plant 9 moves. This is a prediction that includes 9 as a condition.
The display control processing unit 73 combines the image information captured by the imaging unit 71, the image information stored in the past information storage unit 726, and the generated detection target range, and causes the display unit 74 to display the combined information. . Further, the display control processing unit 73 displays the position of the feature point to be matched at the time of shooting in order to facilitate the comparison of the two images.
The shooting direction is determined according to the display of the combined image generated by combining in this way.
Further, the display control processing unit 73 causes the display unit 74 to display the time when the image information stored in the past information storage unit 726 is captured and the current time.
After the shooting direction is determined according to the above, an image in which the detection target image is included in the detection target range is acquired from the image captured by the imaging unit 71.
The growth degree detection unit 721 stores the image information acquired by the imaging unit 71, imaging time information, and subject position information in the past information storage unit 726.

<Procedure 3: Processing for detecting the degree of growth of each part>
The degree-of-growth detection unit 721 obtains difference information of a detection target indicated by the image information from the image information captured by the imaging unit 71 and the image information of the plant 9 captured at a predetermined time interval. This difference information indicates the amount of growth since the last time it was detected. The growth degree detection unit 721, for example, based on the ratio of the growth amount calculated from the difference information of the detection target region and the growth amount expected in the detection target region, Detect the degree of growth.
The display control processing unit 73 causes the display unit 74 to display the degree of growth of the detected part.
The growth degree detection unit 721 detects the degree of growth of each part in the plant 9 by performing the same process as described above at each part.

<Procedure 4: Judgment process of plant growth degree>
Based on the degree of growth calculated from the difference information of each part detected by the degree-of-growth detection unit 721, the growth stage determination unit 722 grows the detection target that is the whole plant 9 or a part of the plant 9. Determine the degree. The growth stage determination unit 722 determines the stage of growth based on the degree of growth of the detection target.
The display control processing unit 73 causes the display unit 74 to display the determined growth stage.

<Procedure 5: Detection target range resetting process>
In accordance with the determination result of the degree of growth of the plant 9 according to the procedure 4, the detection target range setting unit 723 newly sets the detection target range of the plant 9 suitable for the growth state.
Further, the detection target range setting unit 723 may set the detection target range of the plant 9 according to the stage of growth determined based on the growth degree determined by the procedure 4.
Thus, the imaging range of the next imaging is determined based on the newly set detection target range of the plant 9.

  By the procedure described above, the growth degree detection device 70 shown in the present embodiment can detect the degree of growth of the plant 9 and can easily manage the breeding status of the plant.

[Status display by the growth degree detector]
Subsequently, different embodiments will be described with reference to the drawings.
FIG. 22 is a block diagram illustrating an information providing system in which the growth degree detection device 70 according to the present embodiment is a status display terminal.
The same components as those in FIGS. 1 to 3 and 9 are denoted by the same reference numerals.

The information providing system shown in this figure includes an information providing device 800 and a terminal device 90 in addition to the configuration shown in FIG.
In the information providing system shown in this figure, the control unit 500, the information providing device 800, the growth degree detecting device 70, and the terminal device 90 are connected to the communication network 900, and are mutually connected via the communication network 900. To communicate.
The communication network 900 may be configured by either a wired line or a wireless line, or a combination of each.
The information providing apparatus 800 obtains information on the plant 9 cultivated in the plant cultivation plant 4 from the control unit 500 and the growth degree detection apparatus 70, and provides the provided information generated based on the information to the growth degree detection apparatus 70. provide. Further, the information providing device 800 collects evaluation information by the evaluator from the terminal device 90 and provides the evaluation information by the evaluator to the growth degree detection device 70.

The information providing apparatus 800 includes a database unit 810, a provided information generation unit 820, an information provision processing unit 830, an information collection processing unit 840, and a communication processing unit 890.
The database unit 810 stores, for each plant 9 (agricultural crops), image information and growth history information of the growing process, provided information generated for providing, and evaluation information collected from the evaluator.
The provided information generation unit 820 is based on various information stored in the storage unit 230, the image information detected by the growth degree detection device 70, or the growth degree, via the growth degree detection device 70 and the terminal device 90. Various display screens are generated as information to be provided. Various screens generated by the provision information generation unit 820 will be described later.
The provided information generation unit 820 generates a composite image indicating a three-dimensional space based on the collected image information and the imaging position information obtained by capturing the image.
In addition, the provided information generation unit 820 generates an all-around image that displays the state of the entire periphery at an arbitrary point in the passage in the plant cultivation plant 4 from the generated composite image that indicates the three-dimensional space.

The information provision processing unit 830 provides the information (display screen) generated by the provision information generation unit 820 to each device in response to requests from the growth degree detection device 70 and the terminal device 90.
The information collection processing unit 840 collects evaluation information for the plant 9 input by the evaluator from the terminal device 90, and stores the collected information in the database unit 810.
The communication processing unit 890 performs communication processing with the control unit 500, and transmits various types of information such as image information and training history information stored in the storage unit 230 of the determination unit 200 via the control unit 500. Are collected and stored in the database unit 810.
The communication processing unit 890 reflects the information on the individual crops stored in the database unit 810 and the evaluation information collected from the evaluator as conditions for the breeding model design.
The communication processing unit 890 performs communication processing with the growth degree detection device 70, collects various types of information such as image information and training history information stored in the past information storage unit 726, and a database unit 810 is stored.
The communication processing unit 890 performs communication processing with the growth degree detection device 70 and collects image information, imaging time information, imaging position information, and subject position information stored in the growth degree detection device 70. And stored in the database unit 810.

  The communication processing unit 890 transmits the information on the individual crops stored in the database unit 810 and the evaluation information collected from the evaluator to each device in response to a request from the growth degree detection device 70 and the terminal device 90. To do.

The growth degree detection device 70 supplies the information of the detected plant 9 to the information providing device 800. The growth degree detection device 70 displays information provided from the information providing device 800.
The terminal device 90 displays information provided from the information providing device 800. The terminal device 90 obtains evaluation information by the evaluator based on the displayed information, and supplies the evaluation information to the information providing device 800.

Next, an information providing process for displaying information in the growth degree detection device 70 will be described with reference to FIGS. 23 to 26.
The following display screen is generated by the information providing apparatus 800 and displayed on the growth degree detecting apparatus 70.
The information providing apparatus 800 is based on layout information indicating the arrangement of the plant cultivation plant 4, position information given to identify the position of the plant 9, and cultivation information of the plant 9 arranged at each position. Then, a display screen to be displayed on the growth degree detection device 70 is generated. Here, description will be made assuming that communication is performed between the information providing apparatus 800 and the growth degree detection apparatus 70 and a display screen generated by the information provision apparatus 800 is displayed on the growth degree detection apparatus 70.
The growth degree detection device 70 detects an operation input by the user. The control unit 79 of the growth degree detection device 70 transmits operation input information based on the detected operation input to the information providing device 800. The information providing apparatus 800 that has received the operation input information generates a display screen to be displayed on the growth degree detection apparatus 70 according to the received operation input information. The information providing apparatus 800 transmits the generated display screen information to the growth degree detecting apparatus 70.

FIG. 23 is a diagram illustrating an example of a screen displayed when selecting a reference position in a plant cultivation plant.
By referring to this screen, the user of the growth degree detection device 70 can obtain the status of each position without actually moving in the plant cultivation plant 4.
This figure shows an image display area P101 and operation input detection areas P106, P107, P108, and P109. The same components as those in FIG.
In the image display area P101, a plan view showing a state in which the three floors 6 are arranged side by side is shown. The three floors 6 are provided along the right-hand direction of the screen across the passage. Each floor 6 is defined as a row “A to F” that identifies the position facing the passage. Further, numbers are assigned in order from 1 to 30 in the longitudinal direction of the floor 6, that is, the row direction.
Thus, the numbers as the rows and the columns are assigned so as to correspond to the positions (identification positions) where the plants 9 can be referred to from the passage, whereby the plants 9 can be identified according to the identification positions. For example, this identification position is indicated as “identification position B27” by combining numbers as rows and columns. “Identification position B27” indicates 27 columns of B rows.

Furthermore, in the image display area P101, a status display P102, a guidance display P103, and a current status display P104 are shown.
The status display P102 displays the degree of concentration (number of people / unit area) at each identification position. For example, as a situation in the vicinity of “identification position E27”, a situation display “This E27 neighborhood is a little crowded.” Is displayed.
This concentration degree is calculated based on, for example, the number of users who actually work in the passage and the number of robots (such as the mobile detection device 40).
Alternatively, by defining a virtual work space corresponding to the plant cultivation plant 4 and joining the virtual work space from a terminal device or the like connected via a communication line, You may calculate based on the concentration degree in the case of working. However, in the case of a virtual work space, unlike the case of working in an actual passage, the degree of concentration of the virtual work space cannot be visually confirmed. Therefore, since the growth degree detection device 70 displays the concentration degree in such a virtual work space, the user can check the concentrated place. As a result, the user can select a place where he / she is away from a concentrated place.
The guidance display P103 displays guidance for the user. For example, as a situation in the vicinity of “identification position B27”, a guidance “The tree near this B27 has a lot of ripe fruit” is shown.
The current status display P104 displays the status of the harvest target that is currently left in the plant 9. For example, as a situation in the vicinity of “identification position D2”, a guidance “The tree near D2 has less ripe fruit” is shown.

The operation input detection area P106 indicates an area for detecting an operation for switching whether or not to display the detection status display P102. The operation input detection unit 75 detects a user operation in the operation input detection region P106, thereby supplying a control signal for switching whether or not to display the status display P102 to the control unit 79, thereby updating the screen to be displayed. .
The operation input detection area P107 indicates an area for detecting an operation for switching whether or not to display the guidance display P103. The operation input detection unit 75 detects a user operation in the operation input detection region P107 and supplies a control signal for switching whether or not to display the guidance display P103 to the control unit 79 to update the screen to be displayed. .

The operation input detection area P108 indicates an area for detecting an operation for switching whether or not to display the current status display P104. The operation input detection unit 75 detects a user operation in the operation input detection area P108, thereby supplying a control signal for switching whether or not to display the current status display P104 to the control unit 79, thereby updating the screen to be displayed. Let
The operation input detection area P109 indicates an area for detecting an operation of switching a screen to be displayed so as to return to the display of the previously displayed screen. The operation input detection unit 75 detects a user operation in the operation input detection area P109, thereby supplying a control signal for returning to the display of the previously displayed screen to the control unit 79 to update the screen to be displayed. .
When the control signal from the operation input detection unit 75 is detected by the control unit 79, the growth degree detection device 70 transmits the detected control signal to the information providing device 800.

  Further, after displaying this screen, for example, in order to display information in the vicinity of “identification position B27”, a path sandwiched between the B and C lines in the image display area P101 (hereinafter referred to as “path BC”). The user performs an operation of touching the vicinity of “identification position B27.” When the operation input detection unit 75 detects an operation of touching the vicinity of “identification position B27”, the growth degree detection device 70 performs “identification”. A control signal indicating that an operation touching the vicinity of “position B27” is detected is transmitted to the information providing apparatus 800. The information providing apparatus 800 that has received the control signal displays on the growth degree detecting apparatus 70 based on the received control signal. The display of the screen to be transferred is changed to the following screen display.

FIG. 24 is a diagram illustrating an example of a screen that displays a state of a plant along a passage.
This figure shows an image display area P111 and operation input detection areas P116, P117, P118, and P119. The same components as those in FIGS. 1 and 23 are denoted by the same reference numerals.
The image display area P111 shows a state in which the state of the plants 9 on the floor 6 arranged side by side facing the passage is displayed. The passage shown in this figure shows, for example, a range from the 26th row to the 29th row of the passage BC ("identification positions B26 to B29 and identification positions C26 to C29"). In the left half of the image display area P111, the state from the B row 26th column to the 29th column ("identification position B26 to B29") on the floor 6 on the left side of the passage in order from the front is shown. In the right half of the image display area P111, the state from the C row to the 26th column to the 29th column ("identification positions C26 to C29") on the floor 6 on the right side of the passage in order from the front is shown.
The cultivated plants 9 are arranged on the respective floors 6 and the growing state of the plants 9 is displayed and can be visually confirmed.

Further, a position display area P112 and a direction display area P113 are displayed in the image display area P111.
The position display area P112 is an area for displaying identification information for identifying the plants 9 respectively arranged on the floor 6. The label “B26” is sequentially labeled “B29”, and the label “C26” is sequentially labeled “C29”. Shows up to. For example, the label “B26” is a display indicating the plant 9 at the “identification position B26”.
The direction display area P113 indicates an area in which an operation for changing the display position to front, rear, left, and right is detected, detects a user operation, and generates a control signal to update the image moved in the direction specified by the operation.
The operation input detection areas P116, P117, P118, and P119 are the same as the operation input detection areas P106, P107, P108, and P109 in FIG.
When the instruction information from the operation input detection unit 75 is detected by the control unit 79, the growth degree detection device 70 transmits each control signal corresponding to the detected instruction information to the information providing device 800.
When the operation input detection unit 75 detects that the direction change of the “left direction” is performed in the direction display area P113 by the user's operation in such a screen display state, the growth degree detection device 70 Then, a control signal indicating that the operation of the direction change “leftward” is detected by the operation input detection unit 75 is transmitted to the information providing apparatus 800. The information providing apparatus 800 that has received the control signal shifts the display of the screen displayed on the growth degree detection apparatus 70 to the following screen display based on the received control signal.

FIG. 25 is a diagram illustrating an example of a screen that displays the state of the plant facing the passage.
By referring to this screen, the user of the growth degree detection device 70 can obtain the situation at the designated position without actually moving through the plant cultivation plant 4.
This figure shows an image display area P121 and operation input detection areas P126, P127, P128, and P129. The same components as those in FIGS. 1, 23, and 24 are denoted by the same reference numerals.
In the image display area P121, a position display area P122 and a detection target identification display area P123 are displayed.
The position display area P122 indicates an area for displaying identification information indicating the position of each plant 9 arranged, and the label “B26” to the label “B28” are sequentially displayed.
The detection target identification display area P123 indicates an area for displaying identification information for identifying the fruit of the plant 9 at the “identification position B27” indicated by the label “B27” in the position display area P122, and the label “ B2701 "to label" B2708 "are shown. Here, the label “B2705” and the label “B2708”, which are not displayed and are missing numbers, are actually attached labels that are not displayed hidden behind the leaves.

The operation input detection area P126 indicates an area for detecting an operation for switching whether or not to display the status display P133 (FIG. 26). The operation input detection unit 75 supplies a control signal for switching whether or not to display the status display P133 (FIG. 26) to the control unit 79 by detecting a user operation in the operation input detection region P126, and displays the control signal. Update the screen.
The operation input detection area P127 indicates an area for detecting an operation for switching whether or not to display the explanation display P134 (FIG. 26). The operation input detection unit 75 supplies a control signal for switching whether or not to display the explanation display P134 (FIG. 26) to the control unit 79 for display by detecting the user's operation in the operation input detection region P127. Update the screen.
The operation input detection area P128 indicates an area for detecting an operation instructing to harvest the selected fruit. The operation input detection unit 75 detects a user operation in the operation input detection area P128, thereby supplying a control signal instructing to harvest the selected fruit to the control unit 79, thereby updating the screen to be displayed. .
The operation input detection area P129 is the same as the operation input detection area P109 in FIG.
When the instruction information from the operation input detection unit 75 is detected by the control unit 79, the growth degree detection device 70 transmits each control signal corresponding to the detected instruction information to the information providing device 800.
After selecting the plant 9 indicated by the label “B27” in such a screen display state, the operation input detection area P127 is operated. When the operation input detection unit 75 detects an operation of the operation input detection region P127, the growth degree detection device 70 provides a control signal indicating that an operation of the operation input detection region P127 is detected by the operation input detection unit 75. To device 800. The information providing apparatus 800 that has received the control signal shifts the display of the screen displayed on the growth degree detection apparatus 70 to the following screen display based on the received control signal.

FIG. 26 is a diagram illustrating an example of a screen that displays the state of the plant facing the passage.
By referring to this screen, the user of the growth degree detection device 70 can obtain detailed information about the status of the designated position without actually moving through the plant cultivation plant 4.
In this figure, an image display area P131 and operation input detection areas P126, P127, P128, and P129 are shown. The same components as those in FIGS. 1, 23, 24, and 25 are denoted by the same reference numerals.

In the image display area P131, in addition to the position display area P122 and the detection target identification display area P123, an all target display area P132, a status display area P133, and an explanation display area P134 are displayed.
The all target display area P132 is a fruit of the plant 9 shown as the label “B27” in the position display area P122, which is not displayed in FIG. Indicates the area to display. In the all target display area P132, for example, actual positions corresponding to the identification information indicated as the label “B2705” are shown.
The status display area P133 indicates an area for displaying the fruit at the time suitable for harvesting, that is, the time of "eating", and the fruit selected as being at the time suitable for harvesting is indicated by a downward arrow. .
The explanation display area 134 shows an area for displaying the explanation of “the fruit hidden behind the leaves” shown in the all target display area P132.

  In such a display state of the screen, after selecting the actual indicated by the label “B2705” and the label “B2708”, the operation input detection area P127 is operated. When the operation input detection unit 75 detects an operation of the operation input detection region P127, the growth degree detection device 70 provides a control signal indicating that an operation of the operation input detection region P127 is detected by the operation input detection unit 75. To device 800. The information providing apparatus 800 that has received the control signal shifts the display of the screen displayed on the growth degree detection apparatus 70 to the following screen display based on the received control signal.

  In addition to the position display area P122 and the detection target identification display area P123 shown in this figure, the all target display area P132, the status display area P133, the explanation display area P134, and the like correspond to the acceleration detected by the acceleration detector 76. Then, the content of the display information, the size of the area for displaying the display information may be changed, or the display may be interrupted. When the movement direction of the growth degree detection device 70 is not determined or the movement amount per unit time is large, it is easier to recognize the displayed image when the display of information by characters is reduced. Therefore, by limiting the information to be displayed according to the acceleration acting on the growth degree detection device 70, the displayed image can be easily recognized.

FIG. 27 is a diagram illustrating an example of a screen that displays evaluation information about a selected fruit.
By referring to this screen, the user of the growth degree detection device 70 can obtain information (evaluation information) evaluated by a third party on the plant 9 at the designated position from the terminal device 90. Further, time information when the evaluated information is registered is stored in association with the evaluated information.
This figure shows an image display area P141 and operation input detection areas P126, P127, P128, and P129. The same components as those in FIGS. 1, 23, 24, 25, and 26 are denoted by the same reference numerals.
In the image display area P141, a symmetrical plant display area P142, selection target image display areas P143a and P143b, evaluation display areas P144a and P144b, and a growth information display area P145 are shown.
The symmetrical plant display area P142 shows an area for displaying the state of the plant 9 selected in FIG.
Selection target image display areas P143a and P143b indicate areas in which the actual image selected in FIG. 26 is displayed as an image. For example, the selection target image display area P143a has a real (* mark) identified as the label “B2705”, and the selection target image display area P143b has a real (* mark) identified as the label “B2708”. It is shown.

The evaluation display areas P144a and P144b indicate areas for displaying the evaluation information about the real selected in FIG. For example, in the evaluation display area P144a, evaluation information about an actual (* mark) identified as the label “B2705” in the selection target image display area P143a is shown. In addition, the evaluation display P144a shows evaluation information about an actual (◇ mark) identified as the label “B2708” in the selection target image display area P143b.
The evaluation information shown in the evaluation display areas P144a and P144b is posted as information from an expert who is a third party, for example, and the evaluation information is registered in the information providing apparatus 800 via the terminal device 90. It is a thing. With this evaluation information, the user can acquire the knowledge of an expert who is a third party.

  The cultivation information display area P145 shows an area for displaying a division method indicating two individual differences based on the cultivation information of the plant 9. For example, the sweetness and size information for the real (* mark) identified as the label “B2705” and the real (* mark) identified as the label “B2708” in the selection target image display area P143b. Are being compared. In the case shown in this figure, the sugar (* mark) identified as the label “B2705” has a lower sugar content but is larger than the fruit (◇ mark) identified as the label “B2708”. It has been shown. Further, in the graph indicated by the growth curve, the vertical axis indicates the degree of growth and the horizontal axis indicates the number of days. If it is within the range surrounded by the dotted line, it is in a time when harvesting is possible, and the fruit (marked with ◇) identified as the label “B2708” is in a more mature state.

  As described above, the information providing apparatus 800 transmits the screen information generated according to the control information received from the growth degree detection apparatus 70 to the growth degree detection apparatus 70. The growth degree detection device 70 causes the growth degree detection device 70 to display screen information transmitted from the information providing device 800.

(Detection of state in plant cultivation plant 4)
In order to appropriately manage the state of the plant 9 cultivated in the plant cultivation plant 4, it is desirable to periodically collect information on the entire plant cultivation plant 4. However, since it takes labor to collect this information, in this embodiment, it is also possible to collect information mechanically. Hereinafter, one mode will be described.
For example, the mobile detection device 40 (FIG. 2) equipped with an all-around camera circulates in the plant cultivation plant 4. The mobile detection device 40 circulates in the plant cultivation plant 4 and collects surrounding images as moving images or continuous still images while moving along the passage of the plant cultivation plant 4.
The image information in the plant cultivation plant 4 collected by the mobile detection device 40 is supplied to the growth degree detection device 70 or the information providing device 800.
The degree-of-growth detection device 70 or the information providing device 800 uses the image information in the plant cultivation plant 4 collected by the mobile detection device 40 to create a three-dimensional space based on the image information of a plurality of frames included in the image. Generate a model for detection. The growth degree detection device 70 or the information providing device 800 generates a composite image in a three-dimensional space based on a detection target model.
Collecting image information in this way can save labor in photographing individual detection targets individually.
In addition, since the image information to be collected is obtained as image information in all directions, when moving in one direction at the time of imaging, it is confirmed not only in an image that can be confirmed in the traveling direction but also in a direction opposite to the traveling direction. Case images can also be collected together.

The growth degree detection device 70 (display control device) shown in the above embodiment is provided with a plurality of stages for classifying according to the growth of the plant 9 in the growth process of the plant 9. Thus, it is possible to detect the degree of growth of the plant 9 from the state of the plant 9 included in the detection target range set in accordance with the stage of growth, and to make it easier to manage the breeding status of the plant 9. The detection device 70 (display control device) displays display information generated from information related to the growth of the plant 9 stored in the storage area in association with identification information for identifying the plant 9 to be displayed by the display control unit 73. Display in association with the image to be displayed. Thereby, the growth degree detection apparatus 70 (display control apparatus) displays the information of the detected growth degree as information of the specific plant 9 selected from the plurality of plants 9, and manages the growth status of the plant 9 Furthermore, the growth degree detection device 70 (display control device) can be easily operated according to the display on the screen, and the burden on the user to learn the operation method can be reduced. Since the operation of the growth degree detection device 70 is simple, the growth degree detection device 70 can be used regardless of the experience level of plant cultivation, from a person with little experience in plant cultivation to an experienced person. . Therefore, when an experienced person who uses a cultivation method that depends on experience and intuition uses the growth degree detection device 70, the result of the determination can be stabilized, and the growth status of the plant 9 can be easily managed. can do.

  In addition, the number of experienced persons who have cultivated plants tends to decrease year by year as the agricultural population decreases. On the other hand, the spread of the plant cultivation plant 4 is desired, and the plant cultivation plant 4 is expected to be introduced into various places. From such a situation, when the number of facilities of the plant cultivation plant 4 suddenly increases, an experienced person who has cultivated a plant may not manage the plant cultivation plant. Therefore, even a person who has little experience in plant cultivation can detect the growth status of the plant 9 by the growth degree detection device 70, and the detection result can be easily confirmed. In short, the growth degree detection device 70 can make it easier to manage the growth status of the plant 9.

The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes changes and the like without departing from the gist of the present invention.
For example, although the growth degree detection device 70 (display control device) has been described as a portable terminal device, it may be a stationary device.
Moreover, although the growth degree detection apparatus 70 (display control apparatus) was shown as what is comprised as an integral type, you may comprise separately each structure.
In addition, in the plant cultivation system 1 shown in FIG. 3, by adding a part or all of the configuration of the growth degree detection device 70 (display control device) to the control unit 500, based on image information obtained from the imaging unit 300. Then, the processing shown as the growth degree detection device 70 (display control device) may be performed by the control unit 500 to which the configuration is added.

In addition, although it has been described that the user adjusts the positions of the two images for detecting the growth degree, the growth degree detection device 70 (display control device) extracts the feature points of the two images and extracts the extracted feature points. By applying image processing techniques such as pattern matching processing based on the above and subject tracking processing, two images can be compared.
Further, the growth degree detection device 70 (display control device) has been described as detecting the degree of growth based on the size of the captured image of the growth of the plant 9, but the difference in size is determined by the imaging magnification of the imaging unit 71. You may judge from.

Further, when identifying the feature point, the growth degree detecting device 70 (display control device) posts identification information for identifying the feature point in the vicinity of the feature point, and features based on the identification information. Points may be identified. The identification information can be appropriately selected from character information, barcode information, pattern information, and the like. Furthermore, by providing RFID (Radio Frequency IDentification) and an IC tag, the feature points can be easily identified.
Further, the display screen in the growth degree detection device 70 (display control device) shown in the above-described embodiment can be displayed on the terminal device 90 in the same manner as the growth degree detection device 70. Further, the terminal device 90 displays the screen generated by the information providing apparatus 800, receives information from a third party, and registers it in the information providing apparatus 800.

  The plant cultivation system 1, the mobile detection device 40, the growth degree detection device 70, the information providing device 800, and the terminal device 90 described above have a computer system therein. The operation process of each functional unit is stored in a computer-readable recording medium in the form of a program, and the above processing is performed by the computer system reading and executing the program. The computer system here includes a CPU, various memories, an OS, and hardware such as peripheral devices.

Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used.
The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Furthermore, the “computer-readable recording medium” dynamically holds a program for a short time like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory in a computer system serving as a server or a client in that case, and a program that holds a program for a certain period of time are also included. The program may be a program for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system.

1 ... Plant cultivation system,
9 ... plants,
100 ... control target part,
200 ... determination part,
300 ... imaging unit,
400: Imaging position moving unit,
500 ... control unit,
70 ... Growth degree detection device (display control device),
72 ... Growth degree detection unit,
73: Display control unit

Claims (12)

A display control unit configured to display display information generated from information related to the growth of the plant stored in the storage area in association with identification information for identifying the display target plant in association with the image to be displayed; A display control device. The display control unit
The image information of the image captured at the first time interval, the image information of the image to be displayed stored in the image storage area in association with the time-series information based on the order of the captured image, The display control apparatus according to claim 1, wherein the display control apparatus reads out and displays the data in accordance with a second time interval in accordance with the time-series information from a storage area. The display control unit
The display information corresponding to the display target image is displayed in association with the display target image that is sequentially read out from the image storage area according to the time-series information and displayed. Display controller. A display information generating unit that generates the display information related to the display target plant associated with the display target plant based on the identification information;
The display control unit
The display control apparatus according to any one of claims 1 to 3, wherein the generated display information is displayed in association with the display target image. The display information generation unit
The display control apparatus according to claim 4, wherein the display information is generated based on estimated information obtained by estimating a stage of growth of the plant. Time information indicating the time when the image was taken and the image information of the taken image are associated with each other and stored in the image storage area,
The display control unit
The display control apparatus according to claim 4, wherein the time information is displayed in association with an image of the image information associated with the time information. A display unit for displaying the display information;
An acceleration sensor for detecting acceleration acting on the display unit;
With
The display information generation unit
The display control apparatus according to any one of claims 4 to 6, wherein the display information to be generated is changed according to the detected acceleration. The display information generation unit
The display control apparatus according to claim 7, wherein a size of an area for displaying the display information is changed according to the detected magnitude of the acceleration. The display information generation unit
The display control apparatus according to claim 7 or 8, wherein display of the display information is interrupted according to the detected magnitude of acceleration. A display control unit configured to display display information generated from information related to the growth of the plant stored in the storage area in association with identification information for identifying the display target plant in association with the image to be displayed; A display control system. A process in which the display control unit displays display information generated from information relating to the growth of the plant stored in the storage area in association with identification information for identifying the display target plant in association with the display target image. A display control method comprising: A computer included in the display control device,
Display information generated from information relating to the growth of the plant stored in the storage area in association with identification information for identifying the plant to be displayed is caused to function as a display control unit that displays the information in association with the image to be displayed Program for.

Images