JP2023088635A - Information processing device, information processing method, program and recording media - Google Patents

Abstract

To provide one of the techniques for selecting a strain with a specific trait in a predetermined plant from a photographed image of a field.SOLUTION: An information processing device (10) according to one aspect of the present invention comprises: a stock area identification unit (123) that identifies the area of each stock of a predetermined plant using a photographed image of the field in which the predetermined plant is planted; a circle area identification unit (124) that identifies the area of the minimum circumscribed circle that circumscribes a predetermined plant stock for each stock using the image; and a determination unit (125) that determines whether the stock has a specific trait using the ratio of the area of the stock and the area of the minimum circumscribed circle.SELECTED DRAWING: Figure 1

Description

The present invention relates to an information processing device, an information processing method, a program, and a recording medium.

Techniques for determining the growth or abnormal growth of plants including crops by image analysis are known. For example, Non-Patent Document 1 describes a technique for extracting node regions and estimating internode distances using an image of a tomato stem viewed from the side. Also, in recent years, a technology has been developed to determine the growth status of plants in a field by analyzing aerial drone images (for example, Non-Patent Document 2).

Kyosuke Yamamoto (2015), Research on automatic growth diagnosis and high-speed phenotyping of tomatoes by image analysis and machine learning, Graduate School of Agricultural and Life Sciences, University of Tokyo, Doctoral dissertation (https://repository.dl.itc.u-tokyo.ac) .jp/records/8365#.YWOOCGLP0uU) Misaki Tochihara, Yasushi Shibato, "Labor-saving growth diagnosis technology for cabbage fields by drone aerial image analysis", Fukuoka Prefectural Agriculture and Forestry Research Institute 2019 achievement information

Techniques for judging the growth status by image analysis still have many problems for practical application because the judging method and judging criteria differ depending on the type of plant, or even with the same type of plant. ing.

An object of one aspect of the present invention is to realize a technique for selecting a strain having a specific trait in a given plant from a photographed image of a field.

In order to solve the above-described problems, an information processing apparatus according to an aspect of the present invention uses an image obtained by photographing a field in which a predetermined plant is planted to identify the area of each strain of a predetermined plant. A strain area identification unit, a circle area identification unit that identifies the area of a minimum circumscribed circle that circumscribes a predetermined plant strain for each strain using the image, and the area of the strain and the area of the minimum circumscribed circle. a determining unit that determines whether the strain has a specific trait using the ratio.

In order to solve the above-described problems, an information processing method according to an aspect of the present invention uses an image of a field in which a predetermined plant is planted to identify the area of each strain of a predetermined plant. A step of specifying a strain area, a step of specifying a circle area for each strain of a minimum circumscribing circle that circumscribes the strain of a predetermined plant using the image, and a step of specifying the area of the strain and the area of the minimum circumscribing circle. and determining whether the strain has a particular trait using the ratio.

The information processing apparatus according to one aspect of the present invention may be implemented by a computer. In this case, a program for implementing the information processing apparatus on the computer by causing the computer to operate as each unit included in the information processing apparatus. , and a computer-readable recording medium recording it are also included in the scope of the present invention.

According to one aspect of the present invention, it is possible to select strains having a specific trait, such as short-internode trait, in a given plant from a photographed image of a field.

1 is an example of a functional block diagram of an information processing system including an information processing device according to an embodiment of the present invention; FIG. It is an example of a flow chart showing a flow of an information processing method concerning one embodiment of the present invention. FIG. 4 is a diagram showing an example of an image used in the information processing apparatus according to one embodiment of the present invention; FIG. FIG. 4 is a diagram showing an example of an image used in the information processing apparatus according to one embodiment of the present invention; FIG. FIG. 4 is a diagram showing an example of an image used in the information processing apparatus according to one embodiment of the present invention; FIG. FIG. 4 is a diagram showing an example of an image used in the information processing apparatus according to one embodiment of the present invention; FIG. FIG. 10 is a diagram showing an example of an image displayed on the display section including determination results determined by the information processing apparatus according to the embodiment of the present invention;

[1. Configuration example of information processing system]
An embodiment of the present invention will be described in detail below. FIG. 1 is an example of a functional block diagram of an information processing system 1 according to this embodiment. The information processing system 1 is a system that determines whether a plant planted in a field has a specific trait. In this embodiment, one or more ridges are formed in the field, and a plurality of strains of a given plant are planted in the ridges. Plants planted in the field are, by way of example, pumpkins or creeping plants. As creeping plants other than pumpkins, for example, plants such as watermelon, tomato, and the like that produce crops above ground (not limited to plants of the family Cucurbitaceae) are preferable, but plants that produce crops underground, such as sweet potatoes, are also included. .

In particular, an example of a system for determining whether or not a pumpkin strain has short internodes will be described below. Short internodes of squash are characterized by tight internodes from the early to middle stages of growth and a small number of lateral branches. Short internodal squash does not require pinching, trimming, or attracting, and has the advantage of being easy to harvest because the fruits can easily reach near the base of the plant. In other words, it is possible to save labor from the initial stage of growth to harvest. It has also been reported that pumpkin cultivars with internodes can be cultivated with about half the furrow width compared to ordinary cultivars that do not have short internodes. Expected.

It is necessary to find pumpkin germplasm and hybrid progeny growing in the field to select pumpkins with such short internodes. At present, a method is adopted in which a grower walks around a field and selects by visual inspection and measuring the vine length of each strain. As a result, it takes a lot of time and effort, and the number of individuals that can be evaluated is limited, which is a factor in reducing the efficiency of breeding.

In view of this problem, the present inventors have constructed a system that can greatly reduce the time and effort required for selection and evaluation compared to the past and can increase the number of individuals to be evaluated.

An information processing system 1 of the present embodiment includes an information processing device 10 and an aircraft 30, as shown in FIG. The information processing device 10 is a device that identifies (determines) the squash plants having short internodes planted in a field, and is, for example, a personal computer. The information processing device 10 includes a control section 12 , a storage section 22 , an input section 24 and a display section 26 . The control unit 12 is a control device that controls the entire information processing apparatus 10, and also functions as an acquisition unit 121, an outline identification unit 122, a stock area identification unit 123, a circle area identification unit 124, a determination unit 125, and an output unit 126. do.

The acquisition unit 121 acquires, via the input unit 24, an aerial image of the field captured by the flying object 30. FIG. The input unit 24 and the aircraft 30 may be connected wirelessly or by wire.

The contour identification unit 122 analyzes the aerial image (hereinafter sometimes referred to as a captured image) acquired by the acquisition unit 121, and recognizes (identifies) pumpkin plants in the image. As an example, recognition (identification) of pumpkin plants in an image is performed by picking out only the green pixels of the pumpkin from the image. When recognizing (identifying) a pumpkin plant, an image obtained by extracting a part of an aerial photographed image can be used, and conversely, an image obtained by joining a plurality of aerially photographed images can also be used.

In addition, the outline specifying unit 122 specifies the outline of the image area of the identified pumpkin plant for each plant.

The plant area identification unit 123 uses the image to identify the area of each pumpkin plant. In one example, the stock area identification unit 123 identifies the area of each stock based on the contour identified by the contour identification unit 122 .

The circle area specifying unit 124 uses the image to specify the area of the minimum circumscribed circle that circumscribes the pumpkin plant for each plant. Specifically, the circle area identification unit 124 identifies the minimum circumscribed circle for the contour identified by the contour identification unit 122, and identifies the area of the minimum circumscribed circle. A known method can be used to specify the minimum circumscribed circle for the contour.

The determination unit 125 uses the ratio of the area of the pumpkin plant identified by the plant area identification unit 123 and the area of the minimum circumscribed circle identified by the circle area identification unit 124 to identify the pumpkin plant having short internodes. It is determined whether it is (whether it is a strain having a specific trait).

Here, the ratio of the area of the pumpkin strain to the area of the minimum circumscribed circle is greater in the pumpkin strains having short internodes than in the so-called common variety pumpkin strains that do not have short internodes. That is, in the case of strains having short internodes, the strains do not spread over a wide area, and the strains are in a compact form, so the minimum circumscribed circle is small, and the strains are densely packed in the minimum circumscribed circle. On the other hand, the stalks of ordinary squash grow on the ground and extend their vines away from the base of the stalks. Therefore, in ordinary varieties, the area of the minimum circumscribed circle tends to be large, and the strains are not densely packed within the minimum circumscribed circle. From these facts, it is possible to identify (determine) whether or not a pumpkin strain has short internodes by using the ratio of the area of the pumpkin strain to the area of the minimum circumscribed circle.

In short, the determining unit 125 has a threshold value, and compares this threshold value with the ratio of the area of the pumpkin plant to the area of the minimum circumscribed circle to determine whether the plant has short internodes. do. Note that the threshold can be changed by the user.

The output unit 126 outputs information regarding the determination result of the determination unit 125 . The information about the determination result is, for example, an image representing the location of the pumpkin plant having short internodes in the image. As an example, the output unit 126 outputs the information by performing a process of causing the display unit 26 to display an image representing a portion of a pumpkin plant having short internodes.

The storage unit 22 is a recording device that stores various types of information. For example, the aerial image acquired by the acquisition unit 121, or information that the determination unit 125 refers to when identifying a pumpkin plant having short internodes. Store.

The input unit 24 is an interface for operating the information processing apparatus 10 or inputting information. For example, the input unit 24 supplies the captured image of the aircraft 30 input to itself to the acquisition unit 121 . Also, part of the input unit 24 can be implemented as a device such as a keyboard or a mouse that receives an operation to the information processing device 10 .

The display unit 26 is a display panel that displays text, moving images, etc. under the control of the control unit 12 . Note that the display unit 26 may be configured to implement a part of the functions of the input unit 24 as a touch panel.

The flying object 30 is an unmanned flying object 30 realized as a drone, UAV (Unmanned Aerial Vehicle), or the like. The flying object 30 is equipped with a camera (not shown) and takes an aerial image of the field. Hereinafter, the flying object 30 will be described as having a configuration for capturing an RGB image, but the configuration is not necessarily limited to the above configuration. Also, the flying object 30 does not need to autonomously fly along a predetermined route, and may be operated by the user in real time.

[2. Processing example of information processing system]
FIG. 2 is an example of a flowchart showing the flow of the information processing method according to this embodiment. In step S101, the flying object 30 takes a stereo photograph or a plurality of photographs as an aerial image of a field. In this processing example, a plurality of ridges are formed in the field to be photographed, and a plurality of pumpkin strains are planted on each ridge. A space is provided between the ridges so that the pumpkin stalks do not overlap each other. The squash plant has vines that grow and cover part of the ridges with leaves.

In step S<b>102 , the acquisition unit 121 acquires an aerial image captured by the aircraft 30 via the input unit 24 . In addition, when the acquiring unit 121 acquires a plurality of aerial images obtained by photographing a part of a field, an image obtained by joining the plurality of aerial images may be generated. In the case of joining, an image (orthoimage) of the entire field may be formed. The imaging range of one aerial image is not particularly limited as long as the determination unit 125 can perform determination based on the stock image captured in the imaging range. In other words, it is sufficient that at least one pumpkin plant is captured in the aerial image. However, in order for the judging unit 125 to efficiently find a pumpkin plant having short internodes from a field or ridge, it is preferable that a plurality of plants be captured in one aerial image.

FIG. 3 is a diagram illustrating an aerial photographed image. Note that the image PIC of FIG. 3 may be an image obtained by joining together a plurality of aerial images captured by the flying object 30 (FIG. 1). The image PIC of FIG. 3 shows one ridge 300 and a plurality of pumpkin plants 400 growing side by side along the longitudinal direction of the ridge. Note that the image PIC of FIG. 3 can be displayed on the display unit 26 (FIG. 1). Note that the aerial image is actually a color image.

In steps S103 to S109, pumpkin plants having short internodes are selected from the pumpkin plants 400 in the image PIC. Specifically, in step S103, the image obtained in step S102 is analyzed, and the outline identification unit 122 (FIG. 1) recognizes (identifies) the pumpkin plant in the image. As an example, recognition (identification) of pumpkin plants in an image is performed by picking out only the green pixels of the pumpkin from the image. Furthermore, in step S103, the contour specifying unit 122 specifies the contour of the image area of the identified pumpkin plant for each plant.

Here, FIG. 4 is an image PIC1 obtained by recognizing a pumpkin plant in the image PIC from the aerial image PIC shown in FIG. The image PIC1 in FIG. 4 is a black and white image as an example, and the white area in the image PIC1 in FIG. 4 is the pumpkin plant. The image PIC1 in FIG. 4 can also be displayed on the display unit 26, like the aerial image PIC in FIG.

The image PIC used when recognizing (identifying) pumpkin plants and the image PIC2 used when specifying the outline of the image area of the identified pumpkin plants for each plant may be the same image data. , may be duplicated image data. Furthermore, the outline may be specified using only a partial image of the image PIC used when recognizing (identifying) the pumpkin plant. As an example, the part of the image is an extracted (cropped) image of the area identified as the pumpkin plant and its surrounding area.

In step S104 (plant area specifying step), the plant area specifying unit 123 (FIG. 1) uses an image to specify the area of each pumpkin plant. Specifically, in step S<b>104 , the stock area identification unit 123 identifies the area of each stock based on the contour identified by the contour identification unit 122 . Note that the method of specifying the area of the stock is not limited to the method of calculating based on the outline. For example, an area in which the pixel values representing the shade of green of each pixel in the image are included in a predetermined range is specified as the stock area. As an example, the stock area identification unit 123 converts the entire image in the RGB color system into the L*a*b color system, and identifies a region of pixels whose channel a* value is equal to or less than a predetermined value (for example, 120) as a stock area. It is also possible to use the number of pixels obtained by specifying the area of the stock and counting the number of pixels in each area of the stock as the area of the stock.

In step S105 (circle area specifying step), the circle area specifying unit 124 (FIG. 1) uses the image to specify the area of the minimum circumscribed circle that circumscribes the pumpkin plant for each plant. Specifically, in step S105, the minimum circumscribed circle for the contour identified in step S104 is identified, and the area of the minimum circumscribed circle is identified.

In this example, the step S104 of specifying the area of the stock is performed prior to the step S105 of specifying the area of the minimum circumscribed circle, but these two steps do not matter in context. In addition, as described above, when the area of the stock is specified without using the outline in step S104, the step S104 of specifying the area of the stock and the step S103 of specifying the outline may be performed in any context. do not have.

In step S106 (determination step), the determination unit 125 (FIG. 1) uses the ratio of the area of the strain identified in step S104 and the area of the minimum circumscribed circle identified in step S105 to Identify (determine) whether it is a pumpkin strain.

5 and 6 show an image PIC2 in which an image 500P showing the outline of a certain pumpkin plant in the image and an image 500C showing the minimum circumscribed circle circumscribing the outline (image 500P) are synthesized and displayed. indicates FIG. 5 shows a pumpkin strain of a common variety that is not a pumpkin strain having short internodes, and FIG. 6 shows a pumpkin strain having short internodes. Image PIC2 in FIGS. 5 and 6 is actually a color image. Each of FIGS. 5 and 6 shows, as an example, the area of the strain, the area of the minimum circumscribed circle, and their ratio (area ratio) together with the image PIC2. The area ratio in FIG. 6 is larger than the area ratio in FIG. That is, the strain shown in FIG. 6 has shorter vines than the strain shown in FIG. This is clear from the images in FIGS. 5 and 6, and it can be seen that the strain in FIG. 6 has a shorter vine length than the strain in FIG. .

Thus, in step S106, the area ratio between the area of the stock and the area of the minimum circumscribed circle is calculated for each stock, and when the calculated area ratio is larger than the threshold, the process proceeds to step S107. On the other hand, if the calculated area ratio is equal to or less than the threshold, the process proceeds to step S108.

In step S107, the determination unit 125 generates an output signal indicating that the strain to be determined has short internodes. Also, in step S108, the determination unit 125 generates an output signal indicating that the strain to be determined is not a strain having short internodes.

In step S109, the output unit 126 (FIG. 1) acquires the output signals generated in steps S107 and S108, and the output unit 126 outputs information according to the output signals. As an example, the output unit 126 generates an image representing the location of the strain having short internodes, and displays the generated image on the display unit 26 .

FIG. 7 shows an example of an image displayed on the display unit 26. As shown in FIG. FIG. 7 shows two images, and the upper image and the lower image show the difference in determination results when the same aerial image is used and the threshold is changed. Each image in FIG. 7 is actually a color image.

In the upper image of FIG. 7, the threshold is set to 0.69 (69%), and strains with a ratio (area ratio) of 0.69 (69%) or more are displayed as strains having short internodes, and 0.69 (69%). Less than 69 (69%) strains are designated as non-strains with short internodes.

In the upper image of FIG. 7, the threshold is set to 0.60 (60%), and strains with a ratio (area ratio) of 0.60 (60%) or more are displayed as strains having short internodes. Less than 60 (60%) strains are designated as non-strains with short internodes.

In each image of FIG. 7 , the minimum circumscribed circle of the strain having short internodes is displayed in white, and the minimum circumscribed circle of the strain having short internodes is displayed in black. This allows the user to intuitively (visually) find strains having short internodes. The display method is not limited to the color of the minimum circumscribed circle, as long as it is possible to distinguish whether or not the strain is a strain having short internodes. As another example, there may be a mode in which only strains with short internodes are displayed with images of the minimum circumscribed circle or other shape. Moreover, when each strain is assigned an identification number, the identification number of the strain having short internodes may be displayed.

When changing the threshold value as shown in FIG. 7, the threshold value can be easily changed by sliding the slide bar 500S displayed on the display unit 26. FIG. In this case, it is sufficient if the determination result is reflected on the display unit 26 almost simultaneously when the threshold value is changed. This allows the user to examine the threshold value while viewing the determination results displayed on the display unit 26 .

(Effect of this embodiment)
According to this embodiment, it is possible to efficiently select pumpkin strains having short internodes from photographed images of fields.

[Additional notes]
[Appendix 1]
In the above description, one ridge 300 shown in the image PIC of FIG. 3 is taken as an imaging target, but the imaging target is not limited to this, and multiple ridges can be taken as an imaging target. As a result, a wide range of regions can be determined at once by the determination unit 125, and strains having short internodes can be efficiently searched. Moreover, it is not limited to a field in which ridges are formed. In addition, it is not necessary that a plurality of strains are cultivated side by side, but it is acceptable as long as the strains are planted at predetermined intervals so that the strains do not overlap.

[Appendix 2]
The functions of the information processing apparatus 10 according to the above-described embodiments may be implemented by a single device, or may be implemented by a system in which a plurality of devices work together. For example, a first device that implements the acquisition unit 121, a second device that implements the contour identification unit 122, a third device that implements the stock area identification unit 123, and a third device that implements the circle area identification unit 124. 4 and a fifth device implementing the determination unit 125 and the output unit 126 may implement the information processing device 10 .

[Appendix 3]
In the above-described embodiment, the acquisition unit 121 acquires the aerial image captured by the flying object 30 . However, without being limited to this, as an example, the acquisition unit 121 may acquire a satellite image obtained by imaging observation data of a sensor mounted on an artificial satellite.

[Appendix 4]
In the above-described embodiment, a pumpkin strain having short internodes is selected from the ratio of the area of the strain identified based on the outline of the strain and the area of the minimum circumscribed surface. Alternatively, it is also possible to select an individual having a specific trait from the ratio of the strain area specified based on the contour of the strain and the area of the minimum circumscribed surface. For example, it is possible to compare whether the progress of the growth stage is fast or slow, and for example, it is possible to select strains that are early in the growth stage. As an example, it is possible to distinguish between individuals whose vines have begun to grow and those whose vines have not yet begun to grow, even for the same pumpkin variety, based on the ratio of the plant area to the minimum circumscribed surface area. In research focusing on cultivation techniques, the ratio of the area of the plant to the area of the minimum circumscribed surface can be used as a method of comparing individuals of the same variety. When looking at differences in growth due to cultivation techniques, the target individual may be an individual cultivated in a different cultivation environment, rather than an individual growing side by side on the same ridge as in the above-described embodiment. you can

[Appendix 5]
In the above-described embodiment, creeping plants (pumpkins) are targeted, but plants other than creeping plants may be used. For example, plants of the Gramineae family such as rice and wheat may be used. In the grasses such as rice and wheat, the angle between the main stem and the tillers (tiller opening) differs depending on the germplasm when the tillers are increased. When quantifying the appearance (plant type, grass form) of each genetic resource, the same plant area (coverage area) or the same minimum Even genetic resources with circumscribed circle area can show different land coverage.

[Example of realization by software]
The control blocks of the information processing system 1 (especially the acquisition unit 121, the contour identification unit 122, the stock area identification unit 123, the circle area identification unit 124, the determination unit 125 and the output unit 126) are formed in an integrated circuit (IC chip) or the like. It may be implemented by a logic circuit (hardware), or may be implemented by software.

In the latter case, the information processing system 1 includes a computer that executes instructions of programs, which are software that implements each function. This computer includes, for example, one or more processors, and a computer-readable recording medium recording the above program. In the computer, the processor reads the program from the recording medium and executes it, thereby achieving the object of the present invention. As the processor, for example, a CPU (Central Processing Unit) can be used. As the recording medium, a "non-temporary tangible medium" such as a ROM (Read Only Memory), a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, or the like can be used. In addition, a RAM (Random Access Memory) for developing the above program may be further provided. Also, the program may be supplied to the computer via any transmission medium (communication network, broadcast wave, etc.) capable of transmitting the program. Note that one aspect of the present invention can also be implemented in the form of a data signal embedded in a carrier wave in which the program is embodied by electronic transmission.

The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims.

1 information processing system 10 information processing device 12 control unit 22 storage unit 24 input unit 26 display unit 30 flying object 121 acquisition unit 122 outline identification unit 123 stock area identification unit 124 circle area identification unit 125 determination unit 126 output unit 400 pumpkin stock ( predetermined plant strain)

Claims (9)

a stock area identification unit that identifies the area of each stock of a predetermined plant using an image of a field in which a predetermined plant is planted;
A circle area identifying unit that identifies the area of the minimum circumscribed circle that circumscribes the stock of a predetermined plant using the image, for each stock;
a determination unit that determines whether the strain has a specific trait using the ratio of the area of the strain and the area of the minimum circumscribed circle;
including,
Information processing equipment. Using the image, further comprising a contour identifying unit that identifies the contour of a given plant strain,
The stock area identification unit identifies the area of the stock based on the outline of the stock,
The circle area identifying unit identifies the minimum circumscribed circle for the outline of the stock and identifies the area of the minimum circumscribed circle,
The information processing device according to claim 1 . The predetermined plant is a creeping plant,
The information processing apparatus according to claim 1 or 2. The predetermined plant is pumpkin,
The information processing apparatus according to any one of claims 1 to 3. The determination unit determines whether the strain has short internodes,
The information processing apparatus according to claim 4. The determination unit compares the ratio with a threshold value to determine whether the strain has a specific trait,
the threshold is variable;
The information processing apparatus according to claim 4. A stock area identification step of identifying the area of each stock of a predetermined plant using an image of a field in which a predetermined plant is planted;
A circle area specifying step of specifying the area of the minimum circumscribed circle that circumscribes the strain of a predetermined plant using the image for each strain;
A determination step of determining whether the strain has a specific trait using the ratio of the area of the strain and the area of the minimum circumscribed circle;
including,
Information processing methods. A program for causing a computer to function as the information processing apparatus according to claim 1, the program for causing the computer to function as the stock area identifying section, the circle area identifying section, and the determining section. A computer-readable recording medium recording the program according to claim 8 .

Images