JP2020122692A - Image processor and method for processing image - Google Patents

Abstract

To specify a state of an object by using an image of the object.SOLUTION: An image processor 2 acquires first information showing a direction of an edge of an object from an image generated by imaging of the object and specifies at least one of a first region of the image, in which the object is in a specific state, and a second region, in which the object is not in the specific state, by using the first information.SELECTED DRAWING: Figure 1

Description

The present invention relates to an image processing technique for acquiring information on an object from an image obtained by imaging an object such as a plant.

In the cultivation of plants such as paddy rice, the leaf color value, which is a numerical value of the leaf color, is used as an index for determining the growth state of the plant. There are two methods for measuring leaf color values: a method using a leaf color scale and a method using a chlorophyll meter.

In these measuring methods, it has been proposed to use an image obtained by imaging a plant. Patent Document 1 discloses a method of correcting the influence of the illumination spectrum at the time of imaging in order to accurately measure the leaf color value of paddy rice using an image obtained by imaging rice.

JP, 2002-168771, A

However, even if the method disclosed in Patent Document 1 is used, the leaf color value may not be accurately measured when the paddy rice is hindered or lodged by the wind. The reason for this is that the leaf color values are measured as different values because the illumination light hits differently between the paddy rice that has been swollen or lodged by the wind and the paddy rice that does not. Further, even if it is tried to determine whether the paddy rice is dull or lodging due to wind from the measured leaf color values, it is difficult to distinguish it from the case where the leaf color values are partially different due to poor growth or the like.

The present invention provides an image processing apparatus and an image processing method that are effective for specifying the state of an object using a captured image of the object.

An image processing apparatus according to one aspect of the present invention uses a first information acquisition unit that acquires first information indicating an edge direction of an object from an image generated by imaging an object, and the first information. , An area specifying unit that specifies at least one of a first area in which the object is in a specific state and a second area in which the object is not in a specific state.

An image processing method according to another aspect of the present invention uses a step of acquiring first information indicating an edge direction of the object from an image generated by imaging the object, and using the first information, Identifying at least one of a first region in which the object is in a particular state and a second region in which the object is not in a particular state.

A computer program that causes a computer to execute the processing according to the image processing method also constitutes another aspect of the present invention.

According to the present invention, it is possible to specify a region in a captured image of an object according to the state of the object.

The flowchart which shows the process performed in the plant information acquisition system which is an Example of this invention. 6 is a flowchart showing a process of calculating an edge direction feature amount according to the embodiment. The figure which shows the smoothing filter used for the edge emphasis process in FIG. FIG. 3 is a diagram showing a differential filter used for the edge image generation processing in FIG. 2. FIG. 3 is a diagram showing an example of a measurement image, a solid/collapsed area, and a non-silvered/non-cold area in Example 1. 9 is a flowchart showing a process performed in the plant information acquisition system according to the second embodiment. The flowchart which shows the process performed in the plant information acquisition system which is Example 3. FIG. 10 is a diagram showing a relationship between a smooth/collapse region area and a threshold value in Example 3; The flowchart which shows the process performed in the plant information acquisition system which is Example 4. FIG. 13 is a diagram showing a process for determining a solid/falling area in the fourth embodiment. The figure which shows the structure of the plant information acquisition system of an Example.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

In the embodiment of the present invention, an edge direction feature amount (first information) as information indicating an edge direction is acquired (calculated) from an image generated by imaging a plant as an object (hereinafter, referred to as a measurement image). .. Next, using the edge direction feature amount, it is determined whether or not the plant is in a specific state in which it is sluggish in the wind or lodged (hereinafter, referred to as slid/lodging state). Then, at least one of the first region and the second region in which the plant is in the absent/disposed state is extracted (specified) from the measurement image. An image area as the first or second area is used as an information acquisition area, and a leaf color value (second information), which is plant information serving as an index for determining the growth state of a plant, is acquired from the image information of the information acquisition area ( Can be measured).

First, the edge direction feature amount will be described. The flowchart of FIG. 2 shows a process of calculating the edge direction feature amount. The calculation of the edge direction feature amount is executed by four processes of an edge emphasis process (S111), a binarization process (S112), an edge image generation process (S113), and an edge direction calculation process (S114). S means a step.

As a premise, a monochrome image I(x, y) obtained by imaging a plant is used as the measurement image as an input image for which the edge direction feature amount is calculated. I indicates the brightness value of each pixel of the monochrome image. When the image generated by imaging the plant is a multi-band image, an image calculated by the weighted sum of the band images may be used as the monochrome image I(x, y). For example, when the image generated by imaging a plant is an RGB image, it is preferable to use a grayscale image calculated by the weighted sum of the R band image, the G band image, and the B band image. Also, as the monochrome image I(x, y), a representative band image of the multi-band images may be used. For example, when the image generated by capturing an image of a plant leaf is an RGB image, a G band image that well reflects the green color of the plant leaf may be used as the monochrome image I(x, y). When the number of horizontal and vertical pixels of the image is Nx and Ny, respectively, x=1,. ． ． , Nx and y=1,. ． ． , Ny.

In the edge enhancement processing (S111), an edge enhanced image I _enhance (x, y) in which the edges of the monochrome image I(x, y) are enhanced is generated. Specifically, the equation (1) and (2) below, to generate an edge enhanced image edge enhanced image _I enhance (x, y).

Here, I _mean (x, y) is a smoothed image obtained by smoothing the monochrome image I(x, y) using the smoothing filter H _mean , in other words, a local image including each pixel (x, y). The average luminance value of the pixels in the area is shown. The symbol * means convolution integral. That is, the expression (2) means that the brightness value I of each pixel (x, y) of the monochrome image I(x, y) is divided by the average brightness value.

3A and 3B show examples of the smoothing filter H _mean . FIG. 3A shows a smoothing filter when the filter size n=3, and FIG. 3B shows a smoothing filter when the filter size n=5.

When the edge enhancement processing (S111) is performed, the edge can be enhanced even in a region where the local brightness difference is small. As a result, the edge to be detected can be appropriately detected in the subsequent edge image generation processing (S113).

In the binarization process (S112), the binarized image I _binary (x, y) is generated from the edge-enhanced image I _enhance (x, y) obtained by the edge enhancement process using the following formula (3). To do.

When this binarization processing (S112) is performed, the edges of the edge-enhanced image I _enhance (x, y) can be further enhanced, and the edges to be detected are more appropriate in the subsequent edge image generation processing (S113). Can be detected.

In the edge image generation process (S113), the binarized image I _binary (x, y) obtained by the binarization process is differentiated in at least two directions to generate differential images in at least two directions. Here, the differential images G _x (x, y) and G _y (x, y) in the x and y directions will be described as an example. The differential images G _x (x, y) and G _y (x, y) are generated using the following equations (4) and (5).

Here, H _x is a differential filter in the x direction and H _y is a differential filter in the y direction. The symbol * means convolution integral.

Figure 4 (a) ~ (c) shows an example of a y-direction differential filter H _y. As the y-direction differential filter H _y , the Prewitt filter shown in FIG. 4A may be used, or the Sobel filter shown in FIG. 4B may be used. Alternatively, a differential filter as shown in FIG. 4(c) may be used. As shown in FIG. 4C, when the filter size of the differential filter is increased, the gradation of the edge direction feature amount calculated in the edge direction calculation process (S114) can be increased. A filter obtained by transposing the y-direction differential filter H _y may be used as the x-direction differential filter H _x .

In the edge direction calculation process (S114), the edge direction feature amount θ(x, y) is calculated using the following equation (6).

Here, the edge direction feature amount θ(x, y) is a feature amount that changes in the range of −90°≦θ(x, y)≦90°. In the description of the edge image generation process (S113) and the edge direction calculation process (S114), the differential images G _x (x, y) and G _y (x, y) in the x direction and the y direction are given as examples. , The two directions to be differentiated are not limited to the x direction and the y direction as long as they are orthogonal to each other.

The calculation method of the edge direction feature amount θ(x, y) in the present embodiment is such that even if the monochrome image I(x, y) has a luminance distribution or the contrast of the monochrome image I(x, y) is low, the edge Is almost certainly detected, and the edge direction feature quantity θ(x, y) of the detected edge can be accurately calculated.

Then, in this embodiment, the edge/feature amount θ(x, y) thus calculated is used to determine the swinging/accumulation state of the imaged plant. The edge direction feature amount θ(x, y) has a feature that it is unlikely to depend on the brightness and color of the plant in the image. For this reason, it is possible to determine the dull/accelerated state of the plant independently of the brightness and color of the plant.

According to this example, it is possible to favorably distinguish between paddy rice having different leaf color values due to poor growth and the like, and paddy rice having different leaf color values due to the dry/falling state.

Hereinafter, as a specific example, the information on the leaf color value is selected from the measurement image by using the edge direction feature value to determine the plant's laid/lodging state and selecting the information acquisition region using the information on the laid/lodging state ( The plant information acquisition system to be set) will be described.

FIG. 11 shows the configuration of the plant information acquisition system 100 according to the first embodiment. The plant information acquisition system 100 includes an imaging device 1 that images a plant (paddy rice) to generate a measurement image, and a personal computer (PC) 2 as a pixel processing device that acquires the measurement image from the imaging device 1. It is configured.

The flowchart of FIG. 1 shows the processing (image processing method) performed by the PC 2 in this embodiment. The PC 2 performs an edge direction feature amount calculation process (S110), a smooth/accelerated state determination process (S120), and a non-cervical/non-accelerated region (information acquisition region) extraction process (S130) according to a built-in computer program. To do. The PC 2 functions as first information acquisition means and area acquisition means.

In the edge direction feature amount calculation process (S110), the PC 2 calculates the edge direction feature amount θ(x, y) by performing the process described with reference to the flowchart of FIG.

FIG. 5 shows the determination result of the rice paddy/laying state using the edge direction feature amount θ(x, y). The three measurement images in the upper row are grayscale images generated from RGB images generated by imaging rice in the same region from the same point at different times (t1, t2, t3). In the measurement image at the time t2, part of the paddy rice is turbulent in the wind, the brightness value of the quiescent paddy rice is high, and the measured leaf color value is low. The PC 2 calculates the edge direction feature amount θ(x, y) for these measurement images by the edge direction feature amount calculation process (S110).

Next, the PC 2 performs the edge/direction feature amount θ() between the plant in the aggression/accordance state as a specific state and the plant in the non-acquisition/accordance state (S120) in the aspiration/acceleration state determination process (S120). The difference in (x, y) is used to determine whether or not the plant in the measurement image is in a dull/accumulated state. Specifically, the determination is made using the following determination formula (7).

Here, θ _threshold is a threshold value (predetermined value) that is compared with the edge direction feature amount θ(x, y) in order to determine whether the state is the solid/falling state or the non-solid/non-falling state. In the present embodiment, the edge direction feature amount θ(x, y) has a value near 0 in the non-quick/non-yielding state, and the absolute value of the edge direction feature amount θ(x, y) has a value in the non-quick/prone state. The fact that the size becomes large is used to determine whether the user is in a good/prone state or a non-good/non-prone state.

The three images in the middle part of FIG. 5 are images showing the regions determined to be in the laid/lodging state among the above-mentioned three measurement images, and the region in the laid/tilting state is 1 (white), Others are represented by 0 (black). From these images, it can be seen that the area in the prone/prone state is appropriately determined using the edge direction feature amount. In addition, the three images in the lower part of FIG. 5 are images showing the regions determined to be in the non-quick/non-collapsed state among the above-described three measurement images, and the regions in the non-quick/non-collapsed state. Is represented by 1 (white), and the others are represented by 0 (black).

In this embodiment, the size of the smoothing filter is n=7, and the filter shown in FIG. 4C is used as the differential filter. Further, the threshold value θ _threshold =20° is set. The threshold value θ _threshold can be selected according to the type of the plant to be imaged and the degree of the dull/accumbing state to be detected. In the case of paddy rice, the threshold value θ _threshold = 20° to 30° may be set.

Further, in the non-silver/non-yielding region extraction processing (S130), the PC 2 extracts the non-silver/non-yielding region shown in the lower part of FIG. 5 from the measurement image. The PC 2 may display the extracted non-silver/non-collapsed area on a monitor or output it to a printer.

According to the present embodiment, even if the plant in the measurement image is swollen or lodged in the wind, it is possible to extract an image region in which accurate plant information can be acquired.

Next, a second embodiment will be described. The flowchart of FIG. 6 shows the processing performed by the PC 2 in this embodiment. The PC 2 performs an edge direction feature amount calculation process (S210), a smooth/accelerated state determination process (S220), a non-quick/non-accumulated region extraction process (S230), and a plant information acquisition process (S240). S210 to S230 are the same as S110 to S130 described in the first embodiment. The PC 2 functions as a first information acquisition unit, a region specifying unit, and a second information acquisition unit.

In the plant information acquisition process (S240), the PC 2 measures the leaf color value as the plant information (object information) from the extracted image information of the non-silver/non-cold area.

In the present embodiment, a non-silent/non-yielding region that does not include a solid/yield state paddy rice that is not suitable for obtaining plant information is extracted, and the plant information is acquired using the image information of this non-silver/non-yielding region. Therefore, accurate plant information can be obtained.

It is also possible to extract the solid/accelerated region from the image for measurement and obtain the plant information from the image information in the extracted crystalline/accelerated region.

Next, a third embodiment will be described. The flowchart of FIG. 7 shows the processing performed by the PC 2 in this embodiment. The PC 2 performs an edge direction feature amount calculation process (S310) and a squeeze/falling state determination process (S320). S310 and S320 are the same as S110 and S120 described in the first embodiment. Further, the PC 2 performs a smooth/accelerated region extraction process (S330), a smooth/accelerated region area calculation process (S340), a threshold determination process (S350), and a plant information acquisition process (S360). S360 is the same as S240 described in the second embodiment. The PC 2 functions as a first information acquisition unit, a region identification unit, an image determination unit, and a second information acquisition unit.

In the smooth/accelerated area extraction process (S330), the PC 2 extracts the smooth/accelerated area shown in the middle part of FIG. 5 from the measurement image.

Next, in the squeeze/acceleration region area calculation process (S340), the PC 2 calculates the area of the squeeze/acceleration region extracted by the compression/acceleration region extraction process (S330). Specifically, the PC 2 counts the area of the blue/collapse region by the number of pixels. FIG. 8 shows the result of calculating the number of pixels in the solid/collapsed area at the times t1, t2, and t3 shown in FIG.

Subsequently, in the threshold value determination processing (S350), the PC 2 determines whether or not the area of the rough/collapsed region is equal to or larger than the threshold value N _threshold . When the area of the auspicious/accelerated region is greater than or equal to the threshold N _threshold , the measurement image is obtained by imaging the avid/accelerated plant, and is suitable as a measurement image for obtaining plant information. It is determined that the image does not exist. In FIG. 8, since the area of the solid/accelerated region of the measurement image at time t2 is equal to or larger than the threshold N _threshold, it is determined that this measurement image is not suitable as an image for obtaining plant information.

On the other hand, when the area of the solid/accelerated region is smaller than the threshold N _threshold , the PC 2 determines that the measurement image is an image suitable for obtaining the plant information, and acquires the plant information by using the measurement image. A process (S360) is performed. In FIG. 8, the area of the solid/accelerated region of the measurement images at times t1 and t3 is smaller than the threshold value N _threshold , and thus these measurement images are determined to be suitable as images for obtaining plant information.

The threshold value N _threshold can be selected according to the total number of pixels of the measurement image, the vegetation cover ratio of the plant (the ratio of the plant covering a predetermined area), and the like.

According to the present embodiment, it is possible to stably and accurately measure the plant information by not using the measurement image that is not suitable for obtaining the plant information as the measurement image for obtaining the plant information.

Note that the area of the non-smooth/non-collapsed area may be represented by the actual area instead of the number of pixels. In addition, instead of determining whether the measurement image is suitable for obtaining plant information by determining whether the area of the solid/accelerated area is smaller than a threshold value, whether the area of the non-ceramic/non-accumulated area is larger than the threshold value is determined. Alternatively, it may be determined whether or not the measurement image is suitable for obtaining the plant information.

Next, a fourth embodiment will be described. The flowchart of FIG. 9 shows the processing performed by the PC 2 in this embodiment. The PC2 acquires a measurement image (hereinafter referred to as a t1 image and a t2 image) generated by capturing the same region from the same point at different times t1 and t2, respectively, as a t1 image input process (S411). And t2 image input processing (S412). Then, the PC 2 performs edge direction feature amount calculation processing (S421, S422), squeeze/acceleration state determination processing (S431, S432), and squeeze/acceleration area extraction processing (S441) for each of the t1 image and the t2 image. S442) is performed. S411 to S432 are the same as S110 to S120 described in the first embodiment, and S411 and S442 are the same as S330 described in the third embodiment.

Further, the PC 2 performs a difference calculation process (S450) and a squeeze/fallover determination process (S460). The PC 2 functions as a first information acquisition unit, a region identification unit, and an object determination unit.

The difference calculation process (S450) will be described with reference to FIG. In this difference calculation process, the PC 2 calculates the difference between the images of the solid/accelerated regions at the times t1 and t2 shown in FIG. 5, as shown in FIG. 10, to generate a difference image. Specifically, the image of the dynamic/fallover area at time t1 is subtracted from the image of the dynamic/fallover area at time t2.

In the difference image shown in FIG. 10, the area in the quivering state at time t2 is +1 (white), and the areas in the quivering/accumulating state or the non-quickness/non-clining state at times t1 and t2 are 0 (gray), The region in the dull state at time t1 is represented by -1 (black).

Next, in the dynamic/acceleration determination process (S460), the PC 2 multiplies the image obtained by adding 2 to the difference image by the image of the dynamic/accelerated region at the time t1, so that the time t1 shown in the lower part of FIG. To generate a slap/pronouncement determination image. Similarly, the image obtained by adding 2 to the difference image is multiplied by the image of the dynamic/accelerated area at time t2 to generate the dynamic/accelerated determination image at time t2.

In addition, in the dynamic/accelerated determination process, the image of the non-determined/non-accelerated area may be multiplied by the image obtained by adding 2 to the difference image to generate the dynamic/accelerated determination image.

Then, the PC 2 determines whether or not the plant is in the wind or is prone to the wind at each time by comparing the propulsion/accumbing determination images at the times t1 and t2. Specifically, in each of the mesh/acceleration determination images, the area of +3 (white) indicates that it is in the mesh state at the time t2, and the area of +2 (ash) is in the lodging state in which the plant is constantly collapsed. Indicates that. Further, the area of +1 (gray) indicates that it is in a dull state at time t1. Further, the region of 0 (black) indicates that the time is at a time t1 or t2 in a non-quick/non-laid-down state.

According to the present embodiment, by using the measurement image generated by capturing the same region from the same point at different times, it is determined whether the plant is in the wind or lying. be able to.

Note that the measurement images generated by imaging at different times may be a plurality of images, and the more measurement images are used, the more reliable the determination result can be.

In each of the above embodiments, paddy rice was taken as an example of the plant to be imaged, but wheat or other plants may be the subject to be imaged, and an object other than the plant may be the image to be captured. Further, the leaf color value has been taken as an example of the plant information, but plant information different from the leaf color value may be acquired.
(Other embodiments)
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program. It can also be realized by the processing. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

The embodiments described above are merely representative examples, and various modifications and changes can be made to the embodiments when implementing the present invention.

1 Imaging device 2 Personal computer

Claims (10)

First information acquisition means for acquiring first information indicating an edge direction of the object from an image generated by imaging the object;
Area specifying means for specifying at least one of a first area in which the object is in a specific state and a second area in which the object is not in the specific state using the first information; An image processing apparatus comprising: The first information acquisition means is
A process of generating an edge-enhanced image by dividing a brightness value of each pixel of the image by an average brightness value of pixels in a local area including each pixel;
A process of generating a binarized image from the edge-enhanced image,
A process of differentiating the binarized image in at least two directions to generate a differential image;
The image processing apparatus according to claim 1, wherein the first information is acquired by performing a process of obtaining the edge direction from the differential image. The image processing apparatus according to claim 1, wherein the area specifying unit specifies at least one of the first and second areas by comparing the first information with a predetermined value. .. The object is a plant,
The image processing apparatus according to claim 1, wherein the specific state is at least one of a state where the plant is swaying in the wind and a state where the plant is lying down. 5. The information processing apparatus according to claim 1, further comprising a second information acquisition unit that acquires second information different from the first information as the information of the object in the first or second area. The image processing device according to claim 1. The object is a plant,
The image processing apparatus according to claim 5, wherein the second information is a leaf color value of the plant. 7. An image determination means for determining whether or not to acquire the second information from the image according to the area of the first or second region is included. The image processing device according to item. Using a plurality of images generated by imaging the object at different times, the object determining means determines whether or not the object is in the specific state at each of the times. The image processing device according to claim 1. Acquiring first information indicating an edge direction of the object from an image generated by imaging the object,
Identifying at least one of a first region in which the object is in a particular state and a second region in which the object is not in the particular state, using the first information. An image processing method characterized by the above. A computer program for causing a computer to execute a process according to the image processing method according to claim 9.