JP2016165303A - Vegetation determination program, vegetation determination method, and vegetation determination device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a vegetation determination program, a vegetation determination method, and vegetation determination device in which influence of the time of the year when an image is captured is suppressed.SOLUTION: A vegetation determination program allows a computer to execute processing of: severally calculating image feature quantities of at least a predetermined region regarding a first image and a second image which are obtained by imaging a predetermined region at different times of the year; referring to database including information showing different image feature quantities at different times of the year associating with types of plants, based on the times of the year when the images were captured to specify the type of plant included in the predetermined region of the first image and the type of plant included in the predetermined region of the second image; determining that the type of plant in the predetermined region has not been changed between the different times of the year in the case where the type of plant specified regarding the first image matches the type of plant specified regarding the second image, and determining that the type of plant in the predetermined region has been changed between different times of the year in the case where the type of plant specified regarding the first image does not match the type of plant specified regarding the second image.SELECTED DRAWING: Figure 3

Description

  The present disclosure relates to a vegetation determination program, a vegetation determination method, and a vegetation determination apparatus.

  For example, the technique of Patent Document 1 has been proposed as a technique for discriminating the type of plant that has become a subject based on an image of a plant photographed.

  Further, as a technique for classifying trees growing in the photographing range according to the type based on an image obtained by photographing the forest from the sky and acquiring information on the current state of the forest, for example, the technique of Patent Document 2 is used. Proposed.

  The types of plants growing in the shooting range of individual images are based on the techniques disclosed in Patent Documents 1 and 2 described above from images shot at a time when the plants show typical characteristics. Can be determined.

Japanese Patent Laid-Open No. 2002-203242 JP 2010-86276 A

  By the way, multiple images obtained by photographing the same area at different times are compared, and even if a changed area is detected in the image, the location detected from the image and the change of the house or illegal dumping of waste There may be cases where it does not correspond to a place where various land uses have changed.

  This is because the state of the plant greatly changes depending on the season, the progress of farming, and the like, and even the images of the same region have different characteristics such as color depending on the time of shooting. For example, an image taken in the summer of a forest or field where deciduous trees are growing, and an image taken in the winter, the plant that covers the area to be photographed, i.e., there is no change in vegetation, Features including colors of regions corresponding to forests and fields in each image are greatly different. For this reason, when the two images described above are compared, the area corresponding to the forest or the field in the image is the difference between the images even though there is no change in the vegetation of the area reflected in the area. May be detected.

  In addition, in the conventional technique for determining the type of plant to be photographed based on the image, the type of plant that is the subject based on the image photographed when the plant to be photographed does not exhibit typical characteristics. Is difficult to determine.

  As described above, in the related art, it may not be possible to determine whether or not the vegetation in the shooting range has changed depending on the time when each image to be compared was shot.

  In one aspect, an object of the present invention is to provide a vegetation determination program, a vegetation determination method, and a vegetation determination apparatus that suppress the influence of an image capturing time.

  A vegetation determination program according to one aspect calculates an image feature amount of at least the predetermined region for each of the first image and the second image obtained by photographing the predetermined region at different times, A database including information indicating image feature amounts that differ according to time in association with types is referred to based on the time of shooting, and the types of plants included in the predetermined area of the first image and the database When the type of plant included in the predetermined area of the second image is specified, and the type of plant specified for the first image matches the type of plant specified for the second image, It is determined that the plant type in the predetermined region has not changed between the different times, and the plant type specified for the first image is identical to the plant type specified for the second image. If not, the type of plants of the predetermined region is to execute the process of determining that varied between the different times on a computer.

  Further, the vegetation determination method according to another aspect calculates at least the image feature amount of the predetermined region for the first image and the second image obtained by photographing the predetermined region at different times, Types of plants included in the predetermined region of the first image with reference to a database including information indicating image feature quantities that differ according to the time in association with the types of plants based on the time of shooting. The type of plant included in the predetermined area of the second image and the type of plant specified for the first image matches the type of plant specified for the second image In addition, it is determined that the plant type of the predetermined region has not changed between the different times, the plant type specified for the first image, and the plant type specified for the second image Matches If no, the type of plants of the predetermined area is determined to have changed between the different times.

  In another aspect, the vegetation determination device calculates at least the image feature amount of the predetermined region for each of the first image and the second image obtained by photographing the predetermined region at different times. A database containing information indicating image features that differ depending on the time in association with the type of plant and the plant is referred to based on the time taken and included in the predetermined region of the first image A specifying unit for specifying a plant type and a plant type included in the predetermined area of the second image, a plant type specified for the first image, and a plant specified for the second image. When the type matches, it is determined that the plant type in the predetermined region has not changed between the different times, and the plant type specified for the first image and the second image Identify If the the type of plant does not match, the type of plants of the predetermined region has a determining section for determining a varied between the different times.

  According to the vegetation determination program, the vegetation determination method, and the vegetation determination apparatus of the present disclosure, it is possible to suppress the influence of the image capturing time.

It is a figure which shows one Embodiment of a vegetation determination apparatus. It is a figure which shows the example of a plant state database. It is a figure (the 1) which shows the example of the flowchart of the process which determines the presence or absence of the change of vegetation. It is FIG. (2) which shows the example of the flowchart of the process which determines the presence or absence of the change of vegetation. It is a figure which shows the hardware structural example of the difference detection apparatus to which the vegetation determination method is applied. It is a figure which shows the example of the flowchart of a difference detection process. It is a figure which shows the example of the flowchart of the process which discriminates vegetation. It is a figure which shows the example of the flowchart of the process which estimates a feature-value. It is a figure which shows the example of the flowchart of the process which evaluates similarity. It is a figure which shows another hardware structural example of the difference detection apparatus to which the vegetation determination method is applied. It is a figure which shows the example of a map information database. It is a figure which shows another example of the flowchart of a difference detection process. It is a figure which shows another example of the flowchart of the process which determines the change of vegetation using map information.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 shows an embodiment of a vegetation determination apparatus 10. The vegetation determination device 10 receives two images A and B obtained by photographing a predetermined area at different times, determines vegetation at each point included in the above-described area based on each image, and It is determined whether or not the vegetation has changed between the image A and B image capturing periods.

  The vegetation determination apparatus 10 illustrated in FIG. 1 includes a determination unit 11, an estimation unit 12, an evaluation unit 13, and a determination unit 14. The discriminating unit 11 receives the above-described images A and B, and the estimating unit 12 receives plant information indicating the feature amount of the image obtained by photographing each plant for each period from the plant state database DB.

  The discriminating unit 11 discriminates the types of plants photographed in the respective sections based on the feature amounts of the respective sections obtained by classifying the images A and B. Each section may be, for example, each pixel included in each of the images A and B, or may be a block in which a plurality of pixels are collected. For example, the determination unit 11 may determine the type of plant photographed in each section based on information indicating a color that is one of the feature amounts of each section. Here, each section of the images A and B corresponds to each point included in the shooting range, and the plant photographed in the section is a plant covering the point corresponding to the section. Therefore, the kind of plant which the discrimination | determination part 11 discriminate | determined about each division has shown the vegetation of the point corresponding to each division. In the following description, “determining the type of plant photographed in each section” included in the image is referred to as “determining vegetation at a point corresponding to each section”.

  Note that the images A and B described above are images that have been aligned, and the corresponding sections of the images A and B correspond to the same point included in the region described above.

  Here, consider a case where the images A and B are images obtained by shooting the shooting range including the area where the deciduous broad-leaved tree grows in May and December, respectively.

  In this case, the feature amount such as the color of the section included in the region where the above-described region is captured in the image A taken in May indicates a typical feature of the plant that is the subject. Therefore, the determination unit 11 can determine the type of the plant that is the subject based on the feature amount of the section.

  On the other hand, the feature amount of the corresponding section of the image B photographed in December indicates the characteristics such as the color of the trunk and branches of the tree after the fallen leaves. For example, the color of the trunk, branch, etc. of each tree species after falling leaves is not as great as the difference in the leaf color of each tree species. Therefore, it is difficult for the above-described determination unit 11 to determine the type of plant that is the subject for each section of the region where the deciduous broad-leaved forest is shown in the image B.

  Therefore, the discriminating unit 11 can discriminate a plant for the image A but discriminates a plant for the image B with respect to each section included in an area obtained by photographing the deciduous broad-leaved forest that is commonly included in the images A and B. The result of determination that it cannot be obtained. In this case, the image A including the section where the plant type can be determined is an example of the second image, and the image B including the section where the plant type cannot be determined is an example of the first image. .

  Based on the plant information received from the plant state database DB, the estimating unit 12 determines that the type of plant determined for the section included in the above-described region of the image A corresponding to the second image corresponds to the first image. The feature amount of the image indicating the state presented at the shooting time of B is estimated. The feature amount estimated by the estimation unit 12 is, for example, a spectral characteristic indicating the color that the plant specified based on the image A exhibits at the time when the image B is captured. For example, the estimation unit 12 may acquire plant information by searching a plant state database DB as illustrated in FIG. 2 based on the determination result obtained for the image A.

  FIG. 2 shows an example of a plant state database. The plant state database shown in FIG. 2 holds multispectral data representing spectral characteristics indicating colors, which are examples of image feature amounts obtained by photographing the plant at each time for each type of plant. Yes.

  The plant state database DB shown in FIG. 2 holds multispectral data indicating the color of an image obtained when each plant is photographed corresponding to each month included in one year. The plant state database is not limited to the example shown in FIG. 2, but for example, multispectral data indicating the color of an image obtained when each plant is photographed corresponding to each week or each day included in one year. You may do.

  The multispectral data shown in the example of FIG. 2 includes spectral values representing the red band, green band and blue band included in the visible light region, and spectral values representing the near infrared band included in the infrared region. Is included. In FIG. 2, the red band, the green band, the blue band, and the near-infrared band are indicated by the symbols “red”, “green”, “blue”, and “near-infrared”, respectively. Each multispectral data stored in the plant state database DB is not limited to the example of FIG. 2, and may include spectrum values in wavelength bands different from the above-described four wavelength bands, or spectra in different numbers of wavelength bands. It may contain a value.

  The plant state database DB shown in FIG. 2 is obtained when multi-spectrum data obtained when rice is photographed in April and when rice is photographed in December, corresponding to rice that is an example of plant types. It holds information including multispectral data. In the example of FIG. 2, multispectral data obtained when rice is photographed in April are denoted by reference characters R_r4, G_r4, B_r4, and IR_r4. In addition, multispectral data obtained when rice is photographed in December are indicated by symbols R_r12, G_r12, B_r12, and IR_r12.

  Similarly, the plant state database DB shown in FIG. 2 corresponds to a maple which is another example of the kind of plant, multispectral data obtained when a maple is photographed in April, and a maple is photographed in December. It holds information including multispectral data obtained in some cases. In the example of FIG. 2, multispectral data obtained when maple is photographed in April are indicated by reference symbols R_m4, G_m4, B_m4, and IR_m4. Further, multispectral data obtained when a maple is photographed in December are denoted by reference symbols R_m12, G_m12, B_m12, and IR_m12.

  In addition, the plant state database DB shown in FIG. 2 corresponds to the cherry tree which is another example of the plant type, and the multispectral data obtained when the cherry tree is photographed in April and the cherry tree is photographed in December. Holds information including multispectral data obtained. In the example of FIG. 2, multispectral data obtained when a cherry tree is photographed in April are indicated by reference symbols R_c4, G_c4, B_c4, and IR_c4. Further, multispectral data obtained when a cherry tree is photographed in December are denoted by reference symbols R_c12, G_c12, B_c12, and IR_c12.

  For example, when a determination result that the subject plant is a maple is obtained, the estimation unit 12 illustrated in FIG. 1 is the shooting time of the image B from the information corresponding to the maple in the plant database DB described above. Multispectral data corresponding to December can be acquired.

  The estimation unit 12 passes the multispectral data obtained as described above to the evaluation unit 13 as an estimated feature amount. Note that the estimation unit 12 can identify the plant for the image B, but identifies the section for which the determination result indicating that the plant cannot be identified for the image A is obtained based on the identified plant and the shooting time of the image A. Multispectral data may be passed to the evaluation unit 13 as an estimated feature amount. Moreover, the estimation part 12 constructs | assembles the model showing the change of the state of the said plant corresponding to a season or progress of farm work about each plant instead of using plant state database DB, Based on the constructed model, desired You may estimate the feature-value which shows the state of a plant in time.

The evaluation unit 13 can identify the plant for the image A, but includes the feature amount obtained by the estimation unit 12 and the image B for each section where the determination result that the plant cannot be identified for the image B is obtained. The similarity with the feature amount of the image of the section is evaluated. For example, the evaluation unit 13 receives, as the feature amount of the section of the image B, for example, multispectral data indicating the color of the section from the determination unit 11, and the received multispectral data and the multispectrum obtained by the estimation unit 12. The degree of similarity with data may be calculated. Similarly, the evaluation unit 13 can discriminate the plant for the image B, but the feature amount obtained by the estimation unit 12 and the image for each section where the discrimination result that the plant cannot be discriminated for the image A is obtained. The similarity with the feature amount of the image of the section included in B is evaluated. For example, the evaluation unit 13 calculates the Euclidean distance D between the two multispectral data using the formula (1), and the obtained Euclidean distance D is similar indicating that the smaller the value, the higher the similarity. You may pass to the judgment part 14 as a degree.
D = ((Br1-Br2) 2 + (Bg1-Bg2) 2 + (Bb1-Bb2) 2 + (Bir1-Bir2) 2) 1/2 ··· (1)
In Equation (1), the multispectral data estimated by the estimation unit 12 is indicated by symbols Br1, Bg1, Bb1, and Bir1, and the multispectral data corresponding to the section included in the image in which vegetation is not determined is denoted by symbols Br2, Indicated by Bg2, Bb2, and Bir2. In the following description, the Euclidean distance between multispectrals is simply referred to as the distance between multispectral data.

  The determination unit 14 may have a similarity between the feature amount obtained by the estimation unit 12 and the feature amount of a section included in an image in which vegetation is not yet determined, based on the similarity obtained by the evaluation unit 13. If indicated, it is determined that the vegetation at the point corresponding to the section has not changed during the photographing times of the images A and B. For example, when it is evaluated that the multispectral data indicating the color exhibited by the maple in December obtained by the estimation unit 12 and the multispectral data of the section of the image B are similar, the determination unit 14 sets the section to the section. Judge that there is no change in the vegetation at the corresponding point. In other words, the determination unit 14 determines that the difference in the feature amount of the sections included in the images A and B corresponds to the change that the plant obtained as a result of the vegetation determination exhibits during the shooting time of the images A and B. Judge that there is no change in vegetation.

  On the other hand, for the section indicated that there is no similarity between the feature amount obtained by the estimation unit 12 and the feature amount of the section included in the image whose vegetation is not yet determined, , B, it is determined that the vegetation of the section has changed during the photographing period. In other words, the determination unit 14 determines that the difference in the feature amount of the sections included in the images A and B is different from the change that the plant obtained as a result of the vegetation determination exhibits during the shooting time of the images A and B. Judge that there is a change in vegetation.

  In this way, the determination unit 14 also determines whether the difference in the feature amount of the section reflects the change in the state of the plant determined in the other image for the section where vegetation has not been determined in one image. Whether or not there is a change in the vegetation at the corresponding point can be determined.

  When it is determined that the vegetation of the section has not changed during the shooting times of the images A and B, the determination unit 14 determines that the vegetation at the point corresponding to the section included in each of the images A and B is It can be determined that they are the same. That is, the discrimination result of the plant type obtained by the discrimination unit 11 for one image may be passed to the display device 1 as the discrimination result of the section of the image that has not been discriminated.

  In addition, the determination unit 14 receives information indicating the types of plants determined for the sections included in the images A and B from the determination unit 11 and collates the received information, so that the shooting times of the images A and B are obtained. It may be determined whether the vegetation at the point corresponding to the section has changed during the period.

  Thus, according to the vegetation determination apparatus 10 of the present disclosure, the change in vegetation between the image A and B shooting times for the sections in which the determination unit 11 has determined the vegetation for at least one of the images A and B. It can be determined whether or not there has been. Thus, for example, even if a plant photographed in one of the two images does not necessarily exhibit a typical state like a deciduous broad-leaved forest in winter, whether or not there is a change in vegetation in the corresponding region It becomes possible to determine. That is, according to the vegetation determination apparatus 10 of the present disclosure, even when one of the two images to be processed is photographed at a time when a plant included in the photographing range does not exhibit a typical state, It can be determined whether or not there has been a change in vegetation during the time when the images of the images were taken.

  That is, according to the vegetation determination device 10 of the present disclosure, it is possible to determine whether or not there is a change in vegetation in the image capturing range regardless of the time when each of the two images is captured. In addition, two images selected in various combinations from a plurality of images are input to the vegetation determination device 10 of the present disclosure, and vegetation in the image capturing range is captured during the imaging time corresponding to each of the two input images. It is also possible to determine the presence or absence of changes.

  In addition, the process which determines the presence or absence of the change of the vegetation by the vegetation determination apparatus 10 shown in FIG. 1 can be represented by the flowchart shown in FIG. 3 and FIG.

  3 and 4 show examples of flowcharts of processing for determining the presence or absence of vegetation change. The flowchart shown in FIG. 3 and the flowchart shown in FIG. 4 are connected by connectors having reference numerals 1, 2, and 3. Moreover, the vegetation determination apparatus 10 shown in FIG. 1 performs the processing of steps 311 to 317 shown in FIG. 3 and the processing of steps S11 to S15 shown in FIG. 4 for each section included in the images A and B, respectively. Run.

  First, the vegetation determination apparatus 10 determines vegetation at a point corresponding to a processing target section included in each of the images A and B (step 311).

  Next, the vegetation determination apparatus 10 determines whether or not the vegetation determination has succeeded in one of the images A and B and the determination has failed in the other in the process of step 311 (step 312).

  For example, when the vegetation determination succeeds for the section included in the image A and the vegetation determination fails for the section included in the image B, the vegetation determination apparatus 10 performs the step according to the affirmative determination route (YES) of step 312. The process proceeds to 313. On the other hand, when the vegetation determination is successful for both of the sections included in the images A and B, and when both are unsuccessful, the vegetation determination apparatus 10 uses a negative determination route (NO) in step 312 as described later. Execute the process. In the following description, an image in which vegetation at a corresponding point can be determined in the processing of step 311 is referred to as a determined image, and an image that cannot be determined is referred to as an unidentified image.

  In step 313, the vegetation determination apparatus 10 estimates a feature amount indicating a state that the plant obtained by the successful determination process exhibits at the photographing time of the unidentified image, based on the plant information obtained from the plant state database DB. . The vegetation determination apparatus 10 uses the multispectral data corresponding to the shooting time of the undiscriminated image as the estimated feature amount from the information accumulated in the plant state database DB corresponding to the plant indicated by the successful discrimination processing. You may get it. For example, in the section to be processed, when the image A is a determined image and the image B is an unidentified image, the vegetation determination device 10 uses the image B of the information corresponding to the plant obtained for the image A. Multispectral data corresponding to the shooting time is acquired as an estimated feature amount. On the contrary, in the section to be processed, when the image B is a determined image and the image A is an unidentified image, the vegetation determination device 10 selects the image among the information corresponding to the plant obtained for the image B. Multispectral data corresponding to the shooting time of A is acquired as an estimated feature amount.

  Next, the vegetation determination apparatus 10 evaluates the similarity between the feature amount estimated in the processing of step 313 and the feature amount of the processing target section included in the unidentified image (step 314). In the process of step 314, the vegetation determination apparatus 10 may calculate the distance between the multispectral data indicating the two feature amounts, for example, as an index indicating the similarity between the two feature amounts described above.

  Based on the comparison between the similarity obtained in the process of step 314 and a predetermined threshold, the vegetation determination apparatus 10 determines whether the above-described two feature amounts are similar (step 315). The vegetation determination device 10 determines, for the image A, the feature amount of the section of the image B where vegetation is unspecified based on, for example, whether the distance between the multispectral data is equal to or less than a predetermined threshold. It is possible to evaluate whether or not the feature amount estimated for the plant is similar.

  Note that the threshold value used in the process of step 315 is smaller than the distance between the multispectral data of different types of plants, and more than the value corresponding to the fluctuation of the multispectral data indicating the state that each plant exhibits in each period. Is preferably set to a large value. When the vegetation determination apparatus 10 performs the processing of step 315 using such a threshold value, it is assumed that the range of variation in spectral characteristics due to weather conditions at the time of capturing an image or individual plant individual differences is similar. It can be included in the allowable range when judging.

  When it is determined that the above-described two feature quantities are similar (affirmative determination in step 315), the vegetation determination apparatus 10 indicates that there is no change in vegetation between the shooting times of the images A and B at the point corresponding to the section. Is output (step 316).

  On the other hand, when it is determined that the above-described two feature quantities are not similar (negative determination in step 315), the vegetation determination apparatus 10 changes the vegetation between the shooting times of the images A and B at the point corresponding to the section. A determination result indicating that there is a problem is output (step 317).

  The process in step 311 described above is an example of a process for determining the type of plant photographed in each section based on the feature amount of the image in each section.

  Further, the processing in step 312 described above is an example of processing for distinguishing between cases where vegetation can be determined for a section included in one of the images A and B and a section included in the other cannot be determined, and other cases.

  Further, the process in step 313 is an example of a process for estimating the feature amount of an image indicating a state in which a plant discriminated based on one image exhibits at the shooting time of the other image.

  Similarly, the process in step 314 is an example of a process for evaluating the similarity between the estimated feature quantity and the feature quantity of the section to be processed in the other image.

  Moreover, the process of step 315-step 317 is a process of judging whether there has been a change in the vegetation at the point corresponding to the section based on the similarity evaluation result for the case of distinction as described above. It is an example.

  That is, the vegetation determination apparatus 10 executing the processes of Step 311 to Step 317 described above is an example of an embodiment of the vegetation determination program and the vegetation determination method of the present disclosure. According to the vegetation determination program and the vegetation determination method of the present disclosure, even if a plant cannot be identified on one of the images, the difference in the feature amount of the processing target section included in each image is caused by the difference in the shooting time. When the change is reflected, it can be determined that there is no change in vegetation. Thus, according to the vegetation determination program and the vegetation determination method of the present disclosure, it is possible to determine whether or not there is a change in vegetation in the image capturing range regardless of the time when each of the two images is captured.

  Moreover, in the case of the negative determination of step 312 mentioned above, the vegetation determination apparatus 10 performs the process of step S11-step S15 shown in FIG. 4, and the point of the point corresponding to the process target division in the images A and B is shown. You may judge when both vegetation discrimination succeeds or fails.

  First, the vegetation determination apparatus 10 determines whether vegetation has been successfully determined for both of the processing target sections included in the images A and B (step S11).

  When the vegetation determination is successful for both (affirmative determination in step S11), the vegetation determination apparatus 10 collates the types of plants obtained as the determination results for the processing target sections included in the images A and B, respectively. (Step S12). Then, based on the collation result obtained in step 312, it is determined whether or not the discrimination results for the processing target sections included in the images A and B match (step S 13).

  When the types of plants obtained for the sections to be processed included in the images A and B match (affirmative determination in step S13), the vegetation determination device 10 goes to step 316 in FIG. Proceed and output the judgment result that there is no change in vegetation.

  For example, when a determination result is obtained for the processing target sections of the images A and B taken in May and November, respectively, where the vegetation at the point corresponding to the section is a maple (affirmative determination in step S13). The vegetation determination apparatus 10 determines that there is no change in vegetation in the section to be processed.

  Here, the maple in May is in a fresh green state, whereas the maple in November is likely to be in a state of autumn leaves. For this reason, a clear difference appears in the multispectral data itself, which is the feature quantity of the section to be processed, included in the images reflecting the maple states in May and November. However, if the section in each image represents a typical maple state, information indicating the same plant can be obtained as a discrimination result in the process of step 311 described above. As described above, according to the vegetation determination apparatus 10, when a specific result indicating the same plant is obtained for the processing target sections included in the two images, the corresponding section is supported regardless of the difference in the images themselves. It can be judged that there is no change in the vegetation at the point where it is done.

  On the other hand, when the types of plants obtained for the sections to be processed included in the images A and B do not match (negative determination in step S12), the vegetation determination device 10 performs the step of FIG. Proceeding to 317, a determination result indicating that there is a change in vegetation is output.

  By the way, in the case of the negative determination of step S11 mentioned above, the vegetation determination apparatus 10 determines that the determination of vegetation has failed for both. In this case, the vegetation determination apparatus 10 may output a determination result indicating that the determination cannot be made, or by executing the processes of steps S14 and S15 described below, the vegetation determination apparatus 10 can perform vegetation at a point corresponding to the processing target section. It may be determined whether there is a change.

  The vegetation determination apparatus 10 calculates a difference d between the feature amounts of the processing target sections included in the images A and B (step S14), and determines whether the obtained difference d is less than a predetermined threshold Th ( Step S15). In addition, as for threshold value Th, it is desirable to set a value larger than the dispersion | variation in the value obtained as a difference of the feature-value of each division, when there is no change in vegetation. Further, for example, a value similar to the average value of the difference between the feature amounts corresponding to each plant at the time of photographing each of the images A and B may be set. In addition, the minimum value of the difference in the feature amount obtained for the section where the vegetation is changed between the images A and B, or the same value as the maximum value of the feature amount obtained for the section where there is no change. The threshold value Th may be set.

  When the difference d obtained in the process of step S14 is smaller than the threshold value Th described above (positive determination in step S15), the vegetation determination apparatus 10 proceeds to step 316 in FIG. The judgment result that there is no change is output.

  On the other hand, when the difference d obtained in the process of step S14 is greater than or equal to the above-described threshold Th (negative determination of step S15), the vegetation determination apparatus 10 proceeds to step 317 of FIG. The judgment result that there is a change in vegetation is output.

  Here, if the shooting time of at least one of the images A and B shown in FIG. 1 is a time when leaves of trees are growing, at least the image corresponds to the section to be processed in the processing of step 311 described above. Information indicating the vegetation at the point to be obtained is obtained. Therefore, the processes of step S14 and step S15 described above are executed only in very limited cases, such as when both images A and B are taken in winter.

  The vegetation determination apparatus 10, the vegetation determination program, and the vegetation determination method described above are applied to a technique for detecting a difference indicating a location where the land use has changed from two images obtained by photographing a predetermined area at different times. You can also.

  Hereinafter, another embodiment of the vegetation determination program and the vegetation determination method of the present disclosure will be described together with a difference detection device that performs difference detection to which the vegetation determination program and the vegetation determination method of the present disclosure are applied.

  For example, as shown in FIG. 5, the difference detection device 16 that performs the difference detection to which the vegetation determination method of the present disclosure is applied includes a computer system including a client device such as a personal computer and a server device arranged in a network. Can be realized.

  FIG. 5 shows an example of the hardware configuration of the difference detection device 16. The example of FIG. 5 shows an example in which the difference detection device 16 is realized by a computer system in which the server device 20 and the client device 30 are connected by a network NW.

The server device 20 illustrated in FIG. 5 includes a processor 21, a memory 22, a hard disk device 23, and a network interface (I / F: InterFace) 24. The processor 21, the memory 22, the hard disk device 23, and the network interface 24 are connected to each other via a bus, and the server device 20 is connected to the network NW via the network interface 24. The above-described processor 21, memory 22, hard disk device 23, and network interface 24 are included in the difference detection device 16.

  The client device 30 shown in FIG. 5 includes a processor 31, a memory 32, a hard disk device 33, a network interface (I / F) 34, a display unit 35, and a general-purpose interface 36. The processor 31, the memory 32, the hard disk device 33, the network interface 34, the display unit 35, and the general-purpose interface 36 are connected to each other via a bus. The above-described processor 31, memory 32, hard disk device 33, network interface (I / F) 34, and general-purpose interface 36 are included in the difference detection device 16.

  The client device 30 is connected to the network NW via the network interface 34, and is further connected to the imaging device 2 via the general-purpose interface 36. The client device 30 receives an image captured by the imaging device 2 via the general-purpose interface 36, and further receives the received image to the difference detection device 10 in the server device 20 via the network NW in FIG. The images A and B shown can be passed. That is, the image obtained by the imaging device 2 is an example of the images A and B shown in FIG.

  Note that the imaging device 2 is a multispectral sensor having sensitivity to light in a plurality of wavelength bands respectively included in a red band, a green band, and a blue band included in the visible light region, and a near infrared band in the infrared light region. It is desirable to contain. Below, the case where the pixel value of each pixel included in the image obtained by the imaging device 2 is multispectral data including spectral values corresponding to the plurality of wavelength bands described above will be described.

  The memory 32 of the client device 30 stores an application program for the processor 31 to execute processing for exchanging information related to the difference detection processing including the vegetation determination processing described above with the server device 20 together with the operating system. Yes. The application program described above includes an application program for causing the processor 321 to execute processing for acquiring an image from the imaging device 2 via the general-purpose interface 36 and passing the acquired image to the server device 20 via the network NW. The application program described above includes an application program for causing the processor 321 to execute a process of receiving a detection result obtained by the server apparatus 20 executing a process described later via the network NW. The application program described above includes an application program for causing the processor 321 to execute a process of presenting the processing result acquired from the server device 20 to the user via the display unit 35.

  The memory 22 illustrated in FIG. 5 stores an application program for the processor 21 to execute the difference detection process including the vegetation determination process described above together with the operating system of the server device 20. The application program for executing the difference detection process including the vegetation determination process can be recorded and distributed on a removable disk such as an optical disk, for example. Then, by loading this removable disk into an optical drive device (not shown) of the server device 20 and performing a read process, an application program for executing the above-described difference detection process is stored in the memory 22 and the hard disk device 23. Also good. It is also possible to receive an application program for executing the above-described difference detection process from another server device or the like via the network interface 24 and read it into the memory 22 and the hard disk device 23. Moreover, it is desirable to provide the application program for executing the above-described difference detection process together with information for constructing the plant state database DB as shown in FIG.

  The processor 21 may perform the functions of the determination unit 11, the estimation unit 12, the evaluation unit 13, and the determination unit 14 illustrated in FIG. 1 by executing an application program stored in the memory 22. Further, when realizing the functions of the determination unit 11 and the estimation unit 12, the processor 21 may refer to information stored in the plant state database DB constructed in the hard disk device 23 or the like.

  As described above, the vegetation determination device 10 of the present disclosure can be realized by, for example, cooperation of the processor 21, the memory 22, and the hard disk device 23 included in the server device 20 described above.

  FIG. 6 shows an example of a flowchart of difference detection processing executed by the server device 20 shown in FIG. Each process of step 301 to step 309 illustrated in FIG. 6 is an example of a process included in the application program for the difference detection process. In addition, each processing of step 301 to step 309 is executed by the processor 21 of the server device 20.

  First, the processor 21 aligns the two images acquired from the client device 30 illustrated in FIG. 5, and further detects a difference in an area in the image corresponding to the shooting range common to the two images. The target area is specified (step 302).

  Here, the client device 30 stores a plurality of images acquired from the imaging device 2 in the hard disk device 33 in advance, and sends two images from the stored images to the server device 20 via the network NW. May be. When the client device 30 sends two images, it is desirable to send information useful for alignment of these images together with information indicating the shooting date and time of each image. For example, when the two images are both images taken from an aircraft or the like, useful information for alignment includes information indicating the latitude and longitude of the shooting location, the shooting magnification, and the altitude of the aircraft at the time of shooting. It is desirable to include information indicating such as. Note that the alignment of the two images may be performed using an image recognition technique, and in that case, the addition of information indicating the shooting magnification or the altitude of the aircraft may be omitted.

  Further, for example, based on the information received together with the two images, the processor 21 extracts, from each image, an area corresponding to a shooting range common to the two images, and the extracted area is a difference detection target area. It is good. In the process of step 301, the processor 21 is configured so that each pixel included in the difference detection target area specified for each of the two images corresponds to a range of the same size on the ground surface of the shooting range. It is desirable to adjust the size of the image.

  Next, the processor 21 selects one of the sections included in the difference detection target area specified in each image as a target for subsequent processing (step 302). For example, the processor 21 may sequentially select one of the pixels included in the difference detection target area of each image as a processing target section, or select a block including a plurality of pixels as one section. May be.

For the section selected in the process of step 302, the processor 21 first determines whether or not the ground surface range corresponding to the section is covered with a plant based on the feature amount of the section (step 303). . When each section has one pixel, the processor 21 acquires the feature amount of the section to be processed as multispectral data that is an observation value by the multispectral sensor corresponding to one pixel. On the other hand, when each section is a block including a plurality of pixels, the processor 21 obtains the feature value of each section by obtaining the average value of the multispectral data corresponding to the plurality of pixels included in the section. Good. The processor 21 calculates a vegetation index (NDVI: Normalized Difference Vegetation Index) based on, for example, multispectral data obtained as pixel values of pixels to be processed selected from two images. For example, the processor 21 may calculate the vegetation index NDVI using the equation (2) from the near-infrared spectrum value Bir and the red-band spectrum value Br included in the multispectral data.
NDVI = (Bir−Br) / (Bir + Br) (2)
Further, when the obtained vegetation index is equal to or greater than a predetermined value, the processor 21 determines that the ground surface range corresponding to the section is a vegetation region covered with plants. In addition, the predetermined value compared with a vegetation index can set the value about the minimum value of the vegetation index obtained about the image which image | photographed the vegetation area | region, for example.

  Next, the processor 21 determines whether or not the point corresponding to the section is based on whether or not the processing result in step 303 indicates that the processing target section selected from the two images is not a vegetation region. It is determined whether or not there is a possibility of being a vegetation region (step 304).

  In the case of an affirmative determination (YES) in step 304, the processor 21 executes a process for determining a change in vegetation as shown in FIGS. 3 and 4 (step 305), and based on the obtained determination result. Thus, a difference detection result is generated (step 306). When the processor 21 executes the process of step 311 and the process of step 313 shown in FIG. 3 in the course of the process of step 305, it accumulates in the plant state database DB provided in the hard disk device 23 shown in FIG. Information can be referred to. A more preferable example of the processing in step 311 and step 313 will be described later.

  When the processor 21 obtains a determination result indicating that there is no change in vegetation between the shooting times of the two images by executing the processing of step 305, the processing target section is set between the two images. A detection result that is not included in the difference is generated. On the other hand, if it is determined in step 305 that there is a change in vegetation between the shooting times of the two images, the processor 21 determines that the section to be processed is included in the difference between the two images. A detection result is generated.

  As described above, when the processor 21 executes the processing of step 305 and step 306, when there is a change in the vegetation at the corresponding point, the section is detected as a difference between the two images, and there is no change. In some cases can be excluded from the difference.

  As a result, when the change between two images with different shooting times is a change in the state of the plant in the section to be processed, the section is not erroneously detected as a difference between the images. Can do. That is, for each section, the processor 21 executes a process according to the vegetation determination program of the present disclosure, so that a section whose image has changed due to a change in the state of the plant is erroneously detected as a difference between images. False detection can be reduced.

  If the change between two images with different shooting times does not match the change in the state of the plant shown in the processing target section, it is determined that the vegetation at the point corresponding to the section has changed, and the image It can be detected as a difference between them. As a result, for example, as in the example described below, it is possible to detect changes in land use status that may be overlooked in the conventional method of detecting a difference by comparing two images. .

  For example, let us consider a case where a determination result that the vegetation is rice is obtained for the processing target section of the image photographed in May and the vegetation for the processing target section of the image photographed in November cannot be determined. In this case, if weeds that have not withered appear in the processing target section of the image taken in November, the multispectral data that is the feature amount of the processing target section included in the two images described above. There may be no obvious difference in itself. However, in the managed field, the rice in May is in a state of vigorous growth after planting, whereas the rice in November is in a state after cutting. As described above, if the information indicating the state of each rice in the managed field is reflected in the plant state database DB illustrated in FIG. 5, the processor 21 determines the processing target sections of the two images. It can be detected as a change in vegetation that there is little change in the feature amount. Thereby, since the vegetation at the point corresponding to the section has changed regardless of the similarity of the image itself, the section can be detected as a difference between the two images.

  On the other hand, in the case of a negative determination (NO) in step 304, the processor 21 compares the feature quantities of the processing target sections selected from the two images, and based on the comparison result, the processor 21 determines that the section is two sheets. It is determined whether or not to detect as a difference between images (step 307). For example, the processor 21 calculates the distance between the multispectral data corresponding to these sections as the difference between the feature quantities of the sections to be processed included in the two images, and the obtained distance is equal to or greater than a predetermined threshold value. If it is, the section is detected as a difference between the two images. On the other hand, when the obtained distance is less than the predetermined threshold, the processor 21 determines that the section is not included in the difference between the two images.

  Here, the threshold value to be compared with the distance between the multispectral data in the process of step 307 is set to a value larger than the fluctuation of the multispectral data obtained when the land use status is not changed, for example. It is desirable.

  The affirmative determination route processing and the negative determination route processing of step 304 described above merge at step 308. In step 308, the processor 21 generates image information for presenting the difference detection result obtained in step 306 or step 307 to the user corresponding to the processing target section. The processor 21 may return the generated image to the client device 30 via the network interface 24 illustrated in FIG. For example, when the detection result obtained in step 306 or step 307 indicates that the section is a difference between two images, the processor 21 changes the display color of the section. Image information to be highlighted may be generated. The image information generated in this way can be returned to the client device 30 via the network interface 24 shown in FIG. 5 and presented to the user by being subjected to display processing by the display unit 35 of the client device 30. it can.

  Next, the processor 21 determines whether or not the processing of step 302 to step 308 has been completed for all the sections included in the difference detection target area specified in step 302 (step 309).

  When there is an unprocessed partition (negative determination at step 309), the processor 21 returns to the process of step 302, selects one of the unprocessed partitions as a processing target partition, and executes the processes after step 303. To do.

  In this way, the processing of Step 302 to Step 309 is repeatedly executed, and when the processing for all the sections included in the above-described region is completed, the processor 21 described above as an affirmative determination of Step 309. The difference detection process for the entire area is terminated.

  As described above, a difference between images indicating a change in land use status is detected for each section in a difference detection target area specified for two images, and each section detected as a difference is detected as a client device. 30 can be presented to the user.

  The difference between the images presented in this way does not include sections that are erroneously detected because the characteristics of the images have changed due to changes in the state of the plant. Therefore, it is very useful for detecting changes in land use status from images taken at different times of vegetation regions with large seasonal changes such as deciduous broad-leaved forests and farm fields.

  Here, in each section included in the image taken from the sky of the deciduous broad-leaved forest after defoliation and the field after harvest, multispectral data reflecting the characteristics of the chloroplasts of the plant cannot be obtained. Note that is not considered a vegetation area. For this reason, in the method of determining the presence or absence of vegetation based on the vegetation index, when one image is taken in winter, the part showing the deciduous broad-leaved forest is detected as a part that is no longer a vegetation area. End up. In contrast, the difference detection process illustrated in FIG. 6 applies the vegetation determination program and the vegetation determination method of the present disclosure to a section where vegetation is recognized in at least one of the images, so that the image of the section is deciduous. It is possible to realize difference detection in consideration of the possibility of corresponding to the maple.

  By the way, the process of step 311 shown in FIG. 3 can also be implement | achieved when the processor 21 performs the process of step 321-step 328 shown in FIG.

  FIG. 7 shows an example of a flowchart of processing for discriminating vegetation. Each process of step 321 to step 328 illustrated in FIG. 7 is an example of a process of determining the vegetation illustrated in step 311 in FIG. In addition, each processing of step 321 to step 328 is executed by the processor 21.

  First, the processor 21 selects one of the two images as a processing target (step 321), and corrects the shooting time extracted from the tag information of the selected image based on the local climate of the shooting range of the image. (Step 322).

  In step 322, for example, the processor 21 takes a picture according to whether the climate of the photographed area is cold or warm compared to the climate assumed in the plant state database DB shown in FIG. Perform the correction. For example, when the shooting time is included in a period from winter to spring, and the climate of the imaged region is cold, the processor 21 shifts the shooting time acquired from the image in the direction of returning to winter. You may correct to. Further, for example, when the shooting time is included in a period from winter to spring, and the climate of the imaged region is warm, the processor 21 conversely turns the shooting time acquired from the image toward spring. You may correct | amend so that it may shift to the advancing direction.

  Similarly, the shooting time used for the search from the plant state database DB may be corrected in consideration of variations in weather conditions such as the difference between the average temperature before and after the shooting time and the average temperature of the normal year.

  Next, the processor 21 searches the multi-spectral data stored corresponding to the corrected photographing time as the feature amount of each plant from the information registered corresponding to each plant in the plant state database DB. (Step 323).

  Next, the processor 21 calculates the similarity between the feature amount of the processing target section included in the processing target image and the feature amount of each plant obtained in step 321 described above (step 324). When the section to be processed is a single pixel, the processor 21 may perform the process of step 324 using the multispectral data corresponding to the pixel as the feature amount of the section to be processed. Further, when the processing target section includes a plurality of pixels, the processor 21 calculates an average value of the multispectral data corresponding to each pixel, and the obtained value indicates a feature amount corresponding to the section. Spectral data may be used. Further, the processor 21 increases the distance calculated between the multispectral data corresponding to each plant obtained as a result of the search processing in step 323 and the multispectral data corresponding to the processing target section as the value is smaller. You may use as a similarity which shows similarity.

  Next, based on the similarity obtained in step 324, the processor 21 determines whether or not there is a plant that exhibits a state indicated by a feature amount similar to the feature amount of the processing target section at the time of capturing the processing target image. (Step 325). For example, the processor 21 has a plant exhibiting a color similar to the color of the section to be processed based on the comparison result between the minimum value of the distance between the multispectral data calculated corresponding to each plant and a predetermined threshold value. It may be determined whether or not. Note that the threshold value used in the process of step 325 is desirably set to a value approximately equal to the value corresponding to the fluctuation of the multispectral data obtained for the same plant, for example.

  When a plant having a color similar to the color of the section to be processed can be found (affirmative determination in step 325), the processor 21 displays information indicating the type of the plant that is similar to the point corresponding to the section. The result is output as a vegetation discrimination result (step 326).

  On the other hand, when a plant having a color similar to the color of the section to be processed cannot be found (negative determination in step 325), the processor 21 determines the vegetation as a vegetation determination result at the point corresponding to the section. That it cannot be determined (step 327).

  As described above, after the execution of the processing in steps 321 to 327 is completed, the processor 21 determines whether or not the processing for both images has been completed (step 328).

  When there is an unprocessed image (negative determination at step 328), the processor 21 returns to step 321 and starts the processing from step 322 onward with the unprocessed image as a processing target. In this way, when the processing for both images is completed, the processor 21 ends the processing for determining vegetation, and proceeds to the processing of step 312 shown in FIG.

  As described above, by performing the search process using the photographing time corrected in step 322, the processor 21 can acquire a feature amount corresponding to the feature of the photographed region as the feature amount of each plant. . In addition, when correction is not performed by determining whether or not vegetation corresponding to the processing target section can be determined based on the similarity calculated from the feature amount obtained by the search processing using the corrected shooting time As a result, it is possible to obtain a highly accurate discrimination result.

  In addition, when determining the vegetation corresponding to each division by the process of the above-mentioned step 321-step 328, it corresponds to each plant in the plant state database DB, and the multi corresponding to each week or each day included in one year. It is desirable to store spectral data. If such a plant state database DB is prepared, in the processing of step 322 described above, the processor 21 can finely correct the shooting time according to the climate of the region shot in the processing target image. It is. In addition, there are some types of plants in which the state captured as multispectral data changes on a daily basis, such as a cherry blossom at the flowering time. For such plants, for example, if multispectral data is accumulated in the plant state database DB on a daily basis, the correct search results for each section are compared with the case where multispectral data averaged on a monthly basis is accumulated. Is more likely to get. Therefore, in the case of using the plant state database DB in which the weekly or daily feature amount is accumulated corresponding to each plant, compared to the case of using the plant state database DB in which the monthly feature amount is accumulated. The vegetation can be discriminated with higher accuracy.

  Next, with respect to step 313 and step 314 shown in FIG. 3, an example of processing suitable for a case where a plant state database DB in which weekly or daily feature values are accumulated is prepared corresponding to each plant. Will be explained.

  The processing in step 313 shown in FIG. 3 can also be realized by the processor 21 executing, for example, the processing in steps 331 to 333 shown in FIG.

  FIG. 8 shows an example of a flowchart of processing for estimating a feature amount. Each process of step 331 to step 333 shown in FIG. 8 corresponds to an example of a process for estimating the feature amount shown as step 313 in FIG. In addition, each processing of step 331 to step 333 is executed by the processor 21.

  First, the processor 21 obtains the vegetation discrimination result obtained for one image at the point corresponding to the section to be processed in the process of step 311 described above, and information indicating the photographing time of the other image Is acquired (step 331). For example, when the plant a is obtained as the vegetation of the point corresponding to the processing target section for the image A out of the two images A and B, and the vegetation cannot be determined for the image B, the processor 21 Information indicating a and information indicating the shooting time of the image B are acquired.

  Next, the processor 21 corrects the photographing time obtained in step 331 based on the local climate in the photographing range in the same manner as in step 322 in FIG. 7 (step 332). In the process of determining the vegetation shown in FIG. 7, the processor 21 stores the corrected shooting time obtained in the above-described step 322 for the two images corresponding to each image in the memory 22. It is possible to keep it. In this case, the processor 21 may omit the processing of steps 331 and 332 and instead read out the corrected photographing time held in the course of the processing for determining vegetation from the memory 22 or the like.

  Next, the processor 21 corresponds to each period within a predetermined length period including the corrected photographing period from the information accumulated in the plant state database DB corresponding to the type of plant obtained in step 331. The feature amount is searched as an estimated feature amount candidate (step 333). For example, the processor 21 estimates multispectral data corresponding to each period within a predetermined length period including the corrected photographing period from information accumulated in the plant state database DB corresponding to the plant a described above. Obtained as a candidate for the feature amount. Note that the length of the period including the corrected photographing time may be set in advance to, for example, about two weeks in consideration of variations in the growth state of individual plants. In addition, the processor 21 may cause variation in the state of the plant due to the difference between the weather condition at the photographing time of the image B in which vegetation is undetermined and the normal weather condition during the processing of step 333. In consideration of the above, the length of the period described above may be adjusted.

  According to the processing of Step 331 to Step 333 described above, a plurality of multispectral data reflecting the variation in the growth state of the plant at the time of photographing the image where the vegetation has not been determined is estimated together with the deviation due to the climate of the photographed region. Can be obtained as a candidate for a feature amount. That is, according to the processing of Step 331 to Step 333 described above, for example, the estimation result of the state of the plant a at the shooting time of the image B in which the vegetation is undetermined, the deviation due to the climate of the shot region and the change of the weather condition It is possible to provide a width that takes into account variations due to the above.

  As described above, when a plurality of pieces of multispectral data are acquired as estimated feature amount candidates, the processor 21 performs steps 341 to 346 shown in FIG. 9 as the processing of step 314 shown in FIG. It is desirable to execute processing.

  FIG. 9 shows an example of a flowchart of processing for evaluating similarity. Each process of step 341 to step 346 shown in FIG. 9 corresponds to an example of the process of step 314 shown in FIG. In addition, each processing of step 341 to step 346 is executed by the processor 21.

  First, the processor 21 acquires multispectral data corresponding to a processing target section from the image whose vegetation has not been determined as a feature amount of the section (step 341). For example, when the vegetation cannot be determined for the image B out of the two images A and B, the processor 21 uses the multispectrum of the section as information indicating the feature amount of the section to be processed included in the image B. Get the data.

  Next, the processor 21 calculates a distance between the feature quantity acquired in step 341 and one of the feature quantity candidates obtained in the process of estimating the feature quantity shown in FIG. 8 (step 342).

  Next, the processor 21 determines whether the estimated feature quantity candidate is similar to the feature quantity acquired in step 341 based on whether the distance calculated in step 342 is equal to or less than a predetermined threshold. Is determined (step 343). Here, the variation of the state of the plant in the imaging | photography time of the image in which the vegetation has not been discriminated is reflected in the feature quantity candidates obtained by the processes of Step 331 to Step 333 shown in FIG. Therefore, the processor 21 may execute the process of step 343 using, for example, a threshold value set to a value comparable to the value corresponding to the fluctuation of the multispectral data obtained for the same plant.

  When the distance calculated in step 342 is equal to or smaller than a predetermined threshold (affirmative determination in step 343), the processor 21 determines the estimated feature amount and the feature amount of the processing target section included in the image whose vegetation is not yet determined. An evaluation result indicating that and are similar is output (step 344). Thereafter, the processor 21 proceeds to the processing of step 315 shown in FIG.

  On the other hand, when the distance calculated in step 342 is larger than the predetermined threshold (negative determination in step 343), the processor 21 selects all the feature amount candidates obtained by the process of estimating the feature amount shown in FIG. It is determined whether or not the processing has been completed (step 345).

  When there is an unprocessed candidate among the estimated feature amount candidates (No determination in step 345), the processor 21 returns to the process of step 342, and the process after step 342 is performed for one of the unprocessed candidates. Execute.

  When the processes of Steps 342 to 345 described above are repeatedly executed and the process is completed for all of the feature quantity candidates (Yes determination in Step 345), the feature quantity acquired in Step 341 among the feature quantity candidates. Judge that there is nothing similar to. In this case, the processor 21 outputs an evaluation result indicating that the estimated feature quantity is not similar to the feature quantity of the processing target section included in the image whose vegetation has not been discriminated (step 346), and thereafter, FIG. The process proceeds to step 315 shown in FIG.

  When the processor 21 executes the processes of Steps 341 to 346 described above, the state in which the plant identified in one image is presented at the time of photographing the other image and the state of the plant photographed in the other image Similarity can be evaluated with high accuracy.

  In addition, the plant information database DB in which the feature amount in units of weeks or days is stored corresponding to each plant includes a huge amount of information. For this reason, as shown in FIG. 5, it is desirable to arrange the plant state database DB in the server device 20 or the like. In addition, it is desirable that the difference detection program and the service based on the difference detection method to which the vegetation determination program and the vegetation determination method of the present disclosure are applied be implemented as one of cloud services.

  Furthermore, information indicating the vegetation obtained for each section included in each image by the vegetation determination program and the vegetation determination method of the present disclosure can be accumulated as map information regarding a point corresponding to the section. Further, the map information accumulated in this way is transferred from the server device 20 shown in FIG. 5 to the client device 30, and the client device 30, for example, obtains the survey result obtained in the field survey and the received map information. It is also possible to link them. Also, the map information reflecting the above-mentioned field survey result is received from the client device 30 together with the image data representing the difference detection target image, and the difference detection device 16 provided in the server device 20 performs subsequent difference detection processing. You can also use the received map information.

  Hereinafter, a method of using map information for vegetation determination processing and difference detection processing will be described with reference to FIGS. 10 to 13.

  FIG. 10 shows another example of the hardware configuration of the difference detection device 16 to which the vegetation determination method of the present disclosure is applied. Note that, among the components shown in FIG. 10, the same components as those shown in FIG. 5 are denoted by the same reference numerals and description thereof is omitted.

  The computer device 40 shown in FIG. 10 includes a processor 41, a memory 42, a hard disk device 43, a general-purpose interface (I / F) 44, a display control unit 45, and an optical drive 46. The processor 41, the memory 42, the hard disk device 43, the general-purpose interface 24, the display control unit 45, and the optical drive device 46 are connected to one another via a bus. Further, the computer device 40 is connected to the imaging device 2 and the input device 3 via the general-purpose interface 24. The display control unit 45 is connected to the display device 1. The processor 41, the memory 42, and the hard disk device 43 described above are included in the vegetation determination device 10. The processor 41, the memory 42, the hard disk device 43, and the general-purpose interface 44 described above are included in the difference detection device 16. As described above, the difference detection device 16 including the vegetation determination device 10 of the present disclosure can also be realized by the stand-alone computer device 40.

  For example, the input device 3 has a function of inputting, to the computer device 40, information indicating vegetation and the like specified in the field survey for each point subjected to the field survey. The processor 41 of the computer device 40 stores the information input via the input device 3 in the map information database 232 provided in the hard disk device 43, for example, as shown in FIG. It may be accumulated corresponding to such as.

  FIG. 11 shows an example of the map information database 232. The map information database 232 illustrated in FIG. 11 stores the types of plants identified as vegetation, the method of identifying vegetation, and the specific date and time, corresponding to position information indicating the coordinates of each point.

  In the map information database 232 shown in FIG. 11, the symbol Cj indicates the type of plant at the point indicated by coordinates (xj, yj), and the symbol Ck is the plant at the point indicated by coordinates (xk, yk). Shows the kind of. The code Datej indicates the date when the vegetation at the point indicated by the coordinates (xj, yj) is specified. Similarly, the code Datek indicates the date when the vegetation at the point indicated by the coordinates (xk, yk) is specified. Is shown.

  In addition, when vegetation is specified by field survey, it is desirable that the map information database 232 stores the date and time when the field survey was performed as the date and time when vegetation was identified. For example, in the map information database 232 shown in FIG. 11, the information corresponding to the coordinates (xj, yj) is an example of information specified by the field survey, and the specific date and time Datej indicates the date and time when the field survey was performed. Show. On the other hand, when the vegetation is specified based on at least one of the two images that are the difference detection targets in the above-described difference detection process, the date and time when the vegetation is specified is, for example, the latest of the two images. It is desirable to store the shooting date and time of the image taken at the end. For example, the information corresponding to the coordinates (xk, yk) is an example of information specified based on the above-described image subjected to the difference detection process, and the specific date and time Datek is the most recent of the difference detection target images. Indicates the shooting date and time of the captured image.

  The memory 42 of the computer device 40 illustrated in FIG. 10 stores an application program for the processor 41 to execute a vegetation determination process and a difference detection process using map information together with the operating system of the computer apparatus 40. The application program described above can be recorded and distributed on a removable disk 47 such as an optical disk, for example. Then, the above-described application program may be stored in the memory 42 and the hard disk device 43 by attaching the removable disk 47 to the optical drive device 46 and performing a reading process. Further, the above-described application program can be read into the memory 42 and the hard disk device 43 via a network interface (not shown). The application program described above is preferably provided together with information for constructing the plant state database DB described above in the hard disk device 43 or the like.

  FIG. 12 shows another example of the flowchart of the difference detection process. Of the steps shown in FIG. 12, the same steps as those shown in FIG. 6 are designated by the same reference numerals, and the description thereof is omitted. Each process of step 301 to step 310 and step 350 shown in FIG. 12 is an example of a process included in an application program for vegetation determination processing and difference detection processing using map information. In addition, each of these processes of Step 301 to Step 310 and Step 50 is executed by the processor 41 shown in FIG.

  In the processing of the affirmative determination route in step 304, the processor 41 determines whether or not there is map information about a point corresponding to the processing target section prior to the processing for determining the vegetation change in step 305 (step 305). 310). For example, the processor 41 may refer to the map information database 232 and perform the determination process in step 310 based on whether or not map information about a point corresponding to the processing target section is detected. Here, the coordinates of the point corresponding to each section included in the image can be specified in advance based on, for example, information indicating the shooting position added to each image. Therefore, the processor 41 may detect the map information registered in correspondence with the coordinates indicating the position of the point corresponding to the processing target section in the map information database 232.

  When there is no map information about the point corresponding to the processing target section (No determination in step 310), the processor 41 proceeds to the process of step 305, and the vegetation at the point corresponding to the processing target section is It is determined whether or not there is a change between image capturing times.

  On the other hand, when there is map information about the point corresponding to the section to be processed (affirmative determination in step 310), the processor 41 proceeds to the process of step 350 and uses the map information as shown in FIG. To determine changes in vegetation.

  FIG. 13 shows an example of a flowchart of processing for determining changes in vegetation using map information. Of the steps shown in FIG. 13, the same steps as those shown in FIG. 3 are denoted by the same reference numerals and description thereof is omitted. Each process of step 313 to step 317 and step 351 to step 355 shown in FIG. 13 is an example of a process for determining a change in vegetation using the map information shown in step 350 of FIG. In addition, each of these processes of Step 301 to Step 310 and Step 351 to Step 355 is executed by the processor 41 shown in FIG.

  First, the processor 41 determines whether or not the map information includes a vegetation determination result based on one of the processing target images based on information indicating a specific date and time included in the map information (step 351). For example, when one of the two images has already been the target of the difference detection between the images, the map includes the shooting date and time of the image as the specific date and time together with information indicating the vegetation specified based on the image Information may be registered in the map information database 232 in some cases. Therefore, the processor 41 specifies the map information based on one image based on whether one of the photographing dates and times of each difference detection target image matches the specific date and time included in the map information. It can be determined whether or not vegetation is indicated.

  When the detected map information includes a vegetation determination result based on one image (affirmative determination in step 351), the processor 41 has already determined the image that has resulted in the vegetation determination result included in the map information. The other image is an unidentified image (step 352). For example, when the shooting date and time of the image A and the specific date and time included in the map information match between the images A and B, the processor 41 sets the image A as the determined image and sets the image B as the unidentified image. Also good. That is, the processor 41 omits the process of discriminating vegetation for the images A and B, and executes the processes after step 313 using the type of plant indicated by the map information as the discrimination result obtained for the image A.

  Here, considering the use of tracking and investigating the land use situation, two images obtained by photographing the same area every predetermined period are likely to be images to be processed. In this case, for one of the images to be processed, the vegetation is determined for each section in the process of past difference detection processing, and the map information including the determination result of the vegetation corresponding to each section is map information. Accumulated in the database 232. Therefore, when performing the difference detection process in the above-described use, it is highly possible that the process of determining the vegetation shown in step 311 in FIG. 3 can be omitted for many sections included in the processing target image. . Thereby, compared with the case where the process which discriminate | determines vegetation is performed about all the divisions, the time which the process which determines the change of vegetation can be shortened.

  In addition, the map information including the vegetation discrimination result based on one of the two images to be processed is assigned to each section at a second time different from the first time corresponding to the shooting time value of the other image. It is an example of the reference information containing the information which shows the result which specified the vegetation about the corresponding point.

  In addition, executing the process of step 313 using the type of plant indicated by the map information as a vegetation discrimination result based on one image indicates that the plant indicated by the reference information exhibits in the first period described above. It is an example of the process which estimates the feature-value of the image which shows.

  Similarly, executing the process of step 314 in the affirmative determination route of step 351 is similar to the estimated feature quantity and the feature quantity of the processing target section included in the first image that has been determined as an unidentified image. It is an example of the process which evaluates property.

  In addition, executing the processing of step 315 to step 316 in the affirmative determination route of step 351 means that the vegetation at the point corresponding to the processing target section is based on the evaluation result of the similarity and the second time period described above. It is an example of the process which judges whether it changed between 1 time.

  Therefore, each process of the affirmation determination route of step 351 shown in FIG. 13 corresponds to another embodiment of the vegetation determination method and the vegetation determination program of the present disclosure.

  Note that, in the case of the negative determination in step 315 described above, the processor 41 performs steps 321 to 327 shown in FIG. 7 with respect to an image that has been determined as an unidentified image in the processing in step 352 prior to the processing in step 317. You may perform the process which discriminates vegetation. Moreover, when the information which shows the new vegetation corresponding to the division of a process target is obtained by performing the process which discriminates vegetation, you may update the map information database 232 using the obtained information. .

  On the other hand, for example, when the detected map information includes information indicating the vegetation specified by the field survey, the processor 41 executes the processing of step 353 to step 355 according to the negative determination route of step 351.

  In step 353, the processor 41 estimates a feature amount indicating a state that the plant indicated by the determination result included in the map information exhibits at the time of photographing each of the two images to be processed. The processor 41 may acquire the feature amount corresponding to the photographing time of each image by searching the feature amount corresponding to each photographing time from the plant state database DB corresponding to the above-described plant. Further, the processor 41 obtains a plurality of feature quantity candidates as the feature quantity estimated corresponding to the shooting time of each image by applying the processing of step 331 to step 333 shown in FIG. Also good.

  Next, the processor 41 evaluates the similarity between the feature amount estimated corresponding to the shooting time of each image and the feature amount corresponding to the processing target section in each image (step 354). The processor 41 may evaluate the similarity between the feature amount estimated for each image and the actual feature amount of the processing target section included in each image based on the distance between the feature amounts. In addition, the processor 41 actually applies the processing of Step 341 to Step 346 shown in FIG. 9 for each of a plurality of feature amount candidates obtained for each image and the processing target section included in each image. You may evaluate the similarity with the obtained feature-value.

  Next, the processor 41 obtains an evaluation result that the feature amount estimated for both images and the feature amount corresponding to the section to be processed in each image are similar in the process of step 354 described above. It is determined whether or not it has been made (step 355).

  When an evaluation result indicating that both images are similar is obtained (affirmative determination in step 355), the processor 41 indicates the vegetation corresponding to the section included in the two images by map information. It is judged that none has changed since the vegetation. In this case, the processor 41 proceeds to the process of step 316 according to the affirmative determination route of step 360, and outputs a determination result indicating that there is no change in the vegetation at the point corresponding to the section.

  On the other hand, if an evaluation result indicating that at least one image is not similar is obtained (negative determination in step 355), the processor 41 determines that the vegetation of the section included in at least one image is indicated by the map information. Judge that it has changed. In this case, the processor 41 proceeds to the process of step 317 according to the negative determination route of step 355, and outputs a determination result indicating that there is a change in the vegetation at the point corresponding to the partition.

  The process of step 353 described above is a process of setting each image to be processed as a first image whose vegetation has not been determined, and estimating a feature amount of an image indicating a state represented by the plant indicated by the reference information at the shooting time of the first image. It is an example.

  Similarly, in the processing in step 354 described above, each image to be processed is set as a first image whose vegetation has not been determined. The feature amount estimated for each image and the feature amount of the processing target section included in the first image It is an example of the process which evaluates similarity.

  In addition, the processing in step 355 described above is performed during the shooting time of each image based on whether or not the vegetation at the point corresponding to the processing target section included in each image has changed from the vegetation indicated by the reference information. It is an example of the process which judges whether the vegetation of the point corresponding to the said division changed.

  Therefore, each process of the negative determination route of step 351 shown in FIG. 13 also corresponds to another embodiment of the vegetation determination method and the vegetation determination program of the present disclosure.

  The process which determines the change of the vegetation using the information accumulate | stored in the map information database 232 as information which shows the vegetation of the point corresponding to a process target partition, when the processor 41 performs each process shown in FIG. Can be realized. As described above, when map information is used, it is possible to omit the process of determining the vegetation at a point corresponding to the processing target section based on the characteristics of the image. Therefore, when it is determined that there is map information in step 310 shown in FIG. 12, instead of the process in step 305, a process for determining a change in vegetation using the map information described above is executed, whereby a difference is obtained. The time required for the detection process can be shortened.

  In the case of the negative determination in step 355 described above, prior to the processing in step 317, the processor 41 performs processing of steps 321 to 327 shown in FIG. 7 for the processing target sections included in each processing target image. You may perform the process which discriminates vegetation. Moreover, when the information which shows the new vegetation corresponding to the division of a process target is obtained by performing the process which discriminates vegetation, you may update the map information database 232 using the obtained information. .

  Further, the map information database 232 shown in FIG. 11 can be constructed in the hard disk device 33 included in the client device 30 shown in FIG. Then, when two images to be processed are passed from the client device 30 to the server device 20, map information corresponding to each point included in the shooting range of these images may be passed to the server device 20. In addition, the server device 20 executes a process of determining a change in vegetation using the received map information in the course of the difference detection process, and uses information indicating the vegetation obtained in the process of the determination process. Map information may be updated. Further, the updated map information may be returned to the client device 30 together with the difference detection result.

Regarding the above description, the following items are further disclosed.
(Appendix 1)
For each section obtained by dividing each of the two images obtained by photographing a predetermined region at different times, the type of plant photographed in the section is determined based on the feature amount of the image of the section. Discriminate,
The type of the plant cannot be determined for the section included in the first image which is one of the two images, and the type of the plant is determined for the section included in the second image which is the other of the two images. If it can be determined,
Based on the plant information indicating the feature amount of the image obtained by photographing each plant for each period, the type of plant determined for the section included in the second image is the photographing period of the first image. Estimate the feature quantity of the image showing the state presented in
Evaluating the similarity between the estimated feature quantity and the feature quantity of the image of the section included in the first image;
When it is evaluated that there is the similarity, it is determined that the vegetation at the point corresponding to the section has not changed between the shooting time of the second image and the shooting time of the first image, When it is evaluated that there is no similarity, the computer determines that the vegetation at the point corresponding to the section has changed between the shooting time of the second image and the shooting time of the first image. A vegetation judgment program characterized by causing
(Appendix 2)
In the vegetation judgment program described in Appendix 1,
The process of discriminating the type of plant photographed in the section,
A vegetation determination program characterized by including a process of searching for a plant type having a feature quantity similar to the feature quantity of the section from the plant information at the time of capturing an image including the section.
(Appendix 3)
In the vegetation judgment program described in Appendix 3,
The process of searching for the plant type is as follows:
A vegetation judgment program characterized by including a process of correcting a photographing time of an image including the section based on a climate of a region photographed in the image.
(Appendix 4)
For each section obtained by dividing the first image obtained by photographing a predetermined area at the first time, vegetation was specified at a point corresponding to the section at a second time different from the first time. Reference information including information indicating the result,
Based on the plant information indicating the feature amount of the image obtained by photographing each plant for each period, the state of the plant of the type indicated as the vegetation of the section by the reference information in the first period Estimate the feature value of the image shown,
Evaluating the similarity between the estimated feature quantity and the feature quantity of the image of the section included in the first image;
When it is evaluated that there is the similarity, it is determined that the vegetation at the point corresponding to the section has not changed between the second period and the first period, and it is evaluated that there is no similarity. In this case, the vegetation of the point corresponding to the section is caused to cause the computer to execute a process of determining that the vegetation at the point corresponding to the section has changed between the shooting time of the second image and the shooting time of the first image. Vegetation judgment program.
(Appendix 5)
In the vegetation judgment program described in Appendix 1,
The process of estimating the feature amount includes:
Among the plant information, including a process of searching for a feature amount corresponding to the shooting time of the first image from information corresponding to the identified type of plant,
The process of searching for the feature amount includes:
A vegetation judgment program characterized by including a process of correcting the photographing time of the first image based on a climate of a region photographed in the first image.
(Appendix 6)
In the vegetation judgment program described in Appendix 1,
The process of estimating the feature amount includes:
Among the plant information, including a process of searching for a feature amount corresponding to the shooting time of the first image from information corresponding to the identified type of plant,
The process of searching for the feature amount includes:
A vegetation determination program characterized by including a process of extracting a feature amount corresponding to each period within a predetermined period including the photographing time of the first image as an estimated feature amount candidate.
(Appendix 7)
For each section obtained by dividing each of the two images obtained by photographing a predetermined region at different times, the type of plant photographed in the section is determined based on the feature amount of the image of the section. Discriminate,
The type of the plant cannot be determined for the section included in the first image which is one of the two images, and the type of the plant is determined for the section included in the second image which is the other of the two images. If it can be determined,
Based on the plant information indicating the feature amount of the image obtained by photographing each plant for each period, the type of plant determined for the section included in the second image is the photographing period of the first image. Estimate the feature quantity of the image showing the state presented in
Evaluating the similarity between the estimated feature quantity and the feature quantity of the image of the section included in the first image;
When it is evaluated that there is the similarity, it is determined that the vegetation at the point corresponding to the section has not changed between the shooting time of the second image and the shooting time of the first image, When it is evaluated that there is no similarity, it is determined that the vegetation at the point corresponding to the section has changed between the shooting time of the second image and the shooting time of the first image. Vegetation judgment method.
(Appendix 8)
For each section obtained by dividing each of the two images obtained by photographing a predetermined region at different times, the type of plant photographed in the section is determined based on the feature amount of the image of the section. A discriminator for discriminating;
The type of the plant cannot be determined for the section included in the first image which is one of the two images, and the type of the plant is determined for the section included in the second image which is the other of the two images. When it can be determined, based on the plant information indicating the feature amount of the image obtained by photographing each plant for each period, the type of plant determined for the section included in the second image, An estimation unit that estimates a feature amount of an image indicating a state presented at the time of photographing the first image;
An evaluation unit that evaluates the similarity between the estimated feature amount and the feature amount of the image of the section included in the first image;
When it is evaluated that there is the similarity, it is determined that the vegetation at the point corresponding to the section has not changed between the shooting time of the second image and the shooting time of the first image, A judgment unit that judges that the vegetation at the point corresponding to the section has changed between the photographing time of the second image and the photographing time of the first image when it is evaluated that there is no similarity; A vegetation judging device characterized by that.

DESCRIPTION OF SYMBOLS 10 ... Vegetation determination apparatus; 11 ... Comparison part; 12 ... Estimation part; 13 ... Evaluation part; 14 ... Judgment part; 16 ... Difference detection apparatus; 1 ... Display apparatus; , 41 ... Processor; 22, 32, 42 ... Memory; 23, 33, 43 ... Hard disk device; 24, 34 ... Network interface (I / F); 35 ... Display unit; 36, 44 ... General-purpose interface (I / F) 45 ... Display control unit; 46 ... Optical drive device; 47 ... Removable disk; 232 ... Map information database; DB ... Plant state database; NW ... Network

Claims (3)

Calculating at least the image feature amount of the predetermined region for the first image and the second image obtained by photographing the predetermined region at different times,
Types of plants included in the predetermined region of the first image with reference to a database including information indicating image feature quantities that differ according to the time in association with the types of plants based on the time of shooting. And identifying the type of plant included in the predetermined area of the second image,
When the plant type specified for the first image matches the plant type specified for the second image, the plant type of the predetermined region has not changed between the different times. And
When the plant type specified for the first image and the plant type specified for the second image do not match, it is determined that the plant type of the predetermined region has changed between the different times. A vegetation judgment program characterized by causing a computer to execute processing. Calculating at least the image feature amount of the predetermined region for the first image and the second image obtained by photographing the predetermined region at different times,
Types of plants included in the predetermined region of the first image with reference to a database including information indicating image feature quantities that differ according to the time in association with the types of plants based on the time of shooting. And identifying the type of plant included in the predetermined area of the second image,
When the plant type specified for the first image matches the plant type specified for the second image, the plant type of the predetermined region has not changed between the different times. And
When the plant type specified for the first image and the plant type specified for the second image do not match, it is determined that the plant type of the predetermined region has changed between the different times. A vegetation judgment method characterized by: A calculating unit that calculates at least the image feature amount of the predetermined region for the first image and the second image obtained by photographing the predetermined region at different times, and
Types of plants included in the predetermined region of the first image with reference to a database including information indicating image feature quantities that differ according to the time in association with the types of plants based on the time of shooting. And a specifying unit for specifying the type of plant included in the predetermined region of the second image,
When the plant type specified for the first image matches the plant type specified for the second image, the plant type of the predetermined region has not changed between the different times. And when the plant type specified for the first image does not match the plant type specified for the second image, the plant type of the predetermined region is between the different times. A vegetation determination device comprising: a determination unit that determines that a change has occurred.

Images