JP6959114B2 - Misidentification possibility evaluation device, misdiscrimination possibility evaluation method and program - Google Patents

Description

The present invention relates to a erroneous discrimination possibility evaluation device, a erroneous discrimination possibility evaluation method, and a program.

Image classification is discriminated, such as by hand or by using a trained machine learning model to discriminate the tree type that appears in the image. In such discrimination, a person in charge such as an expert may confirm the discrimination result of the image classification in order to improve the discrimination accuracy.

Further, Patent Document 1 describes a ground having a peculiar likelihood among features of interpretation results saved in association with a position in a sensing image based on the result of automatic classification of images based on spectral characteristics, radiation characteristics, and the like. A technique for outputting an object as a candidate for interpretation correction is described.

Japanese Unexamined Patent Publication No. 2009-251793

Regardless of whether the discrimination is performed manually or by using trained machine learning, for an image in which objects of multiple classifications appear in a mixed manner, such as an image taken near the boundary of a tree type. There is a tendency for the possibility of misclassification of image classification to increase. On the other hand, for images in which only objects of a single classification appear, the possibility of misclassification of image classification is low. As described above, the possibility of misclassification of image classification varies depending on the image.

However, regardless of the high possibility of erroneous discrimination, the work of confirming the discrimination result of the image classification by the person in charge is performed in the same manner for all the images, which is a great burden on the person in charge.

Here, if the possibility of erroneous discrimination of image classification can be accurately evaluated, the confirmation work according to the evaluation result becomes possible, so that the efficiency of the confirmation work of the discrimination result of image classification can be expected.

It should be noted that the technique described in Patent Document 1 does not consider the probability that the feature of the interpretation result is a different feature in the identification of the interpretation correction candidate, and therefore whether or not the feature of the interpretation result is a candidate for interpretation correction. Was not evaluated accurately in some cases.

The present invention has been made in view of the above problems, and one of the purposes thereof is an erroneous discrimination possibility evaluation device, a erroneous discrimination possibility evaluation method, and a program capable of accurately evaluating the possibility of erroneous discrimination in image classification. Is to provide.

(1) The erroneous discrimination possibility evaluation device according to the present invention classifies the target image by classifying the target image acquisition means for acquiring the target image to be classified and any one of a plurality of given classifications. The first classification discriminating means for discriminating as the first classification having the highest possibility of being, and the first classification among the plurality of classifications in which the probability that the object of the first classification appears is lower than the first probability. Based on the ratio specifying means for specifying the ratio of the size of the region in the target image to the overall size of the target image, in which the probability that objects of different classifications appear is higher than the second probability, and the ratio. , The erroneous discrimination possibility evaluation means for determining a value indicating the possibility that the discrimination result of the first classification discriminating means is erroneous discrimination.

(2) In the misdiscrimination possibility evaluation device according to (1) above, based on the target image, the classification with the second highest possibility of being the classification of the target image is selected from the plurality of classifications. The second classification discriminating means for discriminating as two classifications is further included, and the ratio specifying means has a lower probability that an object of the first classification appears than the first probability, and an object of the second classification appears. It is possible to specify the ratio of the size of the region in the target image to the size of the target image, which has a higher probability of being present than the second probability.

(3) In the erroneous discrimination possibility evaluation device according to the above (1) or (2), the ratio specifying means appears the object of the first classification in the image occupying the region of interest in the target image. The size of the image in which the probability of being present is lower than the first probability and the probability that an object of a classification different from the first classification appears among the plurality of classifications is higher than the second probability. It can be configured to specify the ratio to.

(4) In the erroneous discrimination possibility evaluation device according to the above (1) or (2), in the ratio specifying means, the object of the first classification appears in the image occupying the region of interest in the target image. The size of the image in the size of the region where the probability of being present is lower than the first probability and the probability that an object of a classification different from the first classification appears among the plurality of classifications is higher than the second probability. The ratio to the first ratio is specified as the first ratio, and the ratio specifying means has the first probability that an object of the first classification appears in an image occupying a region other than the region of interest in the target image. The ratio of the size of the region where the probability that an object of the plurality of classifications different from the first classification appears is lower than the second probability to the size of the image is specified as the second ratio. However, in the erroneous discrimination possibility evaluation means, the discrimination result of the first classification discrimination means may be erroneous discrimination based on the weighted average value of the first ratio and the second ratio based on a predetermined weight. It can be configured to determine a value indicating sex.

(5) In the erroneous discrimination possibility evaluation method according to the present invention, the target image acquisition step for acquiring the target image to be classified is classified, and any one of a plurality of given classifications is classified into the target image. The first classification determination step for determining as the first classification having the highest possibility of being, and the first classification among the plurality of classifications in which the probability that the object of the first classification appears is lower than the first probability. Based on the ratio specifying step that specifies the ratio of the size of the region in the target image to the overall size of the target image, where the probability that objects of different classifications appear is higher than the second probability, and the ratio. The erroneous discrimination possibility evaluation step of determining a value indicating the possibility that the discrimination result in the first classification discrimination step is erroneous discrimination is included.

(6) In the program according to the present invention, it is most likely that the target image acquisition procedure for acquiring the target image to be discriminated by classification or any one of a plurality of given classifications is the classification of the target image. The first classification discrimination procedure for discriminating as a high first classification, the probability that an object of the first classification appears is lower than the first probability, and among the plurality of classifications, an object of a classification different from the first classification appears. In the ratio specifying procedure for specifying the ratio of the size of the region in the target image whose probability of being present is higher than the second probability to the total size of the target image, and in the first classification determination procedure based on the ratio. Have the computer execute a misdiscrimination possibility evaluation procedure for determining a value indicating the possibility that the discriminant result of is misdiscrimination.

According to the present invention, the possibility of erroneous discrimination of image classification can be accurately evaluated.

It is a block diagram of the image classification discrimination apparatus which concerns on one Embodiment of this invention. It is a figure which shows an example of a target image. It is a figure which shows an example of the 1st classification evaluation image. It is a figure which shows an example of the contrast classification evaluation image. It is a figure which shows an example of a misdiscrimination evaluation image. It is a figure which shows another example of a target image. It is a figure which shows another example of the 1st classification evaluation image. It is a figure which shows another example of the contrast classification evaluation image. It is a figure which shows another example of a misdiscrimination evaluation image. It is a figure which shows an example of an image classification discrimination module and a visualization map generation module. It is a functional block diagram which shows an example of the function of the image classification discrimination apparatus which concerns on one Embodiment of this invention. It is a figure which shows an example of the region of interest in the target image shown in FIG. It is a flow figure which shows an example of the flow of the process performed by the image classification discrimination apparatus which concerns on one Embodiment of this invention.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

FIG. 1 is a configuration diagram of an image classification discriminating device 10 according to an embodiment of the present invention. The image classification discrimination device 10 according to the present embodiment is a computer such as a personal computer. As shown in FIG. 1, the image classification determination device 10 includes, for example, a processor 12, a storage unit 14, a display unit 16, and an operation unit 18.

The processor 12 is, for example, a program control device such as a CPU that operates according to a program installed in the image classification determination device 10.

The storage unit 14 is a storage element such as a ROM or RAM, a hard disk drive, or the like. The storage unit 14 stores a program or the like executed by the processor 12.

The display unit 16 is a display device such as a liquid crystal display, and displays various images according to the instructions of the processor 12.

The operation unit 18 is a user interface such as a keyboard or a mouse, receives an operation input of the user, and outputs a signal indicating the contents to the processor 12.

The image classification / discrimination device 10 may include a communication interface such as a network board, an optical disk drive for reading an optical disk such as a DVD-ROM or a Blu-ray (registered trademark) disk, a USB (Universal Serial Bus) port, and the like. ..

In the image classification discriminating device 10 according to the present embodiment, with respect to the image to be discriminated by classification, any one of a plurality of given classifications is discriminated as the first classification most likely to be the classification of the image. The process to be executed is executed. Hereinafter, the classification determined as the nth most likely classification of images will be referred to as the nth classification (n = 1, 2, 3, ...). Further, in the image classification discrimination device 10 according to the present embodiment, a process of evaluating the possibility that the result of the discrimination is erroneous discrimination is executed. Hereinafter, the image that is the target of these processes will be referred to as a target image.

FIG. 2 is a diagram showing an example of the target image 20. The target image 20 shown in FIG. 2 may be one of a plurality of patch images cut out at intervals of 10 meters in each of the vertical and horizontal directions from an aircraft image obtained by photographing a forest or the like. Further, the target image 20 may be an image of 224 pixels in each of the vertical and horizontal directions (corresponding to 35.84 meters in length and width) in which one pixel corresponds to 16 cm in length and width.

Then, in the present embodiment, for example, a process of discriminating the classification of the tree type appearing in the target image 20 and a process of evaluating the possibility that the result of the discrimination is erroneous discrimination are executed. Here, examples of classification by tree type include sugi, cypress, pine, bamboo, deciduous broad-leaved trees, evergreen broad-leaved trees, and out-of-forest. In determining the classification of the tree type, for example, the classification most likely to be the tree type appearing near the center of the target image 20 may be determined. More specifically, for example, the classification most likely to be the tree type appearing in the area occupying a part of the target image 20 having the same center as the target image 20 may be determined.

Further, in the present embodiment, for each pixel in the target image 20, the probability that an object of the first classification appears in the pixel is specified. Hereinafter, the probability will be referred to as a first classification probability. Then, the first classification evaluation image 22 illustrated in FIG. 3 is generated, in which a region in which the first classification probability is lower than the predetermined first probability is represented. In the first classification evaluation image 22, the area occupied by the pixels whose first classification probability is lower than the first probability is indicated by dots. Hereinafter, the region will be referred to as a first region 24. The ratio of the size of the first region 24 to the total size of the first classification evaluation image 22 is the size of the region occupied by the pixels whose first classification probability is lower than the above-mentioned first probability in the target image 20. , Corresponds to the ratio to the total size of the target image 20.

Further, in the present embodiment, for example, for each pixel in the target image 20, the probability that an object of a classification different from the first classification (hereinafter, referred to as a contrast classification) appears is specified. Hereinafter, the probability will be referred to as a contrast classification probability. Then, the contrast classification evaluation image 26 illustrated in FIG. 4 is generated, in which the region in which the contrast classification probability is higher than the predetermined second probability is represented. Here, the second probability may be the same as or different from the first probability described above. In the contrast classification evaluation image 26, the area occupied by the pixels whose contrast classification probability is higher than the second probability is indicated by dots. Hereinafter, the region will be referred to as a comparison region 28. The ratio of the size of the contrast region 28 to the total size of the contrast classification evaluation image 26 is the size of the region occupied by the pixels whose contrast classification probability is higher than the above-mentioned second probability in the target image 20. Corresponds to the ratio of 20 to the total size. Here, the contrast classification may be a classification (second classification) that is determined as the classification having the second highest possibility of being the classification of the target image 20.

The first classification evaluation image 22 and the comparison classification evaluation image 26 according to the present embodiment have the same shape, the same size, and the same number of pixels as the target image 20. Then, the pixels in the target image 20 are associated with the pixels in the first classification evaluation image 22 on a one-to-one basis. Further, the pixels in the target image 20 are associated with the pixels in the contrast classification evaluation image 26 on a one-to-one basis.

Then, in the present embodiment, the erroneous discrimination evaluation image 30 illustrated in FIG. 5 is generated based on the first classification evaluation image 22 and the contrast classification evaluation image 26. The misdiscrimination evaluation image 30 according to the present embodiment has the same shape, the same size, and the same number of pixels as the target image 20. Then, the pixels in the target image 20 are also associated with the pixels in the erroneous discrimination evaluation image 30 on a one-to-one basis.

In the present embodiment, for example, the logical product of the dot portion in the first classification evaluation image 22 and the dot portion in the contrast classification evaluation image 26 is shown as the dot portion in the erroneous discrimination evaluation image 30. Hereinafter, the area occupied by the dot portion in the erroneous discrimination evaluation image 30 will be referred to as a erroneous discrimination region 32. The pixels in the target image 20 associated with the pixels in the erroneous discrimination region 32 have a first classification probability lower than the first probability described above and a contrast classification probability higher than the second probability described above. Will be.

Then, in the present embodiment, for example, based on the ratio of the size of the erroneous discrimination region 32 to the total size of the erroneous discrimination evaluation image 30, the classification most likely to be the classification of the target image 20 is the above-mentioned first classification. A value indicating the possibility that the discrimination result to be classified is erroneous discrimination is determined.

Here, for example, when the ratio is larger than a predetermined threshold value, 1 may be determined as a value indicating the possibility that the above-mentioned discrimination result is erroneous discrimination. On the contrary, when the ratio is smaller than the predetermined threshold value, 0 may be determined as a value indicating the possibility that the above-mentioned discrimination result is erroneous discrimination.

In the erroneous discrimination evaluation image 30 shown in FIG. 5, the ratio of the size of the erroneous discrimination region 32 to the total size of the erroneous discrimination evaluation image 30 is 46.8%. Here, for example, when the threshold value is 30%, 1 may be determined as a value indicating the possibility that the above-mentioned discrimination result is erroneous discrimination.

FIG. 6 is a diagram showing an example of a target image 40 different from the target image 20. FIG. 7 is a diagram showing an example of a first classification evaluation image 42 generated based on the target image 40 shown in FIG. The first classification evaluation image 42 shown in FIG. 7 shows the first region 44. FIG. 8 is a diagram showing an example of a contrast classification evaluation image 46 generated based on the target image 40 shown in FIG. The contrast region 48 is shown in the contrast classification evaluation image 46 shown in FIG. FIG. 9 is a diagram showing an example of an erroneous discrimination evaluation image 50 generated based on the first classification evaluation image 42 shown in FIG. 7 and the contrast classification evaluation image 46 shown in FIG. The erroneous discrimination evaluation image 50 shown in FIG. 9 shows the erroneous discrimination region 52.

In the erroneous discrimination evaluation image 50 shown in FIG. 9, the ratio of the size of the erroneous discrimination region 52 to the total size of the erroneous discrimination evaluation image 50 is 5.8%. Here, for example, when the threshold value is 30%, 0 may be determined as a value indicating the possibility that the above-mentioned discrimination result is erroneous discrimination.

As described above, according to the image classification discrimination device 10 according to the present embodiment, the possibility of erroneous discrimination of image classification can be accurately evaluated. Therefore, for example, by having the person in charge of the confirmation work mainly confirm the target image in which 1 is determined as a value indicating the possibility of erroneous discrimination, the work of confirming the determination result of the classification of the target image by the person in charge is efficient. Be transformed.

Here, for example, as described above, it is assumed that a plurality of patch images cut out from the aircraft image at intervals of 10 meters in each of the vertical and horizontal directions are used as the target image. In this case, a map in which pixels of colors associated with the first classification determined for each target image are arranged vertically and horizontally may be displayed on the display unit 16. Then, in the map, the pixel associated with the target image in which 1 is determined as a value indicating the possibility of erroneous discrimination may be highlighted such as being surrounded by a black frame. Then, the person in charge of the confirmation work may confirm the displayed map.

Further, for example, the ratio of the size of the erroneous discrimination region to the overall size of the erroneous discrimination evaluation image may be determined as a value indicating the possibility of erroneous discrimination of the image classification. In this case, a value indicating the ratio of the size of the erroneous discrimination region to the overall size of the erroneous discrimination evaluation image may be presented to the person in charge of the confirmation work together with the target image. In this case, the person in charge can confirm the target image while referring to the value indicating the possibility of erroneous determination of the image classification, so that the person in charge can confirm the determination result of the classification of the target image more efficiently. Will be done.

Hereinafter, an example of determining the image classification and evaluating the possibility of erroneous determination of the image classification will be further described using the target image 20 shown in FIG. 2 as a subject.

FIG. 10 shows an example of the image classification discrimination module 60 and the visualization map generation module 64 implemented in the image classification discrimination device 10 according to the present embodiment. Here, the image classification determination module 60 is, for example, a module for determining the classification of the target image 20. Further, the visualization map generation module 64 is, for example, a module that generates a visualization map 62 that is the basis of the first classification evaluation image 22 and the comparison classification evaluation image 26.

In the present embodiment, the image classification discrimination module 60 includes, for example, a trained convolutional neural network (CNN). This CNN is, for example, a machine learning model in which learning data including an image having the same shape, the same size, and the same number of pixels as the target image 20 as input data and a label associated with the tree type of the image as teacher data is learned. Here, as the teacher data, a label associated with the tree type appearing in the center of the image which is the input data may be included.

Then, in the present embodiment, for example, the target image 20 is input to the feature extraction unit 66 as the input 68. The feature extraction unit 66 includes, for example, a CNN convolution layer and a pooling layer. Then, the feature extraction unit 66 outputs a plurality of feature maps 70, each of which is associated with a channel (filter), in response to the input 68.

The feature map 70 output by the feature extraction unit 66 is input to one or a plurality of fully connected layers 72 included in the image classification determination module 60. Here, the number of fully bonded layers 72 is not particularly limited. Then, the output from the fully connected layer 72 is input to the softmax layer 73 included in the image classification determination module 60. Then, the softmax layer 73 outputs a probability vector including a probability value associated with the classification of the image as an element as an output 74. Here, for example, as the classification in the scene of discriminating the classification of the tree type appearing in the target image 20, sugi, cypress, pine, bamboo, deciduous broad-leaved tree, evergreen broad-leaved tree, and seven trees outside the forest are adopted. .. In this case, a probability vector including the seven probability values as elements is output.

Then, the classification corresponding to the element showing the largest probability value in the probability vector output in this way is determined as the first classification most likely to be the classification of the target image 20 input as the input 68. Will be done. The contrast classification is also determined here. For example, when the contrast classification is the second classification, the classification associated with the second largest probability value is determined as the contrast classification. Further, for example, when the contrast classification is the third classification, the classification associated with the third largest probability value is determined as the contrast classification.

Then, in the present embodiment, in the visualization map generation module 64, the visualization map 62 is generated by using the technology of Grad-cam (Gradient-weighted Class Activation Mapping) that visualizes the judgment basis of CNN.

In the visualization map generation module 64, for example, the element product 78 of the one-hot vector 76 in which the element corresponding to the first classification is 1 and the element corresponding to the other classification is 0, and the probability vector having the output 74. Is calculated. Then, the calculated element product 78 is back-propagated to the probability vector, the softmax layer 73, and the fully connected layer 72, which are the outputs 74, so that a plurality of gradient maps 80, each of which is associated with a channel (filter), are generated. Will be generated.

Then, for example, using the technology of GAP (Global Average Pooling), a representative value vector including a representative value (for example, the average value of the pixel values of the pixels included in the gradient map 80) associated with each of the plurality of gradient maps 80 as an element. 82 is calculated. Here, the number of elements of the representative value vector 82 is the number of channels of the feature map 70.

Then, for each channel, the individual map corresponding to the feature map 70 is obtained by multiplying the pixel value of each pixel included in the feature map 70 of the channel by the value of the element of the representative value vector 82 associated with the feature map 70. 84 is generated. Then, a comprehensive map 86 is generated by synthesizing the individual maps 84 of all channels. Here, for example, as the pixel values of the pixels included in the comprehensive map 86, the total of the pixel values of the pixels corresponding to the pixels for the individual maps 84 of all channels may be set.

Then, by normalizing the output when the comprehensive map 86 is input to the activation function such as the normalized linear function (ReLU) so as to be 0 or more and 1 or less, the visualization map 62 associated with the first classification is generated. Will be done. By doing so, in the visualization map 62, the pixel value becomes 0 for the pixels associated with the negative gradient. Therefore, a positive pixel value is set in the visualization map 62 only for the pixels that contribute to increasing the probability of being the first classification.

Then, in the same manner in the present embodiment, in the visualization map generation module 64, the one-hot vector 76 in which the element corresponding to the contrast classification is 1 and the element corresponding to the other classification is 0, and the probability vector having the output 74. And the element product 78 of is calculated. Then, the calculated element product 78 is back-propagated to the probability vector, the softmax layer 73, and the fully connected layer 72, which are the outputs 74, so that a plurality of gradient maps 80, each of which is associated with a channel (filter), are generated. Will be generated. Then, the representative value vector 82 including the representative value associated with each of the plurality of gradient maps 80 as an element is calculated.

Then, for each channel, the individual map corresponding to the feature map 70 is obtained by multiplying the pixel value of each pixel included in the feature map 70 of the channel by the value of the element of the representative value vector 82 associated with the feature map 70. 84 is generated. Then, a comprehensive map 86 is generated by synthesizing the individual maps 84 of all channels.

Then, by normalizing the output when the comprehensive map 86 is input to the activation function such as the normalized linear function so as to be 0 or more and 1 or less, the visualization map 62 associated with the contrast classification is generated. Also in the visualization map 62, the pixel value is 0 for the pixels associated with the negative gradient. Therefore, a positive pixel value is set in the visualization map 62 only for the pixels that contribute to increasing the probability of the contrast classification.

Then, in the present embodiment, for example, an interpolation technique such as bilinear interpolation is used to generate a first enlarged visualization image in which the visualization map 62 associated with the first classification is enlarged to the same size as the target image 20. The pixels included in the first enlarged visualization image are associated with the pixels included in the target image 20 on a one-to-one basis. Then, the pixel value of the pixel included in the first enlarged visualization image indicates the first classification probability of the pixel in the target image 20 associated with the pixel. Here, for example, the closer the pixel value of the pixel included in the first enlarged visualization image is to 1, the higher the probability that the tree of the first classification appears in the pixel in the target image 20 associated with the pixel. Further, for example, the closer the pixel value of the pixel included in the first enlarged visualization image is to 0, the lower the probability that the tree of the first classification appears in the pixel in the target image 20 associated with the pixel.

Then, in the present embodiment, for example, based on the first enlarged visualization image, the region occupied by the pixels whose probability indicated by the pixel value is lower than the predetermined first probability is represented, and the first classification evaluation image illustrated in FIG. 3 is represented. 22 is generated.

Further, in the present embodiment, an interpolation technique such as bilinear interpolation is used to generate a contrast magnified visualization image in which the visualization map 62 associated with the contrast classification is enlarged to the same size as the target image 20. The pixels included in the contrast magnified visualization image are associated with the pixels included in the target image 20 on a one-to-one basis. Then, the pixel value of the pixel included in the contrast enlarged visualization image indicates the contrast classification probability of the pixel in the target image 20 associated with the pixel. Here, for example, the closer the pixel value of the pixel included in the contrast enlarged visualization image is to 1, the higher the probability that the tree of the contrast classification appears in the pixel in the target image 20 associated with the pixel. Further, for example, the closer the pixel value of the pixel included in the contrast enlarged visualization image is to 0, the lower the probability that the tree of the contrast classification appears in the pixel in the target image 20 associated with the pixel.

Then, in the present embodiment, for example, the contrast classification evaluation image 26 illustrated in FIG. 4 represents a region occupied by pixels whose probability indicated by the pixel value is higher than a predetermined second probability based on the contrast magnified visualization image. Will be generated.

Then, as described above, the classification most likely to be the classification of the target image 20 is the first classification based on the misdiscrimination evaluation image 30 generated based on the first classification evaluation image 22 and the contrast classification evaluation image 26. A value indicating the possibility that the discrimination result to be classified is erroneous discrimination will be determined.

Hereinafter, the functions of the image classification discrimination device 10 according to the present embodiment and the processing executed by the image classification discrimination device 10 according to the present embodiment will be further described.

FIG. 11 is a functional block diagram showing an example of the functions implemented in the image classification determination device 10 according to the present embodiment. It should be noted that the image classification determination device 10 according to the present embodiment does not need to be equipped with all the functions shown in FIG. 11, and may be equipped with functions other than the functions shown in FIG.

As shown in FIG. 11, functionally, the image classification determination device 10 according to the present embodiment includes a target image acquisition unit 90, a classification determination unit 92, a visualization map generation unit 94, a ratio identification unit 96, and an erroneous determination. Possibility assessment unit 98, is included. The classification / discrimination unit 92 corresponds to the image classification / discrimination module 60 shown in FIG. The visualization map generation unit 94 corresponds to the visualization map generation module 64 shown in FIG. The target image acquisition unit 90, the classification determination unit 92, the visualization map generation unit 94, the ratio identification unit 96, and the erroneous discrimination possibility evaluation unit 98 are mainly equipped with the processor 12 and the storage unit 14.

The image classification discrimination device 10 according to the present embodiment not only serves as a device for discriminating the classification of the target image 20, but also serves as a misclassification possibility evaluation device for evaluating the possibility that the discrimination result is erroneous discrimination. It also plays a role.

The above functions may be implemented by executing the program including the commands corresponding to the above functions installed in the image classification determination device 10 which is a computer on the processor 12. This program may be supplied to the image classification discriminating device 10 via a computer-readable information storage medium such as an optical disk, a magnetic disk, a magnetic tape, a magneto-optical disk, or a flash memory, or via the Internet or the like. good.

In the present embodiment, the target image acquisition unit 90 acquires, for example, the target image 20 which is the discrimination target of the classification.

In the present embodiment, the classification discriminating unit 92 discriminates, for example, any one of a plurality of given classifications as the first classification most likely to be the classification of the target image 20. Here, based on the target image 20, the classification most likely to be the classification of the target image 20 may be determined as the first classification.

Here, it is assumed that the nth classification is defined in advance as a comparison classification. In this case, the classification discrimination unit 92 also executes a process of discriminating the classification having the nth highest possibility of being the classification of the target image 20 as a comparison classification. Here, the second classification may be discriminated as a contrast classification, or the third classification may be discriminated as a contrast classification.

The classification discrimination unit 92 may discriminate a plurality of classifications as comparative classifications. Specifically, for example, the second classification and the third classification may be discriminated as contrast classifications.

Further, in the above description, an example in which the classification discriminating unit 92 is implemented by the image classification discriminating module 60 has been described, but it is not necessary that the classification discriminating unit 92 is implemented by the image classification discriminating module 60.

For example, the classification discriminating unit 92 discriminates one of a plurality of given classifications as the first classification most likely to be the classification of the target image 20 by using another image processing technique such as template matching. You may. Further, the classification determination unit 92 may determine the contrast classification of the target image 20 by using another image processing technique such as template matching.

Further, for example, the classification determination unit 92 may accept data representing the first classification of the target image 20 visually determined by the user via the operation unit 18. Then, the classification determination unit 92 may determine the classification represented by the data as the first classification most likely to be the classification of the target image 20. Similarly, the classification determination unit 92 may accept data representing the contrast classification of the target image 20 visually determined by the user via the operation unit 18. Then, the classification discrimination unit 92 may discriminate the classification represented by the data as a comparison classification of the target image 20.

In the present embodiment, the visualization map generation unit 94 generates, for example, a visualization map 62 associated with the first classification and a visualization map 62 associated with the comparison classification.

The visualization map generation unit 94 normalizes the output when the comprehensive map 86 generated for the first classification as described above is input to the activation function such as ReLU so as to be 0 or more and 1 or less. A visualization map 62 associated with the first category is generated.

Further, the visualization map generation unit 94 normalizes the output when the comprehensive map 86 generated for the contrast classification as described above is input to the activation function such as ReLU so as to be 0 or more and 1 or less. A visualization map 62 associated with the contrast classification is generated.

In the present embodiment, the ratio specifying unit 96 is, for example, the entire target image 20 having a size of a region in the target image 20 having a first classification probability lower than the first probability and a contrast classification probability higher than the second probability. Identify the ratio to size.

Here, based on the above-mentioned first magnified visualization image and contrast magnified visualization image, the target of the size of the region in the target image 20 in which the first classification probability is lower than the first probability and the contrast classification probability is higher than the second probability. It is not necessary to specify the ratio of the image 20 to the total size.

For example, the ratio specifying unit 96 generates the first classification evaluation image 22 in which the area occupied by the pixels whose probability indicated by the pixel value is lower than the first probability is represented in the visualization map 62 associated with the first classification. May be good. Further, the ratio specifying unit 96 may generate a contrast classification evaluation image 26 representing a region occupied by pixels whose probability indicated by the pixel value is higher than the second probability in the visualization map 62 associated with the contrast classification. .. The first classification evaluation image 22 and the comparison classification evaluation image 26 have the same shape, the same size, and the same number of pixels as the visualization map 62.

Then, the erroneous discrimination evaluation image 30 in which the erroneous discrimination region 32 is set may be generated based on the first classification evaluation image 22 and the contrast classification evaluation image 26. The erroneous discrimination evaluation image 30 has the same shape, the same size, and the same number of pixels as the visualization map 62. Here, the pixels in the visualization map 62 corresponding to the pixels in the erroneous discrimination region 32 have a first classification probability lower than the above-mentioned first probability and a contrast classification probability higher than the above-mentioned second probability. It will be a high pixel.

Further, as described above, it is assumed that a plurality of classifications are discriminated as comparative classifications. In this case, the size of the region in the target image 20 in which the first classification probability is lower than the first probability and the contrast classification probability for the contrast classification is higher than the predetermined probability corresponding to the contrast classification for a plurality of contrast classifications. The ratio of the target image 20 to the total size may be specified. For example, the size of the region in the target image 20 where the first classification probability is lower than the first probability, the contrast classification probability of the second classification is higher than the second probability, and the contrast classification probability of the third classification is higher than the third probability. , The ratio to the total size of the target image 20 may be specified. Here, the third probability may be the same as or different from the first probability and the second probability.

Further, the ratio specifying unit 96 may specify the first classification probability and the contrast classification probability for each pixel in the target image 20 by using an image processing technique such as template matching. Then, based on the first classification probability and the contrast classification probability identified in this way, the size of the region in the target image 20 in which the first classification probability is lower than the first probability and the contrast classification probability is higher than the second probability. The ratio of the target image 20 to the total size may be specified.

In the present embodiment, the erroneous discrimination possibility evaluation unit 98 first classifies the classification most likely to be the classification of the target image 20 by the classification determination unit 92 based on the ratio specified by the ratio identification unit 96, for example. Determine a value indicating the possibility that the discrimination result is erroneous discrimination. For example, when the ratio specified by the ratio specifying unit 96 is larger than a predetermined threshold value, the classification with the highest possibility of being the classification of the target image 20 is set as the first classification. 1 may be determined. Further, for example, when the ratio specified by the ratio specifying unit 96 is smaller than a predetermined threshold value, the classification result in which the classification most likely to be the classification of the target image 20 is the first classification is a value indicating the possibility of erroneous discrimination. 0 may be determined as.

The visualization map generation unit 94 may invert the positive / negative of the pixel value (multiply by -1) for all the pixels included in the comprehensive map 86 generated for the first classification, for example. Then, by normalizing the output when the comprehensive map 86 in which the positive and negative of the pixel value is inverted to be input to the activation function such as ReLU is 0 or more and 1 or less, the visualization map 62 associated with the first classification is obtained. May be generated. By doing so, in the visualization map 62, the pixel value becomes 0 for the pixels associated with the positive gradient. Therefore, in the visualization map 62, positive pixel values are set only for the pixels that contribute to lowering the probability of being the first classification. Then, in this case, the ratio specifying unit 96 determines the overall size of the target image 20 of the area in the target image 20 in which the first classification probability is higher than the first probability and the contrast classification probability is higher than the second probability. You may specify the ratio to.

Further, as shown in FIG. 12, the ratio specifying unit 96 has a first classification probability lower than the first probability and a contrast classification probability higher than the second probability in the image 102 occupying the region of interest 100 in the target image 20. The ratio of the size of the region to the size of the image 102 may be specified. Here, the region of interest 100 may be an region including the center of the target image 20. For example, the region of interest 100 may be a region having the same center as the target image 20 and having the size of the region surrounding the target image 20 reduced by a predetermined time.

Further, the size of the image 102 in which the ratio specifying unit 96 occupies the region of interest 100 and has a region in which the first classification probability is lower than the first probability and the contrast classification probability is higher than the second probability. The ratio to is specified as the first ratio. Further, the size of the region in the image 104 in which the ratio specifying unit 96 occupies a region other than the region of interest 100 in the target image 20, the first classification probability is lower than the first probability and the contrast classification probability is higher than the second probability. The ratio to the size of the image 104 may be specified as the second ratio.

Then, the misdiscrimination possibility evaluation unit 98 is most likely to classify the target image 20 by the classification discriminating unit 92 based on the weighted average value of the first ratio and the second ratio based on a predetermined weight. A value indicating the possibility that the discrimination result with the classification as the first classification may be an erroneous discrimination may be determined. Here, the weight with respect to the first ratio may be larger than the weight with respect to the second ratio. In this way, the first ratio is more important than the second ratio in the evaluation of the possibility that the discrimination result in which the classification with the highest possibility of being the classification of the target image 20 is the first classification is erroneous discrimination. ..

Here, for example, when the weighted average value of the first ratio and the second ratio based on the predetermined weight is larger than the predetermined threshold value, the classification most likely to be the classification of the target image 20 is defined as the first classification. 1 may be determined as a value indicating the possibility that the result is a misjudgment. Further, for example, when the weighted average value of the first ratio and the second ratio based on the predetermined weight is smaller than the predetermined threshold value, the classification having the highest possibility of being the classification of the target image 20 is set as the first classification. 0 may be determined as a value indicating the possibility of erroneous determination.

Hereinafter, an example of the flow of processing performed in the image classification determination device 10 according to the present embodiment will be described with reference to the flow diagram illustrated in FIG. In the processing example shown in FIG. 13, the second classification is a contrast classification.

First, the target image acquisition unit 90 acquires the target image 20 (S101).

Then, the classification determination unit 92 determines the classification most likely to be the classification of the target image 20 acquired by the process shown in S101 as the first classification (S102).

Then, the classification determination unit 92 determines the classification having the second highest possibility of being the classification of the target image 20 acquired by the process shown in S101 as a comparison classification (S103).

Then, the visualization map generation unit 94 generates a visualization map 62 associated with the first classification determined by the process shown in S102 (S104).

Then, the visualization map generation unit 94 generates a visualization map 62 associated with the contrast classification determined by the process shown in S103 (S105).

Then, the ratio specifying unit 96 generates a first enlarged visualization image based on the visualization map 62 generated by the process shown in S104 (S106).

Then, the ratio specifying unit 96 generates a contrast enlarged visualization image based on the visualization map 62 generated by the process shown in S105 (S107).

Then, the ratio specifying unit 96 generates the first classification evaluation image 22 based on the first enlarged visualization image generated by the process shown in S106 (S108).

Then, the ratio specifying unit 96 generates a contrast classification evaluation image 26 based on the contrast enlarged visualization image generated by the process shown in S107 (S109).

Then, the ratio specifying unit 96 generates an erroneous discrimination evaluation image 30 based on the first classification evaluation image 22 generated by the process shown in S108 and the comparison classification evaluation image 26 generated by the process shown in S109. (S110).

Then, the ratio specifying unit 96 specifies the ratio of the size of the erroneous discrimination region 32 in the erroneous discrimination evaluation image 30 generated by the process shown in S110 to the overall size of the erroneous discrimination evaluation image 30 (S111).

Then, the classification with the highest possibility of being the classification of the target image 20 determined by the processing shown in S102 based on the ratio specified by the processing shown in S111 by the erroneous discrimination possibility evaluation unit 98 is defined as the first classification. A value indicating the possibility that the discrimination result to be performed is erroneous discrimination is determined (S112). Then, the process shown in this processing example is completed.

The execution order of the processes shown in S101 to S112 is not limited to the above, and these processes may be executed in an execution order different from the order shown above. Specifically, for example, after the processes shown in S102, S104, S106, and S108 are executed, the processes shown in S103, S105, S107, and S109 may be executed.

The present invention is not limited to the above-described embodiment.

Further, the scope of application of the present embodiment is not limited to the scene of determining the classification of the tree type appearing in the target image 20. For example, the present invention may be applied to a scene in which the classification of the land cover appearing in the target image 20 is discriminated from a plurality of given classifications such as man-made objects, water areas, grasslands, trees, and bare land.

Further, the above-mentioned specific character strings and numerical values, and specific character strings in the drawings are examples, and the present invention is not limited to these character strings and numerical values.

10 Image classification and discrimination device, 12 Processor, 14 Storage, 16 Display, 18 Operation, 20 Target image, 22 First classification evaluation image, 24 First area, 26 Comparison classification evaluation image, 28 Comparison area, 30 Misclassification Evaluation image, 32 erroneous discrimination area, 40 target image, 42 first classification evaluation image, 44 first area, 46 comparison classification evaluation image, 48 comparison area, 50 erroneous discrimination evaluation image, 52 erroneous discrimination area, 60 image classification discrimination module , 62 visualization map, 64 visualization map generation module, 66 feature extractor, 68 input, 70 feature map, 72 fully connected layer, 73 softmax layer, 74 output, 76 one-hot vector, 78 element product, 80 gradient map, 82 Representative value vector, 84 Individual map, 86 Comprehensive map, 90 Target image acquisition unit, 92 Classification and discrimination unit, 94 Visualization map generation unit, 96 Ratio identification unit, 98 Misdiscrimination possibility evaluation unit, 100 Focus area, 102 images, 104 images.

Claims (6)

A target image acquisition means for acquiring a target image to be classified and
A first classification discriminating means for discriminating any one of a plurality of given classifications as the first classification most likely to be the classification of the target image.
In the target image, the probability that an object of the first category appears is lower than the first probability, and the probability that an object of a category different from the first category appears among the plurality of categories is higher than the second probability. A ratio specifying means for specifying the ratio of the size of the region to the total size of the target image, and
An erroneous discrimination possibility evaluation means for determining a value indicating the possibility that the discrimination result of the first classification discriminating means is erroneous discrimination based on the ratio, and
A misdiscrimination possibility evaluation device characterized by including. Further including a second classification discriminating means for discriminating the classification having the second highest possibility of being the classification of the target image as the second classification from the plurality of classifications based on the target image.
In the target image, the ratio specifying means has a lower probability that an object of the first category appears than the first probability and a higher probability of an object of the second category appearing than the second probability. Specify the ratio of the size of the region to the size of the target image.
The erroneous discrimination possibility evaluation device according to claim 1. The ratio specifying means has a lower probability that an object of the first category appears in an image occupying a region of interest in the target image than the first probability, and is the first category among the plurality of categories. Specify the ratio of the size of the region where the probability that an object of a different classification appears is higher than the second probability to the size of the image.
The erroneous discrimination possibility evaluation device according to claim 1 or 2. The ratio specifying means has a lower probability that an object of the first category appears in an image occupying a region of interest in the target image than the first probability, and is the first category among the plurality of categories. The ratio of the size of the region where the probability that an object of a different classification appears is higher than the second probability to the size of the image is specified as the first ratio.
In the ratio specifying means, the probability that an object of the first classification appears in an image occupying a region other than the region of interest in the target image is lower than the probability of the first classification, and the first of the plurality of classifications. The ratio of the size of the region where the probability that an object of a classification different from the first classification appears is higher than the second probability to the size of the image is specified as the second ratio.
The misdiscrimination possibility evaluation means determines that the discriminant result of the first classification discriminating means may be misdiscrimination based on the weighted average value of the first ratio and the second ratio based on a predetermined weight. Determine the value to be shown,
The erroneous discrimination possibility evaluation device according to claim 1 or 2. The target image acquisition step to acquire the target image that is the discrimination target of the classification, and
A first classification determination step that determines any one of a plurality of given classifications as the first classification most likely to be the classification of the target image.
In the target image, the probability that an object of the first category appears is lower than the first probability, and the probability that an object of a category different from the first category appears among the plurality of categories is higher than the second probability. The ratio specifying step of specifying the ratio of the size of the region to the total size of the target image, and
Based on the ratio, the erroneous discrimination possibility evaluation step of determining a value indicating the possibility that the discrimination result in the first classification discrimination step is erroneous discrimination, and
A method for evaluating the possibility of misidentification, which comprises. Target image acquisition procedure to acquire the target image that is the discrimination target of classification,
A first classification determination procedure for determining any of a given plurality of classifications as the first classification most likely to be the classification of the target image.
In the target image, the probability that an object of the first category appears is lower than the first probability, and the probability that an object of a category different from the first category appears among the plurality of categories is higher than the second probability. Probability specifying procedure for specifying the ratio of the size of the region to the total size of the target image,
An erroneous discrimination possibility evaluation procedure for determining a value indicating the possibility that the discrimination result in the first classification discrimination procedure is erroneous discrimination based on the ratio.
A program characterized by having a computer execute.

Images