JP7351186B2 - Analysis equipment, analysis program and analysis method - Google Patents

Description

The present invention relates to an analysis device, an analysis program, and an analysis method.

In recent years, analysis techniques have been proposed for analyzing the cause of erroneous inference when an erroneous label is inferred in image recognition processing using CNN (Convolutional Neural Network). An example is the score maximization method (activation maximization). Furthermore, an analysis technique has been proposed for analyzing image parts that are of interest during inference in image recognition processing. Examples include the BP (Back Propagation) method and the GBP (Guided Back Propagation) method.

The score maximization method is a method in which a changed part when an input image is changed so that the correct label for inference has the maximum score is identified as an image part that causes incorrect inference. Furthermore, the BP method and the GBP method are methods for visualizing characteristic parts that reacted during inference by backpropagating from the inferred label and tracing it to the input image.

JP2018-097807A Japanese Patent Application Publication No. 2018-045350 Ramprasaath R. Selvariju, et al.: Grad-cam: Visual explanations from deep networks via gradient-based localization. The IEEE International Conference on Computer Vision (ICCV), pp. 618-626, 2017.

However, in the case of the above-mentioned analysis technique, there is a problem in that it is not possible to specify with sufficient accuracy the image location that causes the incorrect inference.

One aspect of the invention is to improve accuracy in identifying image locations that cause erroneous inferences.

According to one aspect, the analysis device includes:
Refinement that generates a refined image that maximizes the score of the correct label of inference from the incorrect inference image by using the difference between the incorrect inference image in which an incorrect label is inferred during image recognition processing and the refined image. an image generation unit;
a first map indicating pixels of the plurality of pixels of the incorrect inference image that have been changed when generating the refined image whose score is maximized ; and a plurality of pixels of the refined image whose score is maximized. By superimposing a second map that shows the degree of attention of each pixel that was focused on at the time of inference and that is adjusted based on the frequency of appearance of each degree of attention, each pixel for inferring the correct label is a map generation unit that generates a third map indicating the importance of the
In the erroneous inference image, a specifying unit that identifies a set of pixels that causes an erroneous inference by calculating a pixel value of the third map for each set of pixels ,
The refined image generation unit includes:
When the set of pixels is identified by the identification unit, the difference is corrected for the area of the identified set of pixels, and the correct label of the inference is determined from the incorrect inference image using the corrected difference. Regenerate a refined image that maximizes the score .

It is possible to improve the accuracy in identifying image locations that cause incorrect inferences.

FIG. 2 is a diagram showing an example of a functional configuration of an analysis device. FIG. 2 is a diagram showing an example of a hardware configuration of an analysis device. FIG. 2 is a first diagram showing an example of the functional configuration of an erroneous inference cause extraction unit. FIG. 7 is a diagram illustrating a specific example of processing by a refined image generation unit. FIG. 6 is a diagram illustrating a specific example of processing by a map generation unit. FIG. 3 is a diagram showing details of the functional configuration of an important feature map generation section. FIG. 7 is a diagram illustrating a specific example of processing by a selective back-error propagation unit. FIG. 7 is a diagram illustrating processing contents of a non-target pixel offset adjustment section. FIG. 7 is a diagram illustrating a specific example of processing by a non-target pixel offset adjustment unit. FIG. 7 is a diagram illustrating a specific example of processing by a superpixel dividing section. FIG. 7 is a diagram illustrating a specific example of processing by an important superpixel determining unit. FIG. 7 is a diagram illustrating a specific example of processing by a region extracting unit and a combining unit. It is a 1st flowchart which shows the flow of incorrect inference cause extraction processing. It is a 2nd flowchart which shows the flow of incorrect inference cause extraction processing. 3 is a flowchart showing the flow of important feature map generation processing. FIG. 2 is a first diagram illustrating a specific example of incorrect inference cause extraction processing. FIG. 2 is a second diagram illustrating an example of the functional configuration of an erroneous inference cause extraction unit. FIG. 7 is a second diagram showing a specific example of the incorrect inference cause extraction process. FIG. 7 is a third diagram illustrating an example of the functional configuration of an erroneous inference cause extraction unit. FIG. 7 is a third diagram showing a specific example of the incorrect inference cause extraction process. FIG. 4 is a fourth diagram showing an example of the functional configuration of the incorrect inference cause extraction unit. FIG. 2 is a first diagram showing an example of a functional configuration of a detailed cause analysis section. FIG. 2 is a first diagram showing a specific example of processing by a detailed cause analysis unit. It is a 1st flowchart which shows the flow of detailed cause analysis processing. FIG. 5 is a fifth diagram showing an example of the functional configuration of an incorrect inference cause extraction unit. FIG. 2 is a second diagram showing an example of the functional configuration of a detailed cause analysis section. FIG. 7 is a second diagram showing a specific example of processing by the detailed cause analysis unit. It is a 2nd flowchart which shows the flow of detailed cause analysis processing.

Each embodiment will be described below with reference to the accompanying drawings. Note that, in this specification and the drawings, components having substantially the same functional configuration are designated by the same reference numerals, thereby omitting redundant explanation.

[First embodiment]
<Functional configuration of analysis device>
First, the functional configuration of the analysis device according to the first embodiment will be described. FIG. 1 is a diagram showing an example of the functional configuration of an analysis device. An analysis program is installed in the analysis device 100, and by executing the program, the analysis device 100 functions as an inference section 110, an erroneous inference image extraction section 120, and an erroneous inference cause extraction section 140.

The inference unit 110 performs image recognition processing using the trained CNN. Specifically, upon receiving the input image 10, the inference unit 110 infers a label indicating the type of object (in this embodiment, the type of vehicle) included in the input image 10, and uses the inferred label. Output.

The incorrect inference image extraction unit 120 determines whether a label (known) indicating the type of object included in the input image 10 matches the label inferred by the inference unit 110. In addition, the incorrect inference image extraction unit 120 extracts the input image when it is determined that there is no match (when an incorrect label is inferred) as an “incorrect inference image” and stores it in the incorrect inference image storage unit 130. .

The erroneous inference cause extraction unit 140 identifies an image part that causes an erroneous inference in the erroneous inference image, and outputs erroneous inference cause information. Specifically, the incorrect inference cause extraction unit 140 includes a refined image generation unit 141, a map generation unit 142, and a specification unit 143.

The refined image generation unit 141 reads out the incorrect inference image stored in the incorrect inference image storage unit 130. Further, the refined image generation unit 141 generates a score-maximizing refined image in which the score of the correct inference label is maximized from the read incorrect inference image.

The map generation unit 142 generates a map that identifies areas that influence label inference using known analysis techniques that analyze the causes of erroneous inference.

The specifying unit 143 replaces a region of the incorrect inference image that is included in the generated map and that affects label inference with the generated refined image. Further, the specifying unit 143 infers a label by inputting the incorrect inference image in which the region is replaced with a refined image, and determines the effect of the replacement from the score of the inferred label.

Further, the specifying unit 143 infers a label by inputting data while changing the size of a region that affects inference of a label, and specifies an image location that causes incorrect inference from the score of the inferred label. Further, the specifying unit 143 outputs the identified image portion that causes the erroneous inference as erroneous inference cause information.

In this way, when replacing an area that affects label inference with a refined image, by referring to the effect of the replacement, it is possible to accurately identify the image location that causes incorrect inference.

<Hardware configuration of analysis device>
Next, the hardware configuration of the analysis device 100 will be explained. FIG. 2 is a diagram showing an example of the hardware configuration of the analysis device. As shown in FIG. 2, the analysis device 100 includes a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, and a RAM (Random Access Memory) 203. CPU201, ROM202, and RAM203 form what is called a computer.

The analysis device 100 also includes an auxiliary storage device 204, a display device 205, an operating device 206, an I/F (Interface) device 207, and a drive device 208. Note that each piece of hardware in the analysis device 100 is interconnected via a bus 209.

The CPU 201 is a calculation device that executes various programs (for example, an analysis program, etc.) installed in the auxiliary storage device 204. Note that although not shown in FIG. 2, an accelerator (for example, a GPU (Graphics Processing Unit), etc.) may be combined as a calculation device.

ROM202 is a nonvolatile memory. The ROM 202 functions as a main storage device that stores various programs, data, etc. necessary for the CPU 201 to execute various programs installed in the auxiliary storage device 204 . Specifically, the ROM 202 functions as a main storage device that stores boot programs such as BIOS (Basic Input/Output System) and EFI (Extensible Firmware Interface).

The RAM 203 is a volatile memory such as DRAM (Dynamic Random Access Memory) or SRAM (Static Random Access Memory). The RAM 203 functions as a main storage device that provides a work area in which various programs installed in the auxiliary storage device 204 are expanded when executed by the CPU 201 .

The auxiliary storage device 204 is an auxiliary storage device that stores various programs and information used when the various programs are executed. For example, the incorrect inference image storage unit 130 is implemented in the auxiliary storage device 204.

The display device 205 is a display device that displays various display screens including erroneous inference cause information and the like. The operating device 206 is an input device through which a user of the analysis device 100 inputs various instructions to the analysis device 100.

The I/F device 207 is, for example, a communication device for connecting to a network (not shown).

The drive device 208 is a device for setting the recording medium 210. The recording medium 210 here includes a medium for recording information optically, electrically, or magnetically, such as a CD-ROM, a flexible disk, or a magneto-optical disk. Further, the recording medium 210 may include a semiconductor memory or the like that electrically records information, such as a ROM or a flash memory.

The various programs to be installed in the auxiliary storage device 204 can be installed by, for example, setting the distributed recording medium 210 in the drive device 208 and reading out the various programs recorded on the recording medium 210 by the drive device 208. be done. Alternatively, various programs to be installed in the auxiliary storage device 204 may be installed by being downloaded from a network (not shown).

<Functional configuration of incorrect inference cause extraction unit>
Next, the details of the functional configuration of the incorrect inference cause extraction unit 140 among the functions implemented in the analysis device 100 according to the first embodiment will be described. FIG. 3 is a first diagram showing an example of the functional configuration of the incorrect inference cause extraction unit. The details of each part (refined image generation unit 141, map generation unit 142, identification unit 143) of the incorrect inference cause extraction unit 140 will be described below.

(1) Details of refined image generation section First, details of the refined image generation section 141 will be explained. As shown in FIG. 3, the refined image generation unit 141 includes an image refiner unit 301, an image error calculation unit 302, an inference unit 303, and a score error calculation unit 304.

The image refiner unit 301 generates a refined image from the incorrectly inferred image using, for example, CNN as an image generation model.

Note that, when inference is made using the generated refined image, the image refiner unit 301 changes the incorrect inference image so that the score of the correct label is maximized. In addition, in the image refiner unit 301, for example, when generating a refined image using an image generation model, the image refiner 301 uses Generate a refined image. Thereby, the image refiner unit 301 can obtain an image (refined image) that is visually similar to the image before change (erroneous inference image).

Specifically, when the image refiner unit 301 uses CNN as an image generation model,
・Score error, which is the error between the score when inferring using the generated refined image and the score that maximizes the score of the correct answer label,
・An image difference value (for example, image difference (L1 difference), SSIM (Structural Similarity), or a combination thereof) that is the difference between the generated refined image and the incorrectly inferred image,
The CNN is trained to minimize .

The image error calculation unit 302 calculates the difference between the incorrect inference image and the refined image output from the image refiner unit 301 during learning, and inputs the image difference value to the image refiner unit 301. The image error calculation unit 302 calculates an image difference value by, for example, performing a pixel-by-pixel difference (L1 difference) or SSIM (Structural Similarity) calculation, and inputs it to the image refiner unit 301.

The inference unit 303 uses a trained CNN that performs inference using the refined image generated by the image refiner unit 301 or the composite image generated by the important superpixel determining unit 322 (described later) as input, and outputs the score of the inferred label. have A synthesized image is an erroneous inference image in which a region of the erroneous inference image that is included in the map (important feature index map) generated by the map generation unit 142 and that affects the inference of the correct label is replaced with a refined image. No.

Note that the score output by the inference section 303 is notified to the score error calculation section 304 or the important superpixel evaluation section 323.

The score error calculation unit 304 calculates the error between the score notified by the inference unit 303 and the score that maximizes the score of the correct label, and notifies the image refiner unit 301 of the score error. The score error notified by the score error calculation unit 304 is used for CNN learning in the image refiner unit 301.

Note that the refined image output from the image refiner unit 301 during learning of the CNN included in the image refiner unit 301 is stored in the refined image storage unit 305. Learning of the CNN included in the image refiner unit 301 is as follows:
- A predetermined number of learning times (for example, maximum number of learning times = N times), or
・Until the score of the correct label exceeds a predetermined threshold, or
・Until the score of the correct label exceeds the predetermined threshold and the image difference value becomes smaller than the predetermined threshold,
The refined image obtained when the score of the correct label output from the inference unit 303 is maximized is hereinafter referred to as a "score-maximized refined image."

(2) Details of Map Generation Unit Next, details of the map generation unit 142 will be explained. As shown in FIG. 3, the map generation section 142 includes an important feature map generation section 311, a deterioration measure map generation section 312, and a superimposition section 313.

The important feature map generation unit 311 acquires inference unit structure information from the inference unit 303 when inference is made using the score-maximized refined image as input. In addition, the important feature map generation unit 311 generates a “grayscale important feature map” based on the inference unit structure information by using the BP (Back Propagation) method, the GBP (Guided Back Propagation) method, or the selective BP method. generate. The grayscale important feature map is an example of the second map, and is a grayscale map showing the degree of attention of each pixel of the plurality of pixels of the score-maximizing refined image that was focused upon during inference.

Note that in the BP method, the error of each label is calculated from the classification probability obtained by performing inference on input data for which the inferred label is correct (in this case, the score maximization refined image), and the error is back-propagated to the input layer. This is a method of visualizing characteristic parts by creating an image of the magnitude of the obtained gradient. Further, the GBP method is a method of visualizing a characteristic part by imaging only positive values of gradient information as a characteristic part.

Furthermore, the selective BP method is a method in which processing is performed using the BP method or the GBP method after maximizing only the error of the correct label. In the case of the selective BP method, the visualized feature part is a feature part that only affects the score of the correct label.

The deterioration measure map generation unit 312 generates a "deterioration measure map" which is an example of a first map, based on the incorrect inference image and the score maximization refined image. The deterioration scale map shows the degree of change of each pixel that was changed when generating the score-maximizing refined image.

The superimposition unit 313 generates an example of a third map based on the grayscale important feature map generated in the important feature map generation unit 311 and the deterioration measure map generated in the deterioration measure map generation unit 312. "Important feature index map" is generated. The important feature index map indicates the importance of each pixel for inferring the correct label.

(3) Details of Specification Unit Next, details of the specification unit 143 will be explained. As shown in FIG. 3, the identification unit 143 includes a superpixel division unit 321, an important superpixel determination unit 322, and an important superpixel evaluation unit 323.

The superpixel dividing unit 321 divides the incorrectly inferred image into "superpixels" which are regions (sets of pixels) for each part of the object (vehicle in this embodiment) included in the incorrectly inferred image, and divides the incorrectly inferred image into "superpixels", Output. Note that to divide the incorrectly inferred image into superpixels, an existing division function is used, or a CNN or the like that has been trained to divide the image into parts of the vehicle is used.

The important superpixel determining unit 322 adds the values of each pixel of the important feature index map generated by the superimposing unit 313 for each superpixel based on the superpixel division information output by the superpixel division unit 321.

In addition, the important superpixel determination unit 322 extracts, from each superpixel, a superpixel whose summed value of each pixel satisfies a predetermined condition (greater than or equal to the important feature index threshold). Further, the important superpixel determining unit 322 defines a superpixel group that is a combination of superpixels selected from the extracted superpixels as a changeable region (a region to be replaced by the score-maximizing refined image). Furthermore, the important superpixel determining unit 322 defines superpixel groups other than the combined superpixel group as unchangeable areas (areas that cannot be replaced by the score-maximizing refined image).

Furthermore, the important superpixel determination unit 322 extracts an image part corresponding to the unchangeable area from the incorrect inference image, extracts an image part corresponding to the changeable area from the refined image, and combines the two. Generate a composite image. The image refiner unit 301 outputs refined images whose number corresponds to the number of times of learning and which satisfies the output conditions of the image refiner unit 301 . Therefore, the important superpixel determining unit 322 generates a composite image for each of the number of refined images.

Note that the important superpixel determination unit 322 increases the number of superpixels to be extracted by gradually lowering the important feature index threshold used to define the changeable area and the unchangeable area (expands the changeable area, (Narrowing down the areas that are not allowed) Further, the important superpixel determining unit 322 updates the changeable area and the unchangeable area while changing the combination of superpixels selected from the extracted superpixels.

The important superpixel evaluation unit 323 obtains the score of the correct label that is inferred every time the composite image generated by the important superpixel determination unit 322 is input to the inference unit 303. As described above, the important superpixel determining unit 322 generates a number of composite images according to the number of refined images output by the image refiner unit 301, the number of times the important feature index threshold is lowered, and the number of superpixel combinations. do. Therefore, the important superpixel evaluation unit 323 obtains the score of the correct label according to the number. Further, the important superpixel evaluation unit 323 identifies a combination of superpixels (changeable region) that causes an erroneous inference based on the obtained score, and outputs it as erroneous inference cause information.

At this time, the important superpixel evaluation unit 323 specifies the changeable area so that the area is as small as possible. For example, when evaluating the score obtained from the inference unit 303, the important superpixel evaluation unit 323 prioritizes and evaluates the superpixels or combinations of superpixels that have the smallest area before lowering the important feature index threshold. . In addition, in the important super pixel evaluation unit 323, by lowering the important feature index threshold, the changeable region (extracted by the important feature index threshold at the limit at which the correct label can be inferred) is determined at the time when the correct label can be inferred. (changeable area with the smallest area).

<Specific examples of processing of each part of the incorrect inference cause extraction unit>
Next, a specific example of the processing of each part of the incorrect inference cause extraction unit 140 will be described.

(1) Specific example of processing by refined image generation unit First, a specific example of processing by the refined image generation unit 141 will be described. FIG. 4 is a diagram showing a specific example of processing by the refined image generation section. The example on the left side of FIG. 4 shows how, as a result of inference using an incorrect inference image 410 that includes a vehicle with the correct label=“car type A” as input, the incorrect inference is made that the label=“car type B”.

In addition, in the example on the left side of FIG. 4, the score when inference is made using the incorrect inference image 410 as input is
・Score of car type A = 0.0142,
・Score of car type B = 0.4549,
・Score of car type C = 0.0018,
It shows that it was.

On the other hand, the example on the right side of FIG. 4 shows how the refined image generation unit 141 performs processing to generate a refined image from the incorrect inference image 410 and generates a score-maximizing refined image 420. In the example on the right side of FIG. 4, the refined image generation unit 141 generates the score-maximizing refined image 420 by changing the color of the headlights 421 and the color of the road markings 422 with respect to the incorrect inference image 410. It is shown that.

Further, the example on the right side of FIG. 4 shows that when inference is made using the score maximization refined image 420 as input, a label that matches the correct label = "car type A" can be inferred. Furthermore, in the example on the right side of FIG. 4, the score when inferred using the score maximization refined image 420 as input is
・Score of car type A = 0.9927,
・Score of car type B = 0.0042,
・Score of car type C = 0.0022,
It shows that it was.

In this way, the refined image generation unit 141 can infer a label that matches the correct label by changing the incorrect inference image 410, and generates the score-maximizing refined image 420 in which the correct label has the maximum score. can be generated.

Note that, as shown in the example on the right side of FIG. 4, in the case of the score maximization refined image 420 generated by the refined image generation unit 141, even road markings unrelated to the vehicle may be changed with respect to the incorrect inference image 410. There is sex. Error backpropagation in learning that maximizes the score of the correct label affects the CNN path (unit) that affects the score of the correct label, but the affected path (unit) is not necessarily the cause of incorrect inference. This is because it is not necessarily related to

In other words, when trying to identify image parts that cause incorrect inferences based on the changed parts, as in the known score maximization method, there is a problem that it is not possible to identify with sufficient accuracy (for the changed parts, (further narrowing down is necessary). In the erroneous inference cause extraction unit 140 according to the present embodiment, the map generation unit 142 and the identification unit 143 function to perform further narrowing down.

(2) Specific example of processing of map generation unit (2-1) Specific example of processing of entire map generation unit Next, a specific example of processing of map generation unit 142 will be described. First, a specific example of the overall processing of the map generation unit 142 will be described. FIG. 5 is a diagram illustrating a specific example of processing by the map generation unit.

As shown in FIG. 5, in the map generation unit 142, the important feature map generation unit 311 acquires from the inference unit 303 inference unit structure information 501 when the inference unit 303 infers using the score-maximized refined image as input. Further, the important feature map generation unit 311 generates an important feature map using, for example, the selective BP method based on the acquired inference unit structure information 501, and performs offset adjustment on the generated important feature map. Note that the details of these processes by the important feature map generation unit 311 will be described later.

Further, the important feature map generation unit 311 converts the offset-adjusted important feature map into a gray scale to generate a gray scale important feature map 502.

The gray scaled important feature map 502 shown in FIG. 5 is gray scaled with pixel values from 0 to 255. Therefore, in the grayscale important feature map 502, pixels with pixel values close to 255 are pixels that are noticed during inference (pixels of interest), and pixels whose pixel values are close to 0 are pixels that are not noticed during inference (non-target pixels). pixel of interest).

On the other hand, the deterioration scale map generation unit 312 reads the score-maximizing refined image 512 from the refined image storage unit 305 and performs SSIM (Structural Similarity) calculation between it and the incorrect inference image 511. Thereby, the deterioration measure map generation unit 312 generates the deterioration measure map 513. The deterioration scale map 513 takes values from 0 to 1, and the closer the pixel value is to 1, the smaller the image difference is, and the closer the pixel value is to 0, the larger the image difference.

Further, the superimposition unit 313 uses the grayscale important feature map 502 generated by the important feature map generation unit 311 and the deterioration measure map 513 generated by the deterioration measure map generation unit 312 to create an important feature index map 520. generate.

Specifically, the superimposition unit 313 generates the important feature index map 520 based on the following formula.
(Formula 1)
Important feature index map = grayscale important feature map × (1 - deterioration scale map)
In the above equation, the term (1-degradation scale map) takes a value from 0 to 1, and the closer it is to 1, the larger the image difference is, and the closer it is to 0, the smaller the image difference. Therefore, the important feature index map 520 is an image obtained by adding strength to a grayscale important feature map that indicates the degree of attention of a pixel that is noticed during inference among a plurality of pixels of the score-maximizing refined image, based on the magnitude of the image difference. becomes.

Specifically, the important feature index map 520 is
- For parts of the deterioration scale map 513 where the image difference is small, reduce the pixel value of the grayscale important feature map,
・Increasing the pixel value of the grayscale important feature map for the portion where the image difference is large in the deterioration scale map 513,
It is generated by

Note that the important feature index map may be inverted to make it easier to identify and visualize. The important feature index map shown in FIG. 5 is displayed inverted based on the following formula.
(Formula 2)
(Inverted) important feature index map = 255 - [grayscaled important feature map x (1 - deterioration scale map)]
Here, the advantage of superimposing the gray scaled important feature map 502 and the deterioration measure map 513 by the superimposing unit 313 based on the above equation will be explained.

The gray scaled important feature map 502 generated by the important feature map generation unit 311 is a portion of interest calculated by the inference unit 303 when inference is made using the score-maximized refined image as input, and the score of the correct label becomes the maximum. Nothing but.

On the other hand, the deterioration scale map 513 generated by the deterioration scale map generation unit 312 represents the changed part when the incorrect inference image is changed so that the score of the correct label is maximized, and represents the area that causes the incorrect inference. represents. However, the deterioration measure map 513 generated by the deterioration measure map generation unit 312 is not the minimum area necessary to maximize the score of the correct label.

The superimposition unit 313 maximizes the score of the correct label by superimposing the changed part when changing the incorrect inference image and the part of interest calculated by the inference unit 303 so as to maximize the score of the correct label. Narrow down the necessary areas. This makes it possible to narrow down the area that causes the incorrect inference.

(2-2) Specific example of processing of important feature map generation unit Next, further details of the important feature map generation unit 311 of the map generation unit 142 will be explained using FIG. 6 with reference to FIGS. 7 to 9. explain. FIG. 6 is a diagram showing details of the functional configuration of the important feature map generation section.

As shown in FIG. 6, the important feature map generation section 311 includes a selective back error propagation section 611, a non-target pixel offset adjustment section 612, and a gray scale conversion section 613.

The selective back-propagation unit 611 is an example of a generation unit, and acquires the inference unit structure information 501 from the inference unit 303 and generates the important feature map 601 using the selective BP method. FIG. 7 is a diagram illustrating a specific example of processing by the selective back error propagation unit. As shown in FIG. 7, the selective backpropagation unit 611 performs selective backpropagation on the correct label inferred by the inference unit 303 under the constraint that the inference error is not zero. As a result, the selective back error propagation unit 611 generates an important feature map 601. Note that when generating the important feature map 601, the output of the selective back error propagation unit 611 may be used as is, or the absolute value may be used.

The non-target pixel offset adjustment unit 612 is an example of an adjustment unit, and performs offset adjustment so that the pixel value of the non-target pixel becomes zero in the generated important feature map 601. Here, in the important feature map 601 generated by the selective back error propagation unit 611, the pixel value of the non-target pixel may be calculated as zero or non-zero (inference unit 303 (depending on the type of CNN used).

Therefore, the non-target pixel offset adjustment unit 612 performs offset adjustment so that the pixel value of the non-target pixel included in the important feature map 601 becomes zero. Here, the non-attention pixel offset adjustment unit 612 performs offset adjustment on a pixel having a pixel value that has the maximum frequency of occurrence of each pixel value included in the important feature map 601 as a non-attention pixel.

FIG. 8 is a diagram showing the processing contents of the non-target pixel offset adjustment section. As shown in FIG. 8A, when the non-target pixel offset adjustment unit 612 obtains the important feature map 601 from the selective back error propagation unit 611, it generates a histogram indicating the appearance frequency of each pixel value. Subsequently, the non-target pixel offset adjustment unit 612 performs scaling processing so that the minimum value becomes "0" and the maximum value becomes "255" (FIG. 8(b)). Note that when visualizing or applying the signal output of the selective back error propagation unit 611 while maintaining it, a method that does not perform scaling processing may be used. In the following description, unless otherwise specified, scaling will be performed.

Next, in the case where the absolute value of the output of the selective back error propagation unit 611 is used, the non-target pixel offset adjustment unit 612 searches for the pixel value with the maximum frequency of occurrence (FIG. 8(c)). , offset adjustment is performed so that the pixel value with the maximum frequency of appearance becomes zero (FIG. 8(d)).

Subsequently, the non-target pixel offset adjustment unit 612 calculates the absolute value of the pixel value (FIG. 8(e)). As a result, the non-target pixel offset adjustment unit 612 generates an offset important feature map 602 (see FIG. 6).

FIG. 9 is a diagram illustrating a specific example of processing by the non-target pixel offset adjustment section. Of these, FIG. 9A shows a case in which the pixel value of a non-target pixel is calculated as zero in the important feature map generated by the selective back-error propagation unit 611. On the other hand, FIG. 9B shows a case in which the pixel value of a non-target pixel is calculated as non-zero in the important feature map generated by the selective back-error propagation unit 611. Adjusting non-attention pixels with an offset is used, for example, when filtering information or adjusting signal strength according to the degree of attention as described above. This is because by acting on it, it is possible to process it into the intended information. When used for known visualization, it is possible to implicitly read that the unfocused part is some pixel value, but for example, when processed by a computer, it is not handled implicitly. Therefore, offset adjustment as described above is required.

Note that, as described above, which case occurs depends on the type of CNN used in the inference unit 303. Therefore, each offset amount is, for example, an image consisting of uniform pixel values (an image without features (that is, an image consisting of non-target pixels), such as images 901 shown in FIGS. 9(a) and 9(b)). It can be calculated based on Specifically, the inference unit structure information 501 obtained when the image 901 is input to the inference units 303 using different types of CNNs is acquired, and an important feature map is generated using the selective BP method. , the offset amount can be calculated.

FIG. 9A shows a case where the offset amount is "0", and in this case, the non-target pixel offset adjustment unit 612 does not perform offset adjustment (important feature map indicated by reference numeral 902 = offset important feature map). On the other hand, FIG. 9B shows a case where the offset amount is "128", and in this case, the non-target pixel offset adjustment unit 612 performs offset adjustment (important feature map indicated by reference numeral 912→offset indicated by reference numeral 913). important feature map).

Returning to the explanation of FIG. 6. The gray scale conversion unit 613 performs gray scale processing on the offset important feature map 602 whose offset has been adjusted by the non-target pixel offset adjustment unit 612 to generate a gray scale important feature map 502. Specifically, when the offset important feature map 602 is in color, grayscale processing is performed to generate the grayscale important feature map 502. Note that it is also possible to separate each RGB channel, treat each channel as a gray scale, and generate the gray scale important feature map 502 for each channel. However, if the offset important feature map 602 is not in color, the grayscale conversion unit 613 does not perform grayscale processing.

(3) Specific example of processing by specifying unit Next, a specific example of processing by each unit of the specifying unit 143 (superpixel dividing unit 321, important superpixel determining unit 322, important superpixel evaluating unit 323) will be described.

(3-1) Specific Example of Processing of Super Pixel Dividing Unit First, a specific example of processing of the super pixel dividing unit 321 included in the specifying unit 143 will be described. FIG. 10 is a diagram showing a specific example of processing by the superpixel dividing section. As shown in FIG. 10, the superpixel dividing section 321 includes a dividing section 1010 that performs, for example, SLIC (Simple Linear Iterative Clustering) processing. The dividing unit 1010 divides the incorrect inference image 511 into superpixels that are partial images for each vehicle part included in the incorrect inference image 511. Further, the superpixel division unit 321 outputs superpixel division information 1001 generated by division into superpixels by the division unit 1010.

(3-2) Specific example of processing by important super pixel determining unit Next, a specific example of processing by important super pixel determining unit 322 included in identifying unit 143 will be described. FIG. 11 is a diagram showing a specific example of the processing of the important superpixel determining unit.

As shown in FIG. 11, the important superpixel determining section 322 includes a region extracting section 1110 and a combining section 1111.

The important superpixel determining section 322 superimposes the important feature index map 520 output from the superimposing section 313 and the superpixel division information 1001 output from the superpixel division section 321. As a result, the important superpixel determining unit 322 generates the important superpixel image 1101. Note that in FIG. 11, an (inverted) important feature index map is displayed as the important feature index map 520.

Further, the important superpixel determination unit 322 adds the values of each pixel of the important feature index map 520 for each superpixel in the generated important superpixel image 1101.

In addition, the important superpixel determining unit 322 determines whether the added value of each superpixel is equal to or greater than the important feature index threshold, and extracts superpixels for which the added value is determined to be equal to or greater than the important feature index threshold. Note that in FIG. 11, an important superpixel image 1102 clearly shows an example of the added value for each superpixel.

Furthermore, the important superpixel determining unit 322 combines selected superpixels from among the extracted superpixels to define a changeable area, and defines superpixels other than the combined superpixels as an unchangeable area. Further, the important superpixel determining unit 322 notifies the area extracting unit 1110 of the defined changeable area and unchangeable area.

The region extraction unit 1110 extracts an image portion corresponding to an unchangeable region from the incorrect inference image 511, and extracts an image portion corresponding to a changeable region from the refined image 1121.

The combining unit 1111 combines the image portion corresponding to the changeable area extracted from the refined image 1121 and the image portion corresponding to the unchangeable area extracted from the incorrect inference image 511 to generate a combined image.

FIG. 12 is a diagram illustrating a specific example of processing by the region extracting unit and the combining unit. In FIG. 12, the upper row shows how the region extraction unit 1110 extracts an image portion (white portion of the image 1201) corresponding to the changeable region from the refined image 1121.

On the other hand, in FIG. 12, the lower part shows how the region extracting unit 1110 extracts the image portion (white portion of the image 1201') corresponding to the unchangeable region from the incorrect inference image 511. Note that the image 1201' is an image obtained by inverting the white part and the black part of the image 1201 (for convenience of explanation, in the lower part of FIG. 12, the white part is the image part corresponding to the unchangeable area).

As shown in FIG. 12, the combining unit 1111 combines the image portion corresponding to the changeable area of the refined image 1121 and the image portion corresponding to the unchangeable area of the incorrect inference image 511 output from the area extraction unit 1110. The images are combined to generate a composite image 1220.

In this way, according to the specifying unit 143, when generating the composite image 1220, by using the important feature index map 520, the area to be replaced with the refined image 1121 can be specified in units of super pixels.

<Flow of incorrect inference cause extraction process>
Next, the flow of the erroneous inference cause extraction process by the erroneous inference cause extraction unit 140 will be described. 13 and 14 are first and second flowcharts showing the flow of the incorrect inference cause extraction process.

In step S1301, each part of the incorrect inference cause extraction unit 140 performs initialization processing. Specifically, the image refiner unit 301 sets the number of learning times of the CNN to zero, and sets the maximum number of learning times to a value specified by the user. Further, the determining unit 325 sets the important feature index threshold and its lower limit to values specified by the user.

In step S1302, the image refiner unit 301 changes the incorrect inference image and generates a refined image.

In step S1303, the inference unit 303 performs inference using the refined image as input, and calculates the score of the correct label.

In step S1304, the image refiner unit 301 performs CNN learning using the image difference value and the score error.

In step S1305, the image refiner unit 301 determines whether the number of learning times exceeds the maximum number of learning times. If it is determined in step S1305 that the number of learning times does not exceed the maximum number of learning times (No in step S1305), the process returns to step S1302 and continues generating refined images.

On the other hand, if it is determined in step S1305 that the number of times of learning exceeds the maximum number of times of learning (if YES in step S1305), the process advances to step S1306. Note that, at this point, the refined image storage unit 305 stores the refined images (including the score-maximized refined image) generated for each learning.

In step S1306, the important feature map generation unit 311 acquires the inference unit structure information when inference is made using the score maximization refined image as input from the inference unit 303, and converts the important feature map into gray scale based on the acquired inference unit structure information. Generate a feature map (details will be described later).

In step S1307, the deterioration measure map generation unit 312 generates a deterioration measure map based on the incorrect inference image and the score-maximizing refined image.

In step 1308, the superimposition unit 313 generates an important feature index map based on the grayscaled important feature map and the deterioration measure map.

In step S1309, the superpixel division unit 321 divides the incorrectly inferred image into superpixels and generates superpixel division information.

In step S1310, the important superpixel determination unit 322 adds the values of each pixel of the important feature index map in units of superpixels.

In step S1311, the important superpixel determination unit 322 extracts superpixels whose added value is greater than or equal to the important feature index threshold, and defines a changeable region by combining the superpixels selected from the extracted superpixels. Furthermore, the important superpixel determination unit 322 defines superpixels other than the combined superpixels as unchangeable areas.

Subsequently, in step S1401 in FIG. 14, the important superpixel determination unit 322 reads the refined image from the refined image storage unit 305.

In step S1402, the important superpixel determining unit 322 extracts an image portion corresponding to the changeable region from the refined image.

In step S1403, the important superpixel determining unit 322 extracts an image portion corresponding to the unchangeable region from the incorrectly inferred image.

In step S1404, the important superpixel determination unit 322 combines the image portion corresponding to the changeable area extracted from the refined image and the image portion corresponding to the unchangeable area extracted from the incorrect inference image, and generates a composite image. do.

In step S1405, the inference unit 303 performs inference using the composite image as input, and calculates the score of the correct label. Further, the important superpixel evaluation unit 323 obtains the score of the correct label calculated by the inference unit 303.

In step S1407, the image refiner unit 301 determines whether all refined images stored in the refined image storage unit 305 have been read out. If it is determined in step S1407 that there is a refined image that has not been read out (No in step S1407), the process returns to step S1401.

On the other hand, if it is determined in step S1407 that all refined images have been read out (in the case of Yes in step S1407), the process advances to step S1408.

In step S1408, the important superpixel determining unit 322 determines whether the important feature index threshold has reached the lower limit. If it is determined in step S1408 that the lower limit has not been reached (No in step S1408), the process advances to step S1409.

In step S1409, the important superpixel determining unit 322 lowers the important feature index threshold, and then returns to step S1311 in FIG. 13.

On the other hand, if it is determined in step S1408 that the lower limit value has been reached (in the case of Yes in step S1408), the process advances to step S1410.

In step S1410, the important superpixel evaluation unit 323 identifies a superpixel combination (changeable region) that causes incorrect inference based on the score of the acquired correct label, and outputs it as incorrect inference cause information.

<Flow of important feature map generation process>
Next, details of the important feature map generation process in step S1306 in FIG. 13 will be described. FIG. 15 is a flowchart showing the flow of important feature map generation processing.

In step S1501, the selective back error propagation unit 611 obtains inference unit structure information from the inference unit 303.

In step S1502, the selective backpropagation unit 611 generates an important feature map using the selective BP method for the inference unit structure information.

In step S1503, the non-target pixel offset adjustment unit 612 generates a histogram based on the important feature map, and identifies the pixel value with the maximum frequency of appearance.

In step S1504, the non-target pixel offset adjustment unit 612 calculates the offset amount based on the pixel value with the highest frequency of appearance.

In step S1505, the non-target pixel offset adjustment unit 612 performs offset adjustment of the important feature map based on the calculated offset amount, and generates an offset important feature map.

In step S1506, the grayscale conversion unit 613 performs grayscale processing on the offset important feature map to generate a grayscale important feature map.

<Specific example of incorrect inference cause extraction process>
Next, a specific example of the incorrect inference cause extraction process will be described. FIG. 16 is a first diagram showing a specific example of the incorrect inference cause extraction process.

As shown in FIG. 16, first, when the refined image generation unit 141 generates a score-maximizing refined image from the incorrect inference image, the map generation unit 142 generates an important feature index map.

Subsequently, when the superpixel division section 321 generates superpixel division information based on the incorrect inference image, the important superpixel determination section 322 generates an important superpixel image.

Subsequently, the important superpixel determination unit 322 defines a changeable area and an unchangeable area in the important superpixel image based on the important feature index map. At this time, the important superpixel determination unit 322 changes the important feature index threshold and also changes the combination selected from among the superpixels exceeding the important feature index threshold, thereby forming a combination of a plurality of changeable regions and non-changeable regions. generate. Furthermore, the important superpixel determining unit 322 generates a composite image using the generated sets of the plurality of changeable areas and non-changeable areas.

Subsequently, the important superpixel evaluation unit 323 receives the generated composite image as input and obtains the score of the correct label inferred by the inference unit 303. Thereby, the important superpixel evaluation unit 323 identifies the superpixel combination (changeable region) that causes the incorrect inference based on the score of the acquired correct label, and outputs it as incorrect inference cause information.

Note that the composite image generation process may be performed for each of the plurality of refined images generated by the refined image generation unit 141, as shown in FIG.
・You can perform this on the last refined image, or
- It may be performed on the refined image (score-maximizing refined image) with the highest score of the correct label in inference using each refined image as input.

As is clear from the above description, the analysis device 100 according to the first embodiment maximizes the score of the correct label of inference from the incorrect inference image from which an incorrect label is inferred during image recognition processing. Generate a score-maximizing refined image.

Furthermore, the analysis device 100 according to the first embodiment selects non-target pixels at the time of inference among the plurality of pixels of the score-maximizing refined image based on the inference unit structure information when the score-maximizing refined image is generated. Generate a grayscale important feature map whose pixel values are zero.

In addition, the analysis device 100 according to the first embodiment also detects degradation that indicates the degree of change in pixels that have been changed when generating the score-maximizing refined image, based on the difference between the score-maximizing refined image and the incorrect inference image. Generate a scale map.

Furthermore, the analysis device 100 according to the first embodiment superimposes the grayscaled important feature map and the deterioration scale map to obtain an important feature index that indicates the importance of each pixel for inferring a correct label. Generate a map.

Furthermore, the analysis device 100 according to the first embodiment generates superpixels by dividing the incorrect inference image, and adds each pixel value of the important feature index map in units of superpixels. Furthermore, the analysis device 100 according to the first embodiment extracts superpixels whose added value is equal to or greater than an important feature index threshold, and determines a changeable region based on a combination of superpixels selected from the extracted superpixels. A non-changeable area is defined.

Furthermore, the analysis device 100 according to the first embodiment infers a correct label by inputting an incorrect inference image in which the defined changeable region is replaced with a refined image to the inference unit. Furthermore, the analysis device 100 according to the first embodiment performs inference while changing the important feature index threshold and the combination of selected superpixels, and from each score of the inferred correct label, the combination of superpixels that causes incorrect inference ( (areas that can be changed).

In this way, in the first embodiment, superpixels that affect the inference of correct labels are replaced with refined images, and the superpixels are identified while referring to the effect of the replacement, thereby identifying image locations that cause incorrect inferences. Identify. As a result, according to the first embodiment, it is possible to improve the accuracy in identifying image locations that cause erroneous inferences.

[Second embodiment]
In the first embodiment described above, the important feature index map is generated based on the inference unit structure information when the score-maximizing refined image is generated. On the other hand, in the second embodiment, an important feature index map is generated based on each refined image acquired during learning until the score maximization refined image is generated, and an important feature index map is generated based on the generated important feature index map. Then, an average important feature index map is generated. Then, the important superpixel determining unit 322 extracts superpixels whose value is equal to or greater than the important feature index threshold based on the average important feature index map. The second embodiment will be described below, focusing on the differences from the first embodiment.

<Functional configuration of incorrect inference cause extraction unit>
First, the functional configuration of the erroneous inference cause extraction unit of the analysis device according to the second embodiment will be described. FIG. 17 is a second diagram illustrating an example of the functional configuration of the incorrect inference cause extraction unit. The difference from the functional configuration of the incorrect inference cause extraction unit shown in FIG. 3 is the map generation unit 1710. The details of the map generation unit 1710 will be explained below.

As shown in FIG. 17, the map generation section 1710 includes an averaging section 1711 in addition to an important feature map generation section 311, a deterioration measure map generation section 312, and a superimposition section 313.

The map generation unit 1710 sequentially acquires a refined image generated during learning by the image refiner unit 301 and inference unit structure information when the inference unit 303 performs inference using the refined image as input. Furthermore, each time a refined image and inference unit structure information are acquired, in the map generation unit 1710, the important feature map generation unit 311, the deterioration measure map generation unit 312, and the superimposition unit 313 operate to generate an important feature index map.

The averaging unit 1711 calculates the average value of the multiple important feature index maps generated by the superimposing unit 313 each time the important feature map generating unit 311 and the deterioration scale map generating unit 312 acquire the refined image and the inference part structure information. and generate an average important feature index map.

<Specific example of incorrect inference cause extraction process>
Next, a specific example of the incorrect inference cause extraction process will be described. FIG. 18 is a second diagram showing a specific example of the incorrect inference cause extraction process.

As shown in FIG. 18, in the refined image generation unit 141, when the image refiner unit 301 generates a refined image from the incorrect inference image based on the first learning result, the map generation unit 1710 generates an important feature index map. do.

Further, in the refined image generation unit 141, when the image refiner unit 301 generates a refined image from the incorrectly inferred image based on the second learning result, the map generation unit 1710 generates an important feature index map. Thereafter, similar processing is repeated, and when the refined image generation unit 141 generates a score-maximized refined image, the map generation unit 1710 generates an important feature index map.

Subsequently, the averaging unit 1711 obtains multiple important feature index maps that have been generated between the generation of the first refined image and the generation of the score-maximizing refined image. Furthermore, the averaging unit 1711 calculates the average value of the plurality of acquired important feature index maps to generate an average important feature index map.

Subsequently, the superpixel division section 321 generates superpixel division information based on the incorrectly inferred image, and the important superpixel determination section 322 generates an important superpixel image.

Subsequently, the important superpixel determination unit 322 defines a changeable area and an unchangeable area in the important superpixel image based on the average important feature index map. At this time, the important superpixel determination unit 322 changes the important feature index threshold and also changes the combination selected from among the superpixels exceeding the important feature index threshold, thereby forming a combination of a plurality of changeable regions and non-changeable regions. generate. Furthermore, the important superpixel determining unit 322 generates a composite image using the generated sets of the plurality of changeable areas and non-changeable areas.

Subsequently, the important superpixel evaluation unit 323 receives the generated composite image as input and obtains the score of the correct label inferred by the inference unit 303. Thereby, the important superpixel evaluation unit 323 identifies the superpixel combination (changeable region) that causes the incorrect inference based on the obtained correct label and outputs it as incorrect inference cause information.

In addition, in the above-mentioned incorrect inference cause extraction process, the interval at which refined images are acquired and important feature index maps are generated is arbitrary. An important feature index map may be generated. Alternatively, the score of the correct label of the inference unit 303 upon input of the refined image may be evaluated, and if the score is greater than a predetermined threshold, the refined image may be acquired to generate the important feature index map.

Further, the composite image generation process may be performed for each of the plurality of refined images generated by the refined image generation unit 141, as shown in FIG.
・You can perform this on the last refined image, or
- It may be performed on the refined image (score-maximizing refined image) with the highest score of the correct label in inference using each refined image as input.

As is clear from the above description, the analysis device 100 according to the second embodiment calculates the average important feature index map based on the important feature index map generated during learning up to the generation of the score-maximizing refined image. generate. Furthermore, the analysis device 100 according to the second embodiment extracts superpixels having an important feature index equal to or greater than the threshold based on the average important feature index map.

As a result, according to the second embodiment, in addition to the effects of the first embodiment, it is possible to further reduce the influence of fluctuations in the refined image on the important feature index map.

[Third embodiment]
In the first embodiment, when a score-maximizing refined image is generated and an important feature index map is generated, the specifying unit 143 defines a changeable area and an unchangeable area, and defines an image that may cause erroneous inference. The explanation has been made assuming that the process of identifying a location is started.

Furthermore, in the second embodiment, when a score-maximizing refined image is generated and an average important feature index map is generated, the specifying unit 143 defines a changeable area and an unchangeable area, and defines the cause of erroneous inference. The explanation has been made assuming that the process of identifying an image location where .

On the other hand, in the third embodiment, after the specifying unit 143 defines the changeable area and the unchangeable area, the refine image generation unit takes into account the specified changeable area and maximizes the score again. Regenerate the refined image.

According to the third embodiment, by regenerating the score-maximizing refined image with the changeable region taken into consideration, the important feature index that makes the feature parts that influence the inference of the correct label more clear It becomes possible to generate a map (or an average important feature index map). As a result, it is possible to increase the score when inferring a label using a composite image as input.

The third embodiment will be described below, focusing on the differences from the first or second embodiment.

<Functional configuration of incorrect inference cause extraction unit>
First, the functional configuration of the erroneous inference cause extraction unit of the analysis device according to the third embodiment will be described. FIG. 19 is a third diagram showing an example of the functional configuration of the incorrect inference cause extraction unit. The difference from the functional configuration of the incorrect inference cause extraction unit shown in FIG. 17 is the refined image generation unit 1910 and the identification unit 1920. Details of the refined image generation section 1910 and the identification section 1920 will be described below.

(1) Details of refined image generation section First, details of the refined image generation section 1910 will be explained. As shown in FIG. 19, the refined image generation section 1910 includes an image error calculation section 1911 that has a different function from the image error calculation section 302 of the refined image generation section 141.

Similar to the image error calculation unit 302, the image error calculation unit 1911 calculates the difference between the incorrect inference image input to the image refiner unit 301 during learning and the refined image output from the image refiner unit 301 during learning. , and the image difference values are input to the image refiner unit 301. However, in the case of the image error calculation unit 1911, when inputting the image difference value to the image refiner unit 301, the image difference value is corrected for the image portion corresponding to the changeable area notified by the correction unit 1921.

Specifically, the image error calculation unit 1911 corrects the image difference value of the image portion corresponding to the changeable area by multiplying it by a coefficient less than 1. As a result, the image refiner unit 301 relearns the image difference value of the image part corresponding to the changeable area to be weaker than the image difference value of the image part corresponding to the area other than the changeable area, and then It is possible to regenerate a refined image.

(2) Details of Specification Unit Next, details of the specification unit 1920 will be explained. As shown in FIG. 19, the identification unit 1920 includes a correction unit 1921 in addition to a superpixel division unit 321, an important superpixel determination unit 322, and an important superpixel evaluation unit 323.

The correction unit 1921 acquires the changeable area defined by the important superpixel determination unit 322 and notifies the image error calculation unit 1911. As a result, the refined image generation unit 1910 can regenerate the score-maximizing refined image after re-learning with the changeable area taken into consideration.

<Specific example of incorrect inference cause extraction process>
Next, a specific example of the incorrect inference cause extraction process will be described. FIG. 20 is a third diagram showing a specific example of the incorrect inference cause extraction process. The difference from the specific example of the erroneous inference cause extraction process described using FIG. The point is to notify the image error calculation unit 1911 of the changeable area. As a result, the refined image generation unit 1910 re-generates the score-maximizing refined image after re-learning with the changeable region taken into account, and the map generation unit 1710 regenerates the average important feature index map. can.

In addition, in the above-mentioned incorrect inference cause extraction process, the interval at which refined images are acquired and important feature index maps are generated is arbitrary. An important feature index map may be generated. Alternatively, the score of the correct label of the inference unit 303 upon input of the refined image may be evaluated, and if the score is greater than a predetermined threshold, the refined image may be acquired to generate the important feature index map.

Further, the composite image generation process may be performed for each of the plurality of refined images generated by the refined image generation unit 1910, as shown in FIG.
・You can perform this on the last refined image, or
- It may be performed on the refined image (score-maximizing refined image) with the highest score of the correct label in inference using each refined image as input.

As is clear from the above description, the analysis device 100 according to the third embodiment regenerates the score-maximizing refined image by relearning with the changeable region taken into account, and also regenerates the important feature index map ( or the average important feature index map).

As a result, according to the third embodiment, it is possible to generate an important feature index map (or an average important feature index map) in which the feature parts that influence the inference of the correct answer label are made clearer, and the composite image can be It is possible to increase the score when inferring a label as input.

[Fourth embodiment]
In the first to third embodiments described above, the combination of superpixels (changeable area) that causes erroneous inference is specified and is output as erroneous inference cause information. However, the method of outputting the erroneous inference cause information is not limited to this, and for example, important parts within the changeable area may be visualized and output. The fourth embodiment will be described below, focusing on the differences from the first to third embodiments.

<Functional configuration of incorrect inference cause extraction unit>
First, the functional configuration of the erroneous inference cause extraction unit in the analysis device 100 according to the fourth embodiment will be described. FIG. 21 is a fourth diagram illustrating an example of the functional configuration of the incorrect inference cause extraction unit. The difference from the functional configuration of the incorrect inference cause extraction unit 140 shown in FIG. 3 is that it includes a detailed cause analysis unit 2110.

The detailed cause analysis unit 2110 uses the incorrect inference image and the score maximization refinement image to visualize important parts within the changeable region and outputs it as an action result image.

<Functional configuration of detailed cause analysis section>
Next, the functional configuration of detailed cause analysis section 2110 will be explained. FIG. 22 is a first diagram showing an example of the functional configuration of the detailed cause analysis section. As shown in FIG. 22, the detailed cause analysis unit 2110 includes an image difference calculation unit 2201, an SSIM calculation unit 2202, a cutting unit 2203, and an action unit 2204.

The image difference calculation unit 2201 calculates the difference in pixel units between the incorrect inference image and the score-maximizing refined image, and outputs a difference image.

The SSIM calculation unit 2202 outputs an SSIM image by performing SSIM calculation using the incorrect inference image and the score maximization refined image.

The cutting unit 2203 cuts out an image portion corresponding to the changeable area from the difference image. Furthermore, the cutting unit 2203 cuts out an image portion corresponding to the changeable area from the SSIM image. Furthermore, the cutout unit 2203 multiplies the SSIM image by the difference image, which has been cut out from the image portion corresponding to the changeable area, to generate a multiplied image.

The action unit 2204 generates an action result image based on the incorrect inference image and the multiplication image.

<Specific example of processing by the detailed cause analysis unit>
Next, a specific example of processing by the detailed cause analysis unit 2110 will be described. FIG. 23 is a diagram showing a specific example of processing by the detailed cause analysis unit.

As shown in FIG. 23, first, the image difference calculation unit 2201 calculates the difference (=(A)-(B)) between the incorrect inference image (A) and the score-maximizing refined image (B), and is output. The difference image is pixel correction information at a portion of the image that causes incorrect inference.

Next, the SSIM calculation unit 2202 performs SSIM calculation based on the incorrect inference image (A) and the score maximization refined image (B) (y=SSIM((A), (B)). In the calculation unit 2202, the result of the SSIM calculation is inverted (y' = 255 - (y x 255)) to output an SSIM image. This is an image specified by , and a large pixel value indicates a large difference, and a small pixel value indicates a small difference.The process of inverting the result of SSIM calculation is, for example, by calculating y' = 1 - y. This can also be done by doing this.

Subsequently, the cutout unit 2203 cuts out an image portion corresponding to the changeable area from the difference image, and outputs a cutout image (C). Similarly, the cutout unit 2203 cuts out an image portion corresponding to the changeable area from the SSIM image, and outputs a cutout image (D).

Here, the changeable area is a specified area of the image part that causes incorrect inference, and the detailed cause analysis unit 2110 further performs cause analysis at pixel granularity within the specified area. It is an object.

Therefore, the cropping unit 2203 multiplies the cropped image (C) and the cropped image (D) to generate a multiplied image (G). The multiplied image (G) is nothing but pixel correction information in which pixel correction information at a portion of the image that causes erroneous inference is specified with higher precision.

Furthermore, the cutting unit 2203 performs emphasis processing on the multiplied image (G) and outputs an emphasized multiplied image (H). Note that the cutout unit 2203 calculates the emphasized multiplied image (H) based on the following formula.
(Formula 3)
Enhanced multiplication image (H) = 255 × (G) / (max (G) - min (G))
Subsequently, the action unit 2204 visualizes important parts by subtracting the emphasized multiplication image (H) from the incorrect inference image (A), and generates an action result image.

Note that the emphasizing process shown in FIG. 23 is only an example, and the emphasizing process may be performed using any other method as long as it makes it easier to identify important parts when visualized.

<Detailed cause analysis process flow>
Next, the flow of detailed cause analysis processing by the detailed cause analysis unit 2110 will be described. FIG. 24 is a first flowchart showing the flow of detailed cause analysis processing.

In step S2401, the image difference calculation unit 2201 calculates a difference image between the incorrect inference image and the score-maximizing refined image.

In step S2402, the SSIM calculation unit 2202 calculates an SSIM image based on the incorrect inference image and the score maximization refined image.

In step S2403, the cutting unit 2203 cuts out a difference image corresponding to the changeable area.

In step S2404, the cutting unit 2203 cuts out the SSIM image corresponding to the changeable area.

In step S2405, the cutout unit 2203 multiplies the cutout difference image and the cutout SSIM image to generate a multiplied image.

In step S2406, the cutting unit 2203 performs emphasis processing on the multiplied image. Further, the effecting unit 2204 subtracts the emphasized multiplication image from the incorrect inference image, and outputs an effect result image.

As is clear from the above description, the analysis device 100 according to the fourth embodiment generates a difference image and an SSIM image based on an incorrect inference image and a score maximization refined image, and cuts out a changeable region. Multiply by Thereby, according to the analysis device 100 according to the fourth embodiment, it is possible to visually recognize image locations that cause incorrect inferences in pixel units within the changeable region.

[Fifth embodiment]
In the fourth embodiment described above, a case will be described in which image parts that cause incorrect inference are visualized pixel by pixel using a difference image generated based on an incorrect inference image and a score-maximizing refined image and an SSIM image. did.

In contrast, in the fifth embodiment, a grayscale important feature map is further used to visualize image locations that cause erroneous inferences on a pixel-by-pixel basis. The fifth embodiment will be described below, focusing on the differences from the fourth embodiment.

<Functional configuration of incorrect inference cause extraction unit>
First, the functional configuration of the erroneous inference cause extraction unit in the analysis device 100 according to the fifth embodiment will be described. FIG. 25 is a fifth diagram showing an example of the functional configuration of the incorrect inference cause extraction unit. The difference from the functional configuration of the incorrect inference cause extraction unit shown in FIG. 21 is that in the case of FIG. 25, the function of the detailed cause analysis unit 2510 is different from the function of the detailed cause analysis unit 2110 shown in FIG. , the detailed cause analysis unit 2510 acquires inference unit structure information from the inference unit 303.

The detailed cause analysis unit 2510 uses the difference image, SSIM image, and grayscale important feature map generated based on the erroneous inference image, the score-maximizing refined image, and the inference unit structure information to determine the cause of the erroneous inference. Visualize image locations pixel by pixel.

Note that the difference image, SSIM image, and grayscale important feature map used by the detailed cause analysis unit 2510 to visualize image locations that cause erroneous inference in units of pixels have the following attributes.
- Difference image: This is difference information for each pixel, and is information having positive and negative values that indicates how much the pixel should be modified to improve the classification probability of the specified label.
- SSIM image: This is difference information that takes into account changes in the entire image and local regions, and is information that has fewer artifacts (unintended noise) than difference information for each pixel. In other words, it is differential information with higher precision (however, it is information only about positive values).
・Grayscale important feature map: This is a map that visualizes the feature parts that affect the inference of the correct label.

<Functional configuration of detailed cause analysis section>
Next, the functional configuration of detailed cause analysis section 2510 will be explained. FIG. 26 is a second diagram showing an example of the functional configuration of the detailed cause analysis section. The differences from the functional configuration shown in FIG. 22 are that in the case of FIG. 26, an important feature map generation section 2601 is included, and the function of the extraction section 2602 is different from the function of the extraction section 2203 of FIG. 22. .

The important feature map generation unit 2601 obtains inference unit structure information from the inference unit 303 when inference is made using the score maximization refined image as input. Further, the important feature map generation unit 2601 generates a grayscale important feature map based on the inference unit structure information by using the selective BP method.

The cutting unit 2602 cuts out an image portion corresponding to the changeable area from the difference image and the SSIM image, and also cuts out an image portion corresponding to the changeable area from the grayscale important feature map. Further, the cutout unit 2602 multiplies the difference image, the SSIM image, and the grayscale important feature map, which have been cut out from the image portion corresponding to the changeable region, to generate a multiplied image.

In this way, by multiplying the difference image, the SSIM image, and the gray scaled important feature map, it becomes possible to visually recognize image locations that cause erroneous inferences on a pixel-by-pixel basis in the action result image.

Note that by using the difference image during multiplication, the action result image is automatically modified to an image that increases the score of the correct label. Therefore, the difference image may be output as the action result image. Furthermore, if such advantages are not taken into consideration, the detailed cause analysis unit 2510 may perform multiplication using the SSIM image and the grayscaled important feature map (without using the difference image) and output an effect result image. You may.

<Specific example of processing by the detailed cause analysis unit>
Next, a specific example of processing by the detailed cause analysis unit 2510 will be described. FIG. 27 is a second diagram showing a specific example of processing by the detailed cause analysis unit. The difference from the specific example of the processing of the detailed cause analysis unit 2110 in FIG. 23 is that the important feature map generation unit 2601 generates an important feature map using the selective BP method based on the inference unit structure information (I). The point is that generation processing is performed to generate a grayscale important feature map. Another point is that the clipping unit 2602 clips the image portion corresponding to the changeable region from the grayscale important feature map and outputs the clipped image (J). Furthermore, the cutout unit 2602 multiplies the cutout image (C), the cutout image (D), and the cutout image (J) to generate a multiplied image (G).

<Detailed cause analysis process flow>
Next, the flow of detailed cause analysis processing by the detailed cause analysis unit 2510 will be explained. FIG. 28 is a second flowchart showing the flow of detailed cause analysis processing. The differences from the flowchart shown in FIG. 24 are steps S2801, S2802, and S2803.

In step S2801, the important feature map generation unit 2601 obtains inference unit structure information from the inference unit 303 when a label is inferred using the score-maximized refined image as input. Further, the important feature map generation unit 2601 uses the selective BP method to generate a grayscale important feature map based on the inference unit structure information.

In step S2802, the cutting unit 2602 cuts out an image portion corresponding to the changeable region from the grayscale important feature map.

In step S2803, the cropping unit 2602 generates a multiplied image by multiplying the difference image, the SSIM image, and the grayscale important feature map, which are extracted from the image portion corresponding to the changeable region.

As is clear from the above description, the analysis device 100 according to the fifth embodiment converts the difference image, the SSIM image, and the A feature map is generated, and a changeable region is cut out and multiplied. Thereby, according to the analysis device 100 according to the fifth embodiment, it is possible to visually recognize image locations that cause incorrect inferences in pixel units within the changeable region.

Note that the present invention is not limited to the configurations shown here, such as combinations of other elements with the configurations listed in the above embodiments. These points can be modified without departing from the spirit of the present invention, and can be appropriately determined depending on the application thereof.

100: Analysis device 140: Misinference cause extraction unit 141: Refine image generation unit 142: Map generation unit 143: Specification unit 301: Image refiner unit 302: Image error calculation unit 303: Inference unit 304: Score error calculation unit 311: Important Feature map generation unit 312 : Deterioration scale map generation unit 313 : Superposition unit 321 : Super pixel division unit 322 : Important super pixel determination unit 323 : Important super pixel evaluation unit 611 : Selective back error propagation unit 612 : Non-target pixel offset adjustment Section 613: Grayscale section 1110: Area extraction section 1111: Composition section 1710: Map generation section 1711: Averaging section 1910: Refine image generation section 1911: Image error calculation section 1920: Specification section 1921: Correction section 2110: Detailed cause Analysis unit 2201 : Image difference calculation unit 2202 : SSIM calculation unit 2203 : Cutout unit 2204 : Action unit 2510 : Detailed cause analysis unit 2601 : Important feature map generation unit 2602 : Cutout unit

Claims (6)

Refinement that generates a refined image that maximizes the score of the correct label of inference from the incorrect inference image by using the difference between the incorrect inference image in which an incorrect label is inferred during image recognition processing and the refined image. an image generation unit;
a first map indicating pixels of the plurality of pixels of the incorrect inference image that have been changed when generating the refined image whose score is maximized ; and a plurality of pixels of the refined image whose score is maximized. By superimposing a second map that shows the degree of attention of each pixel that was focused on at the time of inference and that is adjusted based on the frequency of appearance of each degree of attention, each pixel for inferring the correct label is a map generation unit that generates a third map indicating the importance of the
In the erroneous inference image, a specifying unit that identifies a set of pixels that causes an erroneous inference by calculating a pixel value of the third map for each set of pixels ,
The refined image generation unit includes:
When the set of pixels is identified by the identification unit, the difference is corrected for the area of the identified set of pixels, and the correct label of the inference is determined from the incorrect inference image using the corrected difference. An analysis device that regenerates a refined image that maximizes the score . The map generation unit identifies the pixel value at which the frequency of appearance of each degree of attention is maximum, and determines the attention of each pixel that was noticed at the time of inference among the plurality of pixels of the refined image so that the identified pixel value becomes zero. The analysis device according to claim 1, which adjusts a map indicating the degree. 2. The map generation unit generates a map indicating the degree of attention of each pixel that is focused upon inference among the plurality of pixels of the refined image using any one of a BP method, a GBP method, or a selective BP method. 2. The analysis device according to 2. an image in which a predetermined area is cut out from a difference image calculated based on the difference between the incorrect inference image and the refined image that maximizes the score ;
an image in which a predetermined region is cut out from an SSIM image obtained by performing SSIM calculation on the incorrect inference image and the refined image that maximizes the score ;
an image cut out from a predetermined area from the second map;
The analysis device according to claim 1, wherein a multiplied image obtained by multiplying is visualized in the incorrect inference image. Generating a refined image that maximizes the score of the correct label of inference from the erroneous inference image using the difference between the erroneous inference image in which an incorrect label is inferred during image recognition processing and the refined image,
a first map indicating pixels of the plurality of pixels of the incorrect inference image that have been changed when generating the refined image whose score is maximized ; and a plurality of pixels of the refined image whose score is maximized. By superimposing a second map that shows the degree of attention of each pixel that was focused on at the time of inference and that is adjusted based on the frequency of appearance of each degree of attention, each pixel for inferring the correct label is generate a third map showing the importance of
causing a computer to perform a process of identifying a set of pixels that causes an erroneous inference by calculating a pixel value of the third map for each set of pixels in the erroneous inference image ;
When the set of pixels is identified, the difference is corrected for the area of the identified set of pixels, and the corrected difference is used to maximize the score of the correct label of the inference from the incorrect inference image. An analysis program that regenerates refined images . Generating a refined image that maximizes the score of the correct label of inference from the erroneous inference image using the difference between the erroneous inference image in which an incorrect label is inferred during image recognition processing and the refined image,
a first map indicating pixels of the plurality of pixels of the incorrect inference image that have been changed when generating the refined image whose score is maximized ; and a plurality of pixels of the refined image whose score is maximized. By superimposing a second map that shows the degree of attention of each pixel that was focused on at the time of inference and that is adjusted based on the frequency of appearance of each degree of attention, each pixel for inferring the correct label is generate a third map showing the importance of
In the erroneous inference image, a computer executes a process of identifying a set of pixels that causes the erroneous inference by calculating a pixel value of the third map for each set of pixels,
When the set of pixels is identified, the difference is corrected for the area of the identified set of pixels, and the corrected difference is used to maximize the score of the correct label of the inference from the incorrect inference image. An analysis method that regenerates refined images .

Images