JP6616536B1 - Visibility evaluation apparatus, visibility evaluation program, and visibility evaluation method - Google Patents

Abstract

A technique for evaluating visibility is provided. One of the representative visibility evaluation apparatuses according to the present invention includes an acquisition unit that acquires an image to be evaluated for visibility as a test image, and a plurality of images obtained by processing the test image with different blur amounts. A processing unit that generates a blurred image of the image, a testing unit that performs an image recognition test on the blurred image and obtains a test result, and obtains a limit blur amount at which image recognition is appropriate based on the test result, And a determination unit that sets a value according to the amount as an evaluation value of visibility (hereinafter, “recognition degree”). [Selection] Figure 1

Description

  The present invention relates to a visibility evaluation device, a visibility evaluation program, and a visibility evaluation method.

  It is known that in the operation of removing a brain tumor such as glioma, the 5-year survival rate of the patient is increased by accurately removing the brain tumor. Therefore, in brain tumor extraction surgery, a test called brain function mapping is performed in order to distinguish between a normal brain region and an abnormal tumor region.

  In this brain function mapping, the operator gives a task related to the brain function of the local site to the patient who is awake while giving a weak electrical stimulus to the local site of the patient's brain surface.

  When the patient can perform the task regardless of the local electrical stimulation, the local site is determined to be a site unrelated to the task (high possibility of a tumor site).

On the other hand, when the patient cannot execute the task due to the local electrical stimulation, it is determined that the local site is a brain site (high possibility of a normal site) necessary for the task that cannot be executed.
One of such tasks is a naming task in which an image card is presented and the name of a picture is answered.

In this naming task, there are some cases where the patient is confused by the image. In that case, it is difficult to immediately determine whether it is a normal site or a tumor site. For this reason, the surgeon took time to map brain functions, such as repeating questions carefully.
For these reasons, in order to improve the certainty and efficiency of brain function mapping, it is important to select an image that can be immediately answered as the image used for the naming task before the operation.

  Non-Patent Document 1 discloses the knowledge that “the more variations of names recalled from images, the more confused which name should be answered and the longer the response time”.

Nishimoto Takehiko “Japanese normative set of 359 pictures” Behavior Research Methods 2005 37 (3) 398-416 (https://link.springer.com/content/pdf/10.3758%2FBF03192709.pdf)

  When an experiment is performed on the naming task, in addition to the image pointed out by Non-Patent Document 1 (an image reminiscent of a plurality of names), there are images that take time to answer.

For example, the image is as follows.
(1) Complex (2) Difficult to capture shape (3) Contrast and color are not clear (4) Excessive information due to extra elements (5) Insufficient information lacking necessary features (6) Not attracting attention (7) Appropriate (8) Deformation (9) Unfamiliar viewpoint and angle

  These images are images that lack ease of visual recognition (hereinafter referred to as “visibility”) such as “easy to see”, “easy to understand”, and “easy to attract eyes”. In the case of an image having low visibility, the subject takes time (or cannot be visually recognized) until the image is visually recognized, and thus it takes time to answer. Therefore, when preparing an image for a naming task, it is preferable to evaluate the high visibility as one of the selection indexes.

  As an image as a visual medium, there are an advertisement, a sign, a pictogram, a signboard, a movie, a photograph, a book, a font, a pictogram, a symbol, a paper, a drawing, an operation screen, and the like. Also in these visual media images, “easy to see”, “easy to understand”, and “easy to attract eyes” are desired. From this point of view, a technique for evaluating the visibility of all images as visual media is desired.

  However, Non-Patent Document 1 described above does not disclose a technique for evaluating the visibility.

  Accordingly, an object of the present invention is to provide a technique for evaluating visibility.

  In order to solve the above problem, one of the representative visibility evaluation apparatuses of the present invention is an acquisition unit that acquires an image to be evaluated for visibility as a test image, and the test image with different blur amounts. A processing unit that generates multiple blurred images by processing, a test unit that performs image recognition tests on the blurred images and obtains test results, and obtains a limit blur amount that makes image recognition valid based on the test results And a determination unit that sets a value corresponding to the limit blur amount as an evaluation value of visibility (hereinafter, “recognition degree”).

According to the present invention, visibility can be evaluated.
Note that problems, configurations, and effects other than those described above will be clarified by the following description of embodiments.

It is a block diagram which shows schematic structure of the visibility evaluation apparatus 100 of Example 1. FIG. 5 is a flowchart illustrating a recognition ease detection process according to the first exemplary embodiment. It is a figure which illustrates the some blurred image G (1) -G (n). It is a figure which illustrates terminal 33 which presents blurred image G (i). 7 is a flowchart showing a setting process of a boundary value BV by a setting unit 45. It is a figure which shows the experimental result of the spatial distribution D. It is a figure which shows a mode that the spatial distribution D was divided into the ranges A and B. It is a flowchart which shows the whole operation | movement of a visibility evaluation apparatus. It is a block diagram which shows schematic structure of the visibility evaluation apparatus 200 of Example 2. FIG. It is a figure explaining the basic composition of the machine learning device 300 of Example 2. FIG. It is a figure which shows a part of training data 400 of the machine learning device 300. 12 is a flowchart illustrating a recognition easy level detection process according to the second embodiment.

  Hereinafter, embodiments of the invention will be described with reference to the drawings.

  Example 1 is an embodiment in which visibility is evaluated using a user test.

(Description of Configuration of Example 1)
FIG. 1 is a block diagram illustrating a schematic configuration of a visibility evaluation apparatus 100 according to the first embodiment.
In the figure, the visibility evaluation apparatus 100 is configured to include functions of an acquisition unit 10, a processing unit 20, a test unit 30, and a determination unit 40.

  The acquisition unit 10 acquires an image to be evaluated for visibility as a test image and transmits the acquired image to the processing unit 20. The acquisition unit 10 can acquire electronic data of a visual medium captured by a user as a test image from an internal camera or an external camera. The acquisition unit 10 can also acquire the test image as an image file, a moving image file, a data file (such as a pdf file, an SVG file, or an HTML file) or other electronic data.

  The processing unit 20 processes the test image with a plurality of different blur amounts to generate two or more blurred images from a slightly blurred image to an image that is so blurred that it cannot be visually recognized. The plurality of blurred images are transmitted to the test unit 30.

  The test unit 30 includes a presentation unit 31 and an input unit 32 in order to perform a user test. The test unit 30 presents a plurality of blurred images to the presentation unit 31 in descending order of the blur amount to the user of the visibility evaluation device 100. The user attempts to visually recognize the presented blurred image (image recognition test). The user inputs the result of the visual recognition (test result) to the input unit 32 of the test unit 30 by operation input, voice input, gesture input, or the like.

  The determination unit 40 obtains the maximum blur amount (hereinafter referred to as “limit blur amount”) for which image recognition is appropriate based on the test result, and sets a value corresponding to the limit as the recognition ease.

  Further, the determination unit 40 evaluates whether or not the test image is “good visibility” by comparing the recognition ease of the test image with the boundary value BV. The determination unit 40 includes a setting unit 45 for setting the boundary value BV. The boundary value BV will be described later.

  The configuration of the visibility evaluation device 100 described above is realized by a dedicated device, and for example, a visibility evaluation program (including an application) is executed on a computer such as a personal computer, a tablet, a smartphone, a mobile terminal, or a camera. It is realized by.

  Further, part or all of the visibility evaluation apparatus 100 may be provided on a server in a network, a cloud, or a data center. In this case, the server authenticates the client terminal (PC, tablet, smartphone, mobile terminal, camera, etc.) connected to the communication and provides the user with a service for evaluating the visibility of the test image via the client terminal. It becomes possible to do.

(Detection of recognition degree in Example 1)
First, the recognition easyness detection process performed by the visibility evaluation apparatus 100 will be described.
FIG. 2 is a flowchart illustrating recognition easy level detection processing according to the first embodiment.
Hereinafter, description will be made along the step numbers shown in FIG.

Step S101: The processing unit 20 performs a blurring process on the test image transmitted from the acquisition unit 10 with the minimum blur amount b (1) to the maximum blur amount b (n), so that n blur images G ( 1) to G (n) are generated. The subscript n is a natural number of 2 or more.

FIG. 3 is a diagram illustrating a plurality of blurred images G (1) to G (n).
In the figure, a blurred image G (1) is an image in which the contour and details of the image to be examined are slightly blurred, and the blurred image G (n) is not visible to the test image. It is a blurred image. The halfway blurred image is an image in which the blurring amount is changed and the blurring process is performed, or the blurring amount is changed depending on the number of times the blurring process is repeated. Incidentally, the main blurred image may be generated by the blur process, and the blur image inserted between the main blur images may be generated by interpolation by the main blur image overlap process.

  The blurring process here may be a blurring process that reduces human visibility. For example, it is possible to reproduce a non-focused state in which human beings are blurred, such as Gaussian blur processing. Further, it is also possible to reproduce a state in which a human line of sight swims or camera shake, such as image flow processing or image fluctuation processing. Furthermore, a process for reducing the spatial resolution, such as a mosaic process, may be used. Moreover, the process which reduces the high frequency spatial frequency component of an image like a spatial low-pass filter may be sufficient. Furthermore, image processing that simulates fogged glass, foggy effect, soft focus effect, lens aberration, color shift processing, image shift multiplexing, noise superimposition, and image distortion may be used. Further, image processing that simulates blurring in a dark place (decrease in visibility), such as lightness reduction processing, processing for crushing gradation in a dark portion, or blackening processing, may be used. Furthermore, image processing that simulates a decrease in visibility with a faint contrast may be used, such as a contrast reduction process or a gradation reduction process. Furthermore, image processing that simulates a reduction in visibility in which colors are blurred, such as color reduction processing or processing for black and white or monochrome conversion to a specific color, may be performed. Further, it may be an image process that simulates a state where the brightness becomes suddenly bright and the visibility is blurred, such as a process of saturation on the high luminance side (such as a whiteout process).

Step S102: Prior to the user test, the test unit 30 substitutes an initial value 0 for the limit blur amount L and substitutes an initial value n for the order i of the blurred images.

Step S103: The test unit 30 presents the blurred image G (i) to the presentation unit 31 for each predetermined time.
As will be described later (step S108), the order i switches from n to 1 in descending order. As a result, the presentation unit 31 is presented with a blurred image G (i) that starts from a blurred image G (n) that cannot be visually recognized in order of the arrows shown in FIG. The

FIG. 4 is a diagram illustrating a terminal 33 that presents the blurred image G (i).
A terminal 33 shown in the figure includes a touch panel screen, and an operation area of the input unit 32 is provided together with a display area of the presentation unit 31. The terminal 33 is also provided with a microphone (not shown) for voice input of the input unit 32 and a light receiving unit or camera (not shown) for gesture input.

Step S104: The user views and approves the blurred image G (i) presented on the presentation unit 31 in the form of a hint quiz (the picture is sequentially shown as a hint and the button is pressed when the answer is known). At this stage, the input unit 32 of the terminal 33 is operated (tap operation, voice input, gesture input, etc.). The operation input to the input unit 32 is transmitted to the test unit 30 as an operation event indicating the intention of “visual approval”.

Step S105: The test unit 30 determines whether or not an operation event “viewing approval” occurs during the presentation period of the blurred image G (i). When the “view authorization” operation event occurs, the test unit 30 shifts the operation to step S106. On the other hand, when the “viewing approval” operation event has not occurred when the presentation period of the blurred image G (i) ends, the test unit 30 shifts the operation to step S107.

Step S106: When the “visual approval” operation event occurs, the test unit 30 substitutes the blur amount b (i) of the blurred image G (i) being presented into the limit blur amount L of the visual approval.
In this user test, the “visual approval” operation event is valid only for the first time, and the substitution process for the limit blur amount L is also performed only for the first time. In that case, since the presentation of the blurred image after the operation event has the meaning of displaying the answer of the quiz, the time required for the user test is shortened by reducing the display interval of the blurred image.

  Note that the user may observe the subsequent clear blurred image G (i), notice that the previous “visual approval” is misunderstanding, and repeat the “visual approval” operation. For a user with many such operations, the substitution process for the limit blur amount L may be an overwrite update process, and the final operation event may be an effective “viewing authorization”.

Step S107: The test unit 30 determines whether or not the presentation of the n blurred images G (n) to G (1) has been completed. If the order i is “1”, the test unit 30 determines that the presentation is complete, and the operation proceeds to step S109. On the other hand, if the order i is greater than “1”, it is determined that the presentation has not been completed, and the operation proceeds to step S108.

Step S108: In order to present the blurred image that becomes clearer in stages, the test unit 30 reduces the order i by one and shifts the operation to step S103.

Step S109: The determination unit 40 sets a value according to the limit blur amount L for viewing approval as the recognition ease.
For example, the determination unit 40 obtains the recognition ease using the following equation.

Ease of recognition R [%] = (limit blur radius) / (pattern radius) × 100
... [1]

The recognizability R shown in the equation [1] is a percentage value obtained by adjusting (normalizing) the size of the limit blur radius [pixel] corresponding to the limit blur amount L with the radius [pixel] of a circle in which the pattern is approximately contained. .
By the series of processes described above, the degree of easy recognition of the test image is obtained.

(Boundary value BV setting process)
Next, processing for setting a boundary value BV indicating a critical boundary for visibility will be described.
FIG. 5 is a flowchart showing the setting process of the boundary value BV by the setting unit 45.
Hereinafter, description will be made along the step numbers shown in FIG.

Step S201: The setting unit 45 collects a plurality of training images for setting the boundary value BV. For the plurality of training images, test images whose visibility has been evaluated in the past by the visibility evaluation apparatus 100 may be used. Moreover, the image newly collected by the development team of the visibility evaluation apparatus 100 may be used, or the image may be collected from the cloud via the Internet (for example, automatically). In addition, about the training image, if the data of the reply time and recognition ease mentioned later are collectable, the collection of the image data of a training image is unnecessary.

Step S202: The setting unit 45 collects answer time (time required for visual recognition), which is an experiment result of the naming task, for each training image.

Step S203: The setting unit 45 collects data on the recognition ease for each training image.

Step S204: The setting unit 45 gives coordinates of (answer time, recognition ease) to each training image, and obtains a spatial distribution D for the group of training images.

FIG. 6 is a diagram illustrating experimental results of the spatial distribution D.
The figure shows experimental results in which training images are plotted with the response time (median value) on the horizontal axis and the ease of recognition (average value) on the vertical axis.

  In the experiment shown in FIG. 6, 32 females in their 20s are subjects, and the response time and the degree of recognition are tested for each training image.

  In addition, when an experiment on the ease of recognition and an experiment on the answer time are performed for one subject on the same training image, an experiment on the later answer time is performed in a state in which the correct answer is known by the previous experiment on the ease of recognition. Therefore, there is a concern that the experimental value may be biased by hindsight.

  Therefore, prior to the experiment, the subjects are grouped into groups Pa and Pb. The training images are also grouped into a training image group Ga and a training image group Gb. And with respect to the test subject of the group Pa, the experiment of the reply time was conducted about the group Ga of the training image, and the experiment of the recognition degree was conducted about the group Gb of the training image. For the subjects of group Pb, an experiment on the answer time was performed for the group Gb of training images, and an experiment on the ease of recognition was performed for the group Ga of training images. By the experiment and the order of the “tacking method”, the deviation of the experimental value due to the hindsight described above was avoided.

As a result of the experiment, a plurality of response times and a plurality of recognition easinesses are obtained in each training image.
Among these, there was a tendency for the experimental value of response time to be extremely delayed in a small number of subjects. In this state, if the average of the response times is a representative value, the extreme delay of a small number of subjects affects the average and biases the representative value. Therefore, the median value (median value) was adopted as a representative value as an experimental result of the response time in order to eliminate the influence of the upper limit side and the lower limit side of the response time.

  On the other hand, as for the degree of recognition, generally stable results were obtained, and no extreme individual differences occurred. Therefore, the average value of the ease of recognition was adopted as the representative value as the experimental result of the degree of recognition.

Step S205: The spatial distribution D created by the data collection of the setting unit 45 shows the characteristics distributed in an L shape as shown in FIG.
The setting unit 45 performs classification processing based on the L-shaped characteristics of the spatial distribution D, and divides the spatial distribution D into the following ranges A and B.
● Recognition range A where the response time is concentrated on the short side
● Recognition range B in which the response time is distributed to the long side
FIG. 7 is a diagram illustrating a state in which the spatial distribution D is divided into ranges A and B.

Step S206: Since the recognition ease ranges A and B have a significant difference in how the response time spreads, the boundary between the ranges A and B is a boundary having critical significance.
Therefore, the setting unit 45 obtains the boundary between the recognition ease ranges A and B and sets the boundary value BV having critical significance.
If the test image is more easily recognized than the boundary value BV, the test image belongs to the range A, so that the response time is estimated to be concentrated on the short side, and it can be evaluated that the visual recognition is good.
The boundary value BV having critical significance obtained from the ranges A and B shown in FIG. 7 is the following value.
Boundary value BV having critical significance = 6.7 [%]
... [2]

Step S207: The setting unit 45 sets the boundary value BV in the determination unit 40 (updates the boundary value BV as needed if the visibility evaluation apparatus 100 is in operation).
With the above operation, the visibility evaluation apparatus 100 can use the boundary value BV for evaluating whether the visibility is good.

(Overall operation of the first embodiment)
Subsequently, the overall operation of the first embodiment will be described.

FIG. 8 is a flowchart showing the overall operation of the visibility evaluation apparatus 100.
Hereinafter, description will be made along the step numbers shown in FIG.

Step S301: The user inputs an image to be evaluated for visibility into the visibility evaluation apparatus 100 as a test image. The acquisition unit 10 transmits the input test image to the processing unit 20.

Step S302: The visibility evaluation device 100 obtains a recognition ease for the test image. The details of this operation have already been described based on the flowchart of FIG.

Step S303: The determination unit 40 returns the recognition ease to the user who has input the test image as the visibility evaluation value.

Step S304: When the determination unit 40 can consider that the recognition degree of the test image is higher than the boundary value BV, the operation proceeds to step S305. In other cases, the determination unit 40 completes the evaluation of the visibility of the test image.

Step S305: Here, since the ease of recognition of the test image can be considered to be higher than the boundary value BV, the test image belongs to the “recognition range A where the answer time is short (see FIG. 7)”. It is estimated to be. Therefore, the determination unit 40 outputs evaluation information indicating that “the visibility of the test image is good” to the user who has input the test image.
With the above operation, the visibility evaluation for the test image is completed.

(Effect of Example 1)
(1) In brain function mapping, the subject is fixed on the skull and lies on the operating table. In this state, the subject's field of vision is easily blurred because he / she becomes sleepy, cannot keep his gaze. Therefore, the inventor of the present application introduces “recognition ease” as an index indicating the blur of the field of view that enables the recognition of the test image. An image having a high degree of recognition is an image that can be easily recognized even if the field of view is blurred. The visibility evaluation apparatus 100 of Example 1 calculates | requires this recognition ease about a test image. Therefore, it becomes possible to evaluate the visibility of the test image.

(2) In the first embodiment, the degree of blur in the field of view is equivalently simulated by varying the amount of blur applied to the test image in a plurality of stages. Usually, the degree of visual blur is a qualitative parameter relating to vision, and it is difficult to quantify it as it is. However, in the first embodiment, the degree of blurring of the field of view is simulated by the blurring amount of the image. This blur amount is a numerical value in image processing. Therefore, the ease of recognition makes it possible to evaluate the visibility based on the numerical value of the blur amount.

(3) A test image with a high degree of recognition is an image that is easy to enter the eye and easily grasped at a glance even if the gaze cannot be maintained and the field of view is not fixed. From this point of view, the degree of recognition is an index for evaluating “easy to see”, “easy to understand”, and “easy to attract eyes” in general images as visual media. Therefore, by making it easy to recognize visual images in general (advertising, signs, pictograms, signs, videos, photos, books, fonts, pictograms, symbols, paper, drawings, operation screens, etc.) It becomes possible to evaluate “easy to understand” and “easy to attract eyes”.

(4) In the first embodiment, the boundary value BV is used to determine the recognition ease. As shown in FIG. 7, this boundary value BV is significantly divided into “recognition range A that concentrates on the side where the answer time is short” and “recognition range B that spreads to the side where the answer time is long”. Value (value having critical significance). Therefore, if the degree of easy recognition of the test image is higher than the boundary value BV, the response time of the test image is likely to be short, and it is possible to determine that the visibility is good. .

(5) In Example 1, the terminal 33 is provided as a user interface. A plurality of blurred images G (n) to G (1) are sequentially switched and presented on the presentation unit 31 of the terminal 33 in descending order of the blur amount. By observing the blurred image that becomes clearer in stages, the user can clearly perceive the stage at which the visual recognition has become possible. For this reason, it is possible to clearly obtain the degree of recognition.

(6) In the first embodiment, the terminal 33 further includes the input unit 32. The input unit 32 acquires the result of the visual test from the user. Therefore, there is no need for an experimental witness to listen to the intention display from the user, and it is possible for even one user to obtain the degree of recognition.

(7) In the evaluation of visibility, a short response time can also be a criterion. However, the answer time user test measures the answer time from seeing the image to be examined until answering the name in units of a comma, and corresponds to the “quick press quiz” format. Therefore, the central nervous system, motor nerve, concentration, and test environment factors greatly influence the test subject, and even if the same test subject tests the same test image, the response time varies every time and the reproducibility is low. Also, once the test subject memorizes the test image and its name, the response time is shortened in a conditional manner. Therefore, when the same subject conducts a test of the answer time for the same test image, it is necessary to take time such as changing the date.
For these reasons, “response time” is often inconvenient as an index for evaluating visibility.

On the other hand, the user test of the ease of recognition uses the blurred image that becomes clearer in stages as a hint, and operates the input unit 32 at the stage when the user is approved for viewing. It corresponds to. Therefore, the user can calmly operate the input unit 32 within the presentation period of one blurred image. Therefore, factors such as the subject's central nerves, motor nerves, concentration, and test environment do not affect so much, and even if the same subject tests on the same test image, recognition is highly reproducible. . Moreover, even if a test subject memorizes a test image once, the degree of recognition is not answered in a condition-reflective manner, but is answered while considering what the blurred image looks like. Therefore, even if the user test is repeated without changing the time, the recognition degree does not change relatively and the reproducibility is high.
For these reasons, “recognition ease” is excellent as an index for objectively evaluating visibility.

  Example 2 is an embodiment in which visibility is evaluated using a machine learning device instead of a user (subject).

(Description of Configuration of Example 2)
FIG. 9 is a block diagram illustrating a schematic configuration of the visibility evaluation apparatus 200 according to the second embodiment.
In the figure, the visibility evaluation apparatus 200 is configured to include functions of an acquisition unit 10, a processing unit 20, a test unit 50, and a determination unit 60. The test unit 50 includes a machine learning device 300.

  In addition, since it is the same as that of Example 1 (refer FIG. 1) except the test part 50 and the determination part 60, duplication description here is abbreviate | omitted.

(Basic configuration of machine learning device 300)
FIG. 10 is a diagram for explaining the basic configuration of the machine learning device 300.
In the figure, video information such as a blurred image is input to the input layer 310 after being subjected to preprocessing (crop, resizing, padding before convolution, etc.) suitable for the analysis content of the image analysis.

  The video information of the input layer 310 is processed through the convolution layer / activation function 321 and the pooling layer 322, and converted into a feature map made up of neurons activated in accordance with local features of the image space. The convolution layer / activation function 321 and the pooling layer 322 are multi-layered by the number of layers suitable for image analysis.

  The neuron values of the feature map via the multi-layered convolution layer / activation function 321 and pooling layer 322 are serialized and then input to the neural network 330. The neural network 330 is configured by connecting (for example, fully connecting) neurons with the number of layers suitable for image analysis. An activation pattern indicating the result of image analysis (likelihood) is output from the plurality of neurons in the output layer 340 at the final end of the neural network 330.

  Here, the basic configuration of the machine learning device 300 has been described with one system. However, depending on the content of the image analysis, the basic configuration may be configured with a plurality of systems with parallel or halfway branches. Further, by providing a layer that holds the state of the past neuron in the middle, it may be possible to recognize an image (moving image) in the time axis direction.

(Machine learning of machine learning device 300)
Next, machine learning of the machine learning device 300 will be described.
FIG. 11 is a diagram illustrating a part of the training data 400 of the machine learning device 300.

A training image 410 illustrated in FIG. 11 is an image group of basic parts such as a circle, a rectangle, and a triangle that are highly likely to be included in the test image.
An activation pattern independent for each basic part is assigned in advance as a teacher value 420 to a plurality of neurons constituting the output layer 340.

  The training data 400 is configured by a group of data sets including these training images 410 and teacher values 420.

  Each of the convolutional layer / activation function 321 of the machine learning device 300 and the neurons in the neural network 330 has a weight array W and a bias bias. These weight array W and bias bias are sequentially updated by repeating machine learning (such as the error back propagation method) using the training data 400, and the output layer 340 outputs an activation pattern close to the learned teacher value. become.

(Detection of recognition degree in the second embodiment)
Next, an operation of automatically detecting the degree of easy recognition using the machine learning device 300 instead of the user (subject) will be described.

FIG. 12 is a flowchart illustrating the recognition ease detection process according to the second embodiment.
Hereinafter, description will be made along the step numbers shown in FIG. The operations that are not specifically described are the same as those in the first embodiment.

Step S501: The processing unit 20 performs a blurring process on the test image transmitted from the acquisition unit 10 with the minimum blur amount b (1) to the maximum blur amount b (n), so that n blur images G ( 1) to G (n) are generated. The subscript n is a natural number of 2 or more.

Step S502: The test unit 50 inputs the test image to the input layer 310 of the machine learning device 300. In the output layer 340 of the machine learning device 300, an activation pattern corresponding to a combination of basic parts (machine learning completed) included in the test image appears as a test result of image recognition. The test unit 50 temporarily stores this activation pattern as a correct value for image recognition of the test image.

Step S503: The test unit 50 substitutes an initial value “1” for the order i of the blurred images.

Step S504: The test unit 50 inputs the blurred image G (i) to the input layer 310 of the machine learning device 300. An activation pattern appears in the output layer 340 of the machine learning device 300 according to the combination of basic parts (machine learning completed) remaining in the blurred image G (i). The test unit 50 acquires the activation pattern of the blurred image G (i) as the image recognition test result.

Step S505: The determination unit 60 obtains the distance between the test result and the correct value (difference sum of squares of the pattern sequence of the activation pattern, average of the roots of difference, the square root thereof, the sum of absolute differences, the average of absolute differences, etc.). Note that this distance may be normalized based on the magnitude of the correct answer value.

Step S506: The determination unit 60 determines whether or not the distance between the test result and the correct value exceeds an appropriate range.
The reasonable range here may be set based on the value of “distance” obtained by collecting the blurred images that have been approved at the last minute in the user test.
Alternatively, a change rate of the distance with respect to the blur amount may be obtained, and an appropriate range may be adaptively determined based on a critical point such as an inflection point of the change rate.
When the distance between the test result and the correct answer value exceeds the appropriate range, the determination unit 60 determines that the blurred image G (i) is not visible, and the operation proceeds to step S509.
When the distance between the test result and the correct answer value is within an appropriate range, the determination unit 60 determines that the blurred image G (i) is view authorization, and the operation proceeds to step S507.

Step S507: The test unit 50 increments the order i by one to advance the image recognition target to the next blurred image.

Step S508: The test unit 50 determines whether the image recognition of the n blurred images G (1) to G (n) is completed. When the order i is “n” or less, it is determined that the image recognition is not completed, and the operation proceeds to step S504. On the other hand, if the order i exceeds “n”, the test unit 50 determines that the image recognition is completed, and shifts the operation to step S509.

Step S509: The determination unit 60 determines that the one before the order i is the limit of viewing authorization, and substitutes the blurring amount b (i−1) for the limit blurring amount L of viewing authorization. When i is “1”, the blurring amount b (1-1) = b (0) = 0.

Step S510: The determination unit 60 sets a value corresponding to the limit blur amount L for viewing authorization as the recognition ease.
Through the processing described above, the degree of ease of recognizing the test image is automatically obtained using the machine learning device 300.

(Other operations of the second embodiment)
Other operations in the second embodiment are the same as the operations in the first embodiment (for example, see FIGS. 5 to 8) except that the recognition ease obtained by the machine learning device 300 is used. A duplicate description is omitted.

(Effect of Example 2)
In the second embodiment, the same effect as that of the first embodiment can be obtained by substituting the image recognition of the user with the image recognition of the machine learning device 300.
Furthermore, Example 2 also has the following effects.

(1) In the second embodiment, an image recognition test is performed on a blurred image using the machine learning device 300 that has performed machine learning for image recognition. As a result, it becomes possible to automatically obtain the recognition ease without requiring a user (human subject) as in the first embodiment.

(2) In the second embodiment, the machine learning device 300 is made to learn basic parts that may constitute a test image as a training image. Therefore, even if the image is unlearned (that is, the first appearance) for the machine learning device 300, the machine learning device 300 can output the image recognition test result in the form of the activation pattern of the basic part.

(3) In Example 2, a physical quantity called a distance between the activation pattern of the test image (correct value for image recognition) and the activation pattern of the blurred image (image recognition test result in a state where the field of view is blurred) Newly introduced. This distance indicates a change in image recognition due to a blurred image (view). Therefore, by determining whether or not this distance is within a reasonable range, it is possible to evaluate on behalf of a human subject whether or not the blurred image is approved for viewing.

(Supplementary items of the embodiment)
In the embodiment described above, a convolutional neural network (see FIG. 10) is used as the machine learning device 300. However, the present invention is not limited to this.

  For example, an FCN (Fully Convolutional Network) in which a convolution layer and an activation function are mainly multilayered may be employed as the machine learning device. This FCN machine-learns the basic parts to extract the features of the basic parts included in the input video information (test image and blurred image), and activates the feature map activation pattern according to the combination of the basic parts ( A kind of image information consisting of activation patterns) is output. Therefore, based on the distance between the feature map of the test image and the feature map of the blurred image (difference sum of squares of difference in pixel of feature map, mean square of difference, square root thereof, sum of absolute differences, mean absolute difference, etc.) It is possible to determine whether or not the image recognition of the blurred image is appropriate.

  Also, for example, such machine learners include decision tree learning, association rule learning, neural network, genetic programming, inductive logic programming, support vector machine, clustering, Bayesian network, reinforcement learning, expression learning, principal component analysis, A learning model based on at least one of extreme learning machines and other machine learning techniques may be employed.

  In the above-described embodiment, the maximum blur amount at which image recognition is valid is adopted as the “limit blur amount at which image recognition is valid”. However, the present invention is not limited to this. For example, the previous blur amount at which image recognition is valid can be adopted as the “limit blur amount at which image recognition is valid”. Further, for example, the blurring amount at the moment when the image recognition is no longer valid from the validity can be adopted as the “limit blurring amount at which the image recognition is valid”.

  In the above-described embodiment, Examples 1 and 2 have been described separately, but the present invention is not limited to this. For example, when the first and second embodiments are executed together, the operation parameters of the machine learning device 300 and the determination unit 60 are automatically or manually set so that the recognition ease by the machine learning device 300 approaches the human recognition ease. You may adjust at any time.

  Furthermore, the present invention is not limited to the above-described embodiments, and includes various modifications. For example, the above-described embodiments have been described in detail for easy understanding of the present invention, and are not necessarily limited to those having all the configurations described.

  Further, a part of the configuration of an embodiment can be replaced with the configuration of another embodiment, and the configuration of another embodiment can be added to the configuration of an embodiment.

  Furthermore, it is possible to add, delete, and replace a part of the configuration described above.

  INDUSTRIAL APPLICABILITY The present invention can be used in industrial applications for evaluating the visibility of all images on visual media.

BV ... Boundary value, 10 ... Acquisition unit, 20 ... Processing unit, 30 ... Testing unit, 31 ... Presentation unit, 32 ... Input unit, 33 ... Terminal, 40 ... Determining unit, 45 ... Setting unit, 50 ... Testing unit, 60 DESCRIPTION OF SYMBOLS ... Determination part, 100 ... Visibility evaluation apparatus, 300 ... Machine learning device, 310 ... Input layer, 321 ... Convolution layer and activation function, 322 ... Pooling layer, 330 ... Neural network, 340 ... Output layer, 400 ... Training data , 410 ... training image, 420 ... teacher value

Claims (6)

An acquisition unit for acquiring an image to be evaluated for visibility as a test image;
A processing unit for generating a plurality of blurred images having different blur amounts from the test image;
A test unit that performs an image recognition test on the blurred image and obtains a test result;
A determination unit that obtains a limit blur amount at which the image recognition is appropriate based on the test result, and sets a value according to the limit blur amount as an evaluation value of visibility (hereinafter, “recognition ease”);
A visibility evaluation apparatus comprising: In the visibility evaluation apparatus according to claim 1,
The determination unit
A distribution is collected about the group of training images with “answer time required for visual recognition” and “recognition ease” as axes, and the distribution is defined as “the range A of recognition ease where the answer time is concentrated”. Including a setting unit for setting or updating boundary values of the ranges A and B obtained by classifying into “the range B of the recognition degree that is distributed to the long side of the answer time”,
The visibility evaluation apparatus characterized by determining that the visibility of the test image is good when the recognition ease obtained for the test image is higher than the boundary value. In the visibility evaluation apparatus of any one of Claims 1-2,
The test section is
A presentation unit that presents the plurality of blurred images in descending order of the blur amount;
An input unit for acquiring a visual recognition result from a user presented with the blurred image. In the visibility evaluation apparatus of any one of Claims 1-2,
The test section is
A machine learning device that performs machine learning of the image recognition using a group of data sets including a training image and a teacher value as training data, and performs the image recognition test on the blurred image using the machine learning device. The visibility evaluation apparatus characterized by this. A visibility evaluation program that causes a computer to function as the acquisition unit, the processing unit, the test unit, and the determination unit according to any one of claims 1 to 4. An input step of acquiring an image to be evaluated for visibility as a test image;
Processing steps for generating a plurality of blurred images having different blur amounts from the test image;
A test step of performing an image recognition test on the blurred image and obtaining a test result;
A determination step of obtaining a limit blur amount at which the image recognition is appropriate based on the test result, and setting a value corresponding to the limit blur amount as an evaluation value of visibility (hereinafter, “recognition ease”);
And providing a visibility evaluation service.

Images