WO2019215994A1

WO2019215994A1 - Image determination system, image determination method, and image determination program

Info

Publication number: WO2019215994A1
Application number: PCT/JP2019/006777
Authority: WO
Inventors: 徳寿伊賀
Original assignee: 日本電気株式会社
Priority date: 2018-05-08
Filing date: 2019-02-22
Publication date: 2019-11-14
Also published as: JPWO2019215994A1; JP6969678B2

Abstract

An image determination system 90 is provided with an identification unit 91 and a threshold value setting unit 92. The identification unit 91 identifies the type of image in which the probability of the type outputted by an estimator for estimating the probability of a type of image to be inputted exceeds a threshold value having been set. The threshold value setting unit 92 sets the threshold value in accordance with the number of unidentifiable estimated images the types of which are estimated to be unidentifiable by the identification unit 91 and the number of personnel to whom identification of types of the unidentifiable estimated images can be assigned.

Description

Image determination system, image determination method, and image determination program

The present invention relates to an image determination system, an image determination method, and an image determination program for determining the type of image.

In order to appropriately perform the subsequent processing corresponding to various types of images, it is generally performed to manually check the contents of the images and classify the images into various types. On the other hand, a method for improving the efficiency of manual confirmation and reducing the work cost has been proposed.

Patent Document 1 describes an automatic discrimination method for image types. In the discrimination method described in Patent Document 1, the type of image is broadly classified into a monochrome image and a color image, and further, the monochrome image is converted into a monochrome photograph, a word processor / newspaper, and a tie-point photograph. Subdivide each. In the method described in Patent Document 1, an image with a known image type is read to create a digital image with a known image type, and an upper limit of saturation of the image, a fractal dimension of brightness and saturation are calculated, and automatic A threshold for determination is determined.

JP-A-06-350861

When there are few images to learn or when similar types of images that are difficult to determine are mixed, it is difficult to make a highly accurate determination for all images. In such a situation, even if the type is automatically determined based on the certainty of the type of image data using the determiner, manual reconfirmation is indispensable.

In order to reduce the number of images that cannot be discriminated, it is conceivable to set a low threshold for automatically discriminating the type of image. However, if the threshold value is set too low, there is a high possibility that a determination error will be induced, and there is a possibility that the work for manual confirmation and the work for correction will increase. On the other hand, if the threshold value is set high from the initial stage, there is a high possibility that the image will not be discriminated by any type, and there is a possibility that the work of manually setting the correct answer label may increase.

In the method described in Patent Document 1, the threshold value is uniquely determined according to the nature of the image type. However, if the number of images to be confirmed increases and the number of images that cannot be identified increases, the assumed personnel may not be able to confirm the content of the images, and the overall processing may be incomplete. . For this reason, it is preferable that the reference for automatically determining the image can be adjusted in accordance with the situation of personnel that can be handled while suppressing the accuracy of automatically determining the image from becoming too low.

SUMMARY OF THE INVENTION An object of the present invention is to provide an image determination system, an image determination method, and an image determination program capable of adjusting a reference for determining an image according to the situation of a person while suppressing a decrease in accuracy of determining an image. And

An image determination system according to the present invention includes: a determination unit that determines the type of an image for which the likelihood of the type output by the estimator that estimates the likelihood of the type of the input image exceeds a set threshold; A threshold setting unit to be set, and the threshold setting unit can be assigned to determine the number of indistinguishable estimated images that are estimated to be the type that the discriminating unit cannot discriminate, and the type of the indistinguishable estimated image The threshold value is set according to the number of.

The image determination method according to the present invention determines the type of an image in which the likelihood of the type output by the estimator that estimates the likelihood of the type of the input image exceeds a set threshold, and cannot determine the type. The threshold value is set according to the number of estimated non-discriminable images that are estimated images and the number of personnel that can be assigned to discriminate the type of the indistinguishable estimated image.

An image determination program according to the present invention is a determination process for determining the type of an image in which the likelihood of the type output by the estimator that estimates the likelihood of the type of image input to the computer exceeds a set threshold value, In addition, the threshold setting process for setting the threshold is executed, and in the threshold setting process, the number of indistinguishable estimation images that are estimated to be the type that cannot be discriminated by the discrimination process and the type of the indistinguishable estimation image A threshold value is set according to the number of assignable personnel.

According to the present invention, it is possible to adjust the reference for discriminating the image according to the situation of the personnel while suppressing the reduction of the accuracy of discriminating the image.

It is a block diagram which shows the structural example of 1st embodiment of the image determination system by this invention. It is a flowchart which shows the operation example of the image determination system of 1st embodiment. It is explanatory drawing which shows the specific example of operation | movement of an image determination system. It is a block diagram which shows the structural example of 2nd embodiment of the image determination system by this invention. It is explanatory drawing which shows the example which matched the non-discriminatory ratio for every threshold value. It is a flowchart which shows the operation example of the image determination system of 2nd embodiment. It is a block diagram which shows the outline | summary of the image determination system by this invention.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

Embodiment 1. FIG.
FIG. 1 is a block diagram showing a configuration example of a first embodiment of an image determination system according to the present invention. The image determination system 100 of the present embodiment includes an image input unit 10, an image type estimation unit 20, a discriminator 30, a relearning data storage unit 31, a test data storage unit 32, and a type setting unit 40. The learning device 50, the evaluation unit 60, and the output unit 70 are provided.

The output unit 70 outputs various results by the image determination system. The output unit 70 is realized by, for example, a display device or a printer.

The image input unit 10 inputs an image used by the image type estimation unit 20 to estimate the type. The image input unit 10 may input an image via a communication network, or may read an image stored in a storage unit (not shown) of another system.

The image type estimation unit 20 estimates the likelihood of the type of the input image using a model that estimates the likelihood of each type represented by the image (hereinafter simply referred to as a model or a learned model). The method of expressing the probability of the type is arbitrary, and may be, for example, the probability of the type or the reliability for each type.

* The type setting method used for image classification is arbitrary. The type may be set for each different type of image, or may be set for the same type of image according to the degree of the object represented by the image. Examples of settings for different types of images include different types of forms (resident card, marriage registration, seal certificate, etc.), maps (road maps, topographic maps, etc.), floor plans (residential, office, etc.) ) Is set. In addition, as an example of setting for each image of the same type, there is a case where the setting is made according to the degree of deterioration of the same building or the degree of progression of lesions.

The mode of the model used by the image type estimation unit 20 is arbitrary, for example, a model generated by deep learning. In this case, the image type estimation unit 20 may estimate the probability for each type by applying the input image to the model. For example, when there are four types of types of image A, that is, types A to D, and the type A is most likely, the image type estimation unit 20 sets “type A: 85%” and “type B: 10 for each type. The probability may be estimated as “%”, “Category C: 3%”, or “Category D: 2%”. A method for learning a model used by the image type estimation unit 20 will be described later.

The discriminator 30 discriminates the image type based on the probability of the image type estimated by the image type estimation unit 20. Specifically, the discriminator 30 discriminates image types whose probability estimated by the model used by the image type estimation unit 20 exceeds a threshold value.

A value may be set for each type of threshold value used for determination, or a common value may be set for the entire type. For example, a threshold that is higher than the thresholds of other types may be set for a type that is easy to estimate.

Further, for example, even when a plurality of types are set, it is only necessary to be able to determine a certain type (for example, type X) and other types, or a type that can be discriminated with a very high probability ( For example, there may be a case where type Y is present. In such a case, when only the type (type X, type Y) needs to be discriminated by the discriminator 30, an appropriate threshold value is set for the type, and an emergency threshold is set for other types. A high threshold may be set. In this way, it is possible to extract a type of image that is preferably discriminated manually.

In the first embodiment, the threshold value used by the discriminator 30 is updated by the evaluation unit 60 described later. In the initial stage, a threshold value may be set based on the user's experience or the like or an existing estimated probability. In addition, a threshold that satisfies the estimated probability may be set by estimating the probability of the type of image to which the correct data is assigned using the current model. A method for updating the threshold will be described later.

The discriminator 30 registers the image whose type has been discriminated (that is, the image whose type has been discriminated) in the test data storage unit 32. Further, the discriminator 30 may transmit an image whose type has been discriminated to subsequent processing. For example, the discriminator 30 may cause the output unit 70 to output an image with the type discriminated together with the type, or may notify the image to a processing unit (not shown) that performs subsequent processing corresponding to the type. .

On the other hand, the discriminator 30 registers an image whose type could not be discriminated (hereinafter referred to as an indistinguishable image) in the relearning data storage unit 31 when the type of the image could not be discriminated. The registered image is used in the subsequent type setting unit 40. Note that the discriminator 30 may directly transmit the indistinguishable image to the type setting unit 40 without registering it in the relearning data storage unit 31.

The type setting unit 40 assigns a type that is correct answer data (label) to an image whose type cannot be determined by the classifier 30 (that is, an image that cannot be determined). For example, the type setting unit 40 may display an indistinguishable image for the user and manually add correct data. In addition, since the indistinguishable image to which correct data is assigned is used as teacher data in subsequent relearning processing, it is preferable that correct data is assigned with high accuracy. Further, since the image with the correct answer label can be used as test data, the type setting unit 40 stores this image in the test data storage unit 32.

The learning device 50 learns a model used in the discriminator 30 (that is, a model for estimating the probability for each type represented by the image) using the learning image data. In the initial stage, the learning device 50 may learn a model using, for example, existing image data as learning data.

Furthermore, the learning device 50 re-learns the model by using the data with the correct answer label assigned to the indistinguishable image as learning image data. A non-discriminable image can be said to be an image for which the model has not been able to estimate the type accuracy at a high level, or an image for which the type could not be estimated with the accuracy required by the discriminator 30 (that is, the accuracy of the set threshold). It is possible to improve the accuracy with which the re-learned model estimates the type by generating learning image data to which a correct answer label is assigned to such an image and performing relearning based on the learning image data. It becomes possible.

The method for the relearning by the learning device 50 is arbitrary. For example, if the type does not increase, the learning device 50 may perform additional learning using the learning image data, or the learning data may be an image used for learning in the past and an indistinguishable image provided with a correct answer label. May be used to learn a new model.

At that time, the data used by the learning device 50 for re-learning can also be selected according to the learning method. The learning device 50 may perform relearning of the model using, for example, all the images that cannot be discriminated to which the correct answer label is assigned as learning image data. Further, in order to shorten the learning time and enhance the estimation accuracy for a specific type, the learning device 50 uses only a specific type of image as learning image data among non-distinguishable images assigned with correct labels. Also good.

For example, the estimation accuracy of a type with a low probability of being estimated to be applicable (that is, a type with a higher probability than other types and a low probability even if it is assumed to be applicable to that type) is improved. I want to let you. For example, when there are types A to D, the estimated probability of type D is 51%. In this case, it is considered that the estimation probability itself is low although it is the type that is estimated to be highest when compared with other types. In this case, the learning device 50 may perform relearning using learning image data in which the type (for example, type D) is set as the correct answer label.

In addition, the learning device 50 may perform relearning using image data of the type that is set most (or most frequently) among the images that cannot be discriminated. This is because such types of images are considered to have low estimation accuracy. By performing relearning using such an image, it is possible to generate a model that can accurately estimate the type of a type with low estimation accuracy.

On the other hand, the timing at which the learning device 50 performs relearning is also arbitrary. The learning device 50 may perform relearning after a lapse of a predetermined period, or may perform relearning when a predetermined number of pieces of learning image data have accumulated so that accuracy can be ensured. In addition, the learning device 50 may perform relearning when the ratio of images whose type cannot be determined by the determination device 30 (hereinafter referred to as an undistinguishable ratio) does not satisfy a predetermined criterion. The learning device 50 calculates, for example, the ratio of the number of images whose type could not be determined with respect to the total number of images to be determined as an indistinguishable ratio, and re-enters when the indistinguishable ratio exceeds a predetermined reference. You may learn. This is preferable because, for example, when the tendency of the input image changes, it becomes possible to automatically relearn.

The evaluation unit 60 evaluates the accuracy of the model learned by the learning device 50. Then, the evaluation unit 60 compares the accuracy of the existing model (the model before relearning) with the model after the relearning, and selects a model with higher accuracy. The evaluation unit 60 may automatically replace the selected model as a model used by the image type estimation unit 20, or may replace it according to an instruction from a user or the like.

The evaluation unit 60 may evaluate the accuracy of the model based on a single index, or may evaluate the accuracy of the model by comprehensively judging a plurality of indices. At that time, the evaluation unit 60 may evaluate the accuracy of the model by using both an image whose type is data stored in the test data storage unit 32 and an unidentifiable image. The accuracy of the model may be evaluated using this data. For example, the evaluation unit 60 discriminates an indistinguishable image in parallel between an existing model and a model after re-learning, and the probability of each type is higher in the model after re-learning than in the existing model. In such a case, the model after re-learning may be selected by determining that the accuracy has increased.

Further, the evaluation unit 60 may calculate the number (or rate) at which discrimination is possible as a result of discriminating images determined to be indistinguishable with the existing model with the model after relearning. Then, when the calculated number (or ratio) satisfies a predetermined criterion, the evaluation unit 60 may determine that the model after re-learning has become a more accurate model and select the model. .

In addition, the evaluation unit 60 applies a plurality of test data to both the existing model and the model after relearning, calculates the certainty for each type, and statistically processes the evaluation result for each test data (for example, The average value of the certainty may be calculated, and the standard deviation of the certainty may be calculated). For example, when the average value is calculated as the statistical process, the evaluation unit 60 may select a model having a high probability average value as a more accurate model. For example, when the standard deviation is calculated as the statistical process, the evaluation unit 60 may select a model with a small standard deviation of probability (that is, less evaluation blur) as a more accurate model. Note that the test data used for the statistical processing may be all the past data of the learned type, or may be data of the number selected in advance.

Furthermore, the evaluation unit 60 may update the threshold used by the discriminator 30 when updating the model used by the image type estimation unit 20. For example, the degree of changing the threshold value during the update may be determined in advance, and the evaluation unit 60 may update the threshold value according to the degree. At this time, the predetermined degree may be for each type or may be common throughout.

Besides, as a result of comparing the existing model with the model after re-learning, the evaluation unit 60 may determine the degree to increase the threshold according to the improved accuracy. For example, the change in the past non-discrimination ratio when the threshold value is changed in accordance with the improved accuracy is learned, and the evaluation unit 60 determines from the relationship between the improved accuracy and the improved threshold value and the non-discrimination ratio, The degree to which the threshold value is changed may be determined.

More specifically, if the estimation probability by the image type estimation unit 20 is improved by updating to a highly accurate model, the indistinguishable ratio decreases. On the other hand, if the discrimination threshold used by the discriminator 30 is increased, the discrimination impossible ratio increases. Therefore, the evaluation unit 60 may increase the threshold according to a predetermined allowable discrimination impossible ratio.

The re-learning data storage unit 31 stores indistinguishable images. The relearning data storage unit 31 may store the indistinguishable image in association with the correct label given by the type setting unit 40. Further, the test data storage unit 32 stores the type determined or added in association with the image. The relearning data storage unit 31 and the test data storage unit 32 are realized by, for example, a magnetic disk device.

The image input unit 10, the image type estimation unit 20, the discriminator 30, the type setting unit 40, the learning unit 50, and the evaluation unit 60 are a CPU (Central Processing) of a computer that operates according to a program (image determination program). Realized by Unit IV). For example, the program is stored in a storage unit (not shown) included in the image determination system, and the CPU reads the program, and in accordance with the program, the image input unit 10, the image type estimation unit 20, the discriminator 30, and the type setting unit. 40, the learning device 50, and the evaluation unit 60 may be operated.

The image input unit 10, the image type estimation unit 20, the discriminator 30, the type setting unit 40, the learning device 50, and the evaluation unit 60 may be realized by dedicated hardware, respectively.

Next, the operation of the image determination system of this embodiment will be described. FIG. 2 is a flowchart illustrating an operation example of the image determination system according to the first embodiment. It is assumed that the model used by the image type estimation unit 20 for estimation in the initial stage is learned in advance.

The image input unit 10 inputs an image to be determined (step S11). The image type estimation unit 20 applies a model to the input image, and estimates the likelihood of the type of the input image (step S12). The discriminator 30 discriminates the image type based on the estimated likelihood of the image type (step S13).

If the image type cannot be determined (No in step S13), the type setting unit 40 assigns correct data to the indistinguishable image (step S14) and stores it in the relearning data storage unit 31. Thereafter, the process of step S15 is performed. On the other hand, when the type of the image can be discriminated (Yes in step S13), the discriminator 30 notifies the image to a processor (not shown) that performs subsequent processing corresponding to the type (step S15). The discriminator 30 stores the discriminated type in the test data storage unit 32 in association with the image.

The learning device 50 re-learns the model by using the data to which the correct label is assigned to the indistinguishable image as the learning image data (step S16). The learning device 50 performs relearning at a timing and method according to a predetermined condition. The evaluation unit 60 evaluates the accuracy of the model learned by the learning device 50 and determines whether the accuracy is higher than that of the existing model (step S17). When the accuracy is high (Yes in step S17), the evaluation unit 60 replaces the existing model used for estimation by the image type estimation unit 20 with a model after re-learning (step S18). Further, the evaluation unit 60 updates the threshold used by the discriminator 30 for discrimination (step S19). On the other hand, if the accuracy is not high (No in step S17), the process is terminated with the existing model as it is.

Next, a specific example of the image determination system of this embodiment will be described. FIG. 3 is an explanatory diagram showing a specific example of the operation of the image determination system of the present embodiment. In the example illustrated in FIG. 3, it is assumed that an image to be determined is stored in the image storage device 11. AI (Artificial Intelligence) 21 illustrated in FIG. 3 corresponds to the image type estimation unit 20 and estimates the probabilities of type A, type B, type C, and type D using a model.

Next, the discriminator 30 compares the probability of the estimated type with a set threshold value, and identifies a type of probability that is greater than the threshold value. Here, an image for which a type greater than the threshold (80%) is specified is classified as discriminated 30x, and an image for which no type is specified is classified as indistinguishable 30y. The user sets types A to D via the type setting unit 40 for images classified as indistinguishable 30y (that is, indistinguishable images). This data is used as relearning data.

The image classified into the discriminated 30x and the indistinguishable image in which the type is set are input to the processors Pa to Pd that perform the respective processes, and each process is performed. Further, these images are stored in the storage device 33 for the discriminated image. The storage device 33 corresponds to the test data storage unit 32.

Thereafter, the AI 51 performs relearning of the model using the relearning data, and generates a relearned model. The AI 51 corresponds to the learning device 50 and performs relearning with a timing and method according to a predetermined condition. The relearning model evaluator 61 compares the accuracy of the existing model pm and the model rm after the relearning, and selects a model with higher accuracy. When the model rm after re-learning is selected, the evaluator 61 replaces the model used by the AI 21. In addition, the evaluator 61 changes the threshold used by the discriminator 30. The evaluator 61 corresponds to the evaluation unit 60.

As described above, in this embodiment, the learning device 50 re-learns the model using the indistinguishable image as learning data, and the evaluation unit 60 determines the accuracy of the model before re-learning and the model after re-learning. Compare and select a model with higher accuracy. Therefore, the accuracy of determining an image according to use can be improved. That is, since a model with higher accuracy is selected according to use, it is possible to improve accuracy by determining an image using the model.

Embodiment 2. FIG.
Next, a second embodiment of the image determination system according to the present invention will be described. There is a limit to the man-hours and labor costs that can be put into the image type determination work, and it is desired that the image type can be determined accurately within the limited resources. That is, in a situation where a large number of personnel can be secured, it is preferable that the correct label can be assigned by manually determining the type including not only images that are difficult to determine but also images that are delicately determined.

On the other hand, in a situation where the number of personnel cannot be secured, if a large number of indistinguishable images occur, the task of assigning a correct label to the indistinguishable image cannot be completed. In order to avoid such a situation, it is preferable that the number of images that cannot be discriminated can be reduced by automatically discriminating images that can be discriminated with an acceptable accuracy based on the model.

In the first embodiment, the case where the threshold value is changed according to the model update has been described. In the present embodiment, a method for adjusting a threshold, which is a reference for discriminating an image, according to the situation of personnel that can be secured for assigning a correct answer label to an indistinguishable image.

FIG. 4 is a block diagram showing a configuration example of the second embodiment of the image determination system according to the present invention. The image determination system 200 according to the present embodiment includes an image input unit 10, an image type estimation unit 20, a discriminator 30, a relearning data storage unit 31, a test data storage unit 32, and a type setting unit 40. , A learning device 50, an evaluation unit 60, a threshold setting unit 62, a storage unit 63, and an output unit 70.

That is, the image determination system 200 of the present embodiment further includes a threshold setting unit 62 and a storage unit 63 as compared with the image determination system 100 of the first embodiment. Other configurations are the same as those in the first embodiment.

Similar to the evaluation unit 60 of the first embodiment, the evaluation unit 60 evaluates the accuracy of the model learned by the learning device 50, and determines the accuracy of the existing model (the model before relearning) and the model after the relearning. Compare and select a model with higher accuracy. In the present embodiment, the case where the threshold setting unit 62 updates the threshold used by the discriminator 30 will be described. However, the evaluation unit 60 may update the threshold as in the first embodiment.

The storage unit 63 stores the correspondence between the ratio of the images for which the classifier 30 cannot determine the type (that is, the ratio that cannot be determined) and the threshold value. In the correspondence relationship, each threshold value and the indistinguishable ratio may be associated with each other, and the relationship between the threshold value and the indistinguishable ratio may be expressed by a mathematical expression. In general, the higher the threshold value, the higher the non-discriminatory ratio. Therefore, the function representing the relationship between the threshold and the non-discriminable ratio is represented by an arbitrary monotonous non-decreasing function.

Also, as described above, a common value may be used for the entire type as the threshold value, or a different value may be used for each type. Further, the storage unit 63 may store the correspondence between the indistinguishable ratio and the threshold for each model used by the image type estimation unit 20.

FIG. 5 is an explanatory diagram showing an example in which an indistinguishable ratio is associated with each threshold value. In the example illustrated in FIG. 5, the threshold value and the indistinguishable ratio are associated with each model and type used by the image type estimation unit 20. For example, when the probability of type A is estimated using the model α, if the threshold is set to 80%, it indicates that the input image is determined to be indistinguishable with an estimated probability of 10%.

The storage unit 63 may store a correspondence relationship between the indistinguishable ratio and the threshold according to past experience and the like. Further, the threshold value setting unit 62 described later may calculate the ratio of non-discriminable images from the discrimination result of the actually input image. The operation of the threshold setting unit 62 will be described later. The storage unit 63 is realized by a magnetic disk device, for example.

The threshold setting unit 62 sets a threshold used when the discriminator 30 discriminates the type of image. Specifically, the threshold setting unit 62 can be assigned to determine the number of images (hereinafter, referred to as indistinguishable estimated images) estimated that the discriminator 30 cannot determine the type and the type of the indistinguishable estimated image. The threshold is set according to the number of personnel. It can be said that the number of personnel that can be assigned to the classification determination is the number of personnel that perform the identification confirmation work. Specifically, the correct label is set to the indistinguishable estimation image by this person.

Suppose N (persons) is the number of personnel that can be invested in the period T (hours) during which the discrimination work is performed. Further, the number of images that can be discriminated in one unit period by one person is assumed to be W (time / person / sheet). When the number of images input during the period T is X (sheets), the number Xn (sheets) of indistinguishable estimated images is expressed by the following formula 1, assuming that the indistinguishable ratio is P. .

Xn = X · P (Formula 1)

In addition, since the number of images that can be discriminated during the period T is T · N / W, in order to check all the indistinguishable estimation images, the following formula 2 must be satisfied.

Xn ≦ T · N / W (Formula 2)

Therefore, the threshold setting unit 62 determines a threshold that satisfies the indistinguishable ratio P ≦ T · N / W · Xn. Then, the threshold setting unit 62 notifies the discriminator 30 of the determined threshold, and sets the notified threshold. In the example illustrated in FIG. 5, for example, it is assumed that the classification impossible ratio of type A is calculated as 8%. In this case, the threshold value setting unit 62 determines the threshold value that sets the highest threshold value, 75%, among the threshold values that satisfy the indistinguishable ratio 8%.

Further, the threshold setting unit 62 may calculate a non-discrimination ratio for each threshold set from the discrimination result of the actually input image. Specifically, when the threshold setting unit 62 receives an instruction to calculate the indistinguishable ratio from the user, the threshold setting unit 62 inputs an image for threshold calculation to the image input unit 10 and cannot determine the number of input images while changing the threshold. An indistinguishable ratio is calculated based on the number of images, and the calculation result is registered in the storage unit 63. For example, image data stored in the test data storage unit 32 may be used as the threshold calculation image.

In addition to receiving an explicit calculation instruction from the user, the threshold setting unit 62 changes the situation for estimating the likelihood of the image type (for example, the model used by the image type estimation unit 20 is updated). Depending on the situation, the indistinguishable ratio for each threshold value may be recalculated. That is, the threshold setting unit 62 determines the non-discrimination ratio for each threshold at the timing when the evaluation unit 60 updates the model, in the same manner as when the evaluation unit 60 updates the model at the timing when the evaluation unit 60 updates the model in the first embodiment. May be recalculated and the information in the storage unit 63 may be updated.

The image input unit 10, the image type estimation unit 20, the discriminator 30, the type setting unit 40, the learning device 50, the evaluation unit 60, and the threshold setting unit 62 are according to a program (image determination program). It is realized by a CPU of an operating computer.

Next, the operation of the image determination system of this embodiment will be described. FIG. 6 is a flowchart illustrating an operation example of the image determination system according to the second embodiment. The threshold value setting unit 62 sets the threshold value of the discriminator 30 according to the number of indistinguishable estimated images and the number of personnel that can be assigned to discriminate the type of the indistinguishable estimated images (step S21). The image type estimation unit 20 estimates the likelihood of the type of the input image (step S22). Then, the discriminator 30 discriminates the image type whose probability of the estimated type exceeds the threshold value based on the set threshold value (step S23).

As described above, in the present embodiment, the threshold value setting unit 62 sets the threshold value according to the number of indistinguishable estimation images and the number of personnel that can be assigned to the discrimination of the indistinguishable estimation image type. Then, the discriminator 30 discriminates image types whose probability estimated by the image type estimation unit 20 exceeds the set threshold value.

With such a configuration, it is possible to adjust the reference for discriminating the image according to the situation of the personnel while suppressing the reduction of the accuracy of discriminating the image. For this reason, it is possible to perform discrimination processing for an image that cannot be discriminated (that is, an image that cannot be discriminated) within the range of personnel that can be assigned to discriminate the type.

Next, the outline of the present invention will be described. FIG. 7 is a block diagram showing an outline of an image determination system according to the present invention. The image determination system 90 (for example, the image determination system 200) according to the present invention has a likelihood of the type output by the estimator (for example, the image type estimation unit 20) that estimates the likelihood of the type of the input image. A discrimination unit 91 (eg, the discriminator 30) that discriminates the type of an image that exceeds the set threshold value, and a threshold setting unit 92 (eg, a threshold setting unit 62) that sets the threshold value are provided.

The threshold setting unit 92 determines the threshold according to the number of indistinguishable estimated images that are estimated to be the type that cannot be discriminated by the discriminating unit 91 and the number of personnel that can be assigned to discriminate the type of the indistinguishable estimated image. Set.

With such a configuration, it is possible to adjust the reference for discriminating the image according to the situation of the personnel while suppressing the reduction of the accuracy of discriminating the image.

Specifically, the image determination system 90 may include a storage unit (for example, the storage unit 63) that stores the correspondence relationship between the determination impossible ratio, which is the ratio of images for which the determination unit 91 cannot determine the type, and the threshold value. Good. Then, the threshold setting unit 92 calculates the number of indistinguishable estimated images for each threshold from the number of images to be discriminated and the indistinguishable ratio, and assigns the calculated number to the personnel that can be assigned to discriminate the indistinguishable estimated images. A threshold value may be set accordingly.

At that time, the threshold value setting unit 92 may set the highest threshold value among the threshold values that can be determined by the assignable personnel for the calculated number of indistinguishable estimation images.

Further, the threshold setting unit 92 may calculate the indistinguishable ratio while changing the threshold based on the number of indistinguishable images with respect to the number of input images, and may register the calculation result in the storage unit.

Further, the threshold value setting unit 92 may calculate an indistinguishable ratio for each threshold value according to a change in the situation for estimating the likelihood of the image type (for example, a change in the model used by the image type estimation unit 20).

As mentioned above, although this invention was demonstrated with reference to embodiment and an Example, this invention is not limited to the said embodiment and Example. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

This application claims priority based on Japanese Patent Application No. 2018-089928 filed on May 8, 2018, the entire disclosure of which is incorporated herein.

DESCRIPTION OF SYMBOLS 10 Image input part 11 Image memory | storage device 20 Image classification estimation part 21 AI
30 Classifier 31 Re-learning Data Storage Unit 32 Test Data Storage Unit 40 Type Setting Unit 50 Learning Unit 51 AI
60 Evaluator 61 Evaluator 62 Threshold Setting Unit 63 Storage Unit 70 Output Unit Pa to

Pb Processor

100, 200 Image Determination System

Claims

A discriminator for discriminating a type of an image in which the likelihood of the type output by the estimator that estimates the likelihood of the type of the input image exceeds a set threshold;
A threshold setting unit for setting the threshold,
The threshold value setting unit is based on the number of indistinguishable estimation images that are estimated to be the type that the discrimination unit cannot discriminate and the number of personnel that can be assigned to discriminate the type of the indistinguishable estimation image. An image determination system characterized by setting a threshold value.
A storage unit that stores a correspondence relationship between a threshold value and a non-discrimination ratio that is a ratio of an image for which the discrimination unit cannot discriminate;
The threshold setting unit calculates the number of indistinguishable estimated images for each threshold from the number of images to be discriminated and the indistinguishable ratio, and determines the threshold according to the number of personnel that can be assigned to discriminate the calculated number of indistinguishable estimated images. The image determination system according to claim 1.
The image determination system according to claim 2, wherein the threshold value setting unit sets the highest threshold value among threshold values that can be determined by the assignable personnel for the calculated number of indistinguishable estimated images.
4. The image determination system according to claim 2, wherein the threshold setting unit calculates an indistinguishable ratio while changing the threshold based on the number of indistinguishable images with respect to the number of input images, and registers the calculation result in the storage unit. .
The image determination system according to claim 4, wherein the threshold setting unit calculates an indistinguishable ratio for each threshold according to a change in a situation in which the probability of the image type is estimated.
Determining the type of image for which the likelihood of the type output by the estimator that estimates the likelihood of the type of the input image exceeds a set threshold;
The threshold value is set according to the number of indistinguishable estimated images that are estimated to be unable to discriminate the type and the number of personnel that can be assigned to discriminate the type of the indistinguishable estimated image. Judgment method.
With reference to a storage unit that stores a correspondence relationship between an indistinguishable ratio, which is a ratio of images whose type cannot be discriminated, and a threshold, the number of indistinguishable estimated images is determined for each threshold by using the number of images to be discriminated and the indistinguishable ratio To
The image determination method according to claim 6, wherein a threshold value is set according to the number of personnel that can be assigned to determine the calculated number of indistinguishable estimated images.
On the computer,
A discriminating process for discriminating the type of image in which the likelihood of the type output by the estimator that estimates the likelihood of the type of the input image exceeds a set threshold; and
Executing threshold setting processing for setting the threshold;
In the threshold value setting process, the number of indistinguishable estimation images that are estimated to be the type that cannot be discriminated in the discrimination process, and the number of personnel that can be assigned to discriminate the type of the indistinguishable estimation image An image judgment program for setting a threshold.
On the computer,
A threshold value setting process refers to a storage unit that stores a correspondence relationship between a threshold value that is a ratio of images whose type cannot be determined and a threshold value. The image determination program according to claim 8, wherein a number is calculated for each of the threshold values, and the threshold value is set according to the number of personnel that can be assigned to determine the calculated number of indistinguishable estimation images.