JP7269013B2 - Image recognition system and image recognition method capable of suppressing false recognition - Google Patents

Description

The present invention relates to an image recognition system and an image recognition method capable of suppressing erroneous recognition in image recognition using deep learning.

Machine learning is a technology that discovers and predicts rules, regularities, judgment criteria, etc. (learning results) from the data set by having the "machine" itself learn based on the prepared data set. . As one of machine learning methods, deep learning has been attracting attention in recent years. Deep learning is a multi-layered neural network (several tens to hundreds of layers), and it is possible to improve the accuracy of the learning process, so it can be applied (or considered for application) to various fields. is done.

A typical application field of deep learning is image recognition. Image recognition processing using deep learning is known to have high generalization ability. Let deep learning learn, and build an image recognition processing system using this deep learning. Even if an image of target X (cat) not used for learning is input to this image recognition processing system, it is possible to appropriately recognize the input image as target X (cat).

By the way, in the field of image recognition processing using machine learning, various techniques have been proposed to suppress erroneous recognition. For example, in Patent Document 1, in a radiation imaging apparatus that detects an object from a radiographic image by image recognition, an object is to suppress erroneous detection (erroneous recognition) when performing image recognition using machine learning. techniques have been proposed. In this technology, learning result data for image recognition is acquired in advance and stored by machine learning using a plurality of rotated images obtained by rotating an image of an object at a plurality of angles, and this learning result data is stored. Based on this, the object is detected by image recognition from the captured radiographic image.

Furthermore, it has recently become clear that unexpected misrecognition occurs in image recognition processing using deep learning. For example, it is known to recognize an image that looks like a noise image to humans as an object X. Alternatively, it has been reported that it is possible to cause deep learning to misrecognise by using an image that is intentionally created to cause misrecognition.

For example, Non-Patent Document 1 reports that erroneous recognition occurs when a trained deep neural network (DNN) is made to recognize an image to which fine perturbations are applied. is called an “Adversarial Example”. In addition, Non-Patent Document 2 reports that an “Adversarial Example” can be created by a method different from that of Non-Patent Document 1.

JP 2017-185007 A

"Intriguing Properties of Neural Networks", Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, Rob Fergus, arXiv org, arXiv:1312.6199 (version 4), December 21, 2013 (version 1 ) "Explaining and Harnessing Adversarial Examples", Ian J. Goodfellow, Jonathon Shlens, Christian Szegedy, arXiv org, arXiv:1412.6572 (version 3), December 20, 2014 (version 1)

Japanese Patent Laid-Open No. 2002-200000 aims at suppressing the occurrence of erroneous recognition caused by various changes in the situation of the surrounding structure, and uses machine learning for this purpose. Specific examples of machine learning include the use of discriminators such as neural networks, support vector machines (SVM), and boosting. One specific example is boosting, which is a type of ensemble learning. ing. However, the method disclosed in Patent Document 1 does not suppress unexpected erroneous recognition or erroneous recognition of an intentionally created image.

In addition, Non-Patent Document 1 and Non-Patent Document 2 report that as described above, it is possible to intentionally create an "Adversarial Example", which is image data that is misrecognized. We propose to suppress misrecognition by using such “Adversarial Examples” as learning data for deep learning. However, there is a possibility that this method may erroneously recognize "Adversarial Examples" created by other methods.

The present invention has been made to solve such problems, and is effective for unexpected erroneous recognition or erroneous recognition of intentionally created images in image recognition using deep learning. The purpose is to suppress

In order to solve the above problems, an image recognition system according to the present invention includes a plurality of parallel image recognition processing units, an input image distribution unit that distributes and supplies input image data to the plurality of image recognition processing units, and a plurality of image recognition processing units. and a recognition result integration processing unit that integrates a plurality of individual recognition results, which are the recognition results of the image recognition processing unit, wherein the image recognition processing unit performs image recognition processing by deep learning that has already learned about a recognition target. and generating the individual recognition results for the recognition candidates included in the input image data, the deep learning in the plurality of image recognition processing units being different from each other, and the recognition result integration processing unit comprising a plurality of When all of the individual recognition results match, the individual recognition result is output as the integrated recognition result of the recognition target, or when the plurality of individual recognition results do not all match, the individual recognition result with the greatest number of matches is selected. It is configured to output as an integrated recognition result of the recognition target.

According to the above configuration, a plurality of image recognition processing units are provided in parallel, and the image recognition processing of these image recognition processing units is performed by deep learning on the recognition target. are learned differently about recognition targets. Then, when integrating the individual recognition results obtained from these image recognition processing units, the individual recognition result that is "unanimous" or the individual recognition result that has the largest number of matches by "majority vote" is selected. , are output as integrated recognition results. As a result, even if an unexpected erroneous recognition of the recognition target occurs in any of the image recognition processing units, or if the image of the recognition target is an image intentionally created to cause erroneous recognition, False recognitions can be virtually eliminated in the process of consensus or majority voting. Therefore, in image recognition using deep learning, it is possible to effectively suppress unexpected erroneous recognition or erroneous recognition of an intentionally created image.

In the image recognition system configured as described above, the image recognition processing unit performs image recognition processing by deep learning, thereby generating a recognition result for each recognition candidate included in the input image data together with a certainty factor. and determining the validity of the recognition result based on the generated recognition result and confidence factor, and outputting the validity of the recognition result together with the recognition result to the recognition result integration processing unit as the individual recognition result. and an individual recognition result determination unit, wherein when the plurality of individual recognition results are integrated, the recognition result integration processing unit votes for each individual recognition result based on at least the validity. It may be configured to determine whether or not, and to generate the integrated recognition result based on the voting result.

Further, in the image recognition system configured as described above, the recognition result integration processing unit may be configured such that when none of the plurality of individual recognition results is valid, or when the invalid individual recognition result is more effective than the valid individual recognition result If there are many, it may be configured to output unknown as the integrated recognition result.

In the image recognition system having the configuration described above, the plurality of image recognition processing units include deep learning in which learning is performed using different teacher images, deep learning in which different initial values for learning are set, or intermediate layers. A configuration using either deep learning in which at least one of the number and the number of cells is different may be used.

Further, in the image recognition system configured as described above, an image supply unit that supplies input image data to the input image distribution unit, and a predetermined processing using the integrated recognition result output from the recognition result integration processing unit. and at least one of a recognition result using unit that executes

Further, in the image recognition processing method according to the present invention, a plurality of image recognition processing units that perform image recognition processing by deep learning that has been trained differently from each other for a recognition target are arranged in parallel, and input image data is sent to these image recognition processing units. distributed and supplied, each image recognition processing unit performs image recognition processing, generates a plurality of individual recognition results for recognition candidates included in the input image data, integrates these individual recognition results, and performs this integration processing. Then, when the plurality of individual recognition results all match, the individual recognition result is output as the integrated recognition result of the recognition target, or when the plurality of individual recognition results do not all match, the individual A recognition result is output as an integrated recognition result of the recognition target.

According to the present invention, with the above configuration, it is possible to effectively suppress unexpected erroneous recognition or erroneous recognition of an intentionally created image in image recognition using deep learning. Play.

1A is a block diagram showing an example of the configuration of an image recognition system according to Embodiment 1 of the present disclosure, and FIG. 1 is a block diagram showing an example of a configuration; FIG. 2 is a flowchart showing a typical example of integration processing by a recognition result integration processing unit provided in the image recognition system shown in FIG. 1(A); FIG. 11 is a flow chart showing a typical example of integration processing by a recognition result integration processing unit included in the image recognition system according to Embodiment 2 of the present disclosure; FIG. FIG. 11 is a block diagram showing a configuration example of an image recognition system according to Embodiment 3 of the present disclosure; 5 is a block diagram showing a configuration example of the image recognition system shown in FIG. 4; FIG.

Hereinafter, typical embodiments of the present invention will be described with reference to the drawings. In the following description, the same or corresponding elements are denoted by the same reference numerals throughout all the drawings, and duplicate descriptions thereof will be omitted.

(Embodiment 1)
As shown in FIG. 1A, an image recognition system 10A according to the present disclosure includes an image recognition unit 11A. The image recognition unit 11A includes an input image distribution unit 12, image recognition processing units 13A to 13C, A recognition result integration processing unit 14 is provided.

Input image data is supplied to the input image distribution unit 12 ("image supply" in the drawing), and this input image data is distributed and supplied to the image recognition processing units 13A to 13C. The input image data may contain an image that can be recognized, but it does not have to be contained. Note that an image that can be a recognition target is called a "recognition candidate" for convenience.

The image recognition processing units 13A to 13C are provided parallel to each other, perform image recognition processing on the input image data distributed and supplied from the input image distribution unit 12, and obtain recognition results for recognition candidates included in the input image data. to generate This image recognition processing is performed by deep learning in which the recognition target has already been learned. The recognition result integration processing unit 14 integrates the recognition results output from the image recognition processing units 13A to 13C, and outputs an "integrated recognition result" which is the final recognition result.

Here, the recognition candidates included in the input image data are recognized by the image recognition processing of the image recognition processing units 13A to 13C as to whether or not they are recognition targets. "candidate recognition result". A plurality of candidate recognition results may be obtained in the image recognition processing units 13A to 13C, and in this case, an appropriate recognition result is adopted from the plurality of candidate recognition results. Therefore, the appropriate recognition results adopted by the image recognition processing units 13A to 13C are called "individual recognition results" to distinguish them from the candidate recognition results. The three individual recognition results obtained from the image recognition processing units 13A to 13C are integrated by the recognition result integration processing unit 14 to become the final recognition result. "integrated recognition result".

The image recognition processing units 13A to 13C perform image recognition processing on recognition candidates that may be included in the input image data by different deep learning. "Different deep learning" here is not particularly limited, but for example, deep learning that uses different learning images, deep learning that sets different initial values for learning, or the number of intermediate layers and Deep learning or the like in which at least one of the numbers of cells is different can be mentioned. Alternatively, a different program development environment for deep learning may be used.

In this embodiment, for example, deep learning using different learning images is used. Specifically, for example, 30,000 target image data are prepared for a recognition target X (for example, a cat), and 10,000 of these target image data are divided into three learning image groups (first image group, second image group, and third image group). divided into three image groups). The first image recognition processing unit 13A performs deep learning using the first image group, the second image recognition processing unit 13B performs deep learning using the second image group, and the third image recognition processing unit 13C Deep learning is performed using the third set of images.

A specific configuration of deep learning in the image recognition processing units 13A to 13C is not particularly limited, and a known configuration can be preferably used. Specific examples include convolutional neural networks (CNN), recurrent neural networks (RNN), LSTM (Long Short-Term Memory), autoencoders, Boltzmann machines, and generative adversarial networks (GAN). In this embodiment, for example, a CNN is used, and dozens of layers may be prepared by alternately arranging convolutional layers for pattern matching and pooling layers for data aggregation.

The image recognition processing units 13A to 13C may perform image recognition processing on recognition candidates that may be included in the input image data by learned deep learning, and the specific configuration thereof is not particularly limited. As shown in FIG. 1B, the image recognition processing units 13A to 13C may have a configuration including an individual recognition processing unit 131 and an individual recognition result determination unit 132. FIG.

The individual recognition processing unit 131 performs image recognition processing on the recognition candidates by deep learning to generate candidate recognition results. In this image recognition processing, the candidate recognition results are also generated along with the degrees of certainty. The individual recognition result determination unit 132 determines the effectiveness of the candidate recognition result based on the generated candidate recognition result and the degree of certainty, and transmits the effectiveness together with the candidate recognition result to the recognition result integration processing unit 14 as the individual recognition result. output as

The specific configuration of the individual recognition processing unit 131 is not particularly limited, and may be deep learning (for example, CNN, etc.) as described above. generate degrees. Confidence of recognition results in deep learning is usually generated as a percentage, but may be generated as a plurality of ranks. In this embodiment, the deep learning of the individual recognition processing unit 131 is assumed to have learned only one type of recognition target X (for example, a cat).

For example, assuming that the input image data contains three recognition candidates, a 90% confidence level is generated for the first recognition candidate (90% confidence is "cat"), and the second recognition candidate A 50% confidence is generated for the candidate (50% confidence is "cat") and a 20% confidence is generated for the third recognition candidate (20% confidence is "cat" ) may be Alternatively, for the first recognition candidate, the confidence level is generated as A rank indicating high confidence, for the second recognition candidate, the confidence level is generated as B rank indicating intermediate confidence, and for the third recognition candidate, , C-rank indicating low confidence.

The specific configuration of the individual recognition result determination unit 132 is not particularly limited, and as described above, the validity of the candidate recognition result is determined from the candidate recognition result and the certainty factor generated by the individual recognition processing unit 131 . The method of determining effectiveness is not particularly limited, either. (3) when the confidence of the candidate recognition result with the highest confidence is sufficiently high and there is a sufficient difference from the candidate recognition result with the second highest confidence, the candidate recognition; A method of adopting the result, etc. can be mentioned.

For example, as described above, if three recognition candidates are included in the input image data, in method (1), the individual recognition result determination unit 132 selects the first recognition candidate with the highest degree of certainty. can be adopted and output as an individual recognition result.

Next, with regard to the method (2), if all of the plurality of candidate recognition results have low certainty, it may not be appropriate to adopt the one with the highest certainty among them. Therefore, in the method (2), the individual recognition result determination unit 132 determines whether or not the certainty of the candidate recognition result is sufficiently high using a threshold or the like, and if it is equal to or higher than the threshold, the candidate recognition result is adopted. output as individual recognition results. For example, as described above, if the certainty is a percentage, the candidate recognition result with a certainty of 50% or more may be adopted as the individual recognition result with the certainty of 50% as the threshold. Also, if the certainty is a rank, then the candidate recognition result with rank B or higher may be adopted as the individual recognition result.

Next, with regard to method (3), if there is a small difference between the candidate recognition result with the highest confidence level and the candidate recognition result with the second highest confidence level among a plurality of candidate recognition results, There is a possibility that it may not be appropriate to adopt the thing of Therefore, in the method (3), the individual recognition result determination unit 132 determines that the high-ranking candidate recognition result has a sufficiently high degree of certainty using a threshold or the like as in the method (2). Only when there is a sufficient difference in confidence between the two, the first candidate recognition result should be adopted and output as an individual recognition result.

Here, in the method (2) and the method (3), if the degree of certainty is less than the threshold value or the difference between the first and second places in the certainty is very small, it is not adopted as an individual recognition result. Therefore, the individual recognition result determination unit 132 should set a flag for the validity of the candidate recognition result. As such a flag, for example, a valid flag that becomes effective when the certainty is 50% or more can be cited. The individual recognition result determination unit 132 initially initializes the valid flag (valid flag: 0), and if the individual recognition processing unit 131 generates a candidate recognition result with a certainty of 50% or more, the valid flag is valid. becomes (valid flag: 1). On the other hand, if a candidate recognition result with a certainty factor of less than 50% is generated, the valid flag is not valid (valid flag: 0), so the candidate recognition result is invalid.

If the image recognition processing units 13A to 13C determine the validity or invalidity of the candidate recognition result in this way, the image recognition processing units 13A to 13C provide the recognition result integration processing unit 14 with valid candidate recognition. Both results and invalid candidate recognition results can be output as individual recognition results. The recognition result integration processing unit 14 may integrate a plurality of individual recognition results output from the image recognition processing units 13A to 13C. At this time, validity or invalidity included in the individual recognition results may be used. can be done.

In the present disclosure, the recognition result integration processing unit 14, as integration processing of individual recognition results, (i) when a plurality of individual recognition results all match, outputs the individual recognition result as an integrated recognition result of the recognition target, (ii) If all of the individual recognition results do not match, the individual recognition result with the greatest number of matches is output as the integrated recognition result of the recognition target. For convenience of explanation, step (i) will be referred to as the "unanimous" step and step (ii) will be referred to as the "majority vote" step.

(i) a unanimity step and (ii) a majority step. A joint processing technique can be used to determine whether or not to vote on a recognition result. The recognition result integration processing unit 14 generates an integrated recognition result based on the voting results for the individual recognition results.

An integration processing method based on voting in the recognition result integration processing unit 14 will be specifically described with reference to FIG. The flowchart shown in FIG. 2 is a schematic step-by-step process of the integration processing method by voting, and in this example, it is composed of a total of 9 steps from step S11 to step S19. Of course, the integrated processing technique is not limited to this.

As shown in FIG. 1A, individual recognition results are output to the recognition result integration processing unit 14 from the image recognition processing units 13A to 13C. Here, the individual recognition result from the first image recognition processing unit 13A is called "first individual recognition result", the individual recognition result from the second image recognition processing unit 13B is called "second individual recognition result", and the The individual recognition result from the three-image recognition processing section 13C will be referred to as "third individual recognition result".

The recognition result integration processing unit 14 determines the validity of all individual recognition results output from the image recognition processing units 13A to 13C, as indicated by the loop of steps S11 and S12 in FIG. For example, in step S11, it is determined whether the first individual recognition result is the recognition target X (for example, a cat), and if it is valid, the first individual recognition result is voted for the recognition target X with the number of votes: 1. If not valid, the first individual recognition result will not be voted on. Next, in step S12, it is determined whether or not the validity of all individual recognition results has been determined. If only the first individual recognition result has been judged (NO in step S12), the process returns to step S11 to judge the validity of the second individual recognition result. Similarly, the validity of the third individual recognition result is judged.

If the validity of all individual recognition results is determined (YES in step S12), the number of votes obtained for each individual recognition result is determined. In the example shown in FIG. 2, in step S13, it is determined whether or not there is an individual recognition result that has obtained votes (whether or not the number of obtained votes is present). If there are individual recognition results that have obtained votes (YES in step S13), in step S14, it is determined whether all the individual recognition results have obtained votes, that is, whether the individual recognition results are unanimous. judge.

When all of the first to third individual recognition results are valid, the recognition result (candidate recognition result) of any recognition candidate included in the input image data obtains three votes for the recognition object X, so this candidate recognition result is It is unanimously determined that the individual recognition result is valid (YES in step S14). Therefore, in step S15, an integrated recognition result is generated that an arbitrary recognition candidate is unanimously the recognition target X (for example, a cat).

On the other hand, when any one of the first to third individual recognition results is invalid, unanimity is not reached (NO in step S14), and any candidate recognition result obtains two votes for recognition object X. . That is, in step S16, it is determined whether the recognition object X has received many votes or whether there are many invalid votes (whether X has the largest number of votes including invalid votes) for any candidate recognition result. If the recognition object X has more votes than the invalid votes, this candidate recognition result is judged to be a valid individual recognition result by majority vote (YES in step S16). Therefore, in step S17, an integrated recognition result is generated in which an arbitrary recognition candidate is the recognition target X (for example, a cat) by majority vote.

On the other hand, even when two of the first to third individual recognition results are invalid, but one is valid, there is no unanimity (NO in step S14), and one vote is obtained for an arbitrary candidate recognition result. become. Looking only at the number of votes, it can be determined that this candidate recognition result is an effective individual recognition result with a majority vote, even though it has one vote. However, if the majority vote is determined by considering the invalid vote as "voted", it can be determined that the candidate recognition result should be "invalid". Therefore, when there are more invalid votes than recognition target X (NO in step S16), the image recognition unit 11A determines that an arbitrary recognition candidate could not be recognized as recognition target X, and in step S18 , an unknown integrated recognition result is generated.

When all of the first to third individual recognition results are invalid, the number of votes obtained for any candidate recognition result is 0 (NO in step S13). Therefore, this candidate recognition result is determined to be invalid. In this case, the image recognition unit 11A cannot recognize any recognition candidate as the recognition target X, so in step S18, an integrated recognition result of unknown is generated.

The integrated recognition result generated in any one of steps S15 to S18 is output from the recognition result integration processing unit 14 as step S19 (see FIG. 1A). This integrated recognition result is used in a recognition result using device (not shown in FIG. 1A). Note that the recognition result using device will be described later.

Thus, the image recognition system or image recognition processing method according to the present disclosure includes a plurality of image recognition processing units (for example, three image recognition processing units 13A to 13C) in parallel. Image recognition processing is performed by deep learning of a recognition target, and each deep learning learns a recognition target differently. Then, when integrating the individual recognition results obtained from these image recognition processing units, the individual recognition result that is "unanimous" or the individual recognition result that has the largest number of matches by "majority vote" is selected. , are output as integrated recognition results.

In the example using the votes shown in FIG. 2 described above, if unanimous, it can be determined that unexpected or intentional erroneous recognition has not occurred in the plurality of image recognition processing units. Also, in the case of a majority vote, if there are more valid votes than invalid votes, there is a possibility that an unexpected or intentional misrecognition has occurred for invalid individual recognition results. Since there is a match, it can be determined that these individual recognition results are appropriate image recognitions.

On the other hand, if all of the individual recognition results are invalid, there is a possibility that unexpected or intentional erroneous recognition has occurred, and there is also a possibility that the candidate is not suitable as a recognition candidate in the first place. Therefore, in this case, by outputting an unknown integrated recognition result, it is possible to avoid outputting inappropriate image recognition. Also, if there are more invalid votes than valid votes, as in the case where all votes are invalid, there is a possibility of unexpected or intentional misrecognition, or that it may not be suitable as a recognition candidate, so the integrated recognition result is unknown. should be output. However, if it can be determined that image recognition is appropriate when even one valid vote is obtained, it is possible to ignore invalid votes and make a decision by majority vote.

Therefore, in the image recognition system or the image recognition processing method according to the present disclosure, an unexpected misrecognition of the recognition target occurs in any of the image recognition processing units, or an image of the recognition target is intentionally misrecognized. The unanimous or majority voting process can virtually eliminate misrecognitions, even for artificially generated images. Therefore, in image recognition using deep learning, it is possible to effectively suppress unexpected erroneous recognition or erroneous recognition of an intentionally created image.

In the image recognition system 10A according to the first embodiment, a total of three image recognition processing units are provided in parallel (image recognition processing units 13A to 13C). Alternatively, four or more image recognition processing units may be provided in parallel.

Further, the input image distribution unit 12, the image recognition processing units 13A to 13C, and the recognition result integration processing unit 14, which constitute the image recognition system 10A according to the first embodiment, are GPU (Graphics Processing) known in the field of deep learning. Unit), FPGA (Field-Programmable Gate Array), ASIC (Application specific integrated circuit), CPU (Central Processing Unit), etc. The image recognition processing units 13A to 13C that implement deep learning or the individual recognition processing unit 131 included therein may be implemented by a computing unit such as a GPU. Also, part of the configuration of the image recognition system 10A may be configured as a logic circuit or the like using known switching elements, subtractors, comparators, etc., or may be configured as an independent device.

(Embodiment 2)
In the first embodiment, the deep learning in the image recognition processing units 13A to 13C learns about only one type of recognition target X (for example, a cat), but in the present disclosure, the image recognition processing units 13A to 13C Deep learning in 13C may be learned about multiple types of recognition targets. For example, the second embodiment will be described with reference to FIG. 3 assuming that deep learning learns about a recognition target X (for example, a cat) and a recognition target Y (for example, a dog).

The configuration of the image recognition system 10A according to the second embodiment is as described in the first embodiment. It is learned. The flowchart shown in FIG. 3, like FIG. 2, is a schematic step-by-step representation of the vote-based integration processing method in the recognition result integration processing unit 14. In this example, a total of nine steps from step S21 to step S29 Configured. Of course, the integrated processing method is not limited to this.

The recognition result integration processing unit 14 determines the validity of all the individual recognition results output from the image recognition processing units 13A to 13C, as indicated by the loop of steps S21 and S22 in FIG. This loop is similar to the loop of steps S11 and S12 in FIG. If it is valid as the recognition target X, the number of votes: 1 for the recognition target X for the first individual recognition result is voted, and if it is valid as the recognition target Y, the recognition target Y for the first individual recognition result Votes for: 1. If not valid, the first individual recognition result will not be voted on. After that, the validity of the second individual recognition result and the third individual recognition result is similarly judged.

If the validity of all individual recognition results is determined (YES in step S22), the number of votes obtained for each individual recognition result is determined. In the example shown in FIG. 3, in step S23, it is determined whether or not there is an individual recognition result that has obtained votes (whether or not the number of obtained votes is present). If there are individual recognition results that have obtained votes (YES in step S23), in step S24, all the individual recognition results have obtained votes for the recognition target X, or all the individual recognition results have been recognized. It is determined whether or not votes have been obtained for object Y, that is, whether or not individual recognition results are unanimous.

When all of the first to third individual recognition results are valid as, for example, the recognition target X (cat), the recognition result (candidate recognition result) of any recognition candidate included in the input image data obtains three votes for X. . Therefore, it is unanimously determined that this candidate recognition result is an effective individual recognition result for X (YES in step S24). Therefore, in step S25, an integrated recognition result is generated that an arbitrary recognition candidate is unanimously the recognition target X (cat). The same applies to recognition target Y (dog).

On the other hand, when the first to third individual recognition results do not unanimously agree (NO in step S24), these individual recognition results include valid recognition target X and recognition target Y. Possibly (possibly including invalid ones that did not get votes). Therefore, in step S26, it is determined whether or not the recognition target X or Y has the highest number of votes among the first to third individual recognition results.

For example, if an arbitrary candidate recognition result obtains two votes as recognition target X and one vote as recognition target Y, if two votes are obtained as recognition target X and the rest are invalid, two votes are obtained as recognition target Y. If one vote is obtained as the recognition target X, if two votes are obtained as the recognition target Y and the rest are invalid, the individual recognition result is valid as the recognition target X by majority vote, or the recognition target Y (YES in step S26). Therefore, in step S27, an integrated recognition result is generated in which any recognition candidate is the recognition target X (cat) or the recognition target Y by majority vote.

Here, if all of the first to third individual recognition results are invalid, neither of the recognition targets X and Y has obtained a single vote (the number of votes obtained is 0) (NO in step S23). . Therefore, this candidate recognition result is determined to be invalid, and an unknown integrated recognition result is generated in step S28.

Alternatively, if one of the first to third individual recognition results obtains one vote as the recognition target X, one of the others obtains one vote as the recognition target Y, and the rest are invalid, the recognition target Either X or Y does not get the most votes (NO in step S26). Similarly, if one vote is obtained for the recognition target X and the rest are invalid, or if one vote is obtained for the recognition target Y and the rest are invalid, the number of invalid votes will be greater. Any of Y does not get the most votes (NO in step S26). In these cases, it cannot be determined whether the candidate recognition result is X or Y, so an unknown integrated recognition result is generated in step S28.

As described above, when the integrated recognition result is generated in any of steps S25, S27, and S28, the integrated recognition result is output from the recognition result integration processing unit 14 to the recognition result using device in step S29.

Thus, in the image recognition system or image recognition processing method according to the present disclosure, a plurality of image recognition processing units using deep learning for a plurality of recognition targets may be provided in parallel. Even in this case, when integrating a plurality of individual recognition results, select the individual recognition result that is "unanimous" or the individual recognition result that has the largest number of matches in the "majority vote", and select it as the integrated recognition result. output.

As a result, even if an unexpected erroneous recognition of the recognition target occurs in any of the image recognition processing units, or if the image of the recognition target is an image intentionally created to cause erroneous recognition, False recognitions can be virtually eliminated in the process of consensus or majority voting. Therefore, in image recognition using deep learning, it is possible to effectively suppress unexpected erroneous recognition or erroneous recognition of an intentionally created image.

(Embodiment 3)
In the first or second embodiment, the image recognition system 10A is configured to include only the image recognition unit 11A including the input image distribution unit 12, the image recognition processing units 13A to 13C, and the recognition result integration processing unit 14. , the disclosure is not limited thereto. For example, as shown in FIG. 4, it may be an image recognition system 10B comprising an image recognition device 11B, an image supply device 15, and a recognition result use device 16A. Alternatively, as shown in FIG. 5, an image recognition system 10C including an image recognition/result usage device 11C and an image supply device 15 may be used.

The image recognition device 11B has the same configuration as the image recognition section 11A described in the first or second embodiment. The image supply device 15 may supply input image data to the image recognition device 11B or the image recognition/result use device 11C. Specifically, for example, it may be a camera (shooting device) that shoots still images or moving images (video), or an information terminal device (personal computer, smartphone, tablet, etc.).

16 A of recognition result using apparatuses should just perform a predetermined process using the integrated recognition result output from the recognition result integration process part 14. FIG. Specifically, an actuator or the like used for various applications (described later) using image recognition can be mentioned. For example, when the image recognition system 10B or 10C is applied to an automatic driving vehicle, a driving determination device is included, and when the image recognition system 10B or 10C is applied to a face authentication system for entry restriction, A locking and unlocking device is mentioned.

The image recognition/result use device 11C is an integration of the image recognition device 11B and the recognition result use device 16A. , a recognition result integration processing unit 14) and a recognition result use unit 16B. The recognition result using unit 16B may be the same as the recognition result using device 16A.

Applications to which the image recognition systems 10B and 10C according to the third embodiment or the image recognition system 10A according to the first or second embodiment can be applied are not particularly limited, and examples of the recognition result using device 16A are given. As described above, any control system, authentication system, diagnostic system, or the like using image recognition may be used.

More specifically, for example, an image recognition system for recognizing the surrounding environment such as other aircraft or equipment during ground taxiing in an autonomous aircraft, and for recognizing the surrounding environment such as other vehicles or pedestrians in an autonomous vehicle. image recognition system for autonomous ships, image recognition system for recognizing the surrounding environment such as other ships or port facilities in autonomous ships, cancer diagnosis system from X-ray images in the medical field, dangerous goods from X-ray images in baggage inspection Automatic identification systems, face authentication systems for restricting access in high security areas, and the like can be mentioned.

In this way, in the image recognition system or image recognition processing method according to the present disclosure, a plurality of image recognition processing units that perform image recognition processing by deep learning that is different from each other for a recognition target are arranged in parallel, and these image recognition processing units The input image data is distributed and supplied to each of the image recognition processing units, and each image recognition processing unit performs image recognition processing to generate a plurality of individual recognition results for the recognition candidates included in the input image data, and integrates these individual recognition results. In this integration process, when the plurality of individual recognition results all match, the individual recognition result is output as the integrated recognition result of the recognition target, and when the plurality of individual recognition results do not all match , the individual recognition result with the largest number of matches is output as the integrated recognition result of the recognition object.

Therefore, the specific configuration of the image recognition system according to the present disclosure is not particularly limited, and may be, for example, the image recognition system 10A shown in FIG. , the image recognition system 10B shown in FIG. 4 described in the third embodiment, or the image recognition system 10C shown in FIG. 5, or a system having a configuration other than that described in the first to third embodiments may be

It should be noted that the present invention is not limited to the description of the above embodiments, and various modifications are possible within the scope of the claims, and different embodiments and multiple modifications are disclosed respectively. Embodiments obtained by appropriately combining the above technical means are also included in the technical scope of the present invention.

INDUSTRIAL APPLICABILITY The present invention can be widely and suitably used in the field of image recognition using deep learning.

10A to 10C: Image recognition system 11A: Image recognition unit 11B: Image recognition device (image recognition unit)
11C: Image recognition/result usage device (image recognition unit, recognition result usage unit)
12: Input image distribution unit 13: Image recognition processing unit 13A: First image recognition processing unit 13B: Second image recognition processing unit 13C: Third image recognition processing unit 14: Recognition result integration processing unit 15: Image supply device 16A: Recognition result usage device (recognition result usage unit)
16B: recognition result using unit 131: individual recognition processing unit 132: individual recognition result determination unit

Claims (6)

A plurality of image recognition processing units that perform image recognition processing on input image data by trained deep learning and generate individual recognition results for recognition candidates included in the input image data, wherein the deep learning is different from each other . , a plurality of image recognition processing units ;
Integrating processing of the individual recognition results output from the plurality of image recognition processing units, and outputting the individual recognition result as an integrated recognition result when the plurality of individual recognition results are all matched, a recognition result integration processing unit for outputting the individual recognition result with the greatest number of matches as an integrated recognition result when all the individual recognition results do not match;
with
The image recognition processing unit generates a recognition result of the recognition candidate and a certainty factor thereof in image recognition processing of the recognition candidate, and evaluates the validity of the recognition result based on the recognition result and the certainty factor. When it is determined that the certainty is high when is equal to or greater than a predetermined threshold,
A method of adopting a first recognition result, which is the recognition result of the recognition candidate with the highest degree of certainty,
A method of adopting a second recognition result that selects the recognition result from the recognition candidates whose certainty is equal to or greater than a predetermined threshold;
A method of determining, as a final recognition result, a recognition candidate selected according to the magnitude of the difference in certainty between the recognition results in the recognition candidate group consisting of the first recognition result and the second recognition result. ,
and outputting the validity as the individual recognition result to the recognition result integration processing unit;
image recognition system. When the plurality of individual recognition results are integrated, the recognition result integration processing unit determines whether or not to vote for each individual recognition result based on at least the validity, and
characterized by generating the integrated recognition result based on the voting result,
The image recognition system according to claim 1. When none of the plurality of individual recognition results are valid, or when the number of invalid individual recognition results is greater than the number of valid individual recognition results, the recognition result integration processing unit outputs an unknown integrated recognition result. characterized by outputting
The image recognition system according to claim 2. In the plurality of image recognition processing units, deep learning is performed using different teacher images, deep learning is performed using different initial values for learning, or deep learning is performed in which at least one of the number of intermediate layers and the number of cells is different. characterized in that one of the
The image recognition system according to any one of claims 1 to 3. further comprising an input image distribution unit that distributes and supplies the input image data to a plurality of the image recognition processing units;
an image supply unit that supplies input image data to the input image distribution unit;
and at least one of a recognition result using unit that executes a predetermined process using the integrated recognition result output from the recognition result integration processing unit,
The image recognition system according to any one of claims 1 to 4. A plurality of image recognition processing units that perform image recognition processing on a recognition target, wherein a plurality of image recognition processing units trained by different deep learning are arranged in parallel ,
Each of the image recognition processing units generates the recognition result of the recognition candidate and its certainty factor in the image recognition process of the recognition target, and determines the validity of the recognition result based on the recognition result and the certainty factor. When it is determined that the degree of certainty is high when the degree is equal to or greater than a predetermined threshold,
A method of adopting a first recognition result, which is the recognition result of the recognition candidate with the highest degree of certainty,
A method of adopting a second recognition result that selects the recognition result from the recognition candidates whose certainty is equal to or greater than a predetermined threshold;
A method of determining, as a final recognition result, a recognition candidate selected according to the magnitude of the difference in confidence between each recognition result in a recognition candidate group consisting of the first recognition result and the second recognition result;
and outputting the effectiveness as the individual recognition result to the recognition result integration processing unit to generate a plurality of individual recognition results for the recognition candidates included in the input image data ,
when all of the plurality of individual recognition results match, outputting the individual recognition result as an integrated recognition result of the recognition target;
When all of the plurality of individual recognition results do not match, outputting the individual recognition result with the greatest number of matches as an integrated recognition result of the recognition target;
Image recognition processing method.

Images