KR20180065889A - Method and apparatus for detecting target - Google Patents

Abstract

A method and apparatus for detecting the liveness of a target based on image quality are disclosed. A method for detecting a target according to an exemplary embodiment includes: a step of determining a quality type of an image; a step of determining the quality type of the image and a convolutional neural network of a corresponding quality type; a step of determining the detection value of the image based on the convolutional neural network; and a step of determining whether the target in the image is a real target based on the detection value of the image. It is possible to detect the target more accurately.

Description

[0001] METHOD AND APPARATUS FOR DETECTING TARGET [0002]

The present invention relates to a computer vision technology domain, and more particularly, the present invention relates to a method and apparatus for detecting targets.

The detection of the target is an important content of the image processing region. When the target in the image is detected as a biometric target registered in advance, the target can be determined as a real target. The target can be a human face or the like, and can be applied to various functions such as a mobile phone unlocking and mobile payment.

Target detection methods can be vulnerable to various forms of fake attacks. For example, according to a conventional target detection method, a paper print image, a photograph, a screen image, a screen video, and a 3D print including a user's face, rather than a real face of a real user, have.

As a target detection method to overcome this, there are a penetration type target detection method and an non-insertion type target detection method.

Existing intrusion target detection methods depend on the user's interaction. For example, the user performs a specified operation such as blinking an eye, waving a head, smile, or the like according to a presentation of a program, and the program identifies a user's action and discriminates whether or not it is a real target. However, this type of method has a disadvantage in that the identification step is relatively cumbersome, takes a long time, and requires an operation designated by the user in a practical application.

In the conventional non-immersive target detection method, features are extracted from image or video information acquired through a photographing apparatus of a terminal equipment, and the authenticity of the target is determined based on the extracted features. For example, with a conventional non-immersive target detection method, there exists a target detection method based on an artificial design feature. According to this method, a designer designs a certain objective algorithm by using the experience of related computer vision and image processing research area to extract the feature of image or video. However, there is a difference in the target image because there is a difference in performance between the imaging devices of different terminal equipments. For example, some images may be slightly overexposed, or the hue may be red. Also, there is a difference between images such as low-illuminance or backlight and images with normal illuminance. When a feature extraction method such as a local binary pattern (LBP) based on an artificial design is used, only the local texture information of the image is considered. In this case, the authenticity of the target can be effectively distinguished under low light conditions or backlight conditions I can not. In other words, according to the conventional non-immersion target detection method, robustness is reduced due to easy occurrence of the true detection target of the true target in a complex and varied real world.

Embodiments provide techniques for increasing robustness by eliminating the need for a user to perform a specified operation, and more accurately detecting the authenticity of a target from a target image obtained in various hardware and / or environments.

Embodiments use a convolutional neural network that is suitable for the quality type of each target image by determining the quality type of each target image and then determining the convolution neural network of the quality type corresponding to the determined quality type. Thus, the embodiments can more accurately detect the authenticity of the target included in each target image.

A method for target detection according to one side comprises the steps of: determining a quality type of a target image; Determining, from a database comprising a plurality of convolutional neural networks, a convolutional neural network of a quality type corresponding to a quality type of the target image; Determining a detection value of the target image based on the convolutional neural network of the corresponding quality type; And determining whether the target in the target image is a real target based on the detected value of the target image.

The quality type of the target image may be determined based on at least one quality value of the target image determined corresponding to at least one quality parameter. The detected value may include a probability that the target image is classified into a positive sample including a real target.

Wherein determining the quality type of the target image comprises determining at least one quality value of the target image corresponding to at least one quality parameter; Determining at least one quality classification of the target image based on the at least one quality value and at least one quality type classification standard preset corresponding to the at least one quality parameter; And determining a quality type of the target image based on the at least one quality classification. Alternatively, determining the quality type of the target image may include performing a blind image quality assessment on the target image to determine a quality type of the target image.

The convolutional neural network may include a cascaded convolutional neural network including convolutional neural networks of at least two stages and at least one threshold value layer.

The cascade convolutional neural network of each quality type requires a different figure of merit and the number of stages included in the cascade convolution neural network of each quality type may be determined by the corresponding figure of merit. The figure of merit required by the cascade convolutional neural network may be satisfied by a combination of the performance indices of the convolutional neural networks of the plurality of stages included in the cascade convolutional neural network.

Wherein determining the detected value of the target image comprises: determining a detected value of the first stage of the target image based on a convolution neural network of a first stage included in the cascade convolution neural network; And a step of determining a detection value of a next stage of the target image by comparing the detected value of the first stage with a predetermined threshold value based on a threshold value layer connected to the convolution neural network of the first stage .

Wherein the step of determining whether the target in the target image is a real target based on a detected value of the target image comprises: determining a value of a mojo evaluation of the current frame when the target image is a current frame in the frame image; A comprehensive detection value of the target image is determined based on a voiced evaluation value of the current frame, a detected value of the current frame, a voiced evaluation value of a plurality of previous frames in the frame image, and detection values of the plurality of previous frames step; And determining whether the target in the current frame is a real target based on the integrated detection value of the target image.

Wherein the step of determining a combined detection value of the target image determines a weighted average value of the detection values using the voiced evaluation values of the current frame and the plurality of previous plural frames as weight values of the corresponding detection values, And determining the value as the comprehensive detection value.

A training method of a convolutional neural network according to one side comprises the steps of: determining a quality type of a sample image; Selecting a convolutional neural network of a quality type corresponding to the determined quality type of convolutional neural networks of a plurality of quality types; And training the convolutional neural network of the selected quality type based on whether the sample image is positive or negative.

If the sample image includes a real target, the sample image corresponds to a positive sample, and if the sample image does not include the real target, the sample image may correspond to a negative sample. The training method of the convolutional neural network may further comprise building a database based on a convolutional neural network of each quality type.

A target detection apparatus according to one side includes an image quality type determination unit for determining a quality type of a target image; A convolutional neural network determiner for determining, from a database comprising a plurality of convolutional neural networks, a convolutional neural network of a quality type corresponding to a quality type of the target image; A detection value determiner for determining a detection value of the target image based on the convolutional neural network of the corresponding quality type; And a target truth determining unit for determining whether the target in the target image is a real target based on the detected value of the target image.

1 is a flow chart of a target detection method according to an embodiment;
Figure 2a is a flow diagram of a method of training a convolutional neural network according to one embodiment.
FIG. 2B is a diagram illustrating a structure of a cascaded convolutional neural network according to an embodiment; FIG.
2C illustrates a structure of a single-stage convolutional neural network according to an embodiment;
3A is a flow chart of a scenario in which a target detection method according to one embodiment is applied to an actual application.
FIG. 3B is a view for explaining a target detection method for a frame image according to an embodiment; FIG.
4 is a block diagram of a target detection apparatus according to one embodiment.
Figure 5 illustrates robustness against various fake attacks according to one embodiment.
Figure 6 illustrates robustness against fraud attacks of the low quality type according to one embodiment;

Specific structural or functional descriptions of embodiments are set forth for illustration purposes only and may be embodied with various changes and modifications. Accordingly, the embodiments are not intended to be limited to the particular forms disclosed, and the scope of the present disclosure includes changes, equivalents, or alternatives included in the technical idea.

The terms first or second, etc. may be used to describe various elements, but such terms should be interpreted solely for the purpose of distinguishing one element from another. For example, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component.

It is to be understood that when an element is referred to as being "connected" to another element, it may be directly connected or connected to the other element, although other elements may be present in between.

The singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, the terms " comprises ", or " having ", and the like, are used to specify one or more of the described features, numbers, steps, operations, elements, But do not preclude the presence or addition of steps, operations, elements, parts, or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in commonly used dictionaries are to be interpreted as having a meaning consistent with the meaning of the context in the relevant art and, unless explicitly defined herein, are to be interpreted as ideal or overly formal Do not.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Like reference symbols in the drawings denote like elements.

1 is a flowchart of a target detection method according to an embodiment. Referring to FIG. 1, a target detection method includes determining a quality type of a target image (S101); Determining (S102) a convolutional neural network of a quality type corresponding to the quality type of the target image; Determining a detected value of the target image based on the convolutional neural network of the corresponding quality type (S103); And determining whether the target in the target image is a real target based on the detected value of the target image (S104).

Embodiments provide a technique for accurately detecting a target regardless of various factors that affect the quality type of the target image, by using different convolution neural networks depending on the quality type of the target image. Here, the detection of the target determines whether the target is true or false, and it can be understood that it is determined, for example, whether or not the target in the target image is a biological target. For example, if the target included in the target image is a biometric face, it is recognized as a real target, and if it is a non-biometric face such as a photograph or a video, it can be recognized as a false target. Hereinafter, the target detection method can be understood as a method of detecting the liveness of the target.

A target is a body part of a creature, such as a human face, a palmprint, a human fingerprint, a human iris, a human limb, an animal's face, an animal's fingerprint, Animal limbs, and the like. The real target is a biological target, and the false target may be a non-biological target.

The quality type of the target image may be determined based on at least one quality value of the target image determined corresponding to the at least one quality parameter. The quality parameter may include a shooting parameter, an attribute parameter, and the like. The shooting parameters may include resolution, ISO (sensitivity), and the like, and the attribute parameters may include color quasi, comparison, brightness, saturation, sharpness, and the like. The quality value can be understood as a value of the target image corresponding to the quality parameter.

According to the target image, the shooting parameter and the attribute parameter may be different. In this case, the target image may have different quality types. For example, the quality type of the target image may vary depending on the performance of the imaging device of the user terminal, and the quality type of the target image may be varied by various imaging environments such as illumination. If the same convolution neural network is used even though the quality types are different, the detection result may be inaccurate. The embodiments described below can ensure accurate detection results even under various terminals or environments that the user can use or experience by using an appropriate convolutional neural network according to the quality type.

Determining (S101) the quality type of the target image comprises determining at least one quality value of the target image corresponding to the at least one quality parameter; Determining at least one quality classification of a target image based on at least one quality type classification standard preset corresponding to at least one quality value and at least one quality parameter; And determining a quality type of the target image based on the at least one quality classification.

The quality type classification standard can be defined for each quality parameter. The quality classification standard classifies the quality classification according to the quality value of the target image for each quality parameter. For example, the quality classification standard of the resolution parameter can classify the quality classification of the target image into a high resolution, a medium resolution, and a low resolution according to the resolution value of the target image.

The quality type of the target image may be determined by a combination of one or more quality indications. For example, the quality type of the target image may be expressed as {resolution: high, brightness: low, sharpness: medium}. As will be described in detail below, the quality type of the target image may be variously modified by a method of determining a representative value among the quality classifications.

Determining (S102) a convolutional neural network of a quality type corresponding to the quality type of the target image comprises determining, from a database comprising convolutional neural networks of the plurality of learned quality types, a quality corresponding to the quality type of the target image Lt; RTI ID = 0.0 > a < / RTI > type of convolutional neural network. As will be described in detail below, a database may be obtained by determining the quality type of a plurality of sample images and training a convolutional neural network of the corresponding quality type based on sample images of each quality type.

According to one embodiment, the convolutional neural network used for target detection may be a single stage convolutional neural network. Based on the output of the convolutional neural network, it can be determined whether the target in the target image is a biometric target. As will be described in detail below, depending on the application of the target detection, it is required that the probability that the sample containing the real target is classified as the normal sample is more than a certain probability, or the probability that the sample containing the false target is classified as the correct sample It may be required to be less than a probability. The performance of the convolutional neural network is improved by adjusting the structure and / or parameters of the convolutional neural network, thereby satisfying the performance requirements required in each application.

According to another embodiment, the convolutional neural network may be a cascaded convolution neural network. The cascaded convolutional neural network may include convolutional neural networks of a plurality of stages. As will be described in greater detail below, when using convolutional neural networks of a plurality of stages, the convolutional neural network of each stage may have a performance level that is lower than the performance requirements required for the entire cascaded convolutional neural network.

(S103) of determining a detection value of a target image based on a convolutional neural network of a corresponding quality type comprises: inputting a target image to a convolutional neural network selected from a database; And obtaining a detection value from the output of the convolutional neural network. As will be described in detail below, the convolutional neural network can output a probability that a target image is classified as a normal sample including a real target, and a probability that the target image is classified as a normal sample including a real target is a detection value Can be used.

The step (S104) of determining whether or not the target in the target image is a real target based on the detected value of the target image may include a step of determining whether the target is true or false by comparing the detected value with a preset threshold value.

Embodiments not only require a user to perform a specific operation to detect a target but also perform robustness to determine an authenticity target by performing accurate target detection on target images acquired under various hardware conditions or shooting environments .

2A is a flow diagram of a training method of a convolutional neural network according to an embodiment. Referring to FIG. 2A, a training method of a convolutional neural network includes determining (S201) a quality type of a plurality of sample images.

Obtaining a quality value corresponding to at least one quality parameter for each of the plurality of sample images; A quality type of the sample image is determined based on at least one quality value of the sample image.

The quality type classification standard for determining a quality type can be predetermined based on experimental data, historical data, empirical data and / or actual conditions. For example, the quality type classification standard of resolution is that when the resolution of a short side of an image (e.g., a vertical resolution) is greater than 1080 pixels each, greater than 720 pixels, not greater than 1080 pixels, and not greater than 720 pixels, And may include a criterion for classifying the quality classification into high-quality resolution, medium-quality resolution, and low-quality resolution, respectively.

According to one embodiment, the quality type of the image can be determined based on the quality classification based on the hierarchical structure of the quality parameters. For example, the quality classification of the imaging parameters is determined according to the parameter having the lowest quality classification among the plurality of parameters belonging to the imaging parameter of the image, and the quality classification of the imaging parameter is determined according to the parameter having the lowest quality classification among the plurality of parameters belonging to the image The quality classification of the attribute parameters can be determined. In this case, the quality type of the image can be determined by a combination of the quality classification of the photographing parameter and the quality classification of the attribute parameter.

As another example, if the quality classification of the arbitrarily drawn parameter among the plurality of parameters belonging to the image pickup parameter and the quality classification of the arbitrarily drawn parameter among the plurality of parameters belonging to the attribute parameter are the same quality classification (for example, high quality) After the lottery is repeated until reaching, the quality type of the image can be determined based on the quality classification.

As another example, the quality type of the image may be determined based on the quality classification of each of the plurality of parameters belonging to the shooting parameter and the plurality of parameters belonging to the attribute parameter.

For example, the resolution quality type of the sample image is determined based on the quality type classification standard of the preset resolution and the resolution of the short side of the sample image; The quality type of the ISO of the sample image is determined based on the ISO of the predetermined ISO quality classification standard and the sample image; The quality type of the comparative chart of the sample image is determined based on the degree of comparison of the quality type classification standard and the sample image of the preset comparative chart.

In this case, the quality type of the sample image is determined based on the resolution of the sample image, the ISO, and the quality type of the comparative chart. For example, when the resolution of the sample image and the degree of comparison with ISO are both high quality, the quality type of the sample image can be determined to be high quality. As another example, when the resolution of the sample image and the degree of comparison with ISO are high quality, medium quality and low quality, respectively, the quality type of the sample image can be determined as medium quality. As another example, the quality type of the sample image may be determined by a quality type of three latitudes including a high quality of resolution of the sample image, a medium quality of ISO, and a low quality of comparison.

According to one embodiment, by performing a blind image quality evaluation on a sample image, the quality type of the sample image can be determined. For example, a method based on BRISQUE (Blind / referenceless image spatial quality evaluator) based on spatial information, a method based on GM-LOG (Gradient Magnitude and Laplacian of Gaussian, slope size and Gaussian Laplacian) Methods based on high order statistics aggregation (HOSA) can be used for blind image quality evaluation.

As an example of a BRISQUE method based on spatial information, a spatial standardization process is performed on a primitive sample image to remove an average value and divided into standard differences; Characterized by a distribution parameter obtained by fitting a parameter distribution of a sample image after the spatial normalization process using a generalized Gaussian distribution; And determining an image quality evaluation result using an evaluation model obtained by pre-training using a method of support vector regression. Here, the evaluation model is a training that is performed using a large number of images labeled with an image quality evaluation value. In the state where image features and quality evaluation values are provided, a mapping relation between the support vector rare learning feature and the quality evaluation value .

Based on the quality classification standard of the image quality evaluation result based on the BRISQUE, the quality type classification of the image quality evaluation result acquired based on the BRISQUE method can be performed to determine the quality type of the sample image.

The training method of the convolutional neural network includes selecting a convolutional neural network of quality type corresponding to the determined quality type among the plurality of quality type convolutional neural networks (S202).

Convolutional neural networks of a plurality of quality types may have the same structure or may have different structures from each other. Convolutional neural networks of different quality types, however, are trained based on different image samples and thus have different parameters (e.g., synapse weights, etc.).

As will be described below, when a cascaded convolution neural network is used, the number of stages may vary as the required performance conditions for different quality types are different. Even in this case, the structures of the single stage convolutional neural networks may be the same or different from each other.

According to one embodiment, prior to training the convolutional neural network, selectable quality types are predetermined, and the structure of the convolutional neural network may be determined for each quality type.

The training method of the convolutional neural network includes training (S203) a convolutional neural network of a selected quality type based on whether the sample image is a positive or a negative sample. The sample image corresponds to a positive sample when the sample image includes a real target, and the sample image corresponds to a negative sample when the sample image does not include a real target.

For each quality type of sample image, a convolution neural network of the corresponding quality type may be trained according to a plurality of sample images of the corresponding quality type. For example, a group of a plurality of quality types in which a plurality of sample images are classified may be formed so that sample images in the same group have the same quality type. In this case, a convolutional neural network of the corresponding quality type is trained based on a plurality of sample images having the same quality type in each group.

For example, if the quality type of a group of sample images includes a high quality resolution, ISO, and a degree of comparison, a high quality cascade convolution neural network is trained based on the sample images of that group. Alternatively, when the resolution of a group of sample images is high quality, the ISO is medium quality, and the comparability is low, a corresponding quality type (sample image resolution is high quality, ISO is medium quality, A low-quality convolution neural network is trained.

According to one embodiment, a database may be established based on the convolutional neural network of each quality type.

2B is a diagram illustrating a structure of a cascaded convolutional neural network according to an embodiment. Referring to FIG. 2B, each quality type of cascaded convolutional neural network includes at least two stages of convolutional neural networks and at least one threshold decision layer. The threshold decision layer of the current stage is connected between the convolution neural network of the current stage and the convolution neural network of the next stage. Specifically, the input node of the threshold value determination layer of the current stage is connected to the output layer of the convolution neural network of the current stage; The output node of the threshold decision layer of the current stage is connected to the input layer of the convolution neural network of the next stage.

In FIG. 2B, CNNs (Convolutional Neural Networks) 1 210 and 2 220 represent first and second stages of convolutional neural networks, respectively; The threshold value 1 (215) indicates the threshold value determination layer of the first stage and is connected between the convolution neural networks of the first and second stages. On the same principle, the threshold value 2 (225) And is connected between the convolution neural network of the second stage and the convolution neural network (not shown) of the third stage.

The training method of the cascaded convolution neural network of each quality type includes a method of determining the number of stages of the cascaded convolution neural network, a training method of the convolution neural network of each stage, and a method of determining the threshold value of the threshold value layer of each stage .

In one embodiment, the output of the convolutional neural network at each stage has a performance index of TPR (True Positive Rate) and FPR (False Positive Rate). The TPR indicates the rate at which the positive sample is correctly classified as a normal sample. FPR refers to the rate at which the sub-sample is incorrectly classified as a normal sample.

A method for determining the number of stages of a cascaded convolutional neural network includes determining a number of stages of a cascade convolutional neural network of the corresponding quality type based on a performance index required by the cascade convolutional neural network of each quality type. The cascade convolutional neural network of each quality type requires a different figure of merit and the number of stages included in the cascade convolution neural network of each quality type can be determined by the corresponding figure of merit.

For example, a cascaded convolutional neural network of each quality type may require TPR = 99.5%, FPR = 0.1%. At this time, the convolution neural network of the first stage can adjust the threshold value to TPR = 99.9% and FPR = 10%, and the convolution neural network of the second stage can adjust the threshold value to TPR = 99.9% and FPR = 10% . The performance index FPR = 10% * 10% = 1%, in which two convolutional neural networks are cascaded, does not satisfy the FPR requirement (0.1%) of the entire cascaded convolution neural network. The FPR requirements of the entire cascaded convolutional neural network can only be reached by further cascading the convolutional neural network of the third stage with the same performance as the convolutional neural networks of the first and second stages. The number of stages in the network may be determined to be three. The performance index of the first, second and third stage convolutional neural networks cascaded meets the performance requirements of the entire cascaded convolutional neural network (TPR = 99.9% * 99.9% * 99.9%> 99.5%, FPR = 10% 10% * 10% < = 0.1%). As such, the performance index required by the cascaded convolutional neural network can be met by a combination of the performance indices of the convolutional neural networks of the plurality of stages included in the cascaded convolutional neural network.

Hereinafter, a training method of the convolutional neural network of each stage will be described.

Specifically, a sample image of a real target contained in a plurality of sample images of each quality type is regarded as a normal sample; A sample image including a fake target is set as a sub-sample. A spoofed target may include a print image of a real target, a photograph of a real target, a screen showing a real target, a 3D print model of a real target, and the like.

The iterative training is performed for the sequential distribution of the convolution neural network of each stage. The parameters of the convolutional neural network of the first stage are repeatedly trained using a back propagation algorithm while inputting positive and negative samples. The TPR of the first stage convolutional neural network may reach one relatively high value (e.g., TPR = 99.9%) and the FPR may not be high (e.g., FPR = 20%). In this case, the sub-sample is partially misclassified as a positive sample. Next, the positive and negative samples classified in the convolution neural network of the first stage are selected and the repeated training is performed on the parameters of the convolution neural network of the second stage. If the final positive and negative samples classified in the convolution neural network of the second stage are selected based on the same principle and the repeated training is performed on the parameters of the last stage convolutional neural network, finally, the angles of the cascade convolution neural network A convolutional neural network of the stage can be obtained. For convenience of explanation, the case where the number of stages is three has been described as an example, but the number of stages can be variously modified.

2C is a diagram illustrating a structure of a single stage convolutional neural network according to an embodiment. Referring to FIG. 2C, the convolutional neural network includes sequentially cascaded input layers, first through sixth subnetworks, an entire connection layer, and an output layer. The first, second, third, fourth or sixth subnetwork includes a convolution layer, a BN (Batch Normalization) layer, a ReLU (Rectified Linear Unit) layer and a Pooling layer. The fifth sub-network includes a convolution layer and a BN layer.

2C, the input image 250 has dimensions of 128X128X3. 128x128 is the resolution of the input image and 3 is the number of image channels (for example, R (Red), G (Green), B (Blue) channel). And 120X120X3 between the input image and the convolution layer 1 261 of the first subnetwork represent the dimensions of the CNN input. In order to obtain the image of the 120X120X3 dimension from the input image, the input image can be edited with three 120X120 matrices, such as a center cut.

In 3X3X3X16 of convolution layer 1 261 3X3 indicates that the unit scan template of convolution layer 1 is a 3X3 pixel point matrix; The third number 3 indicates the number of pixel points of the previous class (i.e., image channels); 16 includes the number of convolution kernels or filters (also referred to as the depth of the convolution layer) included in convolution layer 1. Each of the convolution kernels or filters of the convolution layer 1 261 has a 3x3 pixel point matrix as a unit scan template and sets a predetermined number of pixel points (for example, 1) Scans for a pixel point matrix. During the scan process, each convolution kernel or filter sequentially convolves each 3X3 pixel point in the 120X120 pixel point matrix corresponding to each primary color, and a plurality of first convolution results obtained sequentially are stored in a first (Pixel point matrix) X16 (layer) number of pixel points after the first convolution as a plurality of pixel points after the second convolution. Next, normalization is performed on the pixel points after the first convolution of each layer with the layer BN1 (i.e., the first group normalization) 262 to obtain 16 feature maps after the first convolution, and each feature map is divided into 120X120 (I.e., the definition of 120X120X16 between BN1 262 and the first ReLU 263). Normalization of the BN layer can improve the convergence speed of the convolutional neural network, and shooting conditions such as different illuminance are advantageous to improve the performance of the convolutional neural network by reducing the influence of the convolutional neural network performance.

In FIG. 2C, the first ReLU 263 may be specifically the first activation function and represents one-way activation of the feature maps after the 16 first convolution operations. For example, a value greater than or equal to 0 is maintained in the feature map, and a value less than 0 is reset to 0 and output or activated. This can sparse the parameters of the feature map to weaken the associativity between the parameters and reduce the tendency of the convolution neural network to overfit during training. In FIG. 2C, the first MaxPool (Max Pooling) 264 represents the first pooling. The first pooling layer can be pooled for each of the 16 first convolved feature maps using the maximum pooling method. The size of the pooling zone is 2X2, for example, one maximum value within every 2X2 zones may be selected as the representative value. As a result, sixteen first post-pulling feature maps (i.e., the definition of 60X60X16 between the first MaxPool and convolution layer 2) can be obtained.

Substantially the same as the above, internal structures and operations of the second to sixth subnetworks can be described.

In FIG. 2C, the 4X4X64 shown in the lower column shows a feature map that has passed the 64 sixth subnetwork processes output by the sixth subnetwork. Each feature map includes a 4X4 pixel point matrix. In FIG. 2C, the entire connection layer 271 shown in the lower column indicates all connection layers connected to each other between adjacent layers, and 1024X2 denotes all of the connection layer parameters; The entire link layer converts each pixel of the feature map of 64 4X4 pixel points into a vector of 1X1024 and performs a matrix multiplication operation on the transformed vector and the parameter matrix 1024X2 to obtain a result of 1X2 (Multinomial logistic regression) 272 to the output layer indicated by Softmax (multinomial logistic regression). The output from the output layer includes a probability that the input image 250 is classified as a normal sample and a probability that the input image 250 is classified as a sub-sample. The probability that the input image 250 is classified as a positive sample may be determined as the detected value 280 processed by the convolutional neural network of the input image.

A method of determining the threshold value of the threshold value determination layer of each stage after being connected to the convolutional neural network of each stage will be described below.

When the detection value for the input image is output by the convolutional neural network of each stage according to the embodiment, the detection value may be selected from the threshold value determination layer of the stage. For example, the image of the detection value passed through the selection is provided as the input image of the convolution neural network of the next stage, and the image of the detection value that fails to pass the selection is determined as the false target image, May not be provided.

According to one embodiment, in the training process, the convolution neural network of the initial stage may be trained to partially include mis-determination, and mis-determined sample images may be utilized as a sub-sample of the convolution neural network of the next stage .

The threshold value of the threshold value determination layer of each stage can be set reasonably based on actual conditions such as experimental data, experience data, historical data, and / or actual target identification rate to be finally reached. Through the selection of the threshold value layer, the image input from the convolution neural network of each subsequent stage helps improve the classification accuracy of the convolution neural network of the next stage and can improve the classification accuracy of the true target image as a whole.

In one embodiment, the TPR and FPR performance output by the convolutional neural network of the current stage may be the result of synthesizing the performance of substantially the current stage and the convolutional neural network of all previous stages. For example, the sum of the probability that the input image is classified as a positive sample and the probability that the input image is classified as a negative sample is 1, and the detection value may be a real number larger than 0 and smaller than 1. In one embodiment, threshold values of the threshold value determination layers of the first, second, and third stages may be set to 0.2, 0.3, and 0.2, respectively.

3A is a flowchart of a scenario in which a target detection method according to an embodiment is applied to an actual application. Referring to FIG. 3A, a target image is acquired in an actual application (S301); Determining a quality type of the acquired target image (S302); Determining (S303) a cascade convolution neural network of a quality type corresponding to the quality type of the target image; Determining (S304) the detected value of the target image based on the cascade convolution neural network of the corresponding quality type; Determining whether the target in the target image is a real target based on the detected value of the target image (S305); And performing a corresponding process based on the determined real target (S306).

A terminal equipment according to an embodiment acquires a target image for a target through a photographing apparatus. Here, the target image is a single image. According to the embodiment, the terminal equipment may acquire a continuous image for the target, and may use the image of each target included in the continuous image as a target image. For example, the terminal equipment may acquire a video for a target, and may use a frame image of each target included in the video, that is, a target frame image as a target image.

Determining a quality type of the acquired target image (S302), determining (S303) a cascade convolution neural network of a quality type corresponding to the quality type of the target image, and determining a cascade convolution neural network (S304) of determining the detected value of the target image based on the pixel values of the target image corresponds to steps S101, S102, and S103 in Fig.

In step S304, the determination of the detected value of the current stage of the target image may be performed through the following process. Determine a detected value of a current stage of the target image based on the convolution neural network of the current stage; It is determined whether or not the detected value of the current stage is greater than a real target detection threshold of the current stage based on the threshold value layer of the current stage connected between the current stage and the convolution neural network of the next stage. If the detected value of the current stage is larger than the threshold value of the current stage, the process proceeds to the determination of the detected value of the next stage of the target image and proceeds to the determination of the detected value of the last stage of the target image as the detected value of the target image.

For example, if a target image is input to the cascade convolutional neural network of FIG. 2B described above, the detection value of the first stage of the target image is determined based on the convolution neural network CNN1 of the first stage; It is determined whether or not the detected value of the first stage is larger than the threshold value (threshold value 1) of the threshold value determination layer of the first stage based on the threshold value determination layer of the first stage. When the detected value of the first stage is not larger than the threshold value of the threshold value determination layer of the first stage, the target in the target image is determined as a false target, and the target image is no longer included in the target classification of the convolution neural network of subsequent stages And outputs the detected value of the first stage as the detected value of the target image. If the detected value of the first stage is larger than the threshold of the threshold value determination layer of the first stage, the target image of the first stage, which has passed the convolutional neural network classification of the first stage, (CNN2). When the detection value output from the convolution neural network of the last stage is greater than the threshold value of the threshold value determination layer of the last stage, the detected target may be a real target, and the threshold value determination layer of the last stage The detection value of the last stage larger than the threshold value is set as the detection value of the target video.

The step S305 of determining whether or not the target in the target image is a real target based on the detected value of the target image corresponds to step S104 in Fig.

According to an exemplary embodiment, when the target image is a single image, the target value of the target image is compared with a preset real target detection threshold value; When the detected value of the target image is larger than the threshold value, it is determined that the target of the target image is a real target; When the detected value of the target image is not larger than the threshold value, it is determined that the target in the target image is close to the target. Here, the true target detection threshold can be preset by experiment data, experience data, historical data and / or actual situation; For example, the true target detection threshold may be set to 0.3.

According to an exemplary embodiment, when the target image is a frame image, a comprehensive detection value can be determined using the detected values and the ambiguous evaluation values of each frame. The morphological evaluation value of the target image can be acquired and stored through evaluation of the degree of ambiguity of the target image.

For example, it is possible to evaluate the degree of ambiguity of the target image using a method such as JNB (Just noticeable blur), Cumulative Probability of Blur Detection (CPBD), or cumulative probability of blur detection.

Taking the CPBD method as an example, first, a target image is divided into a plurality of target image blocks; Detect the horizontal edges of each target image block using the Canny (edge) or Soble (edge) edge detection operators. Then, the ratio of edge pixels is calculated. For example, if the ratio of the edge pixels of one target image block is larger than a predetermined number (for example, 0.002), the target image block is determined as the edge image block, and if the ratio of the edge pixels is a predetermined number , 0.002), the corresponding target image block is determined as a non-edge image block.

를 계산한다.Further, using equation (1), the minimum clear edge width based on the comparative diagram C for the edge pixel ei in the edge image block

.

Calculates the actual edge width w (ei) of the edge pixel ei, and calculates the edge blur probability P _blur based on Equation (2). In Equation (2),? Is one fixed parameter.

The percentage of statistics occupied by edge pixels of which P _blur is less than a predetermined number (for example, 0.63) in all edge pixels is taken as an ambiguous detection value. The more ambiguous the image, the lower the pixel ratio of P _blur is, and the corresponding blurred test value becomes smaller. As will be described below, since the comprehensive detection value is determined with the obscured detection value as a weight, the influence on the detection algorithm for the real target (biometric target) can be lowered as the morphological image with the smaller detection value is smaller.

After determining the blurred test value of the current frame target image, the blurred test value is stored. On the same principle, the blurred detection values of a plurality of frame target images before the current frame can be predetermined and stored. Also, it stores the detected value of the current frame target image determined in the above description. For the same reason, the detected values of the plural frame target images before the current frame can be determined and stored in advance.

A comprehensive detection value of the current frame target image is determined based on the detected value and the ambiguous evaluation value of the current frame target image and the previous plural frame target image. For example, a comprehensive detection value of a current frame target image is determined by calculating a weighted average value of each detected value as a weight of a detected value of a current frame target image and a previous evaluation value of a plurality of frame target images .

FIG. 3B is a view for explaining a target detection method for a frame image according to an embodiment. Referring to FIG. 3B, the biometric test value indicates a detected value, the current frame N (311) in the upper row indicates a detected value of the current frame target image, and N is a positive integer. The upper row frame N-i 312 indicates the detected value of the previous frame with the current frame and i frame interval, and i is a positive integer smaller than N. [

In FIG. 3B, the current frame N (321) in the lower row indicates the evaluation value of the current frame target image, and the frame N-i 322 in the lower column indicates the evaluation value of the previous frame with the current frame and the i frame interval. Multiply the result of the multiplication of each frame image by multiplying the detected value and the evaluation value of each frame image in the current frame target image and the previous frame image to obtain a comprehensive detection value 330 of the current frame target image.

Referring back to FIG. 3A, in step S305, it is possible to determine whether the target in the current frame target image is a real target, based on the comprehensive detection value of the current frame target image. For example, the comprehensive detection value of the current frame target image is compared with a preset real target detection threshold value; When the comprehensive detection value is greater than the threshold value, the target in the current frame target image is determined as a real target; When the comprehensive detection value is not larger than the threshold value, the target in the current frame image can be determined as a false target.

In step S306, corresponding processing may be performed depending on whether the target is true or not. If the target in the target image determined in the above step is a real target, the processing step associated with the target image is executed. For example, a settlement step in which a target image is associated, or an unlocking step or the like may be executed. If the target in the target image determined in the above step is a false target, the execution of the associated processing step associated with the target image is rejected. For example, the unlocking step or the execution of the settlement step associated with the target image may be rejected.

4 is a block diagram of a target detection apparatus according to an embodiment. Referring to FIG. 4, the target detection apparatus includes an image quality type determination unit 401, a convolutional neural network determination unit 402, a detection value determination unit 403, and a target truth determination unit 404.

Here, the image quality type determining unit 401 determines the quality type of the target image. The convolutional neural network determination unit 402 determines a convolutional neural network of the quality type corresponding to the quality type of the target image determined by the image quality type determination unit 401. [ The detected value determiner 403 determines the detected value of the target image based on the convolutional neural network of the corresponding quality type determined by the convolutional neural network determiner 402. [ The target truth determining unit 404 determines whether or not the target in the target image is a real target based on the detected value of the target image determined by the detected value determining unit 403. [

According to one embodiment, the convolutional neural network determiner 402 may access a database 406 comprising convolutional neural networks of a plurality of quality types to select a corresponding convolutional neural network. The database 406 may be included in the target detection device or may be wired or wirelessly connected to the target detection device.

According to one embodiment, the convolutional neural network training unit 405 may train convolutional neural networks of a plurality of quality types included in the database 406. [ The convolutional neural network training unit 405 may be included in the target detection apparatus or may be implemented as a separate server or the like.

The convolutional neural network determination unit 402, the detection value determination unit 403, the target truth determination unit 404, the database 406, and the convolutional neural network training unit 405 are connected to the video quality type determination unit 401, the convolutional neural network determination unit 402, The above-described matters may be applied as they are, so that detailed description will be omitted.

The results of the target detection experiment according to one embodiment will be described below. The training database contains a total of 39,1760 sample images, of which 115,115 are true target images (e.g., human face images of a human being) and 276615 false target images (e.g., fraud attack images). The ratio between the real target image and the fake target image is about 1: 3, and the sample images are collected from about 500 individuals. Attack images include prints, screen images, photographs, etc. impersonating a real person's face.

The training database described above is categorized into training sets and test sets (e.g., 80% of the images are used for training and 20% of the images are used for testing), convolution of each stage in the cascaded convolutional neural network The repeated training was sequentially performed on the neural network. The test results are shown in Table 1 below.

Structure of convolution neural network Classification Accuracy of Convolutional Neural Networks Single-Stage CNN TPR = 97.0%, FPR = 1.0% Cascade of multiple stages CNN TPR = 99.2%, FPR = 1.0%

As can be seen from the above-mentioned Table 1, the cascade CNN of the plurality of stages is superior to the single stage CNN in accuracy performance.

5 is a diagram illustrating robustness against various fake attacks according to an embodiment. Referring to FIG. 5, the left four images are attack images including a false target, and the right image is a target image 521 including a real target. Attack video is attack image (511) of mobile phone screen including real person face image, attack image (512) of screen screen including real person face image, attack image (513) including real person face photograph, And an attack image 514 including a face image. In other words, the target in the attack image may not be a real target, but may be a photograph of a real target, a display screen, or a real printed image.

6 is a diagram illustrating robustness against fraud attacks of the low quality type according to one embodiment. Referring to FIG. 6, the left image 611 is an ambiguous image including the real target detected, and the intermediate image 621 and the right image 622 are ambiguous attack images including the detected false target.

In actual application of the target detection system, video frame images are continuously collected through a photographing device included in a terminal device such as a mobile phone, and target detection is performed. As shown in FIG. 6, motion blur and motion may occur in the terminal equipment, and motion blur distortion may appear in the collected frame image. According to the existing algorithm, it is not possible to effectively distinguish between an ambiguous real target image and an attack image (i.e., a false target image). On the other hand, according to the above-described embodiments, it is possible to effectively reduce the probability that the true-point target in the ambiguous image in which the motion-blur distortion appears in the frame image is misjudged, and to accurately detect the real target in the ambiguous image.

The embodiments described above may be implemented in hardware components, software components, and / or a combination of hardware components and software components. For example, the devices, methods, and components described in the embodiments may be implemented within a computer system, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, such as an array, a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For ease of understanding, the processing apparatus may be described as being used singly, but those skilled in the art will recognize that the processing apparatus may have a plurality of processing elements and / As shown in FIG. For example, the processing unit may comprise a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as a parallel processor.

The software may include a computer program, code, instructions, or a combination of one or more of the foregoing, and may be configured to configure the processing device to operate as desired or to process it collectively or collectively Device can be commanded. The software and / or data may be in the form of any type of machine, component, physical device, virtual equipment, computer storage media, or device , Or may be permanently or temporarily embodied in a transmitted signal wave. The software may be distributed over a networked computer system and stored or executed in a distributed manner. The software and data may be stored on one or more computer readable recording media.

The method according to an embodiment may be implemented in the form of a program command that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions to be recorded on the medium may be those specially designed and configured for the embodiments or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

Although the embodiments have been described with reference to the drawings, various technical modifications and variations may be applied to those skilled in the art. For example, it is to be understood that the techniques described may be performed in a different order than the described methods, and / or that components of the described systems, structures, devices, circuits, Lt; / RTI &gt; or equivalents, even if it is replaced or replaced.

Claims (30)

Determining a quality type of the target image;
Determining, from a database comprising a plurality of convolutional neural networks, a convolutional neural network of a quality type corresponding to a quality type of the target image;
Determining a detection value of the target image based on the convolutional neural network of the corresponding quality type; And
Determining whether the target in the target image is a real target based on the detected value of the target image
Gt; The method according to claim 1,
The quality type of the target image is
Is determined based on at least one quality value of the target image determined corresponding to at least one quality parameter. 3. The method of claim 2,
The at least one quality parameter
An image pickup parameter of the target image, and an attribute parameter of the target image. The method according to claim 1,
The measured value
Wherein the target image comprises a probability that the target image has been classified into a positive sample containing a real target. The method according to claim 1,
The step of determining the quality type of the target image
Determining at least one quality value of the target image corresponding to at least one quality parameter;
Determining at least one quality classification of the target image based on the at least one quality value and at least one quality type classification standard preset corresponding to the at least one quality parameter; And
Determining a quality type of the target image based on the at least one quality classification
The method comprising: The method according to claim 1,
The step of determining the quality type of the target image
Determining a quality type of the target image by performing a blind image quality evaluation on the target image
Gt; The method according to claim 1,
The database includes:
Determining a quality type of the plurality of sample images and training the convolutional neural network of the corresponding quality type based on the sample images of each quality type. The method according to claim 1,
The convolutional neural network
And a cascaded convolutional neural network comprising convolutional neural networks of at least two stages and at least one threshold value layer. 9. The method of claim 8,
Wherein a cascaded convolutional neural network of each quality type requires a different figure of merit and the number of stages included in the cascade convolution neural network of each quality type is determined by a corresponding figure of merit. 10. The method of claim 9,
The figure of merit includes a true positive rate (TPR) corresponding to a fraction of a positive sample to a positive sample, and a false positive rate (FPR) corresponding to a fraction of a negative sample to a positive sample , Target detection method. 9. The method of claim 8,
Wherein the figure of merit required by the cascaded convolutional neural network is satisfied by a combination of the figure of merit of the convolutional neural networks of the plurality of stages included in the cascaded convolutional neural network. 9. The method of claim 8,
Wherein the step of determining the detected value of the target image comprises:
Determining a detected value of a first stage of the target image based on a convolutional neural network of a first stage included in the cascaded convolutional neural network; And
Performing a process of determining a detected value of a next stage of the target image by comparing a detected value of the first stage with a predetermined threshold value based on a threshold value determination layer connected to the convolution neural network of the first stage
The method comprising: 13. The method of claim 12,
The step of determining the detection value of the next stage
Sequentially determining a detection value of the last stage in accordance with a result of comparison between a detected value of each stage after the next stage and a corresponding threshold value
Gt; 13. The method of claim 12,
Setting a detected value of the next stage as a detected value of the target image when the detected value of the first stage is larger than the threshold value; And
Determining a detected value of the first stage as a detected value of the target image when the detected value of the first stage is smaller than the threshold value
Further comprising the steps of: The method according to claim 1,
Determining whether a target in the target image is a real target based on the detected value of the target image,
Determining a motion estimation value of the current frame when the target image is a current frame in the frame image;
A comprehensive detection value of the target image is determined based on a voiced evaluation value of the current frame, a detected value of the current frame, a voiced evaluation value of a plurality of previous frames in the frame image, and detection values of the plurality of previous frames step; And
Determining whether a target in the current frame is a real target based on a comprehensive detection value of the target image
Gt; 16. The method of claim 15,
The step of determining the evaluation value
Calculating the morphological evaluation value based on the ambiguity probabilities of the edge pixels in the current frame
The method comprising: 16. The method of claim 15,
Wherein the step of determining a comprehensive detection value of the target image comprises:
Determining a weighted average value of the detected values using the voiced evaluation values of the current frame and the plurality of previous plural frames as weight values of the corresponding detected values, and determining the weighted average value as the integrated detected value
Gt; Determining a quality type of the sample image;
Selecting a convolutional neural network of a quality type corresponding to the determined quality type of convolutional neural networks of a plurality of quality types; And
Training the convolutional neural network of the selected quality type based on whether the sample image is positive or negative,
Gt; a &lt; / RTI &gt; convolutional neural network. 19. The method of claim 18,
Wherein the sample image corresponds to a positive sample when the sample image includes a real target and the sample image corresponds to a negative sample if the sample image does not include the real target. 19. The method of claim 18,
The quality type of the sample image is
Wherein the at least one quality parameter is determined based on at least one quality value of the sample image determined corresponding to at least one quality parameter. 19. The method of claim 18,
The step of determining the quality type of the sample image
Determining at least one quality value of the sample image corresponding to at least one quality parameter;
Determining at least one quality classification of the sample image based on the at least one quality value and at least one quality type classification standard preset corresponding to the at least one quality parameter; And
Determining a quality type of the sample image based on the at least one quality classification
Gt; a &lt; / RTI &gt; convolutional neural network. 19. The method of claim 18,
Establishing a database based on the convolutional neural network of each quality type
Further comprising the steps &lt; RTI ID = 0.0 &gt; of: &lt; / RTI &gt; 19. The method of claim 18,
The convolutional neural network
And a cascaded convolutional neural network including at least two convolutional neural networks and at least one threshold value layer. 24. The method of claim 23,
Wherein the cascade convolutional neural network of each quality type requires a different figure of merit and the number of stages included in the cascade convolutional neural network of each quality type is determined by a corresponding figure of merit, Way. 26. A computer program stored in a medium for executing the method of any one of claims 1 to 24 in combination with hardware. An image quality type determining unit for determining a quality type of a target image;
A convolutional neural network determiner for determining, from a database comprising a plurality of convolutional neural networks, a convolutional neural network of a quality type corresponding to a quality type of the target image;
A detection value determiner for determining a detection value of the target image based on the convolutional neural network of the corresponding quality type; And
A target truth determining unit for determining whether or not the target in the target image is a real target based on the detected value of the target image,
And the target detection device. 27. The method of claim 26,
The convolutional neural network
And a cascaded convolutional neural network including convolutional neural networks of at least two stages and at least one threshold value layer. 28. The method of claim 27,
Wherein the cascade convolutional neural network of each quality type requires a different figure of merit and the number of stages included in the cascade convolution neural network of each quality type is determined by a corresponding figure of merit. 28. The method of claim 27,
The detection value determination unit
Determining a detected value of a first stage of the target image based on a convolutional neural network of a first stage included in the cascaded convolutional neural network and determining a detected value of a first stage of the target image based on a threshold value layer connected to the convolutional neural network of the first stage And performs a process of determining a detection value of a next stage of the target image by comparing the detected value of the first stage with a preset threshold value. 27. The method of claim 26,
The target authenticity determination unit determines,
Determining a motion estimation value of the current frame when the target image is a current frame in the frame image; A comprehensive detection value of the target image is determined based on the voiced evaluation value of the current frame, the detected value of the current frame, the voiced evaluation values of a plurality of previous frames in the frame image, and the detection values of the plurality of previous frames ; And determines whether the target in the current frame is a real target based on the integrated detection value of the target image.

Images