KR20230119574A - Method of operating neural network, training method of neural network, method of detecting whether biometruc information is spoofed using neural network, and electric devece thereof - Google Patents

Abstract

According to an embodiment, a method of operating a neural network including an input layer, a plurality of intermediate layers, and an output layer provides first nodes belonging to an arbitrary first intermediate layer adjacent to an input layer among a plurality of intermediate layers. A first intermediate vector is generated by applying a first activation function, the first intermediate vector is transferred to second nodes belonging to a second intermediate layer adjacent to an output layer among intermediate layers, and the second intermediate vector is transmitted to the second nodes. An activation function is applied to generate a second intermediate vector, the second intermediate vector is applied to an output layer, and the multiplier of the second activation function is such that the peak value of the second activation function is fixed. It is determined by a first hyperparameter associated with the rising slope of and a second hyperparameter associated with the falling slope of the second activation function.

Description

A method of operating a neural network, a method of training a neural network, a method of detecting forgery of biometric information using a neural network, and an electronic device thereof SPOOFED USING NEURAL NETWORK, AND ELECTRIC DEVECE THEREOF}

The following disclosure relates to a method of operating a neural network, a method of training a neural network, a method of detecting forgery of biometric information using a neural network, and an electronic device thereof.

Neural networks are becoming an essential component of modern machine learning. Distributions can be derived from inputs to outputs of a neural network by assuming probability distributions for the parameters of the neural network. Users want the neural network to be modeled more flexibly, but it is common for the neural network to be unable to make good judgments about untrained regions, for example.

According to an embodiment, a method of operating a neural network including an input layer, a plurality of intermediate layers, and an out layer includes any one of the plurality of intermediate layers adjacent to the input layer. generating a first intermediate vector by applying a first activation function to first nodes belonging to 1 intermediate layer; transferring the first intermediate vector to second nodes belonging to a second intermediate layer adjacent to the output layer among the intermediate layers; generating a second intermediate vector by applying a second activation function to the second nodes; and applying the second intermediate vector to the output layer, wherein a multiplier of the second activation function is set such that a peak value of the second activation function is fixed. It may be determined by a first hyper parameter associated with an ascending slope of a second activation function and a second hyper parameter associated with a descending slope of the second activation function.

The dynamic range of the second activation function may be limited to [0, 1].

The second activation function ( ) is expressed by the following equation,

,

Here, a represents a first hyperparameter associated with an ascending slope of the second activation function, and b represents a second hyperparameter associated with a descending slope of the second activation function, denotes an Euler number, the x denotes the input of the second nodes, may represent a Heaviside Step function that makes an output of the second activation function 0 when the x is less than 0.

The first activation function is any one of a step function, a sigmoid function, a hyperbolic tangent function, a ReLU function, and a leaky ReLU function can include

The neural network may include any one of a Convolution Neural Network (CNN) and a Deep Neural Network (DNN).

According to an embodiment, a method for training a neural network including an input layer, a plurality of intermediate layers, and an output layer is a first result value obtained by applying a first activation function to intermediate nodes belonging to each of the plurality of intermediate layers. Extracting; extracting a second result value by applying a second activation function different from the first activation function to additional nodes connected to intermediate nodes belonging to at least one arbitrary layer among the plurality of intermediate layers; and training the neural network based on the difference between the first result value and the second result value.

The second activation function is such that a multiplier of the second activation function is associated with a first hyperparameter associated with an ascending slope of the second activation function and a descending slope of the second activation function such that a peak value of the second activation function is fixed. It can be determined by the second hyperparameter.

The number of additional nodes is equal to the number of intermediate nodes -1, and the additional nodes and the intermediate nodes may be fully connected.

The second activation function ( ) is expressed by the following equation,

,

Here, a represents a first hyperparameter associated with an ascending slope of the second activation function, and b represents a second hyperparameter associated with a descending slope of the second activation function, denotes an Euler number, where x denotes an input to the additional nodes, may represent a Heaviside step function that makes the output of the activation function 0 when the x is less than 0.

The dynamic range of the second activation function may be limited to [0, 1].

The first activation function may include any one of a step function, a sigmoid function, a hyperbolic tangent function, a relu function, and a ricky relu function.

According to an embodiment, a method for training a neural network including an input layer, a plurality of intermediate layers, and an output layer includes learning data input to the input layer, and a first intermediate layer adjacent to the input layer among the intermediate layers. generating a first feature vector by propagating to first nodes belonging to the layer and operating according to a first activation function; primary training of the neural network based on a difference between the first feature vector and the correct answer vector corresponding to the learning data; The first feature vector is propagated to second nodes belonging to a second intermediate layer adjacent to the output layer among intermediate layers of the first-trained neural network and operating according to a second activation function to obtain a second feature vector. generating; and performing secondary training on the first trained neural network based on a difference between an output value obtained by outputting the second embedding vector through the output layer and a correct answer value corresponding to the learning data.

The second activation function is such that a multiplier of the second activation function is associated with a first hyperparameter associated with an ascending slope of the second activation function and a descending slope of the second activation function such that a peak value of the second activation function is fixed. It can be determined by the second hyperparameter.

The second activation function ( ) is expressed by the following equation,

[mathematical expression]

,

Here, a represents a first hyperparameter associated with an ascending slope of the second activation function, and b represents a second hyperparameter associated with a descending slope of the second activation function, denotes an Euler number, the x denotes the second feature vector, and may represent a Heaviside step function that makes an output of the second activation function 0 when the x is less than 0.

The dynamic range of the second activation function may be limited to [0, 1].

The first activation function may include any one of a step function, a sigmoid function, a hyperbolic tangent function, a relu function, and a ricky relu function.

According to an embodiment, a method of detecting whether biometric information is forged or not using a neural network includes a plurality of middle points of the neural network that detects whether or not the biometric information is spoofed from input data including the user's biometric information. extracting one or more first feature vectors from the layers using one or more pretrained first classifiers; detecting whether the biometric information is first forged based on the one or more first feature vectors; calculating a second score by applying an output vector output from the output layer to a pretrained second classifier according to whether the first forgery is detected; and detecting whether the biometric information is second forged or not by a fusion score of the first score calculated based on the one or more first feature vectors and the second score, wherein the one or more first classifiers and In at least one of the second classifiers, a multiplier of the activation function is a first hyperparameter associated with an ascending slope of the activation function and a second hyperparameter associated with a descending slope of the activation function such that a peak value of the activation function for the neural network is fixed. It may be trained by the activation function determined by hyperparameters.

The dynamic range of the activation function may be controlled as [0, 1].

The activation function ( ) is expressed by the following equation,

[mathematical expression]

Here, a represents a first hyperparameter associated with an ascending slope of the activation function, and b represents a second hyperparameter associated with a descending slope of the activation function. represents the Euler number, the x represents the input data, and the may represent a Heaviside step function that makes the output of the activation function 0 when the x is less than 0.

The extracting of the one or more first feature vectors may include extracting a 1-1 feature vector from a first intermediate layer among the plurality of intermediate layers by using a 1-1 classifier among the one or more first classifiers; extracting 1-2 feature vectors from a second intermediate layer after the first intermediate layer by using a 1-2 classifier among the one or more first classifiers; and extracting a feature vector obtained by combining the 1-1 feature vector and the 1-2 feature vector.

The step of detecting whether the biometric information is first forged includes: calculating the first score based on a similarity between at least one of a previously prepared registered feature vector and a fake feature vector and the combined feature vector; and classifying whether the first score is a score determined as fake information or a score determined as real information using the one or more first classifiers.

The biometric information may include any one of the user's fingerprint, iris, and face.

According to an embodiment, an electronic device that detects whether biometric information is forged using a neural network includes a sensor that captures input data including the biometric information of a user; From the input data, one or more first feature vectors are extracted from a plurality of intermediate layers of the neural network that detects whether the biometric information is forged or not, using one or more first classifiers trained in advance, and the one or more first feature vectors are extracted. Based on 1 feature vector, whether the biometric information is first forged or not is detected, and according to whether the first forgery or not is detected, an output vector output from the output layer is applied to a pre-trained second classifier to generate a second classifier. a processor that calculates two scores and detects whether the biometric information is second forged or not based on a fusion score of the first score calculated based on the one or more first feature vectors and the second score; and an output device outputting at least one of the first falsification status and the second falsification status, wherein at least one of the one or more first classifiers and the second classifier has a peak value of an activation function for the neural network A multiplier of the activation function to be fixed is trained based on the activation function determined by a first hyperparameter associated with a rising slope of the activation function and a second hyperparameter associated with a falling slope of the activation function.

The activation function ( ) is expressed by the following equation,

[mathematical expression]

Here, a represents a first hyperparameter associated with an ascending slope of the activation function, and b represents a second hyperparameter associated with a descending slope of the activation function. denotes an Euler number, where x denotes an input to the additional nodes, may represent a Heaviside step function that makes the output of the activation function 0 when the x is less than 0.

1 is a diagram illustrating an environment in which an electronic device including a neural network according to an exemplary embodiment is used.
2 is a flowchart illustrating a method of operating a neural network according to an exemplary embodiment.
3 is a diagram illustrating the structure of a neural network according to an embodiment.
4 is a diagram for explaining a region in which a neural network distinguishes fake information and real information according to an exemplary embodiment.
5 is a diagram for explaining an activation function applied to each layer of a neural network according to an exemplary embodiment.
6 is a flowchart illustrating a method for training a neural network according to an exemplary embodiment.
FIG. 7 is a diagram for explaining the training method of FIG. 6 .
8 is a flowchart illustrating a method for training a neural network according to another embodiment.
FIG. 9 is a diagram for explaining the training method of FIG. 8 .
10 is a flowchart illustrating a method of detecting forgery of biometric information using a neural network according to an embodiment.
FIG. 11 is a diagram for explaining the structure and operation of the neural network of FIG. 10 .
12 is a block diagram of an electronic device that detects forgery of biometric information using a neural network according to an exemplary embodiment.

Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only, and may be changed and implemented in various forms. Therefore, the form actually implemented is not limited only to the specific embodiments disclosed, and the scope of the present specification includes changes, equivalents, or substitutes included in the technical idea described in the embodiments.

Although terms such as first or second may be used to describe various components, such terms should only be construed for the purpose of distinguishing one component from another. For example, a first element may be termed a second element, and similarly, a second element may be termed a first element.

It should be understood that when an element is referred to as being “connected” to another element, it may be directly connected or connected to the other element, but other elements may exist in the middle.

Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as "comprise" or "have" are intended to designate that the described feature, number, operation, operation, component, part, or combination thereof exists, but one or more other features or numbers, It should be understood that the presence or addition of an operation, operation, component, part, or combination thereof is not precluded.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and unless explicitly defined in this specification, it should not be interpreted in an ideal or excessively formal meaning. don't

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. In the description with reference to the accompanying drawings, the same reference numerals are given to the same components regardless of reference numerals, and overlapping descriptions thereof will be omitted.

1 is a diagram illustrating an environment in which an electronic device including a neural network according to an exemplary embodiment is used. Referring to FIG. 1 , an electronic device 100 including a sensor 110 for sensing user's biometric information (eg, fingerprint) and registered fingerprint images 121, 122, and 123 are provided according to an exemplary embodiment. An enrolled fingerprint database 120 comprising an enrolled fingerprint database 120 is shown. Hereinafter, for convenience of description, a case in which the user's biometric information is a fingerprint image will be described as an example, but is not necessarily limited thereto. Biometric information may include various information such as an iris image, a palm print image, and a face image in addition to a fingerprint image.

The electronic device 100 may obtain an input fingerprint image 115 in which the user's fingerprint appears through the sensor 110 . The sensor 110 may be, for example, an ultrasonic fingerprint sensor that captures a user's fingerprint, an optical fingerprint sensor, an electrostatic fingerprint sensor, or an image sensor, but is not limited thereto. The sensor 110 may be, for example, the sensor 1210 of FIG. 12 .

Fingerprint registration may be performed for fingerprint recognition. The registered fingerprint images 121 , 122 , and 123 may be previously stored in the registered fingerprint database 120 through a fingerprint registration process. To protect personal information, the registered fingerprint database 120 stores features or feature vectors extracted from the registered fingerprint images 121, 122, and 123 instead of storing the registered fingerprint images 121, 122, and 123 as they are. can also be saved. The registered fingerprint database 120 may be stored in a memory included in the electronic device 100 (eg, the memory 1270 of FIG. 12 ), or may be stored in a server, a local cache, or a server capable of communicating with the electronic device 100 . It may be stored in an external device (not shown) such as a cloud server.

When the input fingerprint image 115 for authentication is received, the electronic device 100 determines the similarity between the input fingerprint shown in the input fingerprint image 115 and the registered fingerprints shown in the registered fingerprint images 121 to 123. The user of the image 115 may be authenticated, or whether the input fingerprint image 115 is spoofed may be detected. Here, 'falsification' means fake (fake) biometric information, not real (live) biometric information, and can be understood as including, for example, all of the duplication, forgery, and falsification of biometric information.

As will be described in detail below, the electronic device 100 according to an embodiment uses a plurality of pre-prepared real fingerprint features, a pre-prepared unspecified number of counterfeit fingerprint features, and/or a registered fingerprint feature of a device user. Authentication or counterfeiting of the input fingerprint may be determined.

2 is a flowchart illustrating a method of operating a neural network according to an exemplary embodiment. In the following embodiments, each operation may be performed sequentially, but not necessarily sequentially. For example, the order of each operation may be changed, or at least two operations may be performed in parallel.

Referring to FIG. 2 , a neural network including an input layer, a plurality of intermediate layers, and an out layer according to an embodiment may perform steps 210 to 240. there is. The neural network may include, for example, any one of a Convolution Neural Network (CNN) and a Deep Neural Network (DNN), but is not necessarily limited thereto.

In step 210, the neural network generates a first intermediate vector by applying a first activation function to first nodes belonging to a first intermediate layer adjacent to an input layer among a plurality of intermediate layers. The first activation function is, for example, a step function, a sigmoid function, a hyperbolic tangent function, a ReLU function, and a Leaky ReLU function. It may include any one, but is not necessarily limited thereto.

The structure and operation of the neural network will be described in more detail with reference to FIGS. 3 to 5 below.

In step 220, the neural network transfers the first intermediate vector to second nodes belonging to a second intermediate layer adjacent to the output layer among the intermediate layers.

In step 230, the neural network generates a second intermediate vector by applying a second activation function to the second nodes. The second activation function will be described in detail with reference to FIG. 5 below.

At step 240, the neural network applies the second intermediate vector to the output layer.

3 is a diagram illustrating the structure of a neural network according to an embodiment.

Referring to FIG. 3 , a schematic structure of a deep neural network (DNN) 300 corresponding to an example of a neural network is shown. The deep neural network 300 may be trained through deep learning.

The deep neural network 300 may include a plurality of layers 310, 320, and 330 composed of a plurality of nodes. The deep neural network 300 may include connection weights that connect a plurality of nodes included in each of the plurality of layers 310, 320, and 330 to nodes included in other layers. The electronic device obtains the deep neural network 300 from an internal database stored in a memory (eg, see memory 1270 in FIG. 12 below) or an output device (eg, see output device 1250 in FIG. 12). It can be obtained by receiving it from an external server through

For example, the deep neural network 300 may include a large number of artificial neurons connected by line-shaped edges. An artificial neuron is represented by a circular shape in FIG. 3 and may be referred to as a node. Artificial neurons may be interconnected through edges having a connection weight. The connection weight is a specific value that edges have, and may be referred to as 'synapse weight', 'weight', or 'connection strength'.

The deep neural network 300 may include, for example, an input layer 310, a hidden layer 320, and an output layer 330. The input layer 310, the hidden layer 320, and the output layer 330 may include a plurality of nodes. A node included in the input layer 110 may be referred to as an input node, and a node included in the hidden layer 320 may be referred to as a hidden node. The hidden layer 320 may be referred to as an 'intermediate layer' in that it is located in the middle of the input layer 310 and the output layer 330. Hereinafter, 'hidden layer' and 'middle layer' may be understood as the same meaning.

A node included in the output layer 330 may be referred to as an output node.

The input layer 310 may receive input data for performing training or recognition and transfer it to the hidden layer 320 . The output layer 330 may generate an output of the deep neural network 300 based on a signal received from the hidden layer 320 . The hidden layer 320 is located between the input layer 310 and the output layer 330 and can change the training input of training data transferred through the input layer 310 to a value that is easy to predict. Input nodes included in the input layer 310 and hidden nodes included in the hidden layer 320 may be connected to each other through connection lines having connection weights. Hidden nodes included in the hidden layer 320 and output nodes included in the output layer 330 may be connected to each other through connection lines having connection weights.

The hidden layer 320 may include a plurality of layers 320-1, .. and 320-N. For example, assuming that the hidden layer 320 includes a first hidden layer, a second hidden layer, and a third hidden layer, an output of a hidden node belonging to the first hidden layer belongs to the second hidden layer. It can be connected to hidden nodes. Outputs of hidden nodes belonging to the second hidden layer may be connected to hidden nodes belonging to the third hidden layer.

The electronic device may input outputs of previous hidden nodes included in the previous hidden layer to the corresponding hidden layer through connection lines having connection weights. The electronic device may generate outputs of hidden nodes included in the hidden layer based on values to which connection weights are applied to outputs of previous hidden nodes and an activation function. According to an example, when a result of the activation function exceeds a threshold value of the current hidden node, output may be ignited to the next hidden node. In this case, the current hidden node may maintain an inactive state without igniting a signal to the next hidden node until a specific threshold activation strength is reached through input vectors.

An electronic device according to an embodiment may train the deep neural network 300 through supervised learning. An electronic device may be implemented as a software module, a hardware module, or a combination thereof. Supervised learning is a technique of inputting a training input of training data and a corresponding training output together to the deep neural network 300, and updating connection weights of connection lines so that output data corresponding to the training output of the training data is output. Training data is data comprising pairs of training inputs and training outputs.

3 represents the structure of the deep neural network 300 as a node structure, embodiments are not limited to this node structure. Various data structures can be used to store the neural network in memory.

According to an embodiment, the electronic device may determine parameters of nodes through a gradient descent technique based on a loss backpropagated to the deep neural network 300 and output values of nodes included in the deep neural network 300. there is.

For example, the electronic device may update connection weights between nodes through loss back-propagation learning. Loss backpropagation learning, after estimating the loss through forward computation for given training data, is estimated starting from the output layer 330 and in the reverse direction toward the hidden layer 320 and the input layer 110. This is a method of updating connection weights in the direction of reducing the loss while propagating one loss.

The processing of the deep neural network 300 proceeds in the directions of the input layer 310, the hidden layer 320, and the output layer 330, but in the lossy backpropagation training, the update direction of the connection weights is the output layer 330, the hidden layer 330, and the hidden layer 330. It may proceed in the direction of the layer 320 and the input layer 310 . One or more processors may use a buffer memory for storing layers or a series of calculation data in order to process the neural network in a desired direction.

The electronic device may define an objective function for measuring how close to optimum the currently set connection weights are, continuously change the connection weights based on the result of the objective function, and repeatedly perform training. For example, the objective function may be a loss function for calculating a loss between an output value actually output by the deep neural network 300 based on training input of training data and an expected value desired to be output. The electronic device may update connection weights in a direction of reducing the value of the loss function.

The deep neural network 300 according to an embodiment is trained to determine real information or fake information through the network, and the output of the final output layer is trained to derive an optimal result. Each intermediate layer may also have discriminative power for determining real information and counterfeit information. Since the output of the intermediate layer may also have discriminative power, if this is used, it is possible to reduce the execution time by deriving whether or not the biometric information is forged before going to the final output layer 330.

An electronic device according to an embodiment may improve detection speed of whether or not to be counterfeit while minimizing degradation in counterfeit detection accuracy by utilizing an intermediate layer having discrimination power in a step before the deep neural network 300 derives a final result. In addition, the electronic device may minimize accuracy degradation by utilizing results of networks receiving different images as inputs in order to compensate for degradation in terms of accuracy due to utilization of the output of the intermediate layer.

4 is a diagram for explaining a region in which a neural network distinguishes fake information and real information according to an exemplary embodiment. Referring to FIG. 4 , a datagram 400 in which regions 410, 420, 430, and 440 are divided according to a feature distribution classified by a neural network according to an exemplary embodiment is shown.

The neural network may be learned using an unspecified number of real biometric information and an unspecified number of counterfeit biometric information. A vector generated by a neural network embeds feature information of biometric information and may be referred to as an 'embedding vector' or a 'feature vector'.

In the feature distribution shown in FIG. 4, the first area 410 corresponds to an area where features or feature vectors extracted from input data are classified as live information, and the second area 420 is extracted from input data. The resulting feature or feature vector may correspond to an area classified as spoof information.

In addition, the third area 430 between the first area 410 and the second area 420 may correspond to a portion that determines the distinguishing power between real information and counterfeit information. The third region 430 may correspond to a portion that allows some errors to prevent over-fitting from occurring for generalization performance. The third region 430 clarifies whether the score ('first score') corresponding to the input data calculated by the neural network is a score corresponding to live information or a score corresponding to spoof information. It may include a critical range that can be clearly distinguished. In the probability distribution of the first score, the threshold range is, for example, a first threshold corresponding to the maximum probability that the first score is determined to be fake information and a second threshold corresponding to the minimum probability that the first score is determined to be real information. can be determined based on

The first region 410 , the second region 420 , and the third region 430 may correspond to in-distribution regions of a feature distribution corresponding to data or feature vectors trained by the neural network.

In addition, in the datagram 400, the fourth area 440 corresponding to the out-of-distribution (OOD) area of the feature distribution is data that the neural network has never seen (unseen data), that is, the neural network It may correspond to a region corresponding to a feature that has never been learned. In this case, it may be difficult for the neural network to determine whether the feature belonging to the fourth region 440 is fake information or real information.

Augmentation or normalization may be used to determine the fourth area 440, but in order to judge any forged fingerprint well, the neural network needs to have a degree that the fourth area 440 is an uncertain area. It may be desirable to make judgments.

Most methods for detecting an input corresponding to the fourth region 440, which corresponds to the outer (ODD) region of the distribution, formulate the problem by augmenting the classifier with knowledge or rejection classes of the outer (ODD) region of the distribution, or Alternatively, while depending on a specific assumption, in one embodiment, the activation function of the neural network may be changed to process an input corresponding to an outside (ODD) region of the distribution.

5 is a diagram for explaining an activation function applied to each layer of a neural network according to an exemplary embodiment. Referring to FIG. 5 , a diagram illustrating a neural network 500 according to an exemplary embodiment is shown.

In an embodiment, a first intermediate vector is obtained by applying a first activation function to first nodes belonging to an arbitrary first intermediate layer 520 adjacent to the input layer 510 among a plurality of intermediate layers of the neural network 500. can create At this time, the first activation function is, for example, a step function, a sigmoid function, a hyperbolic tangent function, a ReLU function, and a Leaky ReLU It may include any one of functions, but is not necessarily limited thereto.

The neural network 500 transfers the first intermediate vector generated by the first intermediate layer 520 to second nodes belonging to the second intermediate layer 530 adjacent to the output layer 540 among the intermediate layers through radio waves. , and a second intermediate vector may be generated by applying a second activation function to the second nodes. The neural network may output an estimation result by applying the second intermediate vector generated in the second intermediate layer 530 to the output layer 540 .

Under certain assumptions, any neural network with at least one intermediate layer can converge to a Gaussian Process (GP) in the limit of infinite width. A Mattern activation function is available for new nonlinear neural networks that mimics the properties derived by the widely used Matιrn Kernel in the Gaussian Process (GP) model.

The Maternian activation function may have similar properties to the activation function of the Gaussian Process (GP) model. The local fixed properties of the Maternian activation function together with the limited mean square derivative can show excellent performance and uncertainty correction ability in Bayesian deep learning tasks. In particular, local stationarity can help correct for uncertainty in the out-of-distribution (OOD) region.

In one embodiment, for example, an activation function ('second activation function') obtained by improving the Mattern activation function derived from the Matιrn Kernel used in a Gaussian Process (GP) can be used. .

First, the Matern activation function derived from the Matern kernel ( ) is a non-linear function, and may be expressed as, for example, Equation 1 below.

Here, Γ(·) represents the Gamma function, q is a constant, and ν and may be a hyper parameter.

For example, when ν > 1/2, the nonlinear function according to Equation 1 may be smooth, continuous, and continuously differentiable. In contrast, when ν ≤ 1/2, the nonlinear function according to Equation 1 takes the form of a step function that decreases exponentially, so it is not smooth and may match the properties of the Matern kernel.

also, may represent a Heaviside Step function that makes the output of the activation function 0 when the input (x) is less than 0. in Equation 1 may correspond to a part representing the shape of the Matern activation function.

The Matern activation function according to Equation 1 ( ) is improved in terms of stationarity compared to other activation functions for the Bayesian Neural Network, and there may be no restrictions on multipliers.

Multiplier of Equation 1 is derived through complex equation development, but since the degree of freedom may increase, for example, in multipliers for white noise appearing in the middle of logic development or multipliers used in Fourier Transform, In development, the multiplier of Equation 1 may be fixed to an arbitrary constant or may not be fixed. In other words, the multiplier of Equation 1 may correspond to a part with sufficient variability.

When the variable multiplier part is expressed as a constant, the dynamics of the neural network can be fixed to some extent by batch normalization and/or normalization properties for weights. However, when fine-tuning the neural network, the multiplier part must be significantly modified, and if the multiplier part is not modified, it may be difficult for the dynamic range of the activation function to exist between 0 and 1.

In one embodiment, the multiplier of Equation 1 Equation 1 is an activation function ('second activation function') expressed as Equation 2 below, considering the degrees of freedom for ? and hyperparameters from the perspective of a neural network. can be modified with

Here, a may represent a first hyperparameter associated with an ascending slope of the second activation function, and b may represent a second hyperparameter associated with a descending slope of the second activation function. Hyperparameters a,b > 0, and can be defined by the user.

may represent an Euler number. x may represent an input to an activation function (eg, an input of second nodes). may represent a Heaviside Step function that makes the output of the second activation function 0 when the input (x) is less than 0.

Second activation function according to an embodiment is a multiplier of the second activation function so that the peak value of the second activation function is fixed, of the first hyperparameter a and the second activation function associated with the ascending slope of the second activation function. It is determined by the second hyperparameter (b) associated with the descending slope. A peak value of the second activation function may be fixed to '1', and a dynamic range of the second activation function may be limited to [0, 1]. Here, [0, 1] may represent a value between 0 and 1.

In Equation 2, the hyperparameters expressed as ν and λ in Equation 1 can be expressed as a and b by normalizing by limiting the multiplier so that the maximum value of the activation function is 1. When the second activation function according to Equation 2 is used through such normalization, the following advantages can be obtained compared to using the activation function according to Equation 1.

i) Considering the dynamic range of the feature vector (or feature) output from the previous layer according to the characteristics of the neural network 500, the dynamic range of the second activation function is fixed to [0, 1] and used Doing so can provide stable input to the next layer. In addition, ii) the processing speed of the neural network 500 can be improved because the neural network can exhibit a fast convergence rate or fast convergence rate through a stable dynamic range input normalized by the second activation function. there is.

In addition, compared to the first activation function, the second activation function according to Equation 2 makes an uncertain decision about an input belonging to the outer (ODD) region of the distribution corresponding to the fourth region 440 of FIG. By providing an uncertain decision, the probability that the neural network 500 outputs an error with respect to an input belonging to an outside (ODD) region of the distribution can be reduced.

6 is a flowchart illustrating a method for training a neural network according to an exemplary embodiment, and FIG. 7 is a diagram for explaining the training method of FIG. 6 . In the following embodiments, each operation may be performed sequentially, but not necessarily sequentially. For example, the order of each operation may be changed, or at least two operations may be performed in parallel.

Referring to FIGS. 6 and 7 , the training apparatus according to an embodiment may train the neural network 700 through steps 610 to 630 .

In step 610, the training device extracts a first result value obtained by applying a first activation function to intermediate nodes belonging to each of the plurality of intermediate layers 720 and/or 730 of the neural network 700. The first activation function may include, for example, any one of a step function, a sigmoid function, a hyperbolic tangent function, a relu function, and a ricky relu function, but is not necessarily limited thereto.

In step 620, the training device includes additional nodes 715 and/or 745 connected to intermediate nodes 711 and/or 741 belonging to one or more optional layers 710 and/or 740 among a plurality of intermediate layers. ) to extract a second result value by applying a second activation function different from the first activation function. At this time, the number of additional nodes 715 and / or 745 is (the number of intermediate nodes 711 and / or 741 connected to the additional nodes 715 and / or 745 - 1), and the additional nodes 715 and/or 745 and intermediate nodes 711 and/or 741 may be fully connected.

The second activation function is such that the multiplier of the second activation function is fixed to '1', for example, the first hyperparameter associated with the rising slope of the second activation function and the falling of the second activation function so that the peak value of the second activation function is fixed at '1'. It may be determined by a second hyperparameter associated with the slope. The second activation function can be expressed, for example, as in Equation 2 above, where x represents an input to the additional nodes 715 and/or 745, may represent a Heaviside step function that makes the output of the second activation function 0 when the input (x) is less than 0. The dynamic range of the second activation function may be limited to, for example, [0, 1].

In step 630, the training device trains the neural network based on the difference between the first result value extracted in step 610 and the second result value extracted in step 620. The training device may train the neural network such that a difference between the first result value and the second result value is minimized.

In an embodiment, additional nodes 715 and/or 745 corresponding to additional decision neurons are connected to one or more arbitrary layers 710 and/or 740 among intermediate layers of the neural network 700. The neural network 700 is trained by applying a second activation function and applying a decision loss between a first result value obtained by applying the first activation function and a second result value obtained by applying the second activation function. can At this time, designing an additional decision loss by connecting additional nodes 715 and/or 745 to one or more arbitrary layers 710 and/or 740 of the neural network 700 is one or more arbitrary layers 710 and/or 740. Alternatively, the same result as applying a gradient obtained by applying the second activation function directly to 740) may be obtained.

8 is a flowchart illustrating a method for training a neural network according to another embodiment, and FIG. 9 is a diagram for explaining the training method of FIG. 8 .

In the following embodiments, each operation may be performed sequentially, but not necessarily sequentially. For example, the order of each operation may be changed, or at least two operations may be performed in parallel.

Referring to FIGS. 8 and 9 , the training apparatus according to an embodiment may train a neural network 900 through steps 810 to 840 . In FIG. 9 , a neural network 900 shown above the dotted line corresponds to a network trained with a first activation function, and a neural network 900-1 shown below the dotted line uses part of the first activation function as a second activation function. may correspond to the fine-tuned neural network 900-1.

In step 810, the training device applies the training data 905 input to the input layer 910 of the neural network 900 to a first intermediate layer 923 adjacent to the input layer 910 among intermediate layers, A first feature vector is generated by propagating to first nodes operating according to a first activation function. The first activation function may include, for example, any one of a step function, a sigmoid function, a hyperbolic tangent function, a relu function, and a ricky relu function.

In step 820 , the training device performs primary training of the neural network 900 based on the difference between the first feature vector and the correct answer vector corresponding to the learning data 905 . Here, the first training may be referred to as "pre-training".

In step 830, the training device belongs to the second intermediate layer 926 adjacent to the output layer 930 among the intermediate layers 920 of the first trained neural network 900-1, and The second feature vector is generated by propagating to second nodes operating according to the second activation function. The second activation function may be expressed as Equation 2 above, where x represents the second feature vector, may represent a Heaviside step function that makes an output of the second activation function 0 when the second feature vector (x) is less than 0. The dynamic range of the second activation function may be limited to [0, 1].

In step 840, the training apparatus outputs the second embedding vector generated in the second intermediate layer 926 through the output layer 930, and the difference between the output value 940 and the correct value corresponding to the training data 905 Based on , the first trained neural network 900-1 is secondarily trained. Here, the secondary training may be referred to as "fine-tuning".

The neural network 900 outputs an output value 940 corresponding to a result of propagating the training data 905 in a forward direction, and corresponds to the training data 905 corresponding to the expected output of the neural network 900. The difference between the correct answer value and the actual output value 940 may be calculated. The training apparatus may train the neural network 900 by propagating a difference between the correct value and the output value 940 in a backward direction and adjusting weights of the neural network 900 to minimize the difference.

According to an embodiment, a neural network for a task is fine-tuned by applying a second activation function that proposes non-linearity to nodes belonging to a layer adjacent to an output layer even for a general neural network. It is possible to improve the accuracy of the neural network without retraining.

10 is a flowchart illustrating a method of detecting forgery of biometric information using a neural network according to an embodiment, and FIG. 11 is a diagram for explaining the structure and operation of the neural network of FIG. 10 . In the following embodiments, each operation may be performed sequentially, but not necessarily sequentially. For example, the order of each operation may be changed, or at least two operations may be performed in parallel.

Referring to FIGS. 10 and 11 , an electronic device that detects forgery of biometric information using a neural network 1100 according to an embodiment determines whether biometric information is forged through steps 1010 to 1040. can be detected. The neural network 1100 may be, for example, a Convolution Neural Network (Conv) or a deep neural network, but is not necessarily limited thereto. The neural network 1100 may be trained to detect whether biometric information is forged by applying a second activation function to at least some intermediate layers.

In step 1010, the electronic device includes a plurality of intermediate layers 1101, 1102, and 1103 of the neural network 1100 that detect whether the biometric information is spoofed from the input data 1105 including the user's biometric information. ), one or more first feature vectors are extracted using one or more first classifiers 1120 trained in advance. Biometric information may include, for example, any one of a user's fingerprint, iris, and face, but is not necessarily limited thereto. The plurality of intermediate layers 1101 , 1102 , and 1103 may correspond to, for example, convolutional layers, but are not necessarily limited thereto.

The electronic device, for example, from the first intermediate layer 1101 among the plurality of intermediate layers 1101 , 1102 , and 1103 using the 1-1 classifier 1120 - 1 of the one or more first classifiers 1120 . A 1-1 feature vector may be extracted. The electronic device extracts a 1-2 feature vector from the second intermediate layer 1102 after the first intermediate layer 1101 by using the 1-2 classifier 1120-2 among one or more first classifiers 1120. can do. The electronic device may extract a first feature vector 1140 obtained by combining the 1-1st feature vector and the 1-2nd feature vector.

Alternatively, the electronic device may use the 1-3 classifiers 1120-3 of the one or more first classifiers 1120 to obtain the 1-3 feature vectors from the 3rd intermediate layer 1103 after the 2nd intermediate layer 1102. can be extracted. The electronic device may extract a first feature vector 1140 obtained by combining the 1-1 feature vectors, the 1-2 feature vectors, and the 1-3 feature vectors.

In step 1020, the electronic device may detect whether the biometric information is first forged based on the one or more first feature vectors obtained in step 1010. In the electronic device, for example, between at least one of a registered feature vector and a fake feature vector previously provided in the database 1150 and one or more first feature vectors obtained in step 1010 or feature vectors combined in step 1010. A first score may be calculated based on the degree of similarity. Here, 'similarity' is an index indicating how close the input data 1105 is to actual biometric information, and the higher the similarity value, the higher the probability that the input data 1105 is real biometric information (eg, fingerprint or iris).

The first score may be called a “user-dependent similarity score” in that it is determined by a result of training corresponding to a user. The electronic device may classify whether the first score is a score determined as fake information or a score determined as real information using one or more first classifiers 1120 . Since the electronic device calculates the first score according to the ratio between the similarities using in-distribution data such as the registration feature vector and the fake feature vector previously provided in the database 1150, it is robust against the determination of whether to forge or not. (robustness) can be obtained.

For example, an area where the first score can clearly determine whether or not the biometric information falls within a range where the biometric information is determined to be real information or whether the biometric information falls within a range where the biometric information is determined to be fake information (e.g., in FIG. 4 ). If the feature corresponds to the first area 410 and/or the second area 420, the electronic device may directly determine whether the first forgery is forged or not based on the first score. The electronic device may immediately determine the input data 1105 as 'real information' or 'false information' (Early Decision (ED)). In contrast, when the first score corresponds to a feature belonging to an area (eg, the third area 430 and/or the fourth area 440 of FIG. 4 ) in which it is impossible to clearly determine where the biometric information belongs, The electronic device may determine forgery status ('second forgery status') through convergence of the first score and the second score, without immediately determining whether the first forgery has occurred based on the first score.

Unlike conventional deep neural network classifiers that are configured in an end-to-end structure, one or more first classifiers 1120 perform network inference from input data 1105 including biometric information. In the meantime, it is possible to classify whether biometric information is forged or not from feature vectors extracted from the middle layers 1101, 1102, and 1103 of the neural network 1100. One or more first classifiers 1120 may be classifiers trained to classify an input image based on feature vectors. One or more first classifiers 1120 may be composed of, for example, shallow deep neural networks (Shallow DNN), which require less computation than deep neural networks, and reduce speed due to small overhead due to early judgment in the middle layer. It is possible to quickly detect whether or not the first forgery is made without.

When the first forgery of biometric information is detected through one or more first classifiers 1120 that perform an early determination before deriving an output vector from the output layer 1104, the electronic device immediately counterfeits without using the output vector. Since it is possible to detect whether the biometric information is forged or not, the time for determining whether the biometric information is forged can be shortened.

When determining whether biometric information is forged or not, the accuracy of determination and the speed of detecting forgery may correspond to a trade-off relationship. The electronic device sequentially uses one or more first classifiers 1120 for fast determination, but when the detection reliability of the one or more first classifiers 1120 is high, the first forgery information is immediately utilized, and the detection reliability is low. , It is possible to determine whether biometric information is forged (second forgery or not) by using the second score calculated from the output vector by the second classifier 1130 together.

In step 1030, the electronic device applies the output vector output from the output layer 1104 to the pretrained second classifier 1130 according to whether or not the first forgery is detected in step 1020 to obtain a second classifier. Calculate score. The second score may be referred to as an “Image-dependent Decision Score” in that it is a score determined based on an image. In the case of determining forgery only with the second score, the probability of occurrence of an error may be very high due to non-stationarity of unseen data, as shown in the fourth region 440 of FIG. 4 . Accordingly, the electronic device may use the first score and the second score together to determine whether the biometric information is forged or not ('second forgery or not').

For example, if forgery status ('first forgery status') is detected by the first score, the electronic device may end the operation without performing step 1030. On the other hand, the feature corresponding to the first score is included in, for example, the third area 430 and/or the fourth area 440 of FIG. 4 to clarify whether it corresponds to real information or fake information. If it cannot be determined, the electronic device cannot directly determine whether or not it is forged based on the first score. If it is not possible to immediately determine whether or not to be counterfeited by the first score, the electronic device uses the second score calculated from the output vector together with the first score calculated in step 1020 to determine whether or not to have been counterfeited ('second counterfeit status'). can decide

In step 1030, at least one of the one or more first classifiers 1120 and the second classifier 1130 determines that the multiplier of the activation function is equal to the rising slope of the activation function such that the peak value of the activation function for the neural network 1100 is fixed. It may be trained by an activation function determined by a first hyperparameter associated with and a second hyperparameter associated with a descending slope of the activation function. One or more first classifiers 1120 and one or more second classifiers 1130 may be configured of, for example, fully-connected (FC) layers, but are not necessarily limited thereto. At this time, the dynamic range of the activation function may be limited to, for example, [0, 1]. The activation function may be represented by, for example, Equation 2 above. At this time, in Equation 2, x represents the input data 1105, may represent a Heaviside step function that makes the output of the activation function 0 when the input data (x) is less than 0.

In operation 1040, the electronic device detects whether the biometric information is second forged or not based on a fusion score of the first score and the second score calculated based on one or more first feature vectors. For example, the electronic device may calculate a fused score by performing a weighted sum of the first score and the second score. If the fused score is greater than the threshold for detecting the second status, the electronic device may determine the input data 1105 as 'actual information'. In contrast, when the fusion score is less than or equal to the second threshold for forgery detection, the electronic device may determine the input data 1105 as 'counterfeit information'. The electronic device may detect whether or not the second forgery is forged by determining the input data 1105 .

In an electronic device that detects whether biometric information is forged or not, an out-of-distribution (ODD) input corresponding to forged information may frequently occur. The electronic device must reliably make a decision that it is 'false information' for out-of-distribution (ODD) inputs corresponding to spurious information, but it is as if the out-of-distribution (ODD) input is new data that has not been trained by the neural network at all. An over-confidence error may occur in which the neural network is over-confident and judges as it was trained. As such, when an out-of-distribution (ODD) input is applied to a neural network, it is better to make an uncertain decision, such as 'not decided', for an out-of-distribution (ODD) input than to make an incorrect decision about whether it is falsified or not. Could be better.

In an embodiment, the uncertainty of an output for an out-of-distribution (ODD) input may be increased by applying the second activation function to the neural network.

The electronic device utilizes the first score obtained by comparing the similarity between the input data 1105 including the image generated during the authentication attempt and the registered feature vector and the fake feature vector previously provided in the database 1150 together with the second score. Errors can be reduced by outputting the final decision score.

More specifically, if an out-of-distribution (ODD) input is input to the neural network, the first score can be calculated as a reasonable score because it is robust in the similarity calculation method according to the above-described method, but the neural network 1100 is purely Since an overconfidence error is highly likely to occur in the second score calculated using only the output of , the decision score finally output from the neural network 1100 may also have many errors.

In this situation, when the second activation function with uncertainty is applied to the neural network, overconfidence errors for out-of-distribution (OOD) inputs do not occur, and the contribution in the first score is higher, resulting in relatively less error in the final decision score. can happen

12 is a block diagram of an electronic device that detects forgery of biometric information using a neural network according to an exemplary embodiment. Referring to FIG. 12 , an electronic device 1200 according to an embodiment may include a sensor 1210, a processor 1230, an output device 1250, and a memory 1270. The sensor 1210 , processor 1230 , output device 1250 , and memory 1270 may be connected to each other through a communication bus 1205 .

The electronic device 1200 includes, for example, a mobile device such as a mobile phone, a smart phone, a PDA, a netbook, a tablet computer, a laptop computer, a wearable device such as a smart watch, a smart band, and smart glasses, a computing device such as a desktop, a server, and the like. , TVs, smart televisions, home appliances such as refrigerators, security devices such as door locks, medical devices, robotics, IoT (Internet of Things) devices, and smart vehicles, but may be implemented as at least a part, but are not limited thereto, and various types of devices may apply to

Sensor 1210 captures input data including biometric information of the user. The user's biometric information may include, for example, the user's iris, fingerprint, and face, but is not necessarily limited thereto. The sensor 1210 may include, for example, an ultrasonic fingerprint sensor, an optical fingerprint sensor, an electrostatic fingerprint sensor, a depth sensor, an iris sensor, an image sensor, and the like, but is not necessarily limited thereto. As the sensor 1210, any one of these may be used, or two or more may be used. The biometric information sensed by the sensor 1210 may be, for example, the input fingerprint image 115 shown in FIG. 1 , an iris image, or a face image.

The processor 1230 extracts one or more first feature vectors from a plurality of intermediate layers of a neural network that detects whether biometric information is forged or not from input data, using one or more first classifiers trained in advance. The processor 1230 detects whether the biometric information is first forged based on one or more first feature vectors. The processor 1230 calculates a second score by applying an output vector output from the output layer to a pre-trained second classifier according to whether the first forgery is detected. The processor 1230 detects whether or not the second forgery of the biometric information is based on a fusion score of the first score and the second score calculated based on one or more first feature vectors. At this time, in at least one of the one or more first classifiers and the second classifier, the multiplier of the activation function is the first hyperparameter associated with the ascending slope of the activation function and the descending slope of the activation function so that the peak value of the activation function for the neural network is fixed. It may be trained based on an activation function determined by a second hyperparameter associated with . activation function Can be expressed, for example, by Equation 2 above. In Equation 2, x represents the input to the additional nodes, is the activation function when the input (x) to the additional nodes is less than zero It can represent the Heaviside step function that makes the output of 0.

Processor 1230 executes executable instructions contained in memory 1270 . The processor 1230 may execute a program and control the electronic device 1200 . Program codes executed by the processor 1230 may be stored in the memory 1270 .

The output device 1250 outputs at least one of the first forgery and the second forgery detected by the processor 1230 .

The memory 1270 may store input data captured by the sensor 1210 . The memory 1270 may store the first feature vector, the first score, and/or the second score extracted by the processor 1230 . Memory 1270 may store the output vector. The memory 1250 may store whether the processor 1230 detects first forgery or not and/or second forgery or not.

The memory 1270 may store various types of information generated during processing by the processor 1230 . In addition, the memory 1270 may store various data and programs. The memory 1270 may include volatile memory or non-volatile memory. The memory 1270 may include a mass storage medium such as a hard disk to store various types of data.

Also, the processor 1230 may perform at least one method described above with reference to FIGS. 1 to 11 or a technique corresponding to at least one method. The processor 1230 may be an electronic device implemented in hardware having a circuit having a physical structure for executing desired operations. For example, desired operations may include codes or instructions included in a program. For example, the electronic device 1200 implemented as hardware includes a microprocessor, a central processing unit (CPU), a graphic processing unit (GPU), a processor core, and a multiprocessor. - May include a multi-core processor, a multiprocessor, an Application-Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), a Neural Processing Unit (NPU), and the like.

The embodiments described above may be implemented as hardware components, software components, and/or a combination of hardware components and software components. For example, the devices, methods and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate (FPGA). array), programmable logic units (PLUs), microprocessors, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and software applications running on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to execution of software. For convenience of understanding, there are cases in which one processing device is used, but those skilled in the art will understand that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that it can include. For example, a processing device may include a plurality of processors or a processor and a controller. Other processing configurations are also possible, such as parallel processors.

Software may include a computer program, code, instructions, or a combination of one or more of the foregoing, which configures a processing device to operate as desired or processes independently or collectively. You can command the device. Software and/or data may be any tangible machine, component, physical device, virtual equipment, computer storage medium or device, intended to be interpreted by or provide instructions or data to a processing device. , or may be permanently or temporarily embodied in a transmitted signal wave. Software may be distributed on networked computer systems and stored or executed in a distributed manner. Software and data may be stored on computer readable media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination, and the program instructions recorded on the medium may be specially designed and configured for the embodiment or may be known and usable to those skilled in the art of computer software. may be Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. - includes hardware devices specially configured to store and execute program instructions, such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, as well as machine language codes such as those produced by a compiler.

The hardware device described above may be configured to operate as one or a plurality of software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described with limited drawings, those skilled in the art can apply various technical modifications and variations based on this. For example, the described techniques may be performed in an order different from the method described, and/or components of the described system, structure, device, circuit, etc. may be combined or combined in a different form than the method described, or other components may be used. Or even if it is replaced or substituted by equivalents, appropriate results can be achieved.

Therefore, other implementations, other embodiments, and equivalents of the claims are within the scope of the following claims.

1200: electronic device
1205: communication bus
1210: sensor
1230: processor
1250: output device
1270: memory

Claims (25)

A method of operating a neural network including an input layer, a plurality of intermediate layers, and an out layer,
generating a first intermediate vector by applying a first activation function to first nodes belonging to a first intermediate layer adjacent to the input layer among the plurality of intermediate layers;
transferring the first intermediate vector to second nodes belonging to a second intermediate layer adjacent to the output layer among the intermediate layers;
generating a second intermediate vector by applying a second activation function to the second nodes; and
applying the second intermediate vector to the output layer;
including,
The second activation function is
A multiplier of the second activation function is a ratio of a first hyperparameter associated with an ascending slope of the second activation function and the second activation function so that the peak value of the second activation function is fixed. A method of operating a neural network, determined by a second hyperparameter associated with a descending slope. According to claim 1,
The method of operating a neural network, wherein the dynamic range of the second activation function is limited to [0, 1]. According to claim 1,
The second activation function ( ) is expressed by the following equation,
[mathematical expression]
,
Here, a represents a first hyperparameter associated with an ascending slope of the second activation function, and b represents a second hyperparameter associated with a descending slope of the second activation function, denotes an Euler number, the x denotes the input of the second nodes, represents a Heaviside Step function that makes an output of the second activation function 0 when the x is less than 0, the neural network operating method. According to claim 1,
The first activation function is
of a neural network, including any one of a Step function, a Sigmoid function, a Hyperbolic tangent function, a ReLU function, and a Leaky ReLU function how it works. According to claim 1,
The neural network is
A method of operating a neural network, including any one of a Convolution Neural Network (CNN), a Deep Neural Network (DNN), and a Recurrent Neural Network (RNN). A method for training a neural network including an input layer, a plurality of intermediate layers, and an output layer,
extracting a first result value obtained by applying a first activation function to intermediate nodes belonging to each of the plurality of intermediate layers;
extracting a second result value by applying a second activation function different from the first activation function to additional nodes connected to intermediate nodes belonging to at least one arbitrary layer among the plurality of intermediate layers; and
training the neural network based on the difference between the first result value and the second result value;
Including, a training method of a neural network. According to claim 6,
The second activation function is
The multiplier of the second activation function is determined by a first hyper parameter associated with the rising slope of the second activation function and a second hyper parameter associated with the falling slope of the second activation function so that the peak value of the second activation function is fixed. Determined, the training method of the neural network. According to claim 6,
The number of additional nodes is the number of intermediate nodes -1,
The method of training a neural network, wherein the additional nodes and the intermediate nodes are fully connected. According to claim 6,
The second activation function ( ) is expressed by the following equation,
[mathematical expression]
,
Here, a represents a first hyperparameter associated with an ascending slope of the second activation function, and b represents a second hyperparameter associated with a descending slope of the second activation function, denotes an Euler number, where x denotes an input to the additional nodes, represents a Heaviside step function that makes the output of the activation function 0 when the x is less than 0, a neural network training method. According to claim 6,
The method of training a neural network, wherein the dynamic range of the second activation function is limited to [0, 1]. According to claim 6,
The first activation function is
A method for training a neural network, including any one of a step function, a sigmoid function, a hyperbolic tangent function, a relu function, and a ricky relu function. A method for training a neural network including an input layer, a plurality of intermediate layers, and an output layer,
generating a first feature vector by propagating learning data input to the input layer to first nodes belonging to a first intermediate layer adjacent to the input layer among the intermediate layers and operating according to a first activation function;
primary training of the neural network based on a difference between the first feature vector and the correct answer vector corresponding to the learning data;
The first feature vector is propagated to second nodes belonging to a second intermediate layer adjacent to the output layer among intermediate layers of the first-trained neural network and operating according to a second activation function to obtain a second feature vector. generating; and
Secondary training of the first trained neural network based on a difference between an output value of the second embedding vector output through the output layer and a correct answer value corresponding to the training data
Including, a training method of a neural network. According to claim 12,
The second activation function is
The multiplier of the second activation function is determined by a first hyper parameter associated with the rising slope of the second activation function and a second hyper parameter associated with the falling slope of the second activation function so that the peak value of the second activation function is fixed. Determined, the training method of the neural network. According to claim 12,
The second activation function ( ) is expressed by the following equation,
[mathematical expression]
,
Here, a represents a first hyperparameter associated with an ascending slope of the second activation function, and b represents a second hyperparameter associated with a descending slope of the second activation function, denotes an Euler number, the x denotes the second feature vector, and represents a Heaviside step function that makes an output of the second activation function 0 when the x is less than 0, the neural network training method. According to claim 12,
The method of training a neural network, wherein the dynamic range of the second activation function is limited to [0, 1]. According to claim 12,
The first activation function is
A method for training a neural network, including any one of a step function, a sigmoid function, a hyperbolic tangent function, a relu function, and a ricky relu function. A method for detecting forgery of biometric information using a neural network,
One or more first features, using one or more first classifiers trained in advance, from a plurality of intermediate layers of the neural network that detects whether the biometric information is spoofed from input data including the user's biometric information. extracting vectors;
detecting whether the biometric information is first forged based on the one or more first feature vectors;
calculating a second score by applying an output vector output from an output layer of the neural network to a pretrained second classifier according to whether the first forgery is detected; and
Detecting whether the biometric information is second forged or not by a fusion score of the first score calculated based on the one or more first feature vectors and the second score
including,
At least one of the one or more first classifiers and the second classifier
The activation function multiplier is determined by a first hyperparameter associated with an ascending slope of the activation function and a second hyperparameter associated with a descending slope of the activation function such that a peak value of the activation function for the neural network is fixed. A method for detecting forgery of biometric information, which is trained by According to claim 17,
The dynamic range of the activation function is controlled to [0, 1], a method for detecting whether biometric information is forged or not. According to claim 17,
The activation function ( ) is expressed by the following equation,
[mathematical expression]

Here, a represents a first hyperparameter associated with an ascending slope of the activation function, and b represents a second hyperparameter associated with a descending slope of the activation function. represents the Euler number, the x represents the input data, and the represents a Heaviside step function that makes the output of the activation function 0 when the x is less than 0, and detects whether biometric information is forged. According to claim 17,
Extracting the one or more first feature vectors
extracting a 1-1 feature vector from a first intermediate layer among the plurality of intermediate layers by using a 1-1 classifier among the one or more first classifiers;
extracting 1-2 feature vectors from a second intermediate layer after the first intermediate layer by using a 1-2 classifier among the one or more first classifiers; and
Extracting a feature vector obtained by combining the 1-1 feature vector and the 1-2 feature vector
Including, a method for detecting whether the biometric information is forged. According to claim 20,
The step of detecting whether the biometric information is first forged or not
calculating the first score based on a degree of similarity between at least one of a previously provided registered feature vector and a fake feature vector and the combined feature vector; and
Classifying whether the first score is a score determined as fake information or a score determined as real information using the one or more first classifiers
Including, a method for detecting whether the biometric information is forged. According to claim 17,
The biometric information
A method for detecting whether biometric information including any one of the user's fingerprint, iris, and face is forged. A computer program stored in a computer-readable recording medium in order to execute the method of any one of claims 1 to 22 in combination with hardware. An electronic device for detecting forgery of biometric information using a neural network,
a sensor that captures input data including the biometric information of a user;
From the input data, one or more first feature vectors are extracted from a plurality of intermediate layers of the neural network that detects whether the biometric information is forged or not, using one or more first classifiers trained in advance, and the one or more first feature vectors are extracted. Based on 1 feature vector, whether the biometric information is first forged or not is detected, and according to whether the first forgery or not is detected, the output vector output from the output layer of the neural network is passed to a pretrained second classifier. a processor that calculates a second score by applying a second score, and detects whether the biometric information is second forged or not based on a fusion score of the first score calculated based on the one or more first feature vectors and the second score; and
An output device outputting at least one of the first forgery status and the second forgery status
including,
At least one of the one or more first classifiers and the second classifier
The activation function multiplier is determined by a first hyperparameter associated with a rising slope of the activation function and a second hyperparameter associated with a falling slope of the activation function such that a peak value of the activation function for the neural network is fixed. An electronic device that is trained based on a function. According to claim 24,
The activation function ( ) is expressed by the following equation,
[mathematical expression]

Here, a represents a first hyperparameter associated with an ascending slope of the activation function, and b represents a second hyperparameter associated with a descending slope of the activation function. denotes an Euler number, where x denotes an input to the additional nodes, represents a Heaviside step function that makes an output of the activation function 0 when the x is less than 0, the electronic device.

Images