KR20220001889A - Electronic device for biometric authentication, method for operating thereof and storage medium - Google Patents

Abstract

An electronic device includes a sensor module, a communication module, and at least one processor operatively connected to the sensor module and the communication module. The at least one processor may be configured to: obtain biometric information through the sensor module; extract feature data from the biometric information; identify a condition related to whether to transmit the feature data; based on the condition related to whether to transmit the feature data, transmit a first transmission frame including the feature data to an external electronic device interworking with the electronic device through the communication module; and ignore or discard the feature data when the condition related to whether to transmit is not satisfied. Other embodiments may be provided.

Description

Electronic device for biometric authentication, operating method and storage medium thereof

Various embodiments relate to an electronic device for biometric authentication, an operating method thereof, and a storage medium.

As the importance of protecting personal information increases, a user authentication method for enhancing security is used in electronic devices, and various methods such as a password method, a pattern method, and a biometric recognition method may be used for user authentication. Among them, biometric recognition is a method of security authentication, in which user authentication is performed using the user's biometric information, which is a unique physical characteristic such as a user's fingerprint, iris, face, voice, retina, and blood vessels. can

The electronic device provides various services related to user authentication based on the user's biometric information. Biometric information is very effective in terms of security as well as ease of use because there is little risk of theft and imitation. Accordingly, biometric information is used as information for user authentication in many electronic devices. For example, an electronic device such as a smart phone may be connected to the wearable electronic device through various connection methods and authentication methods. Among wearable electronic devices that can be worn on the body, there is also a head mounted electronic device such as a head mounted display (HMD) device.

However, in order to perform user authentication using biometric information between the electronic device and the wearable electronic device, the user is only equipped with a biometric sensor (eg, a fingerprint or iris recognition sensor) for recognizing biometric information in each of the electronic device and the wearable electronic device. Authentication may be possible. In addition, since the user cannot see the front while wearing the head-mounted electronic device, the user authentication method using the electronic device may be cumbersome. In addition, when it is necessary to click the power button to wake up the display off state of the electronic device or perform an additional action to unlock the electronic device, user authentication using the electronic device in a state where it is difficult for the user to see in front of wearing the head-mounted electronic device This method can be very cumbersome and inconvenient.

Therefore, for user authentication based on biometric information between the head mounted electronic device and the electronic device interworking therewith, a method for ensuring the security of biometric information while considering the convenience aspect may be proposed.

According to an embodiment, the electronic device includes a sensor module, a communication module, and at least one processor operatively connected to the sensor module and the communication module, wherein the at least one processor provides biometric information through the sensor module. obtains, extracts feature data from the biometric information, identifies a condition related to whether or not to transmit the feature data, and transmits a first transmission frame including the feature data based on the condition related to whether or not to transmit the feature data to the electronic device It may be configured to transmit to an external electronic device interworking with a device through the communication module and ignore or discard the feature data when a condition related to whether or not to transmit is not satisfied.

According to an embodiment, a method for biometric authentication in an electronic device relates to an operation of acquiring biometric information through a sensor module of the electronic device, an operation of extracting feature data from the biometric information, and whether the feature data is transmitted. An operation of identifying a condition, an operation of transmitting a first transmission frame including the feature data to an external electronic device interworking with the electronic device based on a condition related to whether the transmission is performed, an operation of transmitting the first transmission frame, and When the related condition is not satisfied, the operation of ignoring or discarding the feature data may be included.

According to an embodiment, in a storage medium storing instructions, when the instructions are executed by at least one processor, the at least one processor is configured to perform at least one operation, wherein the at least one operation is , based on an operation of acquiring biometric information through a sensor module of the electronic device, an operation of extracting feature data from the biometric information, an operation of identifying a condition related to whether the characteristic data is transmitted, an operation related to the transmission or not, An operation of transmitting a first transmission frame including the characteristic data to an external electronic device interworking with the electronic device and ignoring or discarding the characteristic data when a condition related to whether or not to transmit the first transmission frame is not satisfied It can include actions.

According to an embodiment, for user authentication based on biometric information between the head mounted electronic device and the electronic device interworking therewith, it is possible to ensure the security of biometric information while considering the aspect of convenience.

According to an embodiment, based on biometric authentication using the head mounted electronic device, user authentication of the electronic device may be simultaneously performed.

According to an embodiment, it is possible to reduce the data communication load by not transmitting the data itself related to the biometric information when forgery occurs by determining whether the biometric information input through the head mounted electronic device has been forged or falsified.

Effects obtainable in the present disclosure are not limited to the above-mentioned effects, and other effects not mentioned may be clearly understood by those of ordinary skill in the art to which the present disclosure belongs from the description below. will be.

1 is a block diagram of an electronic device in a network environment according to various embodiments of the present disclosure;
2 is a perspective view illustrating wearing of a head mounted electronic device according to an exemplary embodiment.
3 is a diagram illustrating a connection between an electronic device and an external electronic device according to an exemplary embodiment.
4 is an internal block diagram of an electronic device for biometric authentication according to an embodiment.
5 is a diagram illustrating a display screen of an electronic device during biometric authentication according to an embodiment.
6 is an internal configuration diagram of an electronic device for user authentication based on biometric information and an external electronic device, respectively, according to an exemplary embodiment.
7 is a diagram for explaining transmission of data for verification according to an embodiment.
8 is a view for explaining whether or not forgery based on a matching result according to an embodiment.
9 is an operation flowchart of an electronic device for biometric authentication according to an embodiment.
10 is a diagram for explaining a process of ignoring or discarding feature data in response to forgery occurrence, according to an embodiment.
11 is a diagram for explaining a process of transmitting invalid data instead of feature data in response to forgery occurrence according to an embodiment.

Terms used in this document are only used to describe specific embodiments, and may not be intended to limit the scope of other embodiments. The singular expression may include the plural expression unless the context clearly dictates otherwise. All terms used herein, including technical or scientific terms, may have the same meaning as commonly understood by one of ordinary skill in the art of the present invention. Terms defined in general used in the dictionary may be interpreted as having the same or similar meaning as the meaning in the context of the related art, and unless explicitly defined in this document, it is not interpreted in an ideal or excessively formal meaning. . In some cases, even terms defined in this document cannot be construed to exclude embodiments of the present invention.

1 is a block diagram of an electronic device 101 in a network environment 100 according to various embodiments of the present disclosure. Referring to FIG. 1 , in a network environment 100 , an electronic device 101 communicates with an electronic device 102 through a first network 198 (eg, a short-range wireless communication network) or a second network 199 . It may communicate with the electronic device 104 or the server 108 through (eg, a long-distance wireless communication network). According to an embodiment, the electronic device 101 may communicate with the electronic device 104 through the server 108 . According to an embodiment, the electronic device 101 includes a processor 120 , a memory 130 , an input module 150 , a sound output module 155 , a display module 160 , an audio module 170 , and a sensor module ( 176), interface 177, connection terminal 178, haptic module 179, camera module 180, power management module 188, battery 189, communication module 190, subscriber identification module 196 , or an antenna module 197 may be included. In some embodiments, at least one of these components (eg, the connection terminal 178 ) may be omitted or one or more other components may be added to the electronic device 101 . In some embodiments, some of these components (eg, sensor module 176 , camera module 180 , or antenna module 197 ) are integrated into one component (eg, display module 160 ). can be

The processor 120, for example, executes software (eg, a program 140) to execute at least one other component (eg, a hardware or software component) of the electronic device 101 connected to the processor 120 . It can control and perform various data processing or operations. According to one embodiment, as at least part of data processing or operation, the processor 120 converts commands or data received from other components (eg, the sensor module 176 or the communication module 190 ) to the volatile memory 132 . may be stored in the volatile memory 132 , and may process commands or data stored in the volatile memory 132 , and store the result data in the non-volatile memory 134 . According to an embodiment, the processor 120 is the main processor 121 (eg, a central processing unit or an application processor) or a secondary processor 123 (eg, a graphic processing unit, a neural network processing unit) a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, when the electronic device 101 includes the main processor 121 and the sub-processor 123 , the sub-processor 123 may use less power than the main processor 121 or may be set to be specialized for a specified function. can The auxiliary processor 123 may be implemented separately from or as a part of the main processor 121 .

The auxiliary processor 123 is, for example, on behalf of the main processor 121 while the main processor 121 is in an inactive (eg, sleep) state, or the main processor 121 is active (eg, executing an application). ), together with the main processor 121, at least one of the components of the electronic device 101 (eg, the display module 160, the sensor module 176, or the communication module 190) It is possible to control at least some of the related functions or states. According to an embodiment, the co-processor 123 (eg, an image signal processor or a communication processor) may be implemented as part of another functionally related component (eg, the camera module 180 or the communication module 190). have. According to an embodiment, the auxiliary processor 123 (eg, a neural network processing device) may include a hardware structure specialized for processing an artificial intelligence model. Artificial intelligence models can be created through machine learning. Such learning may be performed, for example, in the electronic device 101 itself on which artificial intelligence is performed, or may be performed through a separate server (eg, the server 108). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but in the above example not limited The artificial intelligence model may include a plurality of artificial neural network layers. Artificial neural networks include deep neural networks (DNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), restricted boltzmann machines (RBMs), deep belief networks (DBNs), bidirectional recurrent deep neural networks (BRDNNs), It may be one of deep Q-networks or a combination of two or more of the above, but is not limited to the above example. The artificial intelligence model may include, in addition to, or alternatively, a software structure in addition to the hardware structure.

The memory 130 may store various data used by at least one component of the electronic device 101 (eg, the processor 120 or the sensor module 176 ). The data may include, for example, input data or output data for software (eg, the program 140 ) and instructions related thereto. The memory 130 may include a volatile memory 132 or a non-volatile memory 134 .

The program 140 may be stored as software in the memory 130 , and may include, for example, an operating system 142 , middleware 144 , or an application 146 .

The input module 150 may receive a command or data to be used in a component (eg, the processor 120 ) of the electronic device 101 from the outside (eg, a user) of the electronic device 101 . The input module 150 may include, for example, a microphone, a mouse, a keyboard, a key (eg, a button), or a digital pen (eg, a stylus pen).

The sound output module 155 may output a sound signal to the outside of the electronic device 101 . The sound output module 155 may include, for example, a speaker or a receiver. The speaker can be used for general purposes such as multimedia playback or recording playback. The receiver may be used to receive an incoming call. According to one embodiment, the receiver may be implemented separately from or as part of the speaker.

The display module 160 may visually provide information to the outside (eg, a user) of the electronic device 101 . The display module 160 may include, for example, a control circuit for controlling a display, a hologram device, or a projector and a corresponding device. According to an embodiment, the display module 160 may include a touch sensor configured to sense a touch or a pressure sensor configured to measure the intensity of a force generated by the touch.

The audio module 170 may convert a sound into an electric signal or, conversely, convert an electric signal into a sound. According to an embodiment, the audio module 170 acquires a sound through the input module 150 , or an external electronic device (eg, a sound output module 155 ) connected directly or wirelessly with the electronic device 101 . A sound may be output through the electronic device 102 (eg, a speaker or headphones).

The sensor module 176 detects an operating state (eg, power or temperature) of the electronic device 101 or an external environmental state (eg, user state), and generates an electrical signal or data value corresponding to the sensed state. can do. According to an embodiment, the sensor module 176 may include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, It may include a temperature sensor, a humidity sensor, or an illuminance sensor.

The interface 177 may support one or more designated protocols that may be used by the electronic device 101 to directly or wirelessly connect with an external electronic device (eg, the electronic device 102 ). According to an embodiment, the interface 177 may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

The connection terminal 178 may include a connector through which the electronic device 101 can be physically connected to an external electronic device (eg, the electronic device 102 ). According to an embodiment, the connection terminal 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (eg, a headphone connector).

The haptic module 179 may convert an electrical signal into a mechanical stimulus (eg, vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic sense. According to an embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

The camera module 180 may capture still images and moving images. According to an embodiment, the camera module 180 may include one or more lenses, image sensors, image signal processors, or flashes.

The power management module 188 may manage power supplied to the electronic device 101 . According to an embodiment, the power management module 188 may be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

The battery 189 may supply power to at least one component of the electronic device 101 . According to one embodiment, battery 189 may include, for example, a non-rechargeable primary cell, a rechargeable secondary cell, or a fuel cell.

The communication module 190 is a direct (eg, wired) communication channel or a wireless communication channel between the electronic device 101 and an external electronic device (eg, the electronic device 102, the electronic device 104, or the server 108). It can support establishment and communication performance through the established communication channel. The communication module 190 may include one or more communication processors that operate independently of the processor 120 (eg, an application processor) and support direct (eg, wired) communication or wireless communication. According to one embodiment, the communication module 190 is a wireless communication module 192 (eg, a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (eg, : It may include a LAN (local area network) communication module, or a power line communication module). A corresponding communication module among these communication modules is a first network 198 (eg, a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network 199 (eg, legacy It may communicate with the external electronic device 104 through a cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (eg, a telecommunication network such as a LAN or a WAN). These various types of communication modules may be integrated into one component (eg, a single chip) or may be implemented as a plurality of components (eg, multiple chips) separate from each other. The wireless communication module 192 uses the subscriber information (eg, International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module 196 within a communication network such as the first network 198 or the second network 199 . The electronic device 101 may be identified or authenticated.

The wireless communication module 192 may support a 5G network after a 4G network and a next-generation communication technology, for example, a new radio access technology (NR). NR access technology includes high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and access to multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency) -latency communications)). The wireless communication module 192 may support a high frequency band (eg, mmWave band) to achieve a high data rate, for example. The wireless communication module 192 includes various technologies for securing performance in a high-frequency band, for example, beamforming, massive multiple-input and multiple-output (MIMO), all-dimensional multiplexing. It may support technologies such as full dimensional MIMO (FD-MIMO), an array antenna, analog beam-forming, or a large scale antenna. The wireless communication module 192 may support various requirements specified in the electronic device 101 , an external electronic device (eg, the electronic device 104 ), or a network system (eg, the second network 199 ). According to an embodiment, the wireless communication module 192 may include a peak data rate (eg, 20 Gbps or more) for realizing eMBB, loss coverage (eg, 164 dB or less) for realizing mMTC, or U-plane latency for realizing URLLC ( Example: downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less).

The antenna module 197 may transmit or receive a signal or power to the outside (eg, an external electronic device). According to an embodiment, the antenna module 197 may include an antenna including a conductor formed on a substrate (eg, a PCB) or a radiator formed of a conductive pattern. According to an embodiment, the antenna module 197 may include a plurality of antennas (eg, an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network such as the first network 198 or the second network 199 is connected from the plurality of antennas by, for example, the communication module 190 . can be selected. A signal or power may be transmitted or received between the communication module 190 and an external electronic device through the selected at least one antenna. According to some embodiments, other components (eg, a radio frequency integrated circuit (RFIC)) other than the radiator may be additionally formed as a part of the antenna module 197 .

According to various embodiments, the antenna module 197 may form a mmWave antenna module. According to one embodiment, the mmWave antenna module comprises a printed circuit board, an RFIC disposed on or adjacent to a first side (eg, bottom side) of the printed circuit board and capable of supporting a designated high frequency band (eg, mmWave band); and a plurality of antennas (eg, an array antenna) disposed on or adjacent to a second side (eg, top or side) of the printed circuit board and capable of transmitting or receiving signals of the designated high frequency band. can do.

At least some of the components are connected to each other through a communication method between peripheral devices (eg, a bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI)) and a signal ( eg commands or data) can be exchanged with each other.

According to an embodiment, the command or data may be transmitted or received between the electronic device 101 and the external electronic device 104 through the server 108 connected to the second network 199 . Each of the external electronic devices 102 or 104 may be the same as or different from the electronic device 101 . According to an embodiment, all or a part of operations executed in the electronic device 101 may be executed in one or more external electronic devices 102 , 104 , or 108 . For example, when the electronic device 101 is to perform a function or service automatically or in response to a request from a user or other device, the electronic device 101 may perform the function or service itself instead of executing the function or service itself. Alternatively or additionally, one or more external electronic devices may be requested to perform at least a part of the function or the service. One or more external electronic devices that have received the request may execute at least a part of the requested function or service, or an additional function or service related to the request, and transmit a result of the execution to the electronic device 101 . The electronic device 101 may process the result as it is or additionally and provide it as at least a part of a response to the request. For this, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device 101 may provide an ultra-low latency service using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device 104 may include an Internet of things (IoT) device. Server 108 may be an intelligent server using machine learning and/or neural networks. According to an embodiment, the external electronic device 104 or the server 108 may be included in the second network 199 . The electronic device 101 may be applied to an intelligent service (eg, smart home, smart city, smart car, or health care) based on 5G communication technology and IoT-related technology.

2 is a perspective view 200 for explaining a user wearing the HMD device according to an embodiment.

Referring to FIG. 2 , the electronic device 201 may be connected to the electronic device 202 . The electronic device 201 may be connected to the electronic device 202 in a wired/wireless manner. For example, the electronic device 201 may be connected to the electronic device 202 based on a short-distance communication method, but this is merely an example, and it is understood by those skilled in the art that there is no limitation as long as data transmission/reception is possible between both electronic devices 201 and 202 . will be easily understood.

In an embodiment, the electronic device 201 operating as a host may be referred to as a host device, and the electronic device 202 that interworks with the electronic device 201 through a communication connection may be a head mounted electronic device.

The electronic device 202 may be worn on the user's head and be fixed to the user's head even if the user moves. The electronic device 201 may be connected to the electronic device 202 , and accordingly, when the user wears the electronic device 202 to which the electronic device 201 is connected, an image displayed on the display of the electronic device 201 . can watch According to an embodiment, an image displayed on the display of the electronic device 201 may be displayed on the display of the electronic device 202 , and for example, a left-eye image and a right-eye image corresponding to both eyes of the user may be displayed on the display of the electronic device 202 . may be displayed.

According to an embodiment, when the electronic device 202 is a see-closed type head mounted electronic device, two displays are placed in front of the eyes. Content can be viewed on an independent screen. According to an embodiment, the electronic device 202 may display content provided from the electronic device 201 .

According to an embodiment, when the electronic device 202 is a see-through type head mounted electronic device, a virtual object or object is synthesized and combined on the basis of the real world by using the characteristic of a semi-transmissive lens. Additional information that is difficult to obtain in the real world alone can be reinforced and provided.

For example, the head mounted electronic device as described above may provide content based on augmented reality (AR) or virtual reality (VR).

According to an embodiment, the electronic device 202 may acquire the user's biometric information from a series of continuous image frames input from the camera while being connected to the electronic device 201 . For example, the electronic device 202 may include a biometric authentication device that identifies whether biometric information has been forged or altered, and the electronic device 201 may perform user authentication based on biometric information. Accordingly, the electronic device 202 may be used in the electronic device 201 in the same manner as other biometric authentication devices.

According to an embodiment, even when it is difficult for the user to perform face/iris authentication using the electronic device 201 while wearing the head mounted electronic device 202, the head is performed through the user authentication method based on biometric information such as the user's iris. The lock function of the electronic device 201 interworking with the mountable electronic device 202 may be released. Accordingly, for user authentication, the user clicks the power button to wake up the display-off state of the electronic device 201 or to unlock the The lock state of the electronic device 201 may be released simultaneously with user authentication without a separate pattern input. As described above, the user can also release the locked state of the electronic device 201 at the same time as user authentication is completed even by keeping his or her eyes wide open for a certain period of time, thereby increasing convenience and ensuring security.

3 is a diagram 300 illustrating a connection between an electronic device (eg, a wearable electronic device) and an external electronic device (eg, a smart phone) according to an embodiment.

Referring to FIG. 3 , when the electronic device 302 and the external electronic device 301 are connected in a wired/wireless manner, the external electronic device 301 functions as a host (or master), and the electronic device 302 accompanies the electronic device 302 . It can function as a client (or slave) as a companion device. For example, assuming that the wearable electronic device is the electronic device 302 , the smart phone may be referred to as an external electronic device 301 connected to the electronic device. According to an embodiment, the electronic device 302 may be referred to as an accessory device connectable to the external electronic device 301 . According to an embodiment, the electronic device 302 may include at least one of a head mounted electronic device and a watch-type wearable electronic device. In addition to the above, the electronic device 302 may be any device including a camera capable of acquiring the user's iris image. Biometric authentication may be performed by photographing, and user authentication may be performed by the external electronic device 301 interworking with the electronic device 302 . As another example, the electronic device 302 worn by the user (eg, a watch-type wearable electronic device) may perform biometric authentication based on a biosignal such as an electrocardiography (ECG) of the user, and the wearable electronic device The external electronic device 301 that interworks with may perform user authentication based on biometric authentication.

Hereinafter, an external electronic device 301 such as a smart phone is referred to as a device serving as a host, and an electronic device 302 such as a head-mounted electronic device connected to the external electronic device 301 is referred to as a device serving as a client. However, the electronic device 302 or the external electronic device 301 is not limited to a head mounted electronic device or a smart phone.

4 is an internal block diagram 400 of an electronic device for biometric authentication according to an embodiment.

The electronic device 402 of FIG. 4 may correspond to an accessory device that is functionally connected to an external electronic device (eg, the electronic device 301 of FIG. 3 ). For example, the electronic device 402 may correspond to a head mounted electronic device or a watch type wearable electronic device, but may not be limited thereto.

Referring to FIG. 4 , the sensor module 476 may be disposed at a location for acquiring the user's biometric information. According to an embodiment, the biometric information may include information on at least one of an iris, an eye, a fingerprint, a face, a hand, a wrist, and a blood vessel (vein, etc.). Hereinafter, the configuration of the electronic device 402 including components for receiving an input of an iris image is exemplified to describe a case where biometric information is iris information as an example, but includes components for receiving input of fingerprint information instead of iris information It may be implemented to do so. In addition, the configuration of the electronic device 402 may be added to include components for receiving complex biometric information that can receive fingerprint information as well as iris information.

According to an embodiment, when the electronic device 402 is a head-mounted electronic device having the appearance of glasses, the sensor module 476 may be disposed at a position facing the user's eyes when the user wears the head-mounted electronic device. can According to an embodiment, the sensor module 476 may be a camera module for photographing the user's eye 405 . Also, according to an embodiment, at least a portion of the sensor module 476 may be disposed outside or inside the electronic device 402 , and may be configured as two to correspond to both eyes 405 of the user.

According to an embodiment, the sensor module 476 may acquire sensor data (or one or more image frames) about the user as biometric information. For example, the sensor module 476 may acquire an iris image of the user through photographing.

According to an embodiment, the communication module 490 may communicate with an external electronic device (eg, the electronic device 301 of FIG. 3 ). According to an embodiment, the communication module 490 may perform wired or wireless communication, and may transmit feature data related to biometric information to an external electronic device and receive a response corresponding thereto.

According to an embodiment, the memory 430 may store a program (or algorithm) for extracting feature data from biometric information and identifying whether the biometric information or feature data is forged or altered.

According to an embodiment, the display 460 may provide a user interface corresponding to biometric information acquisition. For example, the display 460 may display a guide screen for photographing biometric information and a user authentication screen based on biometric information. If an authentication result indicating that user authentication is successful based on the feature data provided by the electronic device 402 is obtained from the external electronic device, the display 460 may display a screen according to the user authentication success.

According to an embodiment, the processor 420 may obtain data (or image frame) related to biometric information about the user, for example, an iris image of the user from the sensor module 476 . Next, the processor 420 may extract feature data related to biometric information from each image frame input from the sensor module 476 . According to an embodiment, the processor 420 may identify whether forgery or not based on the extracted feature data. Alternatively, the processor 420 may identify whether forgery or not based on the input image frame. For example, the biometric information includes a plurality of iris images, and the processor 420 may extract feature data from at least some of the plurality of iris images.

According to an embodiment, the processor 420 may identify a condition related to whether to transmit the feature data extracted from the biometric information. For example, the processor 420 may identify whether a condition for transmitting the feature data is satisfied. If a condition related to whether to transmit is satisfied, the feature data may be transmitted to the external electronic device 301 . According to an embodiment, the processor 420 may generate a transmission frame including the feature data and transmit it to an external electronic device interworking with the electronic device 402 together with metadata for the transmission frame.

According to an embodiment, the transmission frame includes the characteristic data and metadata, and the metadata includes identification information of the electronic device 402 , a certificate of the electronic device 402 , and a number of characteristic data. may include at least one of an order of feature data representing , and data for verification. Here, the identification information of the electronic device 402 may be used in the external electronic device to determine whether the device at the time of biometric registration is the same as the device at the time of biometric authentication. The certificate of the electronic device 402 may include a root of trust that can be verified in a secure environment of the external electronic device. The secure environment may be a trusted execution environment (TEE) logically separated from the processor of the external electronic device, or may be physically separated secure hardware. Also, the order of the feature data may increase as the feature data are sequentially transmitted one by one.

According to an embodiment, the processor 420 may identify whether forgery or not as a method of identifying a condition related to whether feature data is transmitted. For example, the processor 420 may identify whether the extracted feature data or biometric information from the sensor module 476 is forged or altered.

If forgery or not is detected as a result of identification of whether or not forgery is detected, the processor 420 does not need to perform user authentication based on the feature data in the external electronic device, so that the feature data itself is not transmitted to the external electronic device. can Alternatively, the processor 420 considers that the extracted feature data or biometric information is forged or invalid data when forgery or falsification of more than a predetermined threshold range is detected, and instead of the extracted feature data, predefined invalid data (eg, : dummy data) may be transmitted to the external electronic device 301 . In this case, the processor 420 may transmit the invalid data when a preset condition is satisfied in the same manner as in the method of transmitting the feature data in transmitting the invalid data. According to an embodiment, the preset condition may include at least one of a preset transmission period and a preset number of transmission frames. According to an embodiment, the preset condition may be predefined between the electronic device 402 and the external electronic device or defined during session establishment between the electronic device 402 and the external electronic device.

For example, when forgery is not detected, the processor 420 may transmit the extracted feature data to the external electronic device in a predetermined frame unit (eg, one by one or n by one).

According to an embodiment, the processor 420 may transmit the feature data to the external electronic device when a condition related to whether the transmission is satisfied, and transmit verification data together with the feature data or based on a preset condition. Instead of the feature data, or after transmitting a certain number of the feature data, the feature data may be continuously transmitted. For example, when a preset condition is satisfied while transmitting the feature data in a frame unit, the processor 420 transmits the verification data instead of the feature data at a predetermined turn or based on a steganography method. Thus, the verification data may be transmitted together with the characteristic data by inserting the verification data into the characteristic data. Also, for example, the processor 420 may transmit verification data by including data for verification in metadata of a transmission frame.

The verification data according to an embodiment may be data randomly generated by the electronic device 402 for secure transmission between the electronic device 402 and an external electronic device when actually transmitting feature data. The verification data may be transmitted from the electronic device 402 to the external electronic device according to a predetermined transmission method between the electronic device 402 and the external electronic device, and may be predetermined or characteristic data to improve security according to the transmission. It may be data that is newly generated every time it is transmitted.

For example, the processor 420 may induce an intended result by transmitting false data such as verification data, not actual feature data, to the external electronic device according to a predetermined method. As described above, by manipulating the transmission of the feature data using the verification data according to a rule (or preset condition) between the electronic device 402 and the external electronic device, the feature data transmitted from the electronic device 402 to the external electronic device security can be improved.

According to an embodiment, the preset condition for transmitting the verification data may be preset when the electronic device 402 or the external electronic device is manufactured. According to another embodiment, when a session is established (or session connection) between the electronic device 402 and the external electronic device, a preset condition for transmitting verification data may be mutually determined. According to another embodiment, whenever there is a request from the external electronic device while connected to the external electronic device, the electronic device 402 may transmit the verification data in response to the request. For example, the electronic device 402 inserts verification data into the number of transmission frames (or transmission order) (eg, every fifth) requested by the external electronic device and transmits the data to the external electronic device.

When the verification data is received by the external electronic device as described above, it may be identified whether the data is transmitted based on the preset condition, for example, the preset number of transmission frames. By verifying the verification data in the external electronic device, it is possible to check whether an abnormal attempt is made during the feature data transmission. For example, an external electronic device may prevent abnormal attempts such as a replay attack and a man-in-the-middle-attack by using the verification data. Here, the retransmission attack is an attack masquerading as a legitimate user by selecting and copying valid data on a protocol and retransmitting it later, and the man-in-the-middle attack may mean an attack that intercepts feature data.

According to an embodiment, the processor 420 may receive a response (or feedback) corresponding to the transmission of the feature data or the verification data from the external electronic device. A response corresponding to the transmission of the feature data or data for verification may include a matching result for the feature data. The matching result may indicate whether user authentication is successful.

According to an embodiment, the response may include a matching result for the characteristic data, and the processor 420 may manage a history of accumulating whether or not matching with respect to each characteristic data is successful. Accordingly, the processor 420 may be used to determine whether forgery or not based on the accumulated history. For example, when the processor 420 determines whether forgery or not using n previously sent feature data, the next m feature data to be sent together with the accumulated history, that is, the matching result for the n feature data, when the next forgery is determined is available. For example, the processor 420 may detect whether or not forgery has been performed using the n pieces of feature data that have been successfully matched among the n pieces of feature data and the m pieces of feature data to be transmitted next. In this way, the processor 420 can increase the success probability of forgery detection by determining whether or not forgery is based on more feature data than before.

According to an embodiment, the electronic device (eg, the electronic device 202 of FIG. 2 , the electronic device 302 of FIG. 3 , and the electronic device 402 of FIG. 4 ) includes a sensor module 476 and a communication module 490 . ) and at least one processor 420 operatively connected to the sensor module 476 and the communication module 490, wherein the at least one processor 420 is Obtaining information, extracting feature data from the biometric information, identifying a condition related to whether to transmit the feature data, and generating a first transmission frame including the feature data based on the condition related to whether or not to transmit the feature data An external electronic device interworking with the electronic device (eg, the electronic device 201 of FIG. 2 , the electronic device 301 of FIG. 3 ) is transmitted through the communication module 490 and a condition related to whether the transmission is not satisfied Otherwise, it may be set to ignore or discard the feature data.

According to an embodiment, the at least one processor 420 transmits the first transmission frame including the characteristic data to the external electronic device when a condition related to whether to transmit the transmission or not is satisfied. It can be set to transmit through .

According to an embodiment, the at least one processor 420 identifies whether or not spoofing has been performed on the feature data, and based on whether or not the feature data is forged or not, configured to identify a condition related to whether or not to transmit the feature data can be

According to an embodiment, the at least one processor 420 transmits a second transmission frame including data different from the characteristic data to the external electronic device through the communication module 490 in response to the occurrence of the forgery. It can be set to transmit to

According to an embodiment, the biometric information obtained through the sensor module 476 includes a plurality of iris images, and the at least one processor 420 may generate the feature from at least some of the plurality of iris images. It can be set to extract data.

According to an embodiment, when a preset condition is satisfied, the at least one processor 420 may be configured to transmit verification data to the external electronic device through the communication module 490 .

According to an embodiment, the preset condition may include at least one of a preset transmission period and a preset number of transmission frames.

According to an embodiment, when the preset condition is satisfied, the at least one processor 420 may be configured to transmit the verification data included in the feature data.

According to an embodiment, when the preset condition is satisfied, the at least one processor 420 may be configured to transmit the first transmission frame including the verification data instead of the feature data to the external electronic device. can

According to an embodiment, the at least one processor 420 receives an authentication result for the feature data from the external electronic device in response to the first transmission frame, and the authentication result for the feature data is forged or forged. It can be used to identify whether

According to an embodiment, the first transmission frame includes the feature data and metadata, and the metadata includes identification information of the electronic device, a certificate of the electronic device, an order of the feature data, and verification data. may include at least one of

5 is a diagram 500 illustrating a display screen of an electronic device during biometric authentication according to an embodiment.

Referring to FIG. 5 , the electronic device 402 may be, for example, the head mounted electronic device 202 of FIG. 2 . The electronic device 402 is connected to an external electronic device (eg, the electronic device 201 of FIG. 2 and the electronic device 301 of FIG. 3 ) and uses the electronic device 402 to use the external electronic device. The user's biometric information may be input.

The electronic device 402 may include a display, and may output a screen in front of the user's eyes while worn on the user's face. For example, in the case of the see-through type, the electronic device 402 may provide VR to the user by using the characteristics of the semi-transmissive lens, and in the case of the see-closed type, may provide AR.

According to an embodiment, the electronic device 402 may provide the user's biometric information, for example, a user interface 510 corresponding to iris imaging. For example, the electronic device 402 may display a preview screen for iris imaging or a guide screen for iris imaging. For example, the user can know that iris-based user authentication is being performed as a guide screen is displayed notifying that the user has to keep both eyes wide for a certain period of time for iris photography and a guide screen notifying that authentication is being performed. have. At this time, since the user authentication is not successful, information indicating that the locked state of the external electronic device is maintained may be displayed together.

For example, when the iris-based user authentication is successful, information that allows the user wearing the head mounted electronic device 402 to immediately know the user authentication result may be displayed on the screen of the display. Accordingly, a separate pattern for clicking the power button or releasing the lock state to wake up the display-off state of the external electronic device by having the user wearing the head mounted electronic device 402 stare with both eyes open for user authentication It is possible to use the external electronic device simultaneously with user authentication without input. As described above, the user can check the current user authentication status through the contents displayed on the screen even if the user keeps their eyes wide open for a certain period of time, thereby increasing user convenience.

6 is an internal configuration diagram 600 of each of an electronic device for user authentication based on biometric information and an external electronic device according to an exemplary embodiment.

According to an embodiment, the external electronic device 601 (eg, the external electronic device 301 of FIG. 3 ) may include a user authentication module 610 and a communication module 691 .

According to an embodiment, the electronic device 602 (eg, the head mounted electronic device 202 of FIG. 2 , the electronic device 302 of FIG. 3 ) includes a sensor module 676 , a biometric authentication module 620 , and a communication module. (690).

According to an embodiment, the biometric authentication module 620 may perform authentication on the biometric information obtained by the sensor module 676 . For example, the biometric authentication module 620 may perform authentication to identify whether the obtained biometric information is biometric information about a real user or an imposter user. To this end, the biometric authentication module 620 according to an embodiment may include a feature extraction unit 621 , a forgery detection unit 622 , an adjustment unit 623 , and a feature data processing unit 624 . Although the configuration of the biometric authentication module 620 is illustrated in FIG. 6 , the operation of the biometric authentication module 620 may be performed by a processor (eg, the processor 320 of FIG. 3 ).

According to an embodiment, when a plurality of image frames corresponding to biometric information are received from the sensor module 676 , the feature extraction unit 621 may extract feature data from each frame. According to an embodiment, the feature extractor 621 may detect an eye region from each image frame, and if the eye region is detected, determine the outline between the pupil, the iris, and the sclera, and then extract the iris portion. According to an embodiment, the feature extractor 621 may extract only the iris portion and convert the iris information into a feature pattern such as an iris template or an iris code. Here, the characteristic information of the iris is that one characteristic pattern is generated only for the iris part based on data related to the characteristics of the iris, and the characteristic pattern may have the form of an identification code such as an iris code or a template. have.

According to an embodiment, the forgery detection unit 622 may identify whether forgery or not based on a spoofing method.

According to an embodiment, the forgery detection unit 622 may identify whether the biometric information obtained from the sensor module 676 has been forged or falsified or whether the forgery has been identified using the feature data transmitted from the feature extraction unit 621. .

According to an embodiment, the forgery detection unit 622 may identify whether forgery or not with respect to at least a portion of a plurality of image frames representing biometric information obtained from the sensor module 676 . For example, the forgery detection unit 622 may identify whether a biological eye image is included in the eye region in each of the plurality of images. For example, by converting each pixel value of the eye region into a frequency domain and identifying whether a distribution in the frequency domain is within a preset threshold range, it is possible to identify whether the detected eye image is forgery or not. Here, the threshold range may be determined using a frequency value of at least a portion (eg, a pupil region) among the plurality of eye regions.

According to an embodiment, the forgery detection unit 622 may detect forgery by using a forgery score indicating a degree of similarity between previously stored biometric information-related feature data and the acquired feature data as a score. For example, the forgery score is used to determine the obtained iris image as an image of the actual iris, and the forgery detection unit 622 may compare the forgery score with a threshold value to detect forgery. Here, a threshold value for determining whether forgery or not may be set by a manufacturer.

According to an embodiment, the forgery detection unit 622 may detect whether forgery is based on a Neural Network (NN)-engine, and various known algorithms for detecting whether forgery or not may be used in addition to the above.

According to an embodiment, the forgery detection unit 622 may determine whether the forgery or forgery is based on a matching result (or authentication result) provided from the matching unit 631 of the external electronic device 601 .

For example, when the forgery detection unit 622 determines whether forgery or not using the previously sent n feature data, when the next forgery determination is made, the authentication result, that is, the matching result for the n feature data, and the m features to be sent next data is available. For example, in response to the reception of the authentication result, the forgery detection unit 622 may determine whether forgery or not based on n + m pieces of feature data. Accordingly, the success probability of forgery detection by the forgery detection unit 622 may be increased.

According to an embodiment, the controller 623 is a component that connects the electronic device 602 and the external electronic device 601 , and may adjust feature data to be transmitted to the external electronic device 601 . To this end, the adjusting unit 623 may include a buffer serving as a waiting space before transmitting the feature data transmitted from the feature extracting unit 621 . If, based on a condition related to the transmission of the characteristic data, the characteristic data in the buffer is transmitted to the external electronic device 301 through the communication module 390 based on the result of the identification of whether the forgery detection unit 622 has forged or not. can be discarded. According to an embodiment, since the feature data is extracted from each image frame, the feature data may be transmitted frame by frame to the external electronic device 301 through the communication module 390 .

According to an embodiment, the controller 623 may generate a transmission frame including the feature data corresponding to the one frame and transmit it to the external electronic device 301 .

According to an embodiment, the feature data processing unit 624 may control transmission of the feature data accumulated in the buffer in the adjusting unit 623 based on a condition related to the transmission of the feature data. According to an embodiment, when the characteristic data processing unit 624 satisfies a condition related to transmission of the characteristic data, a transmission frame including the characteristic data may be generated and transmitted to the external electronic device 301 . For example, when forgery or not is not detected, the feature data processing unit 624 may generate a first transmission frame including the feature data and transmit it to the external electronic device 301 . According to an embodiment, the feature data processing unit 624 may transmit metadata including identification information indicating that feature data is included when the first transmission frame is transmitted. The metadata may include data related to transmission of the first transmission frame.

According to an embodiment, the feature data processing unit 624 determines that there is no need to transmit the feature data when the condition related to whether or not to transmit the feature data is not satisfied, and the feature data processing unit 624 determines to include other data instead of the actually extracted feature data. 2 A transmission frame may be generated and transmitted to the external electronic device 601 . For example, when forgery is detected, the feature data processing unit 624 may ignore or discard the feature data, but prevents an attempt from the outside for forgery or forgery using whether data is transmitted from the electronic device 602 or not. To do this, the second transmission frame including invalid data instead of the feature data may be transmitted to the external electronic device 601 .

Here, the invalid data included in the second transmission frame is data unrelated to the feature data, and may be data defined in advance with the external electronic device 601 or data obtained by modifying the feature data. According to an embodiment, the feature data processing unit 624 may include and transmit metadata including identification information indicating that data is invalid when the second transmission frame is transmitted.

According to an embodiment, for secure transmission between the electronic device 602 and the external electronic device 601 during actual feature data transmission, the feature data processing unit 624 is configured in advance between the electronic device 602 and the external electronic device 601 . Data for verification may be generated according to the transmission method. Alternatively, the feature data processing unit 624 may identify whether a preset condition for the transmission of the feature data is satisfied, and transmit the verification data and the feature data according to the preset transmission method. For example, the characteristic data processing unit 624 may sequentially transmit the characteristic data and the verification data accumulated in the buffer in the control unit 623 according to a predetermined transmission order.

Accordingly, the characteristic data processing unit 624 may transmit the characteristic data accumulated in the buffer in the control unit 623 to the external electronic device 601 in a predetermined frame unit. In this case, the feature data processing unit 624 may transmit verification data together with the feature data when it is time to transmit verification data based on a preset condition, but transmit verification data instead of the feature data. may be For example, the feature data processing unit 624 may transmit the verification data by hiding it in the feature data, and may transmit the verification data whenever a predetermined number of the feature data is transmitted. As described above, a method of transmitting verification data for security of the feature data when transmitting the feature data is determined in advance between the electronic device 602 and the external electronic device 601 , and the type may not be limited thereto.

According to an embodiment, the feature data processing unit 624 may receive a response (or feedback) including a user authentication result from the external electronic device 601 . The response may include a matching result for the characteristic data sent to the external electronic device 601 , and the characteristic data processing unit 624 may manage a history of accumulating whether or not matching of each characteristic data is successful. Accordingly, the forgery detection unit 622 may identify whether the forgery or forgery is based on the accumulated history including whether the matching for each characteristic data is successful.

Meanwhile, the user authentication module 610 of the external electronic device 301 may receive a transmission frame including feature data transmitted from the electronic device 602 through the communication module 691 . According to an embodiment, the user authentication module 610 may perform user authentication based on a result of biometric authentication in the electronic device 601 . For example, when the biometric authentication result of the electronic device 601 is biometric information about a real user, the external electronic device 602 may receive feature data related to the biometric information, and match a template to the feature data It is possible to perform authentication as to whether the user's biometric information of the actually registered external electronic device 602 is through .

According to an embodiment, upon receiving the transmission frame, the user authentication module 610 determines whether the data included in the transmission frame received from the electronic device 602 is actual feature data or invalid data based on the metadata. can be identified. According to an embodiment, the identification operation for the transmission frame may be performed by the matching unit 631 or the determining unit 632 .

According to an embodiment, if the determination unit 632 identifies the received transmission frame as the second transmission frame including invalid data based on the metadata, the determination unit 632 controls the second transmission frame You can ignore or discard invalid data contained in .

According to an embodiment, if the determination unit 632 identifies the received transmission frame as the first transmission frame including the actual feature data based on the metadata, the matching unit 631 includes it in the first transmission frame. Template matching may be performed using the characteristic data and the characteristic pattern registered in the template storage unit 630 .

For example, the matching unit 631 may compare with characteristic data of biometric information about a user registered in the external electronic device 301 using characteristic data corresponding to a predetermined number of frames. If the compared matching ratio is equal to or greater than a certain ratio (or a threshold value), it is determined that the characteristic data and the registered characteristic data are for the same user, and the determination unit 632 may determine that the user authentication is successful. Accordingly, the matching unit 631 may feed back the matching result to the electronic device 602 through the communication module 391 . The matching result may include whether to match feature data corresponding to each of the predetermined number of frames.

According to an embodiment, when user authentication is successful, the external electronic device 601 may process a corresponding operation. For example, when the external electronic device 601 is in a locked state, unlocking may be performed in response to user authentication. If the display of the external electronic device 601 is in an off state, the external electronic device 601 may turn on the display. For example, when user authentication is required while a payment application is executed, reception The authentication result for the biometric information obtained can be used as information for user authentication. In this way, the user's biometric information acquired in the electronic device 602 is used as data for user authentication in the external electronic device 601 , and the external electronic device 601 includes not only unlocking but also electronic signatures and payment information. The biometric information may be used for the purpose.

7 is a diagram 700 for explaining transmission of data for verification according to an embodiment.

Referring to FIG. 7 , the horizontal axis represents time, and each frame may correspond to feature data. 7 , the electronic device 602 may transmit verification data to the external electronic device 601 based on the first method 710 or the second method 720 .

The first method 710 according to an embodiment may be a method of transmitting verification data based on a preset condition between the electronic device 602 and the external electronic device 601 . Here, the preset condition may include at least one of a preset transmission period and a preset number of transmission frames (or transmission order).

According to an embodiment, when a session is established (or session connection) between the electronic device 602 and the external electronic device 601 , the preset condition for sending verification data may be mutually determined.

For example, the electronic device 602 may transmit the verification data 711 , 712 , 713 , and 714 every five frames, for example, a preset number of transmission frames based on the time stamp. According to an embodiment, the verification data may be invalid data transmitted instead of the actual feature data. Alternatively, the fifth verification data may be sent in a way that is hidden in the actual fifth feature data.

The second method 720 according to an embodiment may be a method in which an order in which verification data is transmitted is determined in response to a request from the external electronic device 601 . For example, the electronic device 602 may transmit verification data 721 and 722 according to location designation in response to a request from the external electronic device 601 . For example, the electronic device 602 inserts verification data into the number of transmission frames (or transmission order) (eg, 8th and 12th) requested by the external electronic device 601 and transmits the data to the external electronic device 601 . can

According to an embodiment, when verification data is not received in a preset transmission order or a requested transmission order, the external electronic device 601 may determine that there is an attack by an invalid user or an external server. Accordingly, the external electronic device 601 may fail the user authentication based on the feature data, and notify the electronic device 602 of the user authentication failure.

8 is a diagram 800 for explaining whether or not forgery is based on a matching result according to an embodiment.

Referring to FIG. 8 , the electronic device 602 may receive a matching result from the external electronic device 601 , and the matching result may be used by the forgery detection unit 622 to detect a forgery attempt.

For example, based on the matching result, the electronic device 602 generates some feature data including fifth and sixth feature data among the feature data in the buffer in the controller (eg, the controller 623 of FIG. 6 ). It is possible to accumulate and manage the history of successful matching only for . As described above, the electronic device 602 may manage a history of accumulating whether or not matching with respect to each feature data is successful. Accordingly, the electronic device 602 may perform forgery detection based on the accumulated history. For example, since the electronic device 602 detects whether or not forgery or not with respect to feature data corresponding to more frames together with a matching result for previously transmitted feature data, a forgery result with higher accuracy may be obtained.

9 is an operation flowchart 900 of an electronic device for biometric authentication according to an embodiment.

Referring to FIG. 9 , the operating method may include operations 905 to 930 . Each step/operation of the operating method of FIG. 9 includes an electronic device (eg, the electronic device 202 of FIG. 2 , the electronic device 302 of FIG. 3 , the electronic device 602 of FIG. 6 , and at least one of the electronic device It may be performed by at least one of the processors (eg, the processor 420 of Fig. 4 ) In one embodiment, at least one of operations 905 to 930 is omitted, the order of some operations is changed, or another operation is performed. This can be added.

Referring to FIG. 9 , an electronic device 602 communicates with an external electronic device (eg, the electronic device 201 of FIG. 2 , the external electronic device 301 of FIG. 3 , and the external electronic device 301 of FIG. 6 ). It can be connected to communication wirelessly. According to an embodiment, when the electronic device 602 is connected to the external electronic device 601 , the electronic device 602 performs authentication with the external electronic device 601 for authentication related to the use of the external electronic device 601 . you can request According to an embodiment, in order to authenticate the user of the external electronic device 601 , the electronic device 602 may transmit the user's biometric information required for user authentication to the external electronic device 601 .

According to an embodiment, in a state in which the electronic device 602 is connected to the external electronic device 601 , the electronic device 602 uses a sensor module (eg, the sensor module 476 of FIG. 4 ) in operation 905 to allow a user biometric information can be obtained. As a method of acquiring the user's biometric information, a method of photographing the user's iris image using the camera of the electronic device 602 may be used.

In operation 910, the electronic device 602 may extract feature data from the biometric information. Next, in operation 915 , the electronic device 602 may identify a condition related to whether to transmit the extracted feature data. As a method for identifying whether it is possible to transmit the feature data, the electronic device 602 according to an embodiment may determine whether the feature data is forged or altered. For example, since an invalid user may try to authenticate the user using a forgery technique, the electronic device 602 identifies whether the user who has entered biometric information is a genuine user or an imposter user. It can be determined whether forgery or not. For example, the electronic device 602 may determine whether forgery or not in order to filter out an authentication attempt by an invalid user using a substitute such as an iris photograph or a model.

As a result of identifying the condition related to whether the feature data is transmitted, the electronic device 602 may identify whether the condition related to whether or not to transmit the feature data is satisfied in operation 920 . If the condition related to whether or not to transmit the feature data is not satisfied, the electronic device 602 determines that there is no need to transmit the feature data and ignores or discards the feature data in operation 925 .

For example, when a condition related to whether the characteristic data is transmitted or not is not satisfied, a case in which the characteristic data is determined to be invalid data (eg, forged biometric information) as a result of a forgery determination may correspond. When it is determined that a forgery attempt has occurred, the electronic device 602 does not need to proceed to the user authentication step, so it may ignore or discard the feature data.

According to an embodiment, the electronic device 602 may provide an early rejection function by ignoring or dropping the feature data. Such a fast rejection function may reduce a data transmission load between the electronic device 602 and the external electronic device 601 . In addition, for example, if a forgery attempt on the feature data is confirmed within the threshold time for biometric authentication, the electronic device 602 processes the biometric authentication failure before the threshold time is reached, thereby providing the biometric authentication result within a short time. can be printed out. Accordingly, when the electronic device 602 determines whether the forgery or not, and determines that the data is invalid, it does not proceed to the user authentication step or ultimately treats the user authentication as a failure regardless of the result of user authentication, thereby reducing the time required for user authentication. can be shortened

According to another embodiment, if the condition related to whether to transmit the feature data is not satisfied, the electronic device 602 determines that there is no need to transmit the feature data and includes a second data other than the actually extracted feature data. The transmission frame may be transmitted to the external electronic device 601 .

For example, when a forgery attempt occurs, the characteristic data may be ignored or discarded, but the side attempting forgery may try another method of forgery using whether data is transmitted from the electronic device 602 or not. Therefore, in order to prevent this, the electronic device 602 may transmit a second transmission frame including invalid data instead of the feature data to the external electronic device 601 . Here, the electronic device 602 may transmit invalid data defined in advance with the external electronic device 601 or data obtained by modifying the feature data regardless of the feature data. According to an embodiment, the second transmission frame may include metadata including identification information indicating invalid data.

Meanwhile, as a result of identifying the condition related to whether or not to transmit the feature data, if the electronic device 602 satisfies the condition related to whether to transmit the feature data in operation 920 , the electronic device 602 performs a first function including the feature data. The transmission frame may be transmitted to the external electronic device 601 interworking with the electronic device 602 .

According to an embodiment, the external electronic device 601 may identify whether data included in a transmission frame received from the electronic device 602 is actual feature data or invalid data based on the metadata.

According to an embodiment, when the external electronic device 601 identifies that the received transmission frame is a second transmission frame including invalid data, it ignores or discards invalid data included in the second transmission frame. can do.

According to an embodiment, when the external electronic device 601 identifies that the received transmission frame is the first transmission frame including the actual feature data, the external electronic device 601 determines the characteristic included in the first transmission frame. Template matching can be performed using data. For example, the external electronic device 601 may use the feature data to compare it with feature data of biometric information about a user registered in the external electronic device 601 . If the compared matching ratio is equal to or greater than a certain ratio, it is determined that the characteristic data and the registered characteristic data are for the same user, and the external electronic device 601 may determine that user authentication is successful. Accordingly, the external electronic device 601 may perform a predetermined operation using the user authentication result.

For example, if user authentication using biometric information is successful, functions such as unlocking, application execution, payment approval, and data transmission may be performed, and the external electronic device 601 may display a screen according to the execution of the function. have. Also, in response to this, the electronic device 602 may receive content according to the function execution of the external electronic device 601 and display the screen.

According to an embodiment, the external electronic device 601 may transmit a matching result, that is, a response (or feedback) including a user authentication result, to the electronic device 602 . According to an embodiment, the response may include a matching result for the feature data, and the electronic device 602 may manage a history of accumulating whether or not matching with respect to each feature data is successful. Accordingly, the electronic device 602 may identify whether forgery or not based on an accumulated history including whether or not matching with respect to each characteristic data is successful. As described above, by receiving the matching feedback from the external electronic device 601, the electronic device 602 can refer to the matching result in determining whether or not forgery has been made. The more the matching results are referred, the higher the probability of success in detecting forgery. can

According to an embodiment, a method for biometric authentication in an electronic device relates to an operation of acquiring biometric information through a sensor module of the electronic device, an operation of extracting feature data from the biometric information, and whether the feature data is transmitted. An operation of identifying a condition, an operation of transmitting a first transmission frame including the feature data to an external electronic device interworking with the electronic device based on a condition related to whether the transmission is performed, an operation of transmitting the first transmission frame, and When the related condition is not satisfied, the operation of ignoring or discarding the feature data may be included.

According to an embodiment, the transmitting to the external electronic device may include transmitting the first transmission frame including the characteristic data to the external electronic device when a condition related to whether the transmission is satisfied. can

According to an embodiment, the operation of identifying a condition related to whether the characteristic data is transmitted may include an operation of identifying whether the characteristic data is spoofed.

According to an embodiment, the method may further include transmitting a second transmission frame including data different from the characteristic data to the external electronic device in response to the occurrence of the forgery.

According to an embodiment, the method further includes receiving an authentication result for the feature data from the external electronic device in response to transmission of the first transmission frame, wherein the authentication result for the feature data is It can be used to identify whether forgery or not.

According to an embodiment, the method may further include transmitting verification data to the external electronic device when a preset condition is satisfied.

According to an embodiment, the preset condition may include at least one of a preset transmission period and a preset number of transmission frames.

According to an embodiment, in the transmitting of the first transmission frame to the external electronic device, when the preset condition is satisfied, the first transmission frame including the verification data instead of the feature data is transmitted to the external electronic device. It may include the operation of transmitting to the device.

10 is a diagram 1000 for explaining a process of ignoring or discarding feature data in response to forgery occurrence according to an embodiment.

Referring to FIG. 10 , an electronic device (eg, the electronic device 202 of FIG. 2 , the electronic device 302 of FIG. 3 , the electronic device 402 of FIG. 4 , and the electronic device 602 of FIG. 6 ) (hereinafter, 602) is connected to an external electronic device (eg, the electronic device 201 of FIG. 2 , the external electronic device 301 of FIG. 3 , and the external electronic device 601 of FIG. 6 ) (hereinafter referred to as 601 ) in a state in which a connection is established. , 1010 , the electronic device 602 may obtain biometric information required for user authentication by driving the sensor module 676 .

As a plurality of image images (eg, N frames) are input from the sensor module 676 in operation 1010 , in operation 1020 , the electronic device 602 (eg, the forgery detection unit 622 ) provides information about the plurality of image images. Forgery can be identified. Also, the feature extractor 621 may extract feature data from at least some of the plurality of image images. For example, the feature extraction unit 621 may extract feature data for every frame. In addition, a plurality of image images from the sensor module 676 may be transmitted to the forgery detection unit 622, otherwise the forgery detection unit 622 receives the feature data from the feature extraction unit 621, Forgery can also be detected.

The electronic device 602 may determine whether to transmit the feature data. According to an embodiment, whether the electronic device 602 is forgery or not is a condition for determining whether to transmit the feature data from the feature extraction unit 621 accumulated in a buffer in the control unit 623 to the external electronic device 602 , for example. results are available.

Accordingly, when forgery does not occur in operation 1020 , the electronic device 602 may sequentially transmit the feature data accumulated in the buffer in operation 1040 to the external electronic device 601 . For example, since the feature data may be extracted for each image frame, the feature data may be transmitted to the external electronic device 601 one by one. According to an embodiment, the electronic device 602 may generate a transmission frame including feature data from one image frame, and transmit the generated transmission frame to the external electronic device 601 . According to an embodiment, the transmission frame may include the feature data and metadata. The metadata may include at least one of identification information of the electronic device 602 , a certificate of the electronic device 602 , information indicating the number of feature data, for example, an order of the feature data, and data for verification. .

As the transmission frame is received, the external electronic device 601 may identify whether feature data is included based on metadata of the transmission frame. If it is identified that the feature data is included, the external electronic device 601 may determine whether user authentication is successful based on the feature data. For example, the external electronic device 601 may perform authentication as to whether it is feature data related to the biometric information of the user of the actually registered external electronic device 602 through template matching for the feature data.

Meanwhile, the electronic device 602 may receive feedback in operation 1060 in response to the transmission of the transmission frame. For example, the feedback may include a matching result for the feature data, which may correspond to a user authentication result. In operation 1070, the electronic device 602 may update the biometric information processing result based on the authentication result. Here, the biometric information processing result may be a matching result for the feature data related to the biometric information. For example, the electronic device 602 may cumulatively manage whether or not matching is successful for each feature data included in the authentication result, and may use the accumulated history for each feature data to identify whether or not forgery has occurred.

On the other hand, when forgery occurs in operation 1020, for example, when it is determined that there is a forgery attempt (eg, a spoofing attack), for example, when an invalid user is an authentication attempt using an alternative such as an iris photo or a model, the user authentication step Since there is no need to proceed to , the electronic device 601 may ignore or discard the feature data extracted from the biometric information accumulated in the buffer for transmission in operation 1030 .

11 is a diagram 1100 for explaining a process of transmitting invalid data instead of feature data in response to forgery occurrence according to an embodiment.

Operations 1110 to 1120 and 1140 of FIG. 11 are the same as operations 1010 to 1020 and 1040 of FIG. 10 , and thus detailed descriptions thereof will be omitted. However, in FIG. 11 , when forgery occurs, for example, when it is determined that there is a forgery attempt (eg, a spoofing attack), in the electronic device 601 , other data (eg: Invalid data) may be transmitted in the second transmission frame. For example, since an invalid user may attempt another attack by using whether or not the feature data is transmitted, invalid data may be transmitted in the same manner as when transmitting the feature data.

According to an embodiment, invalid data included in the transmission frame may be data randomly filled in the transmission frame instead of the feature data. An identification flag (eg, a decoy flag) for preventing the external electronic device 301 from performing user authentication using invalid data may be assigned to the transmission frame or metadata for the transmission frame. For example, an identification flag may be disposed at the end of a transmission frame, and the identification flag may indicate data for verification. Accordingly, when receiving the first transmission frame in operation 1150 , the external electronic device 602 determines whether user authentication is successful based on the feature data including the first transmission frame, but a second transmission frame including invalid data Upon receiving , the external electronic device 602 may not proceed to the user authentication step.

According to an embodiment, invalid data included in the transmission frame may include, for example, data of a pattern that cannot be actually generated, such as '00000000' or '11111', instead of feature data.

The electronic device according to various embodiments disclosed in this document may have various types of devices. The electronic device may include, for example, a portable communication device (eg, a smart phone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a home appliance device. The electronic device according to the embodiment of the present document is not limited to the above-described devices.

The various embodiments of this document and terms used therein are not intended to limit the technical features described in this document to specific embodiments, but it should be understood to include various modifications, equivalents, or substitutions of the embodiments. In connection with the description of the drawings, like reference numerals may be used for similar or related components. The singular form of the noun corresponding to the item may include one or more of the item, unless the relevant context clearly dictates otherwise. As used herein, "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C", and "A , B, or C" each may include any one of, or all possible combinations of, items listed together in the corresponding one of the phrases. Terms such as "first", "second", or "first" or "second" may be used simply to distinguish an element from other elements in question, and may refer elements to other aspects (e.g., importance or order) is not limited. It is said that one (eg, first) component is "coupled" or "connected" to another (eg, second) component, with or without the terms "functionally" or "communicatively". When referenced, it means that one component can be connected to the other component directly (eg by wire), wirelessly, or through a third component.

The term “module” used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and is interchangeable with terms such as, for example, logic, logic block, component, or circuit. can be used as A module may be an integrally formed part or a minimum unit or a part of the part that performs one or more functions. For example, according to an embodiment, the module may be implemented in the form of an application-specific integrated circuit (ASIC).

According to various embodiments of the present document, one or more instructions stored in a storage medium (eg, internal memory 136 or external memory 138) readable by a machine (eg, electronic device 101) may be implemented as software (eg, the program 140) including For example, a processor (eg, processor 120 ) of a device (eg, electronic device 101 ) may call at least one command among one or more commands stored from a storage medium and execute it. This makes it possible for the device to be operated to perform at least one function according to the called at least one command. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' only means that the storage medium is a tangible device and does not contain a signal (eg, electromagnetic wave), and this term refers to the case where data is semi-permanently stored in the storage medium and It does not distinguish between temporary storage cases.

According to an embodiment, the method according to various embodiments disclosed in this document may be included and provided in a computer program product. Computer program products may be traded between sellers and buyers as commodities. The computer program product is distributed in the form of a machine-readable storage medium (eg compact disc read only memory (CD-ROM)), or via an application store (eg Play Store ^TM ) or on two user devices ( It can be distributed (eg downloaded or uploaded) directly between smartphones (eg: smartphones) and online. In the case of online distribution, at least a part of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium such as a memory of a server of a manufacturer, a server of an application store, or a relay server.

According to various embodiments, each component (eg, module or program) of the above-described components may include a singular or a plurality of entities, and some of the plurality of entities may be separately disposed in other components. have. According to various embodiments, one or more components or operations among the above-described corresponding components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (eg, a module or a program) may be integrated into one component. In this case, the integrated component may perform one or more functions of each component of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. . According to various embodiments, operations performed by a module, program, or other component are executed sequentially, in parallel, repeatedly, or heuristically, or one or more of the operations are executed in a different order, or omitted. or one or more other operations may be added.

In the storage medium storing instructions, the instructions are configured to cause the at least one processor to perform at least one operation when executed by the at least one processor, wherein the at least one operation is a sensor module of an electronic device An operation of obtaining biometric information through the method, an operation of extracting feature data from the biometric information, an operation of identifying a condition related to whether the characteristic data is transmitted, and an operation of including the characteristic data based on a condition related to the transmission or not It may include an operation of transmitting the first transmission frame to an external electronic device interworking with the electronic device, and an operation of ignoring or discarding the feature data when a condition related to whether to transmit the first transmission frame is not satisfied. .

And, the embodiments of the present invention invented in the present specification and drawings are merely provided for specific examples to easily explain the technical contents according to the embodiments of the present invention and help the understanding of the embodiments of the present invention, and the scope of the embodiments of the present invention It is not intended to limit Therefore, in the scope of various embodiments of the present invention, in addition to the embodiments disclosed herein, it should be construed that all changes or modifications derived from the technical ideas of various embodiments of the present invention are included in the scope of various embodiments of the present invention. .

Claims (20)

In an electronic device,
sensor module;
communication module; and
at least one processor operatively coupled to the sensor module and the communication module, the at least one processor comprising:
Obtaining biometric information through the sensor module,
extracting feature data from the biometric information, identifying a condition related to whether to transmit the feature data,
Transmitting a first transmission frame including the characteristic data to an external electronic device interworking with the electronic device through the communication module based on a condition related to whether or not the transmission is performed;
The electronic device is configured to ignore or discard the feature data when a condition related to whether to transmit or not is satisfied.
The method of claim 1, wherein the at least one processor comprises:
The electronic device is configured to transmit the first transmission frame including the feature data to the external electronic device through the communication module when a condition related to whether the transmission is satisfied.
The method of claim 1, wherein the at least one processor comprises:
The electronic device is configured to identify whether spoofing has been performed on the feature data, and to identify a condition related to whether or not to transmit the feature data based on whether the feature data is forged or not.
The method of claim 3, wherein the at least one processor comprises:
an electronic device configured to transmit a second transmission frame including data different from the characteristic data to the external electronic device through the communication module in response to the occurrence of the forgery.
According to claim 1,
The biometric information obtained through the sensor module includes a plurality of iris images,
the at least one processor,
and extract the feature data from at least some of the plurality of iris images.
The method of claim 1, wherein the at least one processor comprises:
When a preset condition is satisfied, the electronic device is configured to transmit verification data to the external electronic device through the communication module.
According to claim 6, The preset condition,
An electronic device comprising at least one of a preset transmission period and a preset number of transmission frames.
The method of claim 6, wherein the at least one processor comprises:
When the preset condition is satisfied, the electronic device is configured to include and transmit the verification data in the feature data.
The method of claim 6, wherein the at least one processor comprises:
When the preset condition is satisfied, the electronic device is configured to transmit the first transmission frame including the verification data instead of the feature data to the external electronic device.
The method of claim 1, wherein the at least one processor comprises:
In response to transmission of the first transmission frame, receiving an authentication result for the feature data from the external electronic device,
An electronic device, wherein the authentication result for the feature data is used to identify whether or not forgery.
According to claim 1, wherein the first transmission frame,
including the feature data and meta data;
The metadata includes at least one of identification information of the electronic device, a certificate of the electronic device, an order of the feature data, and verification data.
A method for biometric authentication in an electronic device, comprising:
acquiring biometric information through a sensor module of the electronic device;
extracting feature data from the biometric information;
identifying a condition related to whether the feature data is transmitted;
transmitting a first transmission frame including the feature data to an external electronic device that interworks with the electronic device based on a condition related to whether or not to transmit the first transmission frame; and
and disregarding or discarding the feature data when a condition related to whether the transmission is not satisfied.
The method of claim 12, wherein the transmitting to the external electronic device comprises:
and transmitting the first transmission frame including the feature data to the external electronic device when a condition related to whether the transmission is satisfied.
The method of claim 12, wherein the identifying a condition related to whether the feature data is transmitted comprises:
A method for biometric authentication comprising the operation of identifying whether the feature data is spoofed.
15. The method of claim 14,
The method of claim 1, further comprising transmitting a second transmission frame including data different from the characteristic data to the external electronic device in response to the forgery occurrence.
13. The method of claim 12,
The method further includes receiving an authentication result for the feature data from the external electronic device in response to the transmission of the first transmission frame,
A method for biometric authentication, wherein the authentication result for the feature data is used to identify whether forgery or not.
13. The method of claim 12,
When a preset condition is satisfied, the method further comprising the operation of transmitting verification data to the external electronic device.
The method of claim 17, wherein the preset condition comprises:
A method for biometric authentication, comprising at least one of a preset transmission period and a preset number of transmission frames.
The method of claim 17, wherein the transmitting of the first transmission frame to the external electronic device comprises:
and transmitting the first transmission frame including the verification data instead of the feature data to the external electronic device when the preset condition is satisfied.
A storage medium storing instructions, wherein the instructions are configured to cause the at least one processor to perform at least one operation when executed by the at least one processor, wherein the at least one operation comprises:
acquiring biometric information through a sensor module of the electronic device;
extracting feature data from the biometric information;
identifying a condition related to whether the feature data is transmitted;
transmitting a first transmission frame including the feature data to an external electronic device that interworks with the electronic device based on a condition related to whether or not to transmit the first transmission frame; and
and disregarding or discarding the feature data when a condition related to whether the transmission is not satisfied.

Images