KR20220028559A - Electronic device and method for determining fake fingerprint based on photoacoustic characteristics - Google Patents

Abstract

Various embodiments of the present invention provide a method and a device, the device includes a fingerprint sensor, a light source, a memory including a user's fingerprint feature information, and a processor operatively connected to the fingerprint sensor, the light source, and the memory, wherein the processor is configured to: output light via the light source; receive, via the fingerprint sensor, an optoacoustic signal generated from an object by the output light; extract feature information from the received optoacoustic signal; compare the extracted feature information and the fingerprint feature information stored in the memory so as to determine whether a fingerprint is authenticated. Various embodiments are possible. Therefore, it is possible to secure a security-enhanced authentication means by supplementing the weakness of the fingerprint sensor against forged fingerprints.

Description

Fake fingerprint determination method and device based on optoacoustic characteristics

Various embodiments of the present invention are disclosed with respect to a method and apparatus for determining a fake fingerprint based on optoacoustic characteristics.

With the development of digital technology, various types of electronic devices such as mobile communication terminals, personal digital assistants (PDAs), electronic notebooks, smart phones, tablet PCs (personal computers), and wearable devices are widely used. In order to support and increase functions of the electronic device, the hardware part and/or the software part of the electronic device are continuously being improved.

For example, the electronic device provides a photographing function, a document editing function, an Internet function, and a payment function, so that a user can take a picture using the electronic device instead of a camera, edit a document using the electronic device instead of a laptop computer, or perform Internet can be used, and payment can be made using an electronic device instead of a card or cash. In particular, when performing a financial transaction or payment through the electronic device, the electronic device may perform user authentication using a biometric authentication sensor (eg, fingerprint, iris). Since important information such as information for a financial transaction or a card number for payment is stored in the electronic device, user authentication is used as a very important factor in using the electronic device.

Among authentication through biometric sensors, fingerprint authentication is widely used in finance, access control, and public fields in an accurate and efficient way. The electronic device performs fingerprint authentication by extracting feature points of unique ridges and valleys of a finger from an image acquired from a fingerprint sensor. The fingerprint authentication method may be divided into an optical method, an ultrasonic method, or a capacitive method, and optical and ultrasonic methods are currently being commercialized. However, since the resolution of a general fingerprint sensor is low, it is not sufficient to recognize the minute features of the human body, so it may be difficult to determine forgery of the fingerprint.

In various embodiments, a method and apparatus for arranging a light source for generating light in an electronic device, authenticating a fingerprint using a fingerprint sensor, and determining a forged fingerprint using optoacoustic characteristics may be disclosed. there is.

An electronic device according to various embodiments includes a fingerprint sensor, a light source, a memory including user's fingerprint characteristic information, and a processor operatively connected to the fingerprint sensor, the light source, and the memory, the processor comprising: Outputs light through the light source, receives an optoacoustic signal generated from an object by the output light through the fingerprint sensor, extracts characteristic information from the received photoacoustic signal, and It may be set to determine whether the fingerprint is authenticated by comparing the fingerprint characteristic information stored in the memory.

According to various embodiments of the present disclosure, a method of operating an electronic device including a light source and a fingerprint sensor includes outputting light through the light source and receiving a photoacoustic signal generated from an object by the light output through the fingerprint sensor operation, extracting characteristic information from the received optoacoustic signal, and determining whether a fingerprint is authenticated by comparing the extracted characteristic information with fingerprint characteristic information stored in the memory.

According to various embodiments, it is possible to secure a security-enhanced authentication method by supplementing the weakness of the fingerprint sensor against a forged fingerprint.

According to various embodiments, it is possible to simply distinguish whether a forged fingerprint exists by analyzing the frequency component of the reflected signal without a separate imaging process for the signal reflected by the output of light.

1 is a block diagram of an electronic device in a network environment according to various embodiments of the present disclosure;
2A to 2C are diagrams illustrating a light source and a display included in an electronic device according to various embodiments of the present disclosure;
3A to 3D are other views illustrating a light source and a display included in an electronic device according to various embodiments of the present disclosure;
4 is a diagram illustrating an example of optoacoustic characteristics according to various embodiments of the present disclosure;
5 is a flowchart illustrating a method of operating an electronic device according to various embodiments of the present disclosure;
6 is a diagram illustrating an example in which light is absorbed by a user's body according to various embodiments of the present disclosure;
7A to 7C are diagrams illustrating comparison examples of a user's fingerprint and a forged fingerprint according to various embodiments of the present disclosure;
8 is a diagram illustrating an example of comparing frequency characteristics of a user's fingerprint and a forged fingerprint according to various embodiments of the present disclosure;
9 is a flowchart illustrating a method of determining a forged fingerprint of an electronic device according to various embodiments of the present disclosure;

The electronic device according to various embodiments disclosed in this document may have various types of devices. The electronic device may include, for example, a portable communication device (eg, a smart phone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a home appliance device. The electronic device according to the embodiment of the present document is not limited to the above-described devices.

The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, but it should be understood to include various modifications, equivalents, or substitutions of the embodiments. In connection with the description of the drawings, like reference numerals may be used for similar or related components. The singular form of the noun corresponding to the item may include one or more of the item, unless the relevant context clearly dictates otherwise. As used herein, "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C", and "A , B, or C" each may include any one of, or all possible combinations of, items listed together in the corresponding one of the phrases. Terms such as “first”, “second”, or “first” or “second” may simply be used to distinguish the component from other such components, and may refer to components in other aspects (eg, importance or order) is not limited. It is said that one (eg, first) component is "coupled" or "connected" to another (eg, second) component, with or without the terms "functionally" or "communicatively". When referenced, it means that one component can be connected to the other component directly (eg by wire), wirelessly, or through a third component.

The term “module” used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and is interchangeable with terms such as, for example, logic, logic block, component, or circuit. can be used as A module may be an integrally formed part or a minimum unit or a part of the part that performs one or more functions. For example, according to an embodiment, the module may be implemented in the form of an application-specific integrated circuit (ASIC).

1 is a block diagram of an electronic device 101 in a network environment 100 according to various embodiments of the present disclosure.

Referring to FIG. 1 , in a network environment 100 , the electronic device 101 communicates with the electronic device 102 through a first network 198 (eg, a short-range wireless communication network) or a second network 199 . It may communicate with the electronic device 104 or the server 108 through (eg, a long-distance wireless communication network). According to an embodiment, the electronic device 101 may communicate with the electronic device 104 through the server 108 . According to an embodiment, the electronic device 101 includes a processor 120 , a memory 130 , an input module 150 , a sound output module 155 , a display module 160 , an audio module 170 , and a sensor module ( 176), interface 177, connection terminal 178, haptic module 179, camera module 180, power management module 188, battery 189, communication module 190, subscriber identification module 196 , or an antenna module 197 may be included. In some embodiments, at least one of these components (eg, the connection terminal 178 ) may be omitted or one or more other components may be added to the electronic device 101 . In some embodiments, some of these components (eg, sensor module 176 , camera module 180 , or antenna module 197 ) are integrated into one component (eg, display module 160 ). can be

The processor 120, for example, executes software (eg, the program 140) to execute at least one other component (eg, hardware or software component) of the electronic device 101 connected to the processor 120 . It can control and perform various data processing or operations. According to one embodiment, as at least part of data processing or operation, the processor 120 converts commands or data received from other components (eg, the sensor module 176 or the communication module 190 ) to the volatile memory 132 . may be stored in the volatile memory 132 , and may process commands or data stored in the volatile memory 132 , and store the result data in the non-volatile memory 134 . According to an embodiment, the processor 120 is a main processor 121 (eg, a central processing unit (CPU) or an application processor (AP)) or an auxiliary processor capable of operating independently or together with it ( 123) (eg, a graphic processing unit (GPU), a neural network processing unit (NPU), an image signal processor (ISP), a sensor hub processor, or a communication processor (CP, communication processor)) may be included. For example, when the electronic device 101 includes the main processor 121 and the sub-processor 123 , the sub-processor 123 uses less power than the main processor 121 or is set to be specialized for a specified function. can The auxiliary processor 123 may be implemented separately from or as a part of the main processor 121 .

The auxiliary processor 123 is, for example, on behalf of the main processor 121 while the main processor 121 is in an inactive (eg, sleep) state, or the main processor 121 is At least one of the components of the electronic device 101 (eg, the display module 160 , the sensor module 176 , or At least some of functions or states related to the communication module 190 may be controlled. According to an embodiment, the coprocessor 123 (eg, an image signal processor or a communication processor) may be implemented as part of another functionally related component (eg, the camera module 180 or the communication module 190 ). there is. According to an embodiment, the auxiliary processor 123 (eg, a neural network processing device) may include a hardware structure specialized for processing an artificial intelligence model. Artificial intelligence models can be created through machine learning. Such learning may be performed, for example, in the electronic device 101 itself on which artificial intelligence is performed, or may be performed through a separate server (eg, the server 108). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but in the above example not limited The artificial intelligence model may include a plurality of artificial neural network layers. Artificial neural networks include deep neural networks (DNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), restricted boltzmann machines (RBMs), deep belief networks (DBNs), bidirectional recurrent deep neural networks (BRDNNs), It may be one of deep Q-networks or a combination of two or more of the above, but is not limited to the above example. The artificial intelligence model may include, in addition to, or alternatively, a software structure in addition to the hardware structure.

The memory 130 may store various data used by at least one component of the electronic device 101 (eg, the processor 120 or the sensor module 176 ). The data may include, for example, input data or output data for software (eg, the program 140 ) and instructions related thereto. The memory 130 may include a volatile memory 132 or a non-volatile memory 134 .

The program 140 may be stored as software in the memory 130 , and may include, for example, an operating system (OS) 142 , middleware 144 , or an application 146 . there is.

The input module 150 may receive a command or data to be used by a component (eg, the processor 120 ) of the electronic device 101 from the outside (eg, a user) of the electronic device 101 . The input module 150 may include, for example, a microphone, a mouse, a keyboard, a key (eg, a button), or a digital pen (eg, a stylus pen).

The sound output module 155 may output a sound signal to the outside of the electronic device 101 . The sound output module 155 may include, for example, a speaker or a receiver. The speaker can be used for general purposes such as multimedia playback or recording playback. The receiver may be used to receive an incoming call. According to one embodiment, the receiver may be implemented separately from or as part of the speaker.

The display module 160 may visually provide information to the outside (eg, a user) of the electronic device 101 . The display module 160 may include, for example, a control circuit for controlling a display, a hologram device, or a projector and a corresponding device. According to an embodiment, the display module 160 may include a touch sensor configured to sense a touch or a pressure sensor configured to measure the intensity of a force generated by the touch.

The audio module 170 may convert a sound into an electric signal or, conversely, convert an electric signal into a sound. According to an embodiment, the audio module 170 acquires a sound through the input module 150 , or an external electronic device (eg, a sound output module 155 ) connected directly or wirelessly with the electronic device 101 . Sound may be output through the electronic device 102 (eg, a speaker or headphones).

The sensor module 176 detects an operating state (eg, power or temperature) of the electronic device 101 or an external environmental state (eg, a user state), and generates an electrical signal or data value corresponding to the sensed state. can do. According to an embodiment, the sensor module 176 may include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, It may include a temperature sensor, a humidity sensor, or an illuminance sensor.

The interface 177 may support one or more specified protocols that may be used by the electronic device 101 to directly or wirelessly connect with an external electronic device (eg, the electronic device 102 ). According to an embodiment, the interface 177 may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, a secure digital (SD) card interface, or an audio interface.

The connection terminal 178 may include a connector through which the electronic device 101 can be physically connected to an external electronic device (eg, the electronic device 102 ). According to an embodiment, the connection terminal 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (eg, a headphone connector).

The haptic module 179 may convert an electrical signal into a mechanical stimulus (eg, vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic sense. According to an embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

The camera module 180 may capture still images and moving images. According to an embodiment, the camera module 180 may include one or more lenses, image sensors, image signal processors, or flashes.

The power management module 188 may manage power supplied to the electronic device 101 . According to an embodiment, the power management module 188 may be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

The battery 189 may supply power to at least one component of the electronic device 101 . According to one embodiment, battery 189 may include, for example, a non-rechargeable primary cell, a rechargeable secondary cell, or a fuel cell.

The communication module 190 is a direct (eg, wired) communication channel or a wireless communication channel between the electronic device 101 and an external electronic device (eg, the electronic device 102, the electronic device 104, or the server 108). It can support establishment and communication performance through the established communication channel. The communication module 190 may include one or more communication processors that operate independently of the processor 120 (eg, an application processor) and support direct (eg, wired) communication or wireless communication. According to one embodiment, the communication module 190 is a wireless communication module 192 (eg, a cellular communication module, a short-range communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (eg, : It may include a LAN (local area network) communication module, or a power line communication module). A corresponding communication module among these communication modules is a first network 198 (eg, a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network 199 (eg, legacy It may communicate with the external electronic device 104 through a cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (eg, a telecommunication network such as a LAN or a wide area network (WAN)). These various types of communication modules may be integrated into one component (eg, a single chip) or may be implemented as a plurality of components (eg, multiple chips) separate from each other. The wireless communication module 192 uses the subscriber information (eg, International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module 196 within a communication network such as the first network 198 or the second network 199 . The electronic device 101 may be identified or authenticated.

The wireless communication module 192 may support a 5G network after a 4G network and a next-generation communication technology, for example, a new radio access technology (NR). NR access technology includes high-speed transmission of high-capacity data (eMBB, enhanced mobile broadband), minimization of terminal power and access to multiple terminals (mMTC (massive machine type communications), or high reliability and low latency (URLLC, ultra-reliable and low-latency) communications) The wireless communication module 192 may support a high frequency band (eg, mmWave band) in order to achieve a high data rate, for example. The wireless communication module 192 may support Various techniques for securing performance, for example, beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), It may support technologies such as an array antenna, analog beam-forming, or a large scale antenna. The wireless communication module 192 includes the electronic device 101 and the external electronic device ( For example, it may support various requirements stipulated in the electronic device 104) or the network system (eg, the second network 199.) According to an embodiment, the wireless communication module 192 is configured to implement a Peak for eMBB realization. data rate (e.g. 20 Gbps or more), loss coverage (e.g. 164 dB or less) for realization of mMTC, or U-plane latency (e.g., downlink (DL) and uplink (UL) of 0.5 ms or less for realization of URLLC); or round trip 1ms or less).

The antenna module 197 may transmit or receive a signal or power to the outside (eg, an external electronic device). According to an embodiment, the antenna module 197 may include an antenna including a conductor formed on a substrate (eg, a PCB) or a radiator formed of a conductive pattern. According to an embodiment, the antenna module 197 may include a plurality of antennas (eg, an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network such as the first network 198 or the second network 199 is connected from the plurality of antennas by, for example, the communication module 190 . can be chosen. A signal or power may be transmitted or received between the communication module 190 and an external electronic device through the selected at least one antenna. According to some embodiments, other components (eg, a radio frequency integrated circuit (RFIC)) other than the radiator may be additionally formed as a part of the antenna module 197 .

According to various embodiments, the antenna module 197 may form a mmWave antenna module. According to one embodiment, the mmWave antenna module comprises a printed circuit board, an RFIC disposed on or adjacent to a first side (eg, bottom side) of the printed circuit board and capable of supporting a designated high frequency band (eg, mmWave band); and a plurality of antennas (eg, an array antenna) disposed on or adjacent to a second side (eg, top or side) of the printed circuit board and capable of transmitting or receiving signals of the designated high frequency band. can do.

At least some of the components are connected to each other through a communication method between peripheral devices (eg, a bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI)) and a signal ( eg commands or data) can be exchanged with each other.

According to an embodiment, the command or data may be transmitted or received between the electronic device 101 and the external electronic device 104 through the server 108 connected to the second network 199 . Each of the external electronic devices 102 or 104 may be the same as or different from the electronic device 101 . According to an embodiment, all or a part of operations executed in the electronic device 101 may be executed in one or more external electronic devices 102 , 104 , or 108 . For example, when the electronic device 101 is to perform a function or service automatically or in response to a request from a user or other device, the electronic device 101 may perform the function or service itself instead of executing the function or service itself. Alternatively or additionally, one or more external electronic devices may be requested to perform at least a part of the function or the service. One or more external electronic devices that have received the request may execute at least a part of the requested function or service, or an additional function or service related to the request, and transmit a result of the execution to the electronic device 101 . The electronic device 101 may process the result as it is or additionally and provide it as at least a part of a response to the request. For this, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device 101 may provide an ultra-low latency service using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device 104 may include an Internet of things (IoT) device. Server 108 may be an intelligent server using machine learning and/or neural networks. According to an embodiment, the external electronic device 104 or the server 108 may be included in the second network 199 . The electronic device 101 may be applied to an intelligent service (eg, smart home, smart city, smart car, or health care) based on 5G communication technology and IoT-related technology.

According to various embodiments of the present document, one or more instructions stored in a storage medium (eg, internal memory 136 or external memory 138) readable by a machine (eg, electronic device 101) may be implemented as software (eg, the program 140) including For example, the processor (eg, the processor 120 ) of the device (eg, the electronic device 101 ) may call at least one command among one or more commands stored from a storage medium and execute it. This makes it possible for the device to be operated to perform at least one function according to the called at least one command. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' only means that the storage medium is a tangible device and does not contain a signal (eg, electromagnetic wave), and this term refers to the case where data is semi-permanently stored in the storage medium and It does not distinguish between temporary storage cases.

According to one embodiment, the method according to various embodiments disclosed in this document may be included in a computer program product (computer program product) and provided. Computer program products may be traded between sellers and buyers as commodities. The computer program product is distributed in the form of a machine-readable storage medium (eg compact disc read only memory (CD-ROM)), or via an application store (eg Play Store ^TM ) or on two user devices ( It can be distributed online (eg download or upload), directly between smartphones (eg smartphones). In the case of online distribution, at least a part of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium such as a memory of a server of a manufacturer, a server of an application store, or a relay server.

According to various embodiments, each component (eg, module or program) of the above-described components may include a singular or a plurality of entities, and some of the plurality of entities may be separately disposed in other components. there is. According to various embodiments, one or more components or operations among the above-described corresponding components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (eg, a module or a program) may be integrated into one component. In this case, the integrated component may perform one or more functions of each component of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. . According to various embodiments, operations performed by a module, program, or other component are executed sequentially, in parallel, repeatedly, or heuristically, or one or more of the operations are executed in a different order. , may be omitted, or one or more other operations may be added.

2A to 2C are views illustrating a light source and a display included in an electronic device according to various embodiments of the present disclosure;

2A is a diagram illustrating an arrangement structure of a light source and a display included in an electronic device according to various embodiments of the present disclosure;

Referring to FIG. 2A , an electronic device (eg, the electronic device 101 of FIG. 1 ) according to various embodiments of the present disclosure includes a light source 260 under the display module 160 (eg, the display module 160 of FIG. 1 ). and a fingerprint sensor 270 . The light source 260 may generate light through a diode having nano-pulse characteristics. The light source 260 may include a photodiode (or light pulsed diode) or a pixel diode. A photodiode may be used for the light source 260 disposed separately from the display module 160 . The light source 260 may output light of a wavelength band including a visible ray region to an infrared region.

The fingerprint sensor 270 may be classified into an optical method, an ultrasonic method, or a capacitive method. The optical method recognizes and classifies the difference in the amount of light reflection on the ridge and the bone surface (yin, zero), the ultrasonic method electronically recognizes the difference in the amount of ultrasonic reflection on the ridge and the bone surface and distinguishes it, and the capacitive method recognizes the difference in the amount of light reflection on the ridge and the bone surface. It can be distinguished by recognizing the difference in capacitance between the bone and bone. Hereinafter, a case in which the fingerprint sensor 270 is implemented using an ultrasonic method may be described as an example. However, the present invention is not limited by the description. The fingerprint sensor 270 is made of piezoelectric elements, and receives a transmitter that generates an ultrasonic signal for fingerprint recognition and a reflected signal that is reflected by the object 201 (eg, a user's finger). It may include a receiver that The receiver of the fingerprint sensor 270 may receive (or acquire) a photoacoustic signal generated from the object 201 by the light output from the light source 260 .

A filter layer 280 may be disposed between the display module 160 and the fingerprint sensor 270 . The filter layer 280 may block some of the reflected light from which the light output from the light source 260 is reflected by the object 201 . The filter layer 280 may be made of a material capable of ultrasonic propagation. The light (or wavelength) blocked by the filter layer 280 may be determined based on a wavelength band of the light output from the light source 260 . When the wavelength band of the light is different, the wavelength band of the light to be blocked may be different.

The light source 260 may be disposed on the rear surface (eg, downward direction) of the display module 160 in a direction to irradiate light to the object 201 . The light source 260 may be disposed on the same layer as the fingerprint sensor 270 . One or more light sources 260 may be disposed to surround the fingerprint sensor 270 . For example, the light source 260 surrounds the fingerprint sensor 270 and includes a first photodiode (or light pulsed diode) 261 , a second photodiode 262 , a third photodiode 263 and a fourth photodiode 263 . A diode 264 may be included. The figure shows an example in which the light source 260 is disposed so that the fingerprint sensor 270 can output light in four different directions considering that it is a rectangle. When the shape of the fingerprint sensor 270 is not a rectangle, it may be arranged differently from the drawing. The drawings and description are only examples for helping understanding of the present invention, and the present invention is not limited by the description.

2B is a diagram illustrating a block diagram of a light source and a display included in an electronic device according to various embodiments of the present disclosure;

Referring to FIG. 2B , the display module 160 may include a display driver IC 210 (display driver IC; DDI), a touch circuit 220 , or a display 230 (or a display panel). The display driver IC 210 may include image data from the processor 120 (eg, the main processor 121 (eg, an application processor) or the auxiliary processor 123 operated independently of the function of the main processor 121), or the Image information including an image control signal corresponding to a command for controlling image data may be received. The display driver IC 210 may control the display 230 . At least some pixels of the display 230 are driven based on, for example, the voltage value or the current value, so that visual information (eg, text, image, or icon) corresponding to the image data is displayed on the display 230 . can be

The display driver IC 210 may communicate with the touch circuit 220 . The touch circuit 220 may include a touch sensor and a touch sensor IC for controlling the touch sensor. The touch sensor IC controls the touch sensor to, for example, measure a change in a signal (eg, voltage, light amount, resistance, or charge amount) for a specific location of the display 230 to input a touch input to the specific location. Alternatively, a hovering input may be sensed, and information (eg, location, area, pressure, or time) regarding the sensed touch input or hovering input may be provided to the processor 120 . According to an embodiment, at least a part of the touch circuit 220 (eg, a touch sensor IC) is a part of the display driver IC 210 , or the display 230 , or other components disposed outside the display module 160 . It may be included as part of an element (eg, the coprocessor 123 ). The touch circuit 220 may be disposed between pixels of the pixel layer of the display 230 , or above or below the pixel layer.

The fingerprint sensor module 250 (eg, the sensor module 176 of FIG. 1 ) may include a fingerprint IC 251 , a diode controller 253 , a light source 260 , and a fingerprint sensor 270 . The fingerprint sensor module 250 may be driven under the control of the processor 120 . The fingerprint IC 251 may control the fingerprint sensor 270 . The diode controller 253 may control the light output from the light source 260 . The diode controller 253 may include a power controller 255 and a timing controller 257 . The power control unit 255 may adjust the intensity of the light. The timing controller 257 may adjust the pulse width of the light. The light source 260 may output light under the control of the diode controller 253 . The fingerprint sensor 270 may include a transmitter that generates an ultrasonic signal for fingerprint recognition and a receiver that receives a reflected signal from which the generated ultrasonic signal is reflected by the object 201 . The receiver of the fingerprint sensor 270 may receive a photoacoustic signal generated from the object 201 by the light output from the light source 260 .

2C is a diagram illustrating another block diagram of a light source and a display included in an electronic device according to various embodiments of the present disclosure;

Referring to FIG. 2C , the display module 160 may include a display driver IC 210 (DDI), a touch circuit 220 , or a display 230 . Since the block diagram of the display module 160 is the same as that of FIG. 2B , a detailed description thereof may be omitted. The fingerprint sensor module 250 may include a fingerprint IC 251 and a fingerprint sensor 270 . The fingerprint IC 251 may control the fingerprint sensor 270 . The fingerprint sensor 270 may include a transmitter that generates an ultrasonic signal for fingerprint recognition, and a receiver that receives a reflected signal that is reflected by the generated ultrasonic signal by an object (eg, the object 201 in FIG. 2A ). there is.

The light source 260 may be operated independently of the fingerprint sensor module 250 . The diode controller 253 may include a power controller 255 and a timing controller 257 . The power control unit 255 may adjust the intensity of the light. The timing controller 257 may adjust the pulse width of the light. The light source 260 may output light under the control of the diode controller 253 .

3A to 3D are other views illustrating a light source and a display included in an electronic device according to various embodiments of the present disclosure;

3A is a diagram illustrating an arrangement structure of a light source and a display included in an electronic device according to various embodiments of the present disclosure;

Referring to FIG. 3A , an electronic device (eg, the electronic device 101 of FIG. 1 ) according to various embodiments includes a light source 260 inside a display module 160 (eg, the display module 160 of FIG. 1 ). and a fingerprint sensor (eg, the fingerprint sensor 270 of FIGS. 2A to 2C ) under the display module 160 . The light source 260 may generate light through a diode having nano-pulse characteristics. When the light source 260 is disposed inside the display module 160 , a pixel diode may be used as the light source 260 . The light source 260 may output light of a wavelength band including a visible ray region to an infrared region. The fingerprint sensor 270 may include a transmitter that generates an ultrasonic signal for fingerprint recognition and a receiver that receives a reflected signal that is reflected by the generated ultrasonic signal by the object 201 (eg, a user's finger). . The receiver of the fingerprint sensor 270 may receive a photoacoustic signal generated from the object 201 by the light output from the light source 260 .

A filter layer 280 may be disposed between the display module 160 and the fingerprint sensor 270 . The filter layer 280 may block some of the reflected light from which the light output from the light source 260 is reflected by the object 201 . The filter layer 280 may be made of a material capable of ultrasonic propagation. The light (or wavelength) blocked by the filter layer 280 may be determined based on a wavelength band of the light output from the light source 260 . When the wavelength band of the light is different, the wavelength band of the light to be blocked may be different.

3A is different from FIG. 2A in that the light source 260 is disposed inside the display module 160, and the rest may be similar.

3B is a diagram illustrating a light source inside a display module according to various embodiments of the present disclosure;

Referring to FIG. 3B , the display module 160 may include glass 301 , an adhesive layer 303 (optical clearance adhesive, OCA), and a display 230 (eg, the display 230 of FIGS. 2B and 2C ). there is. The display 230 includes a light source 260 , and a filter layer 280 may be disposed between the display 230 and the fingerprint sensor 270 . The light source 260 may be disposed between pixels (eg, R, G, and B) of a pixel layer of the display 230 . The light source 260 may be disposed around or adjacent to pixels of the display 230 . The structural diagram 310 may be a cross-sectional view of the electronic device 101 cut from top to bottom.

The plan view 330 may be a view when the display 230 is viewed from the top down. Referring to the plan view 320 , the light source 260 may include a plurality of pixel diodes (eg, a first pixel diode 261 and a second pixel diode 262 ). The plurality of pixel diodes may be disposed between the plurality of pixels R, G, and B included in the display 230 .

3C is a diagram illustrating a light source inside a display module according to various embodiments of the present disclosure;

Referring to FIG. 3C , the display module 160 may include a glass 301 , an optical clearance adhesive (OCA), and a display 230 . The display 230 includes a light source 260 , and a filter layer 280 may be disposed between the display 230 and the fingerprint sensor 270 . The light source 260 may be formed by being integrated with the display 230 . For example, the light source 260 may be disposed together with pixels (eg, R, G, and B) of the display 230 . The light source 260 together with pixels (eg, R, G, and B) of the display 230 in an outer partial region that does not overlap the region corresponding to the fingerprint sensor 270 (eg, on the fingerprint sensor 270 ). can be placed. The structural diagram 350 may be a cross-sectional view of the electronic device 101 cut from top to bottom.

The plan view 370 may be a view when the display 230 is viewed from the top down. Referring to the plan view 320 , the light source 260 may include a plurality of pixel diodes (eg, a first pixel diode 261 , a second pixel diode 262 , and a third pixel diode). The plurality of pixel diodes may be disposed to surround the outer edges of the plurality of pixels R, G, and B included in the display 230 .

3C is different from FIG. 3B in the arrangement structure or position in which the light source 260 is disposed on the display 230, and the rest may be similar.

3D is a diagram illustrating another block diagram of a light source and a display included in an electronic device according to various embodiments of the present disclosure;

Referring to FIG. 3D , the display module 160 may include a display driver IC 210 , a touch circuit 220 , a display 230 , or a light source 260 . The display driver IC 210 is controlled by the processor 120 and may include a diode controller 253 . The diode controller 253 may include a power controller 255 and a timing controller 257 . The power control unit 255 may adjust the intensity of the light. The timing controller 257 may adjust the pulse width of the light. The light source 260 may output light under the control of the diode controller 253 . The fingerprint sensor module 250 may include a fingerprint IC 251 and a fingerprint sensor 270 . The fingerprint IC 251 may be controlled by the processor 120 and may control the fingerprint sensor 270 . The fingerprint sensor 270 may include a transmitter that generates an ultrasonic signal for fingerprint recognition, and a receiver that receives a reflected signal that is reflected by the generated ultrasonic signal by an object (eg, the object 201 in FIG. 2A ). there is. The receiver of the fingerprint sensor 270 may receive a photoacoustic signal generated from the object 201 by the light output from the light source 260 .

Since the block diagrams of the display module 160 and the fingerprint sensor module 250 are similar to or identical to those of FIGS. 2B and 2C , a detailed description thereof may be omitted.

4 is a diagram illustrating an example of optoacoustic characteristics according to various embodiments of the present disclosure; 4 shows a 1-D light generation model of an absorber (eg, a user's finger) "Efficient photoacoustic conversion in optical nanomaterials and composites." Advanced Optical Materials 6.24 (2018) was referred to.

Referring to FIG. 4 , f(t) is a temporal heating function due to the duration τ _l of a laser pulse, and g(z) is a spatial light absorption function due to a light-absorption coefficient α. τ may mean a delay time (τ=tz/c). z is the distance in the z-axis direction, and c is the speed of sound. The photoacoustic signal 401 (eg, p(t)) may refer to a signal obtained by a convolution sum of f(t) and g(z). The light 410 output from the light source (eg, the light source 260 of FIGS. 2A to 3D ) may be absorbed by an absorber (eg, the object 201 of FIG. 2A ). The object 201 may generate a signal 420 by the light 410 . Assuming that the object 201 consists of continuous slices in small (smile) units, the optoacoustic signal 401 generated by the object 201 is a superimposed signal of sound pressures generated in countless minute slices. can be seen as The pulse width of the photoacoustic signal 401 can be mathematically approximated by τ _l +1/c (speed of sound)*α (optical absorption coefficient).

When the absorption thickness of the object 201 is thinner than the duration τ _l of the laser pulse, an optoacoustic signal having the same pulse width and high amplitude as the laser pulse width may be generated. When the absorption thickness of the object 201 is thicker than τ _l , a photoacoustic signal having a relatively wide pulse width and low amplitude may be generated. Considering the thickness of the fake fingerprint, the absorption thickness is thicker than the pulse width, so in the case of the final pulse width, the τ _l factor can be neglected and it can be simplified to 1/cα. Since the pulse width 403 of the photoacoustic signal is inversely proportional to the speed of sound and the light-absorption coefficient, the frequency component of the photoacoustic signal may vary depending on the object 201 . For example, a frequency component of an optoacoustic signal generated by an actual user's finger and a photoacoustic signal of a fake fingerprint composed of a component (or material) different from that of the body may be different.

The electronic device (eg, the electronic device 101 of FIG. 1 ) according to various embodiments includes a fingerprint sensor (eg, the fingerprint sensor 270 of FIGS. 2A to 3D ) and a light source (eg, the light source of FIGS. 2A to 3D ) 260), a memory including the user's fingerprint characteristic information (eg, the memory 130 of FIG. 1 ), and a processor operatively connected to the fingerprint sensor, the light source, and the memory (eg, the memory 130 of FIG. 1 ) a processor 120), wherein the processor outputs light through the light source, receives an optoacoustic signal generated from an object by the output light through the fingerprint sensor, and receives the received optoacoustic signal It may be set to extract characteristic information from the , and determine whether the fingerprint is authenticated by comparing the extracted characteristic information with the fingerprint characteristic information stored in the memory.

The processor may be configured to perform fingerprint recognition using the fingerprint sensor.

The electronic device may further include a diode controller (eg, the diode controller 253 of FIG. 2B ) for controlling the intensity of the light output from the light source or the pulse width of the light.

The diode control unit may be included in at least one of the processor, the display module (eg, the display module 160 of FIG. 1 ), a display driver IC (eg, the display driver IC 210 of FIG. 2B ) included in the display module, or the fingerprint sensor. can

The electronic device may further include a display module, and the light source may be configured to be disposed on a rear surface of the display module in a direction for irradiating light to the object or disposed inside the display module.

The fingerprint sensor is disposed on the rear surface of the display module, and the electronic device is a filter layer between the fingerprint sensor and the display module that blocks some of the reflected light from which the light output by the light source is reflected by the object (For example, the filter layer 280 of FIG. 2A may be further included.

When the light source is disposed on the rear surface of the display module, the light source may be set to surround the fingerprint sensor on the same layer as the fingerprint sensor.

When the light source is disposed inside the display module, the light source may be set to be disposed adjacent to or around a plurality of pixels included in the display module.

The light source may be set to be disposed between a plurality of pixels included in the display module.

The light source may be set to be disposed on a plurality of pixels included in the display module in an outer partial area that does not overlap an area corresponding to the fingerprint sensor.

The light source may include a photodiode or a pixel diode.

The processor may be configured to determine whether the fingerprint is forged when fingerprint authentication using the fingerprint sensor is successful.

The processor may be configured to perform fingerprint recognition when the fingerprint recognition area and the counterfeit determination area are different from each other, and provide a user interface for guiding the forgery determination area.

The processor acquires a fingerprint image by using the fingerprint sensor, compares the characteristic points of the fingerprint image stored in the memory with the characteristic points of the obtained fingerprint image to authenticate the fingerprint, and the extracted characteristic information is added to the fingerprint characteristic information. It may be set to determine whether the fingerprint corresponds to the forgery.

The processor may be configured to determine fingerprint forgery using the photoacoustic signal generated from the object by outputting light through the light source when the fingerprint image is acquired according to the fingerprint authentication.

5 is a flowchart 500 illustrating a method of operating an electronic device according to various embodiments of the present disclosure.

Referring to FIG. 5 , in operation 501 , the processor (eg, the processor 120 of FIG. 1 ) of the electronic device (eg, the electronic device 101 of FIG. 1 ) according to various embodiments may receive a fingerprint authentication request. there is. The fingerprint authentication request may be requested by a user input or execution of an application. For example, when the electronic device 101 is in the lock mode, the user may request unlocking. When unlocking is requested in the lock mode, the processor 120 may request fingerprint recognition. Alternatively, when user authentication is required during a financial transaction or payment, an executed application (eg, a bank application) may request fingerprint recognition.

In operation 503 , the processor 120 may acquire a fingerprint image using a fingerprint sensor (eg, the fingerprint sensor 270 of FIGS. 2A to 3D ). The fingerprint sensor 280 outputs an ultrasonic signal for fingerprint recognition, and the ultrasonic signal is reflected by an object (eg, the object 201 in FIG. 2A ) (eg, a user's finger) based on a reflected signal that enters the fingerprint image can be obtained. The processor 120 may be logically divided into a general area and a security area. A trust zone (secure area) is an area having high security (or reliability) and may refer to a trusted execution environment (TEE). The normal zone (normal area) is an area having lower security than the security area, and may refer to a rich execution environment (REE). The processor 120 may acquire a fingerprint image using a fingerprint sensor within the security area.

In operation 505, the processor 120 may authenticate the user based on the obtained fingerprint image. The processor 120 may perform user authentication by comparing the feature points (or feature information) of the fingerprint image stored in the memory (eg, the memory 130 of FIG. 1 ) with the feature points of the acquired fingerprint image to authenticate the fingerprint. there is. The electronic device 101 may store the feature points of the fingerprint image in the memory 130 in advance. For example, the electronic device 101 obtains at least one fingerprint image from the user through a fingerprint registration operation, extracts a feature point from the fingerprint image, and stores the acquired feature point of the at least one fingerprint image in the memory 130 . can be saved The feature point (or feature information) of the fingerprint image is information requiring high security, and the electronic device 101 may store it in the security area of the memory 130 . The security area of the memory 130 may be the same as or similar to the security area of the processor 120 .

According to various embodiments, when the user authentication is successful (eg, the feature points of the acquired fingerprint image match with those of the stored fingerprint image), the processor 120 may perform operation 507 . Alternatively, if the user authentication is not successful (eg, the feature points of the acquired fingerprint image do not match the stored fingerprint image), the processor 120 may perform operation 503 to acquire the fingerprint image again. The processor 120 continues to acquire and compare the fingerprint image until user authentication is successful, or when the operation to acquire and compare the fingerprint image exceeds a set number of times (eg, 3 times, 5 times, 10 times). In this case, fingerprint recognition may be stopped for a certain period of time (eg, 10 minutes).

In operation 507, the processor 120 may determine fingerprint forgery based on the optoacoustic characteristic. The processor 120 may output light through a light source (eg, the light source 260 of FIGS. 2A to 3D ) and receive (or obtain) an optoacoustic signal generated from the object 201 by the output light. there is. The processor 120 may perform a Fourier transform (eg, fast furrier transform (FFT)) on the photoacoustic signal to extract a frequency component (or frequency feature) as a feature point (or feature information). The frequency component may include a center frequency or bandwidth of the optoacoustic signal. The processor 120 may determine whether the frequency component is within a preset range stored in the memory 130 . Frequency information (or fingerprint characteristic information) indicating a frequency characteristic (or characteristic) of an actual user's fingerprint may be stored in the security area of the memory 130 . The frequency information includes the characteristics of the photoacoustic signal generated by the actual user's finger by the light output from the light source 260 , and the frequency information may vary according to the wavelength band of the output light. In addition, since there may be a difference in generating a photoacoustic signal for each user, the frequency information may be stored as a range value in consideration of an error range. When the extracted frequency component is within a preset range, the processor 120 determines the user fingerprint, and when the extracted frequency component is outside the preset range, determines the forged fingerprint.

According to various embodiments, operation 507 may be performed regardless of whether the user authentication is successful. For example, operation 507 may be performed simultaneously with operation 503 or operation 505 . Alternatively, when the user authentication is successful and the extracted frequency component is within a preset range, the processor 120 may complete the operation (or function) according to the fingerprint authentication request. For example, the processor 120 may release the lock mode of the electronic device 101 , perform a financial transaction, or proceed with payment.

According to various embodiments, the processor 120 may request an additional operation for identifying a fake fingerprint from the user separately from user authentication (or fingerprint authentication). For example, when the fingerprint recognition area and the forgery determination area are different from each other, the processor 120 may provide a user interface for guiding the forgery determination area after performing fingerprint recognition. The fingerprint recognition area may include an area in which the fingerprint sensor 270 is disposed, and the counterfeit determination area may include an area in which the light source 260 is disposed. The counterfeit determination area may be the same as or smaller than the fingerprint recognition area. The user interface may display a counterfeit determination area in a display area of a display (eg, the display module 160 of FIG. 1 ) to be distinguished from other areas. The user may touch the finger to the counterfeit determination area according to the guidance.

Although it has been described in FIG. 5 that fingerprint forgery is determined after user authentication is first performed, fingerprint forgery may be first determined and user authentication may be performed depending on implementation.

6 is a diagram illustrating an example in which light is absorbed by a user's body according to various embodiments of the present disclosure;

Referring to FIG. 6 , the human body may have various light absorption coefficients according to wavelength bands. The human skin is composed of melanin, hemoglobin, moisture, lipids, etc., and various wavelength bands may be selected according to the light absorption coefficient characteristics of these components. For example, in a specific wavelength band (eg, 500 nm), the light absorption coefficient of melanin and hemoglobin in the human body is high, whereas the light absorption coefficient of water and lipids may be very low. When the skin is irradiated with light, absorption can occur mostly by melanin and hemoglobin. In the case of the palms and soles of the human body, a very small amount of melanin can be distributed. For example, when light of a specific band is irradiated to a specific part (eg, finger) (eg, object 201) of the user through the laser 600, light absorption may occur by the hemoglobin 605 in the blood. . The optoacoustic signal 603 may be generated (or generated) from the blood vessel 601 .

According to various embodiments, the forged fingerprint may have a three-dimensional shape or a printed form of a single material, which may have characteristics different from those of the human body. Light absorption is achieved by the properties of the pigment (color) constituting the material, and thermal expansion may be achieved according to the properties of the material itself. A fake fingerprint can be expected to generate a different type of optoacoustic signal as it has a heat sink different from that of a real finger.

7A to 7C are diagrams illustrating comparison examples of a user's fingerprint and a forged fingerprint according to various embodiments of the present disclosure;

7A is a diagram illustrating the magnitude of photoacoustic signals of a user's fingerprint and a forged fingerprint according to various embodiments of the present disclosure;

Referring to FIG. 7A , the first graph 710 shows a photoacoustic signal generated when light of the same wavelength band is irradiated to different objects (eg, the first object 701 to the eighth object 708 ). For example, the first object 701 is black rubber, the second object 702 is a user's real finger, the third object 703 is apricot silicone, and the third object 703 is apricot silicone. 4 object 704 is white apricot silicone, a fifth object 705 is blue silicone, a sixth object 706 is transparent gelatin, and a seventh object ( 707 may be paper, and the eighth object 708 may be an overhead project (OHP) film The first object 701 has the largest photoacoustic signal, and the second object 702 has the second light It can be seen that the size of the acoustic signal is large, and the magnitude of the photoacoustic signal of the third object 703 to the fifth object 705 is similar, and the sixth object 706 to the eighth object 708 is It can be seen that the photoacoustic signal is not detected, and thus it can be seen that the magnitude of the photoacoustic signal generated according to the object is different.

7B illustrates waveforms of photoacoustic signals of a user's fingerprint and a forged fingerprint according to various embodiments of the present disclosure;

Referring to FIG. 7B , the second graph 730 shows waveforms of the photoacoustic signal of the first object 701 to the fifth object 705 in which the photoacoustic signal is detected. For example, the amplitude of the optoacoustic signal of the first object 701 is the largest, the amplitude of the optoacoustic signal of the second object 702 is the second largest, and the third object 703 to the fifth object 705 have the largest amplitude. ), it can be seen that the amplitudes of the photoacoustic signals are similar. Accordingly, it can be seen that the amplitude of the photoacoustic signal generated according to the object is different.

7C illustrates frequency characteristics of a user's fingerprint and a forged fingerprint according to various embodiments of the present disclosure;

Referring to FIG. 7C , the third graph 750 shows frequency characteristics obtained from the photoacoustic signals of the first to fifth objects 701 to 705 in which the photoacoustic signals are detected. For example, the first object 701 has a large photoacoustic signal at 5 MHz, the second object 702 has a large photoacoustic signal at 2 MHz, and the third objects 703 to fifth objects (705) shows that the magnitude of the photoacoustic signal is large at 1.8 MHz. Accordingly, it can be seen that the frequency characteristics of the photoacoustic signal generated according to the object are different from each other.

8 is a diagram illustrating an example of comparing frequency characteristics of a user's fingerprint and a forged fingerprint according to various embodiments of the present disclosure;

Referring to FIG. 8 , a graph 800 shows frequency characteristics extracted from photoacoustic signals generated when light of the same wavelength band is irradiated to different objects. The first object 801 to the eighth object 808 may represent frequency characteristics of photoacoustic signals obtained from fingers of different users. For example, the first object 801 is the first user's first finger (eg, thumb), the second object 802 is the first user's second finger (eg, index finger), and the third object 803 is is the first finger of the second user, the fourth object 804 is the second finger of the second user, the fifth object 805 is the first finger of the third user, and the sixth object 806 is the third user's finger. The second finger and the seventh object 807 may indicate the first finger of the fourth user, and the eighth object 808 may indicate the second finger of the fourth user. Looking at the first object 801 to the eighth object 808 , the center frequency 810 may have 2.3 to 3 MHz, and the bandwidth 830 may have 4.9 to 6.6 MHz.

The ninth object 811 is apricot-colored silicon, the tenth object 812 is light apricot-colored silicon, the eleventh object 813 is blue silicon, the twelfth object 814 is black rubber, and the thirteenth object 815 is Transparent gelatin, the fourteenth object 816 may be paper, and the fifteenth object 817 may be an OHP film. The center frequency 820 of the ninth object 811 to the eleventh object 813 that is silicon-based may be 1.3 to 1.6 MHz, and the bandwidth 830 may be 1.3 to 2.8 MHz. The center frequency 820 of the twelfth object 814 may be 5 MHz, and the bandwidth 830 may be 8.6 MHz. Since the photoacoustic signal is not detected in the thirteenth object 815 to the fifteenth object 817 , the center frequency and bandwidth cannot be obtained. Accordingly, it can be seen that the frequency characteristics of the photoacoustic signal generated according to the object are different from each other.

According to various embodiments, the electronic device (eg, the electronic device 101 of FIG. 1 ) stores frequency information for each material in a memory (eg, the memory 130 of FIG. 1 ) in order to determine a forged fingerprint, or Frequency information related to a user's fingerprint may be stored.

9 is a flowchart 900 illustrating a method for determining a forged fingerprint of an electronic device according to various embodiments of the present disclosure.

Referring to FIG. 9 , in operation 901 , the processor (eg, the processor 120 of FIG. 1 ) of the electronic device (eg, the electronic device 101 of FIG. 1 ) according to various embodiments receives a light source (eg, FIG. 2A ) The light may be output through the light source 260 of FIG. 3D . For example, the processor 120 may control the light to be output through the diode controller (the diode controller 253 of FIG. 2B ). For example, the diode controller 253 may include a power controller 255 that adjusts the intensity of light and a timing controller 257 that adjusts the pulse width of light. The processor 120 may transmit a command (or instruction) designating the intensity of the light to be output or the pulse width of the light to be output to the diode controller 253 . The diode controller 253 may control the light source 260 under the control of the processor 120 to adjust the intensity or pulse width of light. The light source 260 may output light under the control of the diode controller 253 . The light source 260 may include at least one of a photodiode configured separately from the display (eg, the display module 160 of FIG. 1 ) or a pixel diode configured inside the display module 160 .

According to various embodiments, frequency information (or frequency range) of an actual fingerprint may be stored in a security area of the memory (eg, the memory 130 of FIG. 1 ) of the electronic device 101 . The light intensity or pulse width may be optimized and set according to frequency information of an actual fingerprint. That is, when the output light is irradiated to the user's finger, the intensity or the pulse width of the light so that the frequency characteristic obtained from the photoacoustic signal generated corresponds to the frequency information stored in the memory 130 may be optimized and set. According to various embodiments of the present disclosure, when the electronic device 101 fails to output with the set light intensity according to the state (eg, aging, display contamination), the processor 120 controls the light to be output with an intensity greater than the set light intensity. can do.

In operation 903 , the processor 120 may acquire an optoacoustic signal. The fingerprint sensor of the present invention (eg, the fingerprint sensor 270 of FIGS. 2A to 3D ) includes a transmitter that generates an ultrasonic signal for fingerprint recognition and the generated ultrasonic signal is reflected by an object (eg, a user's finger). It may include a receiver for receiving the incoming reflected signal. The receiver of the fingerprint sensor 270 may receive a photoacoustic signal generated from an object by the light output from the light source 260 . The processor 120 may acquire an optoacoustic signal by activating the receiver of the fingerprint sensor 270 .

In operation 905, the processor 120 may extract a frequency characteristic (or characteristic information) from the optoacoustic signal. The processor 120 may perform a Fourier transform on the photoacoustic signal to extract a frequency component as a feature point (or feature information). The frequency component may include a center frequency or bandwidth of the optoacoustic signal.

In operation 907, the processor 120 may determine whether the extracted frequency feature is within a preset range (or fingerprint feature information). Frequency information (or frequency range) of an actual fingerprint may be stored in the memory 130 of the electronic device 101 . The processor 120 may determine whether the frequency characteristic is within a preset range stored in the memory 130 . The processor 120 may perform operation 909 when the extracted frequency characteristic is within a preset range, and may perform operation 911 when the extracted frequency characteristic is outside the preset range.

When the extracted frequency characteristic is within a preset range, in operation 909, the processor 120 may determine the user fingerprint. According to various embodiments, when fingerprint authentication is successful through the fingerprint sensor 270 and the extracted frequency characteristic is within a preset range, the processor 120 completes an operation (or function) according to the fingerprint authentication request. can do. For example, the processor 120 may release the lock mode of the electronic device 101 , perform a financial transaction, or proceed with payment.

When the extracted frequency characteristic is out of a preset range, in operation 911, the processor 120 may determine that the fingerprint is a forged fingerprint. According to various embodiments, when it is determined that the forged fingerprint is the forged fingerprint, the processor 120 may request the user to re-authenticate the forged fingerprint. For example, the processor 120 may request fingerprint authentication and forged fingerprint authentication when re-authentication of a forged fingerprint. The processor 120 may request the fingerprint authentication again if it is determined that the fingerprint is a forged fingerprint even if it is confirmed as the user's fingerprint during fingerprint authentication, and may request forged fingerprint authentication when the fingerprint authentication is completed (or successful). The processor 120 may provide a user interface for re-authentication of the forged fingerprint. Alternatively, the processor 120 may request forged fingerprint authentication without re-requesting fingerprint authentication when it is determined that the fingerprint is the user's fingerprint during fingerprint authentication, or when it is determined that the fingerprint is forged.

An electronic device (eg, in FIG. 1 ) including a light source (eg, the light source 260 of FIGS. 2A to 3D ) and a fingerprint sensor (eg, the fingerprint sensor 270 of FIGS. 2A to 3D ) according to various embodiments The method of operating the electronic device 101 includes an operation of outputting light through the light source, an operation of receiving a photoacoustic signal generated from an object by the light output through the fingerprint sensor, and an operation of receiving the photoacoustic signal from the received photoacoustic signal. It may include an operation of extracting characteristic information, and an operation of determining whether a fingerprint is authenticated by comparing the extracted characteristic information with fingerprint characteristic information stored in the memory.

The determining may include determining whether the fingerprint is forged when fingerprint authentication using the fingerprint sensor is successful.

The method may further include performing fingerprint recognition and providing a user interface guiding the forgery determination area when the fingerprint recognition area and the forgery determination area are different from each other.

The method includes an operation of acquiring a fingerprint image using the fingerprint sensor, an operation of authenticating a fingerprint by comparing characteristic points of the fingerprint image stored in the memory with characteristic points of the obtained fingerprint image, and the extracted characteristic information is the fingerprint characteristic The method may further include an operation of determining whether the fingerprint is forgery by determining whether it corresponds to the information.

The determining may include, when acquiring the fingerprint image according to the fingerprint authentication, outputting light through the light source and determining forgery of the fingerprint using a photoacoustic signal generated from the object.

Various embodiments of the present invention disclosed in the present specification and drawings are merely provided for specific examples in order to easily explain the technical contents of the present invention and help the understanding of the present invention, and are not intended to limit the scope of the present invention. Therefore, the scope of the present invention should be construed as including all changes or modifications derived based on the technical idea of the present invention in addition to the embodiments disclosed herein are included in the scope of the present invention.

101: electronic device
120: processor
130: memory
160: display module
260: light source
270: fingerprint sensor

Claims (20)

In an electronic device,
fingerprint sensor;
light source;
a memory including user's fingerprint characteristic information; and
a processor operatively coupled to the fingerprint sensor, the light source, and the memory, the processor comprising:
outputting light through the light source,
Receives a photoacoustic signal generated from an object by the light output through the fingerprint sensor,
extracting feature information from the received photoacoustic signal,
An electronic device configured to determine whether a fingerprint is authenticated by comparing the extracted characteristic information with the fingerprint characteristic information stored in the memory.
The method of claim 1, wherein the processor comprises:
An electronic device configured to perform fingerprint recognition using the fingerprint sensor.
The method of claim 1,
The electronic device further comprising a diode controller for controlling the intensity of the light output from the light source or the pulse width of the light.
4. The method of claim 3,
The diode control unit,
An electronic device included in at least one of the processor, the display driver IC included in the display module, and the fingerprint sensor.
The method of claim 1,
Further comprising a display module,
The light source is
An electronic device configured to be disposed on a rear surface of the display module in a direction for irradiating light to the object or disposed inside the display module.
6. The method of claim 5,
The fingerprint sensor is disposed on the rear surface of the display module,
The electronic device further comprising a filter layer between the fingerprint sensor and the display module to block some of the reflected light from the light output by the light source is reflected by the object.
6. The method of claim 5,
When the light source is disposed on the rear surface of the display module, the light source is configured to surround the fingerprint sensor on the same layer as the fingerprint sensor.
6. The method of claim 5,
When the light source is disposed inside the display module, the light source is
An electronic device configured to be disposed adjacent to or around a plurality of pixels included in the display module.
According to claim 8, wherein the light source,
An electronic device configured to be disposed between a plurality of pixels included in the display module.
According to claim 8, wherein the light source,
An electronic device configured to be disposed on a plurality of pixels included in the display module in an outer partial area that does not overlap an area corresponding to the fingerprint sensor.
According to claim 1, wherein the light source,
An electronic device comprising a photodiode or pixel diode.
The method of claim 1, wherein the processor comprises:
An electronic device configured to determine whether the fingerprint is forged when fingerprint authentication using the fingerprint sensor is successful.
The method of claim 1, wherein the processor comprises:
An electronic device configured to perform fingerprint recognition and provide a user interface for guiding the forgery determination area when the fingerprint recognition area and the forgery determination area are different from each other.
The method of claim 1, wherein the processor comprises:
Obtaining a fingerprint image using the fingerprint sensor,
Authenticate the fingerprint by comparing the feature points of the fingerprint image stored in the memory with the feature points of the acquired fingerprint image,
An electronic device configured to determine whether the extracted characteristic information corresponds to the fingerprint characteristic information to determine forgery of a fingerprint.
15. The method of claim 14, wherein the processor,
The electronic device is configured to output light through the light source to determine fingerprint forgery using the photoacoustic signal generated from the object when the fingerprint image is acquired according to the fingerprint authentication.
A method of operating an electronic device including a light source and a fingerprint sensor, the method comprising:
outputting light through the light source;
receiving a photoacoustic signal generated from an object by the light output through the fingerprint sensor;
extracting feature information from the received photoacoustic signal; and
and determining whether the fingerprint is authenticated by comparing the extracted characteristic information with the fingerprint characteristic information stored in the memory.
The method of claim 16, wherein the determining comprises:
and determining whether the fingerprint is forged when fingerprint authentication using the fingerprint sensor is successful.
17. The method of claim 16,
When the fingerprint recognition area and the counterfeit determination area are different from each other, performing fingerprint recognition and providing a user interface for guiding the forgery determination area.
17. The method of claim 16,
acquiring a fingerprint image using the fingerprint sensor;
authenticating the fingerprint by comparing the feature points of the fingerprint image stored in the memory with the feature points of the acquired fingerprint image; and
and determining whether the extracted characteristic information corresponds to the fingerprint characteristic information to determine forgery of the fingerprint.
The method of claim 19, wherein the determining comprises:
and outputting light through the light source when acquiring the fingerprint image according to the fingerprint authentication and determining forgery of a fingerprint using a photoacoustic signal generated from the object.

Images