KR20240025422A - Method for detecting performance of fingerprint sensor and electronic device supporting the same - Google Patents

Abstract

Embodiments of the present disclosure provide a method for detecting a change in performance of a fingerprint sensor due to a protective material and an electronic device supporting the same. An electronic device according to an embodiment of the present disclosure may include a display, a fingerprint sensor formed below the display, memory, and a processor. According to one embodiment, the processor may be operative to detect designated triggers related to testing the performance of a fingerprint sensor. According to one embodiment, the processor may control the fingerprint sensor to acquire a designated image based on detecting a designated trigger. According to one embodiment, the processor may extract reference data preset in memory. According to one embodiment, the processor may operate to determine a state related to a protective material attachable to the display based on a designated image and reference data acquired through a fingerprint sensor. According to one embodiment, the processor may operate to determine whether the performance of the fingerprint sensor changes based on a state related to the protective material. According to one embodiment, the processor may operate to perform a designated operation based on whether the performance of the fingerprint sensor changes. Various embodiments may be possible.

Description

Method for detecting performance of fingerprint sensor and electronic device supporting same {METHOD FOR DETECTING PERFORMANCE OF FINGERPRINT SENSOR AND ELECTRONIC DEVICE SUPPORTING THE SAME}

Embodiments of the present disclosure provide a method for detecting changes in performance of a fingerprint sensor and an electronic device supporting the same.

With the development of digital technology, various types of electronic devices such as smart phones and/or wearable devices are widely used. In order to support and increase the functionality of electronic devices, the hardware and/or software portions of electronic devices are continuously being developed.

Electronic devices provide various security functions to protect user's personal information or privacy, and user-specific authentication methods are being developed for each electronic device. Among these security features may include fingerprint authentication methods. For example, the fingerprint authentication method is easy to implement in electronic devices with a small form factor and has the advantage of relatively high accuracy and security, so it is widely applied as an authentication method for electronic devices.

An electronic device can recognize a fingerprint using a fingerprint sensor (or fingerprint reader). Fingerprint sensors included in electronic devices may be implemented as small fingerprint sensors that may be smaller than the user's fingerprint to reduce the effective surface area. Recently, in response to the expansion of the display area, the fingerprint sensor can be implemented as an optical fingerprint sensor (e.g., an optical in-display fingerprint sensor), and the fingerprint sensor can be placed under the display. In the case of an optical fingerprint sensor, it may have a UPS (under panel sensor) (or UDS, under display sensor) structure. According to one embodiment, an optical fingerprint sensor may represent a sensor that photographs (or scans) a user's fingerprint using a light source generated from a display.

For example, a fingerprint sensor (e.g., an optical fingerprint sensor) is disposed in the interior space of the electronic device, below the display, and captures (or scans) an external subject (e.g., a fingerprint) through at least a portion of the display. ) and obtain related images.

Meanwhile, the fingerprint sensor can undergo calibration during the manufacturing stage of the electronic device. For example, because the fingerprint sensor is placed under the display, it is necessary to use the display's laminated structure (display panel layer, touch screen panel (TSP) layer, polarizer layer, adhesive layer, or glass ( It can be greatly affected by the glass layer (e.g., the influence of structural characteristics), and correction work may be performed during the manufacturing stage of the electronic device to maintain optimal performance depending on the display lamination structure. For example, in a fingerprint sensor, the light generated from the light source of the display is reflected from the user's fingerprint and passes through several layers of the display before entering the fingerprint sensor, causing a lot of loss and distortion to obtain a fingerprint image. Additionally, the unique performance of each fingerprint sensor and display may vary depending on the electronic device. Accordingly, in order to minimize the above-mentioned error and fix and extract the specified standard value of the fingerprint sensor, correction work on the fingerprint sensor may be performed during the manufacturing stage of the electronic device.

Meanwhile, users of electronic devices can attach a protective material such as a protective film or protective glass to the display to protect the display. In this case, due to the attachment of a protective film or protective glass, a laminated structure with new characteristics may be added, and a change in the recognition performance of the fingerprint sensor (e.g., performance degradation) may occur. For example, a fingerprint sensor can achieve optimal performance only when the user uses it under the same conditions as the calibration process. However, if many users attach protective materials to protect the display, changes in performance may occur due to the addition of a laminated structure with new characteristics.

In one embodiment of the present disclosure, a method and support for detecting a change in performance (e.g., deterioration) of the fingerprint sensor due to a protective material (e.g., protective film or protective glass) of the display using the fingerprint sensor of an electronic device is provided. An electronic device that can be provided can be provided.

In one embodiment of the present disclosure, a method for detecting a protective material attached to a display of an electronic device and providing a guide for changes in the performance of a fingerprint sensor based on the characteristics of the detected protective material, and an electronic device supporting the same can be provided.

The technical problem to be achieved in this document is not limited to the technical problem mentioned above, and other technical problems not mentioned can be clearly understood by those skilled in the art from the description below. There will be.

An electronic device according to an embodiment may include a display, a fingerprint sensor formed below the display, a memory, and a processor operatively connected to the display, the fingerprint sensor, and the memory. The processor according to one embodiment may operate to detect a designated trigger related to a performance test of the fingerprint sensor. The processor according to one embodiment may control the fingerprint sensor to acquire a designated image based on detecting the designated trigger. The processor according to one embodiment may operate to extract reference data preset in the memory. The processor according to an embodiment may operate to determine a state related to a protective material attachable to the display based on a designated image acquired through the fingerprint sensor and the reference data. The processor according to one embodiment may operate to determine whether the performance of the fingerprint sensor changes based on a state related to the protective material. The processor according to one embodiment may operate to perform a designated operation based on whether the performance of the fingerprint sensor changes.

A method of operating an electronic device according to an embodiment may include performing an operation for detecting a designated trigger related to a performance test of a fingerprint sensor formed below the display. The method of operation may include performing an operation of controlling the fingerprint sensor to acquire a designated image based on detecting the designated trigger. The operating method may include performing an operation of extracting reference data preset in the memory of the electronic device. The operating method may include performing an operation to determine a state related to a protective material attachable to the display based on a designated image acquired through the fingerprint sensor and the reference data. The operating method may include performing an operation to determine whether performance of the fingerprint sensor has changed based on a state related to the protective material. The operation method may include performing a designated operation based on whether the performance of the fingerprint sensor changes.

In order to solve the above problems, an embodiment of the present disclosure may include a computer-readable recording medium on which a program for executing the method on a processor is recorded.

According to one embodiment, a non-transitory computer readable storage medium (or computer program product) storing one or more programs is described. According to one embodiment, one or more programs, when executed by a processor of an electronic device, perform an operation of detecting a designated trigger related to a performance test of a fingerprint sensor formed under the display, and based on detecting the designated trigger, a designated trigger. An operation of controlling the fingerprint sensor to acquire an image, an operation of extracting reference data preset in the memory of the electronic device, based on the reference data and a designated image acquired through the fingerprint sensor, a device that can be attached to the display An operation of determining a state related to a protective material, an operation of determining whether the performance of the fingerprint sensor changes based on the state related to the protective material, and an operation of performing a designated operation based on whether the performance of the fingerprint sensor changes. It may contain commands to be executed.

Additional scope of applicability of the present disclosure will become apparent from the detailed description that follows. However, since various changes and modifications within the spirit and scope of the present disclosure may be clearly understood by those skilled in the art, the detailed description and specific embodiments, such as preferred embodiments of the present disclosure, should be understood as being given by way of example only.

According to an electronic device, its operating method, and a recording medium according to an embodiment of the present disclosure, when a protective material (e.g., a protective film or protective glass) is attached to a display of the electronic device, this is detected, and a fingerprint sensor according to the protective material is attached. It is possible to guide users about the recognition performance of .

According to an electronic device, its operating method, and a recording medium according to an embodiment of the present disclosure, a fingerprint sensor (e.g., an optical in-display fingerprint sensor) is used to determine whether a protective material is attached to a display in a user's intentional or unintentional situation. It can be detected.

According to an electronic device, its operating method, and a recording medium according to an embodiment of the present disclosure, the electronic device can distinguish protective materials that are not suitable for the recognition performance of a fingerprint sensor and provide guidance on this to the user.

According to an electronic device, its operating method, and a recording medium according to an embodiment of the present disclosure, a guide related to a protective material allows a user to easily recognize changes in fingerprint recognition performance of a fingerprint sensor and helps improve performance when performance deteriorates. You can.

The effects that can be obtained from the present disclosure are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the description below. will be.

In relation to the drawing description, the same or similar reference numerals may be used for the same or similar components.
1 is a block diagram of an electronic device in a network environment according to various embodiments.
FIG. 2 is a reference diagram schematically showing an example of an electronic device according to an embodiment of the present disclosure.
FIG. 3 is a reference diagram schematically illustrating an example of a display and a fingerprint sensor being arranged in an electronic device according to an embodiment of the present disclosure.
FIG. 4 is a reference diagram illustrating an example of a polarized light transmission direction of a display in an electronic device according to an embodiment of the present disclosure.
FIG. 5 is a reference diagram illustrating an example of a pattern for measuring signal characteristics in an electronic device according to an embodiment of the present disclosure.
FIG. 6 is a diagram schematically showing the configuration of an electronic device according to an embodiment of the present disclosure.
FIG. 7 is a flowchart illustrating a method of operating an electronic device according to an embodiment of the present disclosure.
FIG. 8 is a flowchart illustrating a method of operating an electronic device according to an embodiment of the present disclosure.
FIG. 9 is a flowchart illustrating a method of operating an electronic device according to an embodiment of the present disclosure.
FIG. 10 is a flowchart illustrating a method of operating an electronic device according to an embodiment of the present disclosure.
FIGS. 11A and 11B are reference views for explaining an example of detecting a protective material using a background image in an electronic device according to an embodiment of the present disclosure.
FIG. 12 is a reference diagram illustrating an example of signal characteristics of a fingerprint sensor depending on the presence or absence of a protective material in an electronic device according to an embodiment of the present disclosure.
FIG. 13 is a reference diagram illustrating an example of an image processing result of an image acquired through a fingerprint sensor in an electronic device according to an embodiment of the present disclosure.
FIG. 14 is a reference diagram illustrating an example of a change in the characteristics of a protective material in an electronic device according to an embodiment of the present disclosure.
FIG. 15 is a reference diagram illustrating an example of a change in the characteristics of a protective material in an electronic device according to an embodiment of the present disclosure.
FIG. 16 is a reference diagram illustrating an example of distinguishing characteristics of a protective material in an electronic device according to an embodiment of the present disclosure.
FIG. 17 is a flowchart illustrating a method of operating an electronic device according to an embodiment of the present disclosure.
FIG. 18 is a flowchart illustrating a method of operating an electronic device according to an embodiment of the present disclosure.
FIG. 19 is a reference diagram for explaining an example of detecting a protective material using a fingerprint image in an electronic device according to an embodiment of the present disclosure.
FIGS. 20A and 20B are reference views showing examples of image patterns extracted based on a reference map and a fingerprint image in an electronic device according to an embodiment of the present disclosure.
FIG. 21 is a reference diagram illustrating an example of comparing an image pattern based on a fingerprint image with a reference map in an electronic device according to an embodiment of the present disclosure.

FIG. 1 is a block diagram of an electronic device 101 in a network environment 100 according to various embodiments.

Referring to FIG. 1, in the network environment 100, the electronic device 101 communicates with the electronic device 102 through a first network 198 (e.g., a short-range wireless communication network) or a second network 199. It is possible to communicate with at least one of the electronic device 104 or the server 108 through (e.g., a long-distance wireless communication network). According to one embodiment, the electronic device 101 may communicate with the electronic device 104 through the server 108. According to one embodiment, the electronic device 101 includes a processor 120, a memory 130, an input module 150, an audio output module 155, a display module 160, an audio module 170, and a sensor module ( 176), interface 177, connection terminal 178, haptic module 179, camera module 180, power management module 188, battery 189, communication module 190, subscriber identification module 196 , or may include an antenna module 197. In some embodiments, at least one of these components (eg, the connection terminal 178) may be omitted or one or more other components may be added to the electronic device 101. In some embodiments, some of these components (e.g., sensor module 176, camera module 180, or antenna module 197) are integrated into one component (e.g., display module 160). It can be.

The processor 120, for example, executes software (e.g., program 140) to operate at least one other component (e.g., hardware or software component) of the electronic device 101 connected to the processor 120. It can be controlled and various data processing or calculations can be performed. According to one embodiment, as at least part of data processing or computation, the processor 120 stores instructions or data received from another component (e.g., sensor module 176 or communication module 190) in volatile memory 132. The commands or data stored in the volatile memory 132 can be processed, and the resulting data can be stored in the non-volatile memory 134. According to one embodiment, the processor 120 is a main processor 121 (e.g., a central processing unit (CPU) or an application processor (AP)) or an auxiliary processor (e.g., a central processing unit (CPU) or an application processor (AP)) that can be operated independently or together. 123) (e.g., graphic processing unit (GPU), neural processing unit (NPU), image signal processor (ISP), sensor hub processor, or communication processor (CP, communication processor)) may be included. For example, if the electronic device 101 includes a main processor 121 and a secondary processor 123, the secondary processor 123 may be set to use lower power than the main processor 121 or be specialized for a designated function. You can. The auxiliary processor 123 may be implemented separately from the main processor 121 or as part of it.

The auxiliary processor 123 may, for example, replace the main processor 121 while the main processor 121 is in an inactive (e.g., sleep) state, or when the main processor 121 While in an active (e.g., application execution) state, at least one of the components of the electronic device 101 (e.g., the display module 160, the sensor module 176, or At least some of the functions or states related to the communication module 190 can be controlled. According to one embodiment, co-processor 123 (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module 180 or communication module 190). there is. According to one embodiment, the auxiliary processor 123 (eg, neural network processing device) may include a hardware structure specialized for processing artificial intelligence models. Artificial intelligence models can be created through machine learning. For example, such learning may be performed in the electronic device 101 itself on which the artificial intelligence model is performed, or may be performed through a separate server (e.g., server 108). Learning algorithms may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but It is not limited. An artificial intelligence model may include multiple artificial neural network layers. Artificial neural networks include deep neural network (DNN), convolutional neural network (CNN), recurrent neural network (RNN), restricted boltzmann machine (RBM), belief deep network (DBN), bidirectional recurrent deep neural network (BRDNN), It may be one of deep Q-networks or a combination of two or more of the above, but is not limited to the examples described above. In addition to hardware structures, artificial intelligence models may additionally or alternatively include software structures.

The memory 130 may store various data used by at least one component (eg, the processor 120 or the sensor module 176) of the electronic device 101. Data may include, for example, input data or output data for software (e.g., program 140) and instructions related thereto. Memory 130 may include volatile memory 132 or non-volatile memory 134.

The program 140 may be stored as software in the memory 130 and may include, for example, an operating system (OS) 142, middleware 144, or applications 146. there is.

The input module 150 may receive commands or data to be used in a component of the electronic device 101 (e.g., the processor 120) from outside the electronic device 101 (e.g., a user). The input module 150 may include, for example, a microphone, mouse, keyboard, keys (eg, buttons), or digital pen (eg, stylus pen).

The sound output module 155 may output sound signals to the outside of the electronic device 101. The sound output module 155 may include, for example, a speaker or a receiver. Speakers can be used for general purposes such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part of it.

The display module 160 can visually provide information to the outside of the electronic device 101 (eg, a user). The display module 160 may include, for example, a display, a hologram device, or a projector, and a control circuit for controlling the device. According to one embodiment, the display module 160 may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of force generated by the touch.

The audio module 170 can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module 170 acquires sound through the input module 150, the sound output module 155, or an external electronic device (e.g., directly or wirelessly connected to the electronic device 101). Sound may be output through the electronic device 102 (e.g., speaker or headphone).

The sensor module 176 detects the operating state (e.g., power or temperature) of the electronic device 101 or the external environmental state (e.g., user state) and generates an electrical signal or data value corresponding to the detected state. can do. According to one embodiment, the sensor module 176 includes, for example, a gesture sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, It may include a temperature sensor, humidity sensor, or light sensor.

The interface 177 may support one or more designated protocols that can be used to connect the electronic device 101 directly or wirelessly with an external electronic device (eg, the electronic device 102). According to one embodiment, the interface 177 may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, a secure digital (SD) card interface, or an audio interface.

The connection terminal 178 may include a connector through which the electronic device 101 can be physically connected to an external electronic device (eg, the electronic device 102). According to one embodiment, the connection terminal 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (eg, a headphone connector).

The haptic module 179 can convert electrical signals into mechanical stimulation (e.g., vibration or movement) or electrical stimulation that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

The camera module 180 can capture still images and moving images. According to one embodiment, the camera module 180 may include one or more lenses, image sensors, image signal processors, or flashes.

The power management module 188 can manage power supplied to the electronic device 101. According to one embodiment, the power management module 188 may be implemented as at least a part of, for example, a power management integrated circuit (PMIC).

The battery 189 may supply power to at least one component of the electronic device 101. According to one embodiment, the battery 189 may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

Communication module 190 is configured to provide a direct (e.g., wired) communication channel or wireless communication channel between electronic device 101 and an external electronic device (e.g., electronic device 102, electronic device 104, or server 108). It can support establishment and communication through established communication channels. Communication module 190 operates independently of processor 120 (e.g., an application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module 190 may be a wireless communication module 192 (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (e.g., : LAN (local area network) communication module, or power line communication module) may be included. Among these communication modules, the corresponding communication module is a first network 198 (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network 199 (e.g., legacy It may communicate with an external electronic device 104 through a cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a telecommunication network such as a LAN or wide area network (WAN)). These various types of communication modules may be integrated into one component (e.g., a single chip) or may be implemented as a plurality of separate components (e.g., multiple chips). The wireless communication module 192 uses subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module 196 to communicate within a communication network such as the first network 198 or the second network 199. The electronic device 101 can be confirmed or authenticated.

The wireless communication module 192 may support 5G networks after 4G networks and next-generation communication technologies, for example, NR access technology (new radio access technology). NR access technologies include high-speed transmission of high-capacity data (eMBB, enhanced mobile broadband), minimization of terminal power and access to multiple terminals (mMTC, massive machine type communications), or high-reliability and low-latency (URLLC, ultra-reliable and low-latency). communications) can be supported. The wireless communication module 192 may support high frequency bands (eg, mmWave bands), for example, to achieve high data rates. The wireless communication module 192 uses various technologies to secure performance in high frequency bands, for example, beamforming, massive array multiple-input and multiple-output (MIMO), and full-dimensional multiplexing. It can support technologies such as input/output (FD-MIMO, full dimensional MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module 192 may support various requirements specified in the electronic device 101, an external electronic device (e.g., electronic device 104), or a network system (e.g., second network 199). According to one embodiment, the wireless communication module 192 supports Peak data rate (e.g., 20 Gbps or more) for realizing eMBB, loss coverage (e.g., 164 dB or less) for realizing mmTC, or U-plane latency (e.g., 164 dB or less) for realizing URLLC. Example: Downlink (DL) and uplink (UL) each of 0.5 ms or less, or round trip 1 ms or less) can be supported.

The antenna module 197 may transmit or receive signals or power to or from the outside (eg, an external electronic device). According to one embodiment, the antenna module 197 may include an antenna including a radiator made of a conductor or a conductive pattern formed on a substrate (eg, PCB). According to one embodiment, the antenna module 197 may include a plurality of antennas (eg, an array antenna). In this case, at least one antenna suitable for the communication method used in the communication network, such as the first network 198 or the second network 199, is connected to the plurality of antennas by, for example, the communication module 190. can be selected. Signals or power may be transmitted or received between the communication module 190 and an external electronic device through the at least one selected antenna. According to some embodiments, in addition to the radiator, other components (eg, radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module 197.

According to various embodiments, the antenna module 197 may form a mmWave antenna module. According to one embodiment, a mmWave antenna module includes a printed circuit board, an RFIC disposed on or adjacent to a first side (e.g., bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., mmWave band); And a plurality of antennas (e.g., array antennas) disposed on or adjacent to the second side (e.g., top or side) of the printed circuit board and capable of transmitting or receiving signals in the designated high frequency band. can do.

At least some of the components are connected to each other through a communication method between peripheral devices (e.g., bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI)) and signal ( (e.g. commands or data) can be exchanged with each other.

According to one embodiment, commands or data may be transmitted or received between the electronic device 101 and the external electronic device 104 through the server 108 connected to the second network 199. Each of the external electronic devices 102 or 104 may be of the same or different type as the electronic device 101. According to one embodiment, all or part of the operations performed in the electronic device 101 may be executed in one or more of the external electronic devices 102, 104, or 108. For example, when the electronic device 101 needs to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device 101 may perform the function or service instead of executing the function or service on its own. Alternatively, or additionally, one or more external electronic devices may be requested to perform at least part of the function or service. One or more external electronic devices that have received the request may execute at least part of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device 101. The electronic device 101 may process the result as is or additionally and provide it as at least part of a response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology can be used. The electronic device 101 may provide an ultra-low latency service using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device 104 may include an Internet of Things (IoT) device. Server 108 may be an intelligent server using machine learning and/or neural networks. According to one embodiment, the external electronic device 104 or server 108 may be included in the second network 199. The electronic device 101 may be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

Electronic devices according to various embodiments disclosed in this document may be of various types. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to embodiments of this document are not limited to the above-described devices.

The various embodiments of this document and the terms used herein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various changes, equivalents, or replacements of the embodiments. In connection with the description of the drawings, similar reference numbers may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the above items, unless the relevant context clearly indicates otherwise. As used herein, “A or B”, “at least one of A and B”, “at least one of A or B”, “A, B or C”, “at least one of A, B and C”, and “A Each of phrases such as “at least one of , B, or C” may include any one of the items listed together in the corresponding phrase, or any possible combination thereof. Terms such as "first", "second", or "first" or "second" may be used simply to distinguish one element from another, and may be used to distinguish such elements in other respects, such as importance or order) is not limited. One (e.g. first) component is said to be "coupled" or "connected" to another (e.g. second) component, with or without the terms "functionally" or "communicatively". Where mentioned, it means that any of the components can be connected to the other components directly (e.g. wired), wirelessly, or through a third component.

The term “module” used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and is interchangeable with terms such as logic, logic block, component, or circuit, for example. It can be used as A module may be an integrated part or a minimum unit of the parts or a part thereof that performs one or more functions. For example, according to one embodiment, the module may be implemented in the form of an application-specific integrated circuit (ASIC).

Various embodiments of the present document are one or more instructions stored in a storage medium (e.g., built-in memory 136 or external memory 138) that can be read by a machine (e.g., electronic device 101). It may be implemented as software (e.g., program 140) including these. For example, a processor (e.g., processor 120) of a device (e.g., electronic device 101) may call at least one command among one or more commands stored from a storage medium and execute it. This allows the device to be operated to perform at least one function according to the at least one instruction called. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. A storage medium that can be read by a device may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' only means that the storage medium is a tangible device and does not contain signals (e.g. electromagnetic waves), and this term refers to cases where data is semi-permanently stored in the storage medium. There is no distinction between temporary storage cases.

According to one embodiment, methods according to various embodiments disclosed in this document may be provided and included in a computer program product. Computer program products are commodities and can be traded between sellers and buyers. The computer program product may be distributed in the form of a machine-readable storage medium (e.g. compact disc read only memory (CD-ROM)) or through an application store (e.g. Play StoreTM) or on two user devices (e.g. It can be distributed (e.g. downloaded or uploaded) directly between smart phones) or online. In the case of online distribution, at least a portion of the computer program product may be at least temporarily stored or temporarily created in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

According to various embodiments, each component (e.g., module or program) of the above-described components may include a single or plural entity, and some of the plurality of entities may be separately placed in other components. there is. According to various embodiments, one or more of the components or operations described above may be omitted, or one or more other components or operations may be added. Alternatively or additionally, multiple components (eg, modules or programs) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each component of the plurality of components in the same or similar manner as those performed by the corresponding component of the plurality of components prior to the integration. . According to various embodiments, operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order. may be removed, omitted, or one or more other operations may be added.

FIG. 2 is a reference diagram schematically showing an example of an electronic device according to an embodiment of the present disclosure.

Referring to FIG. 2, the electronic device 101 (e.g., the electronic device 101 of FIG. 1) according to an embodiment includes a housing 210 and a display 220 (e.g., the display module 160 of FIG. 1). ), sound output module 230 (e.g., sound output module 155 in FIG. 1), sensor module 260 (e.g., sensor module 176 in FIG. 1), camera module 270 (e.g., FIG. 1 It may include a camera module 180), a key input device 280 (e.g., the input module 150 of FIG. 1), an indicator 290, and/or a fingerprint sensor 200. In some embodiments, the electronic device 101 may omit at least one of the components (eg, the key input device 280 or the indicator 290) or may additionally include another component.

According to one embodiment, in FIG. 2, at least one of the sensor module 260, the sound output module 230, and/or the camera module 270 including a proximity sensor and/or an illumination sensor is installed on the front of the electronic device 101. An example of mounting on a plate (e.g., housing 210) is shown, but is not limited thereto. For example, although not shown, the sensor module 260, the audio output module 230, and/or the camera module 270 are located below the display 220 of the electronic device 101 (e.g., under panel). It may also be implemented in a structure placed in . For example, the sensor module 260, the audio output module 230, and/or the camera module 270 are mounted on the display 220 on the front of the housing 210 between the front of the housing 210 and the back opposite to the front. ), it may be placed below the display 220. In some embodiments, at least a portion of the sensor module 260, the audio output module 230, and/or the camera module 270 may be disposed in at least one area of the display 220.

According to one embodiment, the sensor module 260, the sound output module 230, and/or the camera module 270 are disposed in a space formed by the housing 210 of the electronic device 101, and the housing 210 ) and/or may be exposed to the outside through at least one hole (or opening) formed in the display 220. For example, the display 220 of the electronic device 101 may be implemented with a design that increases the area occupied on the front (e.g., bezel-less display), and various components disposed on the front (e.g., sensor module 260 ), the sound output module 230 and/or the camera module 270) are placed between the display 220 and the rear (e.g., second side) (or below the display 220), or the display 220 It can be placed inside. For example, the electronic device 101 may be implemented with an under display sensor structure to place various components between the display 220 and the rear (or under the display 220 (e.g., under panel)). You can. According to one embodiment, the electronic device 101 may be implemented with an in-display sensor structure in which various components are integrated with the display 220 and formed within the display 220.

According to one embodiment, the housing 210 may refer to an external portion surrounding the electronic device 101. For example, the housing 210 has a first side (or front side), a second side (or back side) opposite the first side, and a side (e.g., side side) surrounding the space between the first side and the second side. absence) may be included. According to one embodiment, the first side may be formed at least in part by a substantially transparent front plate (eg, a glass plate including various coating layers, or a polymer plate). According to one embodiment, the second side may be formed by a substantially opaque back plate. According to one embodiment, the back plate is coated or colored glass, ceramic, polymer, metal (e.g., aluminum, stainless steel (STS), or magnesium), or at least one of the foregoing materials. It can be formed by a combination of the two. According to one embodiment, the side may be formed by a side bezel structure (or side member) that combines with the front plate and the back plate and includes metal and/or polymer.

According to one embodiment, the display 220 (e.g., the display module 160 of FIG. 1) may be disposed on the first side (e.g., front plate) of the housing 210 and exposed to the outside. there is. The display 220 is combined with a touch sensor (or touch detection circuit), a pressure sensor capable of measuring the intensity (pressure) of touch, and/or a digitizer that detects a magnetic field type digital pen (e.g., stylus pen). Can be placed adjacent to each other. According to one embodiment, the display 220 may be implemented in various forms including a liquid crystal display (LCD), an organic light emitting diode (OLED), or an active matrix OLED (AMOLED). The display 220 may display various images (e.g., moving images and/or still images) based on the control of a processor (e.g., processor 120 of FIG. 1), and may display various external objects (e.g., images) on the display 220. : Input by human hands and/or digital pen) can be received. The display 220 may include a touch sensor (not shown) to receive input from various external objects.

According to one embodiment, the touch sensor may be composed of a layer independent of the display panel of the display 220, or may be implemented in a structure integrated with the display panel. The touch sensor is a first touch implemented by direct contact between an external object and the display 220 (e.g., contact touch input), or a first touch implemented by proximity (e.g., hovering) but the external object does not contact the display 220. 2 A touch (e.g. proximity input) can be received.

According to one embodiment, the sound output module 230 may include a speaker. The speaker may include a receiver for calls. According to one embodiment, the sound output module 230 is placed at the edge of the bezel area of the electronic device 101, or is an under panel speaker that uses the display 220 (e.g., OLED) as a diaphragm. may include. For example, the under panel speaker may be placed below the display 220 and may not be visible from the outside.

According to one embodiment, the sensor module 260 may generate an electrical signal or data value corresponding to the internal operating state of the electronic device 101 or the external environmental state. The sensor module 260 may include, for example, a first sensor module 240 (e.g., a proximity sensor) and/or a second sensor module 250 (e.g., an illuminance sensor) disposed on the first side of the housing 210. ), and/or a third sensor module (not shown) (eg, a fingerprint sensor) disposed on the second side of the housing 210. According to one embodiment, the first sensor module 240 (eg, proximity sensor) may detect an external object approaching the electronic device 101. The first sensor module 240 may include a light emitting unit 241 that emits infrared rays and a light receiving unit 243 that receives infrared rays reflected by an external object. According to one embodiment, the second sensor module 250 (e.g., illuminance sensor) may measure the illumination (e.g., ambient light) of the surroundings (or surroundings) of the electronic device 101. there is. According to one embodiment, the second sensor module 250 may measure the illuminance by measuring the amount of light through a previously created hole.

According to one embodiment, the camera module 270 includes a first camera module 270 disposed on the first side of the electronic device 101, and/or a second camera module (not shown) disposed on the second side. , and/or flash (not shown). The camera module 270 may include one or more lenses, an image sensor, and/or an image signal processor. According to one embodiment, the flash may include a light emitting diode or a xenon lamp. According to one embodiment, two or more lenses (eg, wide-angle and telephoto lenses) and image sensors may be disposed on one side of the electronic device 101.

According to one embodiment, the key input device 280 may be placed on the side of the housing 210. In one embodiment, the electronic device 101 may not include some or all of the key input devices 280, and the key input devices 280 that are not included may be in other forms, such as soft keys, on the display 220. It can be implemented as: In one embodiment, the key input device 280 may be implemented using a touch sensor and/or a pressure sensor included in the display 220.

According to one embodiment, the indicator 290 may be disposed on the first side of the housing 210. The indicator 290 may provide status information of the electronic device 101 in the form of light. For example, the indicator 290 may be provided as an LED status indicator (pilot lamp) (or indicating lamp). In one embodiment, the indicator 290 may provide a light source linked to the operation of the camera module 270. In one embodiment, indicator 290 may include a light emitting device such as an LED, IR LED, and/or a xenon lamp.

Although not shown, the electronic device 101 may include an input module (not shown) and/or a connector hole (not shown). The input module may include a plurality of microphones arranged to detect the direction of sound. The connector hole includes a first connector hole that can accommodate a connector (e.g., USB connector) for transmitting and receiving power and/or data with an external electronic device, and/or a connector for transmitting and receiving an audio signal with an external electronic device. It may include a second connector hole (or earphone jack). According to one embodiment, the electronic device 101 includes a sensor module not shown (e.g., the sensor module 176 in FIG. 1), for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, It may further include at least one of a grip sensor, a color sensor, an infrared (IR) sensor, a biometric sensor, a temperature sensor, or a humidity sensor.

According to one embodiment, the electronic device 101 may provide a fingerprint recognition (e.g., FOD (fingerprint on display) or in-display fingerprint) (hereinafter referred to as 'display fingerprint recognition') function on the display 220. there is. For example, the electronic device 101 may include a fingerprint sensor in a portion of the space between the display 220 and the second surface (or below (e.g., under panel) the display 220) (or in the space formed by the housing 210). (200) (e.g., an in-display fingerprint sensor). According to one embodiment, the fingerprint sensor 200 may be disposed between the display 220 located on the front (eg, first side) of the electronic device 101 and the rear (eg, second side). According to one embodiment, the electronic device 101 may further include a button-type fingerprint sensor (e.g., a fingerprint sensor disposed on a physical button on the second surface or side) along with the display fingerprint sensor.

According to one embodiment, the electronic device 101 may provide fingerprint recognition based on the fingerprint sensor 200. According to one embodiment, the fingerprint sensor 200 is disposed below the display 220 (e.g., under panel) and may be implemented to recognize a fingerprint using the light source of the display 220 without being exposed to the outside. there is. For example, the electronic device 101 places the fingerprint sensor 200 on the back of the display (e.g., the bottom of the display panel), and the fingerprint touched (or touched) on the front of the display (e.g., the display surface (or screen)). can be recognized. According to one embodiment, the fingerprint sensor 200 may use an optical method to acquire a fingerprint image (or video) reflected in visible light. For example, the optical method may refer to a method of shining light on a platen and recognizing the image of the fingerprint reflected according to the shape of the fingerprint of the fingertip placed on the platen.

FIG. 3 is a reference diagram schematically illustrating an example of a display and a fingerprint sensor being arranged in an electronic device according to an embodiment of the present disclosure.

FIG. 4 is a reference diagram illustrating an example of a polarized light transmission direction of a display in an electronic device according to an embodiment of the present disclosure.

FIG. 5 is a reference diagram illustrating an example of a pattern for measuring signal characteristics in an electronic device according to an embodiment of the present disclosure.

According to one embodiment, FIG. 3 may illustrate an example in which the fingerprint sensor 200 (eg, an optical in-display fingerprint sensor) is disposed below the display 220.

As illustrated in FIG. 3, the display 220 according to one embodiment includes glass 310, adhesive 320, polarizer 330, and touch screen panel (TSP) 340. ), a display panel 350, and a copper sheet 360.

According to one embodiment, the fingerprint sensor 200 may be implemented to recognize a fingerprint using the light source of the display 220. For example, light generated from the display panel 350 within the structure of the display 220 may be transmitted through several layers of the display 220 (e.g., TSP 340, polarizer 330, adhesive 320, and After passing through the glass 310 layer, it may be incident on the user's fingerprint 300 in contact with the surface of the display 220. The refractive index of the glass 310, which is in direct contact with the fingerprint 300, is similar to that of the user's skin. There may be a relatively large difference from the refractive index of the air layer. For example, the refractive index of the glass 310 may be about 1.5, the refractive index of the skin may be about 1.4, and the refractive index of the air layer may be about 1.0. there is.

According to one embodiment, light is reflected as the difference in refractive index becomes larger and passes as it is similar, so most of the light incident on the fingerprint 300 is reflected in the valley of the fingerprint 300 and can be directed back toward the display 220. And, most of the light incident on the ridge of the fingerprint 300 can be absorbed by the user's finger. The light reflected from the valleys of the fingerprint 300 passes through all layers of the display 220 and is received by the fingerprint sensor 200 to obtain a fingerprint image.

In this way, in the process of reaching the user's fingerprint, the light generated from the display panel 350 reaches the user's fingerprint 300 through several layers within the display 220, and is reflected again from the valley to pass through several layers of the fingerprint. Sensor 200 can be reached. Among the various layers of the display 220, the display panel 350 and the polarizer 330 may have a significant impact on the fingerprint sensor 200. For example, the display panel 350 is made of a specific metal pattern, and is involved in the transmission characteristics, frequency characteristics, and noise characteristics of light received by the fingerprint sensor 200 and may affect image quality.

For example, the polarizer 330, which is applied to make the black color more accurate in the display 220 and to accurately distinguish colors outdoors, transmits light with a random direction generated from the light source of the display panel only in a certain direction. It can be made with linear polarization characteristics and transmitted to the user's fingerprint. The light reflected from the user's fingerprint may pass through the polarizer 330 again and be transmitted to the fingerprint sensor 200.

As illustrated in FIG. 4 , the transmission angle 400 of the polarizer 330 may be generally included in the display 220 at a transmission angle of about 45 degrees or about 135 degrees. Depending on the transmission angle 400 of the polarizer 330, only light polarized in a specific direction may pass through while light passes through the polarizer 330, and light in other directions may be blocked. For example, light may change to linear polarization while passing through the polarizer 330, which is generally aligned at about 45 degrees.

As described above, the image (e.g., fingerprint image) acquired through the fingerprint sensor 200 passes through several layers until the light generated from the light source of the display panel 350 is reflected from the user's fingerprint 300 and enters the fingerprint sensor 200. As it passes through, loss and distortion may be included. Additionally, the unique performance of the fingerprint sensor 200 and the display 220 may vary when manufacturing the electronic device 101. Accordingly, a calibration process may be performed on the fingerprint sensor 200 in the manufacturing stage of the electronic device 101 in order to minimize errors in these areas and fix and extract a desired standard value.

According to one embodiment, during the calibration process of the fingerprint sensor 200, image capture time information, sensor signal/noise information, background image, and sensor pixel are collected. A variety of information can be obtained, such as information, lens magnification information, and/or additional information that can be confirmed through the image. In one embodiment, the image capture time may indicate information regarding the light intensity of the display 220, transmission characteristics, and/or the sensitivity of the sensor. In one embodiment, the sensor signal/noise may represent information about the contrast (or difference) of a specific pattern that is finally obtained. In one embodiment, the background image may represent an image represented by a display pattern, light quantity, and/or display stack.

According to one embodiment, the electronic device 101 may use at least one calibration kit (or groove tool) to obtain the various information in the calibration process.

According to one embodiment, when manufacturing the electronic device 101, the calibration operation is performed. For example, the calibration kit includes a reflective color groove tool (e.g., an apricot color groove tool) and an absorbing color groove tool (e.g., a black groove tool). , and/or a specific pattern tool of a reflected color (e.g., an apricot stripe tool). According to one embodiment, the reflected color may be white and the absorbed color may be black.

According to one embodiment, a reflective color home tool may represent a tool or a corresponding environment that can measure light reflected from the home tool. According to one embodiment, the absorbing color groove tool may represent a tool for measuring light on the back side of the display 220 or a corresponding environment (eg, a dark room). According to one embodiment, the absorbing color groove tool may be used to obtain a background image through the fingerprint sensor 200. According to one embodiment, a specific pattern tool of reflected color can be used to obtain information for determining signal characteristics and patterns.

According to one embodiment, various information including saturation time of the fingerprint sensor 200, magnification, presence of foreign matter attached, signal characteristics, and display back pattern characteristics (or back image characteristics) of the fingerprint sensor 200 through a calibration operation (e.g., reference data) can be obtained. In one embodiment, various information may be stored in the memory 130 or a memory (not shown) provided within the fingerprint sensor 200. In one embodiment, the saturation time may represent the exposure time for reaching the maximum number of electrons that a pixel of the fingerprint sensor 200 can store. In one embodiment, the magnification may represent the ratio between the actual fingerprint size and the image size received by the fingerprint sensor 200. In one embodiment, whether foreign matter is attached may be a process of checking whether there is foreign matter or damage in the fingerprint sensor 200 or the path through which light passes during the manufacturing process of the electronic device 101.

In one embodiment, the signal characteristic may indicate the level of the signal characteristic incident on the fingerprint sensor 200. For example, signal characteristics can be measured using a designated pattern (e.g., a pattern where white and black alternate), as illustrated in FIG. 5 . For example, black may be in contact with the display 220 and serve as a ridge for a fingerprint, and white may serve as a valley for a fingerprint with an air layer. In these patterns, signal characteristics can be distinguished depending on how accurately the distinction between white and black is expressed.

In one embodiment, the display back pattern may represent a shape created when a light source passes through the polarizer 330 and the display panel 350 as background information when imaging the fingerprint sensor 200. This shape can act as a fixed shape and can be used when a user attempts fingerprint recognition to obtain a fingerprint image to improve image quality. According to one embodiment, in the case of the fingerprint sensor 200 of the electronic device 101, the quality of the acquired image may be affected by the characteristics of the display 220, as described above, and a correction operation to compensate for this may be performed. You can proceed.

As discussed above, the fingerprint sensor 200 may be greatly affected by the stacked structure of the display 220. Therefore, the fingerprint sensor 200 can exhibit optimal performance when used under the same conditions as those in which correction work is performed in the manufacturing stage. However, in order to protect the display 220, users of the electronic device 101 may attach a protective material such as a protective film or protective glass to the display 220. In this case, due to the addition of a protective material, a layered structure with new characteristics may be formed, and the performance of the fingerprint sensor 200 may change.

According to one embodiment, the protective film may include a structure in which an adhesive component and PET (polyester) are laminated on the top of the display 220. According to one embodiment, the protective glass may include a structure in which an adhesive component, an anti-shattering film, and glass are stacked on top of the display 220. In these protective materials, for example PET or anti-shatter films and some adhesive components may have polarizing properties (eg retarding properties (eg phase retardation)). For example, a protective material may develop retarding properties in a particular direction due to various factors (e.g., expansion, heat, and/or pressure) during its manufacturing process.

In one embodiment, the retarding characteristic may represent a phenomenon in which the phase of a wave traveling in a direction of a specific axis (eg, X axis and/or Y axis) is delayed. For example, the retarding characteristic may represent the phenomenon of delaying the phase of light in a specific direction when light passes through a specific type of molecular structure. According to one embodiment, in the case of a protective material, there is no loss of signal intensity due to the retarding characteristic, but the phase in a specific axis direction may be delayed. For example, in the case of a protective material, retarding properties may occur in the width and length directions. In one embodiment, the specific direction in which light travels can be delayed by about 90 degrees (eg, linearly polarized light -> circularly polarized light, or circularly polarized light -> linearly polarized light) by the protective material.

According to one embodiment, as described above, in the case of the fingerprint sensor 200, the recognition performance of the fingerprint sensor 200 may change (e.g., recognition performance deterioration) due to the retarding characteristics of the protective material. .

According to one embodiment, the retarding characteristic of the protective material is that if the direction is not aligned with the polarizer 330 (e.g., linear polarizer) of the display 220, it has linear polarization characteristics emitted from the polarizer 330. The specific direction of the light source can be phase delayed (or retarded) to change it to elliptical polarization characteristics. The changed elliptical polarization characteristics may be reflected from the fingerprint and changed to linear polarization characteristics as they pass through the polarizer 330 of the display 220.

For example, the polarizer 330 (eg, a linear polarizer) of the display 220 may pass a linearly polarized signal traveling at a specified angle without loss. However, when an elliptically polarized signal is input to the polarizer 330, the elliptically polarized signal may be attenuated. For example, the signal may be lost as the elliptical polarization characteristic changes to the linear polarization characteristic, which may cause a decrease in signal to noise ratio (SNR) and affect the recognition performance of the fingerprint sensor 200. For example, the resolution of the fingerprint sensor 200 to distinguish between ridges and valleys in a fingerprint image may be poor.

In this way, the retarding properties generated from the protective material may differ depending on the process and fabric of each manufacturer of the protective material, and cannot be distinguished by appearance. Therefore, if the polarization characteristics of the protective material do not match the direction of passage of the polarizer 330 inside the display 220 of the electronic device 101, the signal transmitted to the fingerprint sensor 200 may be significantly reduced. . These results may cause changes in the recognition performance of the fingerprint sensor 200 (e.g., performance deterioration).

The electronic device 101 according to an embodiment of the present disclosure can detect the presence or absence of a protective material using the polarization characteristics of the display 220 under specified conditions. When detecting attachment of a protective material, the electronic device 101 according to an embodiment of the present disclosure may determine characteristics of the protective material that may affect the fingerprint recognition performance of the fingerprint sensor 200. When the electronic device 101 according to an embodiment of the present disclosure determines that the recognition performance of the fingerprint sensor 200 has deteriorated based on the characteristics of the protective material, it provides guidance to the user regarding changes in fingerprint recognition performance due to the protective material. can do. Through this, it is possible to maintain or improve the recognition performance of the fingerprint sensor 200 by allowing the user to cope with the deterioration of fingerprint recognition performance due to the protective material.

FIG. 6 is a diagram schematically showing the configuration of an electronic device according to an embodiment of the present disclosure.

The electronic device 101 according to an embodiment of the present disclosure may include various devices in which a fingerprint sensor can be placed under the display 220. According to one embodiment, the electronic device 101 may include various devices such as a smart phone, wearable device, digital camera, multimedia player, or portable computer. In one embodiment of the present disclosure, for convenience of explanation, it is explained that the electronic device 101 is a smart phone as an example, but various embodiments according to the present disclosure are not limited to smart phones.

According to one embodiment, the fingerprint sensor 200 may recognize a fingerprint input from a user. According to one embodiment, the fingerprint sensor 200 may acquire an image (or fingerprint characteristics) of a finger print indicating differences in the user's unique characteristics. According to one embodiment, the fingerprint sensor 200 may image a fingerprint in various ways, such as optical, semiconductor device sensing capacitance or electrical conduction, ultrasonic, thermal, non-contact, and/or a combination of these methods. can be obtained.

According to one embodiment, the fingerprint sensor 200 may include a sensor (eg, an optical fingerprint sensor) that receives light and converts it into an electrical signal. According to one embodiment, the light source used as a signal in the fingerprint sensor 200 may be a light source within the display 220. According to one embodiment, the first light source used in the calibration operation related to the fingerprint sensor 200 in the manufacturing stage of the electronic device 101 and the performance of the fingerprint sensor 200 in an environment in which the user uses the electronic device 101 The second light source used during testing has the same color (e.g., a combination of R (red), G (green), and B (blue)) and/or brightness (e.g., a brightness within the range in which the display 220 can occur). may be substantially the same.

According to one embodiment, the processor 120 may control the operation of the electronic device 101. According to one embodiment, the processor 120 may control operations related to performing functions according to fingerprint recognition. According to one embodiment, the processor 120 is operatively connected to the display module 160, the fingerprint sensor 200, and/or the memory 130, and the processor 120 is connected to the display module 160, the fingerprint sensor 200, and/or the memory 130. The memory 130 can be controlled. According to one embodiment, the processor 120 executes one or more programs (or instructions) stored in the memory 130 to control the operation of the electronic device 101 of various embodiments. It can be implemented with one or more processors. According to one embodiment, the processor 120 may include at least one processor among an application processor (AP), a communication processor (CP), and/or a supplementary processor.

Referring to FIG. 6, the electronic device 101 according to an embodiment of the present disclosure includes a processor 120, a memory 130, a fingerprint sensor 200, a display 220, a sensor module 260, and/or It may include a camera module 270. According to one embodiment, the electronic device 101 may not include at least one component (eg, the camera module 270). According to one embodiment, the electronic device 101 may include one or more other components (e.g., the communication module 190, power management module 188, and/or battery 189 of FIG. 1). For example, the electronic device 101 may include all or at least some of the components of the electronic device 101 as described in the description with reference to FIGS. 1, 2, and 3.

According to one embodiment, the display 220 (e.g., the display module 160 of FIG. 1 or the display 220 of FIG. 2) visually provides various information to the outside of the electronic device 101 (e.g., the user). can do. According to one embodiment, the display 220 may visually provide various information related to changes in recognition performance of the fingerprint sensor 200 due to a protective material under the control of the processor 120.

According to one embodiment, the display 220 includes a touch detection circuit (or touch sensor) (not shown), a pressure sensor capable of measuring the intensity of touch, and/or a touch that detects a magnetic field-type stylus pen. May include panels (e.g. digitizers). According to one embodiment, the display 220 receives a signal (e.g., voltage, light amount, resistance, electromagnetic signal, and/or charge amount) for a specific position of the display 220 based on a touch detection circuit, a pressure sensor, and/or a touch panel. ) can detect touch input and/or hovering input (or proximity input). According to one embodiment, the display 220 may include a liquid crystal display (LCD), an organic light emitted diode (OLED), or an active matrix organic light emitted diode (AMOLED). According to some embodiments, the display 210 may include a flexible display.

According to one embodiment, the memory 130 may correspond to the memory 130 described in the description with reference to FIG. 1 . According to one embodiment, the memory 130 may store various data used by the electronic device 101. In one embodiment, data may include, for example, input data or output data for an application (e.g., program 140 of FIG. 1) and instructions associated with the application.

In one embodiment, the data may include various sensor data (eg, acceleration sensor data, gyro sensor data, or barometric pressure sensor data) acquired from the sensor module 260. In one embodiment, the data may include data acquired from the fingerprint sensor 200 (eg, a fingerprint image and/or a background image). In one embodiment, the data includes various reference data (e.g., reference background image) associated with the fingerprint sensor 200 that is preset in memory 130 to identify changes in recognition performance of the fingerprint sensor 200. , a reference fingerprint image, and/or a reference map).

According to one embodiment, the memory 130 may store instructions that cause the processor 120 to operate when executed. For example, an application may be stored as software (eg, program 140 in FIG. 1) on the memory 130 and may be executable by the processor 120. According to one embodiment, the application may be a variety of applications that can provide various services (eg, a performance test service of the fingerprint sensor 200) in the electronic device 101. In one embodiment, the performance test service may include a service (or application) for determining whether a change occurs in the recognition performance of the fingerprint sensor 200 due to the protective material and providing feedback to the user according to the determination result. You can. In one embodiment, performance testing may be performed intentionally or unintentionally by the user.

According to one embodiment, the sensor module 260 may correspond to the sensor module 176 as described in the description with reference to FIG. 1. According to one embodiment, the sensor module 260 may include various sensors such as an illumination sensor 261, an acceleration sensor 263, a gyro sensor 265, and an air pressure sensor 267. According to one embodiment, the sensor module 260 may include an attitude sensor that can replace the acceleration sensor 263 and/or the gyro sensor 265. According to one embodiment, the electronic device 101 uses the fingerprint sensor 200 based on sensor data using the acceleration sensor 263, gyro sensor 265, and/or barometric pressure sensor 267 of the sensor module 260. You can perform operations related to performance testing. For example, the processor 120 may determine a trigger for detecting and configuring a designated environment for testing the performance of the fingerprint sensor 200 based on at least one sensor data of the sensor module 260.

According to one embodiment, the processor 120 may perform an application layer processing function required by the user of the electronic device 101. According to one embodiment, the processor 120 may provide commands and control of functions for various blocks of the electronic device 101. According to one embodiment, the processor 120 may perform operations or data processing related to control and/or communication of each component of the electronic device 101. For example, the processor 120 may include at least some of the components and/or functions of the processor 120 of FIG. 1 . According to one embodiment, the processor 120 may be operatively connected to components of the electronic device 101. According to one embodiment, the processor 120 loads commands or data received from other components of the electronic device 101 into the memory 130, processes the commands or data stored in the memory 130, and , the resulting data can be saved.

According to one embodiment, the processor 120 may include processing circuitry and/or executable program elements. According to one embodiment, processor 120 may control (or process) operations related to detecting the presence or absence of adhesion of a protective material on display 220 based on processing circuitry and/or executable program elements. According to one embodiment, the processor 120 determines a change in recognition performance of the fingerprint sensor 200 when detecting a protective material, based on processing circuits and/or executable program elements, and provides feedback to the user accordingly. You can control (or process) operations related to it.

According to one embodiment, the processor 120 may operate to detect designated triggers related to a performance test of the fingerprint sensor 200 located below the display 220. According to one embodiment, the processor 120 may control the fingerprint sensor 200 to acquire a designated image based on detecting a designated trigger. According to one embodiment, the processor 120 may extract reference data (eg, first reference data or second reference data) set in the memory 130. According to one embodiment, the processor 120 determines the information related to the protective material attached on the display 220 based on a designated image (e.g., background image or fingerprint image) and reference data acquired through the fingerprint sensor 200. It can be operated to determine the status. According to one embodiment, the processor 120 may operate to determine whether the performance of the fingerprint sensor 200 changes based on a state related to the protective material. According to one embodiment, the processor 120 may operate to perform a designated operation based on whether the performance of the fingerprint sensor 200 changes.

According to one embodiment, the processor 120 may control operations related to performing a first performance test or a second performance test to detect whether the performance of the fingerprint sensor 200 changes due to the protective material.

According to one embodiment, the processor 120 may obtain a background image from the user environment during the first performance test for the fingerprint sensor 200. According to one embodiment, the processor 120 may use a reference background image set in the manufacturing stage as a comparison image for the background image. According to one embodiment, the processor 120 may perform a first performance test based on processing (eg, comparative processing) of the acquired background image and the comparison image (eg, reference background image). According to one embodiment, the processor 120 may check whether a protective material is attached by comparing background images obtained at the manufacturing stage and in the user environment. According to one embodiment, the processor 120 may infer differences between background images and/or characteristics of a protective material (eg, protective film or protective glass) depending on the shape of the background image.

According to one embodiment, when performing a second performance test for the fingerprint sensor 200, the processor 120 may acquire a fingerprint image in an environment where fingerprint authentication is performed rather than as a background image. According to one embodiment, the processor 120 may extract a specific image pattern from a fingerprint image. According to one embodiment, the processor 120 uses numerical data (e.g., reference map) quantified (e.g., binarized/ranged) based on the averaged fingerprint image in the manufacturing stage as a comparison image for the fingerprint image. You can. According to one embodiment, the processor 120 may perform a second performance test based on processing (eg, comparative processing) of an image pattern extracted from an acquired fingerprint image and a comparison image (eg, reference map). According to one embodiment, the processor 120 may check the similarity as a number based on the image pattern and reference map extracted by patterning the fingerprint image.

According to an embodiment of the present disclosure, the processor 120 performs a correction operation (e.g., performance test) related to the fingerprint sensor 200 performed in the manufacturing stage of the electronic device 101 when the user uses the electronic device 101. Even in the environment, it can be performed at regular intervals or in designated situations and the results can be provided to the user. According to one embodiment, the processor 120 includes a designated calibration kit (or groove tool) (e.g., a color that absorbs light (e.g., black) for calibrating the fingerprint sensor 200. You can perform correction work in a designated environment (e.g., darkroom environment) that is the same or similar to the process of performing correction work using the home tool in . According to one embodiment, the processor 120 acquires an image (e.g., background image) in a designated environment, and an image (e.g., absorbs light) acquired in a correction operation in the manufacturing stage of the electronic device 101. It can be compared with the background image obtained through the home tool of the color. According to one embodiment, the processor 120 may detect whether a protective material is attached to the display 220 based on the comparison result. According to one embodiment, the processor 120 may determine the characteristics of the protective material when detecting the protective material and provide related feedback to the user.

According to one embodiment of the present disclosure, the processor 120 performs a correction operation (e.g., performance test) related to the fingerprint sensor 200 based on an image (e.g., fingerprint image) acquired while the user uses the electronic device 101. ) can proceed. According to one embodiment, the processor 120 may acquire the user's fingerprint image based on the user's intention or non-intention while the user uses the electronic device 101. According to one embodiment, the processor 120 may extract (eg, binary or range) a pattern based on the fingerprint image. According to one embodiment, the processor 120 may compare the extracted pattern with a reference map set in the electronic device 101. According to one embodiment, the processor 120 may detect whether a protective material is attached to the display 220 based on the comparison result. According to one embodiment, the processor 120 may determine the characteristics of the protective material when detecting the protective material and provide related feedback to the user.

According to one embodiment, detailed operations of the processor 120 of the electronic device 101 will be described with reference to the drawings described later.

According to one embodiment, operations performed by the processor 120 may be implemented as a recording medium (or computer program product). For example, the recording medium may include a non-transitory computer-readable recording medium on which a program for executing various operations performed by the processor 120 is recorded.

Embodiments described in this disclosure may be implemented in a recording medium readable by a computer or similar device using software, hardware, or a combination thereof. According to hardware implementation, the operations described in one embodiment include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), and field programmable gate arrays (FPGAs). ), processors, controllers, micro-controllers, microprocessors, and/or other electrical units to perform functions. .

In one embodiment, the recording medium (or computer program product) includes an operation for detecting a designated trigger related to a performance test of a fingerprint sensor 200 formed below the display 220, based on detecting the designated trigger, An operation of controlling the fingerprint sensor 200 to acquire a designated image, an operation of extracting reference data set in the memory 130 of the electronic device 101, the designated image acquired through the fingerprint sensor 200 and the An operation of determining a state related to a protective material attachable to the display 220 based on reference data; an operation of determining whether the performance of the fingerprint sensor 200 changes based on the state related to the protective material; And it may include a computer-readable recording medium that records a program for executing a designated operation based on whether the performance of the fingerprint sensor 200 changes.

The electronic device 101 according to an embodiment of the present disclosure includes a display (e.g., the display module 160 of FIG. 1 or the display 220 of FIG. 2 or 6) and a fingerprint sensor formed below the display (e.g. : Fingerprint sensor 200 of FIG. 2 or FIG. 6), memory (e.g., memory 130 of FIG. 1 or FIG. 6), and a processor operatively connected to the display, the fingerprint sensor, and the memory (e.g. It may include the processor 120 of FIG. 1 or FIG. 6).

According to one embodiment, the processor 120 may operate to detect a designated trigger related to a performance test of the fingerprint sensor. According to one embodiment, the processor 120 may control the fingerprint sensor to acquire a designated image based on detecting the designated trigger. According to one embodiment, the processor 120 may operate to extract reference data set in the memory. According to one embodiment, the processor 120 may operate to determine a state related to a protective material attachable to the display based on a designated image acquired through the fingerprint sensor and the reference data. According to one embodiment, the processor 120 may operate to determine whether the performance of the fingerprint sensor changes based on a state related to the protective material. According to one embodiment, the processor 120 may operate to perform a designated operation based on whether the performance of the fingerprint sensor changes.

According to one embodiment, the processor 120 may operate to detect a designated trigger related to the performance test in a user's intended situation or a user's unintended situation.

According to one embodiment, the designated trigger may include a first trigger for performing a first performance test based on a background image acquired in a designated environment. According to one embodiment, the designated trigger may include a second trigger for performing a second performance test based on a fingerprint image acquired in a usage environment of the electronic device.

According to one embodiment, the processor 120 may operate to identify the type of designated trigger based on a performance test environment. According to one embodiment, the processor 120 may operate to perform a first performance test using the background image based on determining that the designated trigger is the first trigger. According to one embodiment, the processor 120 may operate to perform a second performance test using the fingerprint image based on determining that the designated trigger is the second trigger.

According to one embodiment, the processor 120 may operate to obtain the background image through the fingerprint sensor in a designated environment. According to one embodiment, the processor 120 may operate to obtain first reference data set in the memory. According to one embodiment, the processor 120 may operate to detect whether the protective material is attached based on comparison between the background image and the first reference data.

According to one embodiment, the first reference data may include at least one reference background image acquired using a designated calibration kit when calibrating the fingerprint sensor.

According to one embodiment, the processor 120 may operate to obtain a plurality of background images at designated regular time intervals. According to one embodiment, the processor 120 may operate to determine similarity between the plurality of background images.

According to one embodiment, the processor 120 may operate to obtain a background image in a similar environment corresponding to the environment during calibration of the fingerprint sensor, based on the sensor module of the electronic device.

According to one embodiment, the processor 120 may operate to obtain the fingerprint image through the fingerprint sensor. According to one embodiment, the processor 120 may operate to extract an image pattern based on the fingerprint image. According to one embodiment, the processor 120 may operate to obtain second reference data set in the memory. According to one embodiment, the processor 120 may operate to detect whether the protective material is attached based on comparison between the image pattern and the second reference data.

According to one embodiment, the second reference data may include at least one reference map that quantifies an image pattern extracted based on an averaged fingerprint image during calibration of the fingerprint sensor.

According to one embodiment, the processor 120 may operate to extract the image pattern by quantifying an image area in a range corresponding to the reference map in the fingerprint image.

According to one embodiment, the processor 120 may calculate similarity between the image pattern and the second reference data. According to one embodiment, the processor 120 may operate to compare the calculated similarity with a specified threshold value. According to one embodiment, the processor 120 may operate to determine that a protective material is present on the display when the similarity is greater than or equal to the specified threshold.

According to one embodiment, the processor 120 may operate to determine the presence or absence of a protective material on the display based on the designated image and the reference data. According to one embodiment, the processor 120 may operate to analyze the characteristics of the protective material when determining that the protective material is present. According to one embodiment, the processor 120 may operate to determine whether the performance of the fingerprint sensor changes based on the characteristics of the protective material.

According to one embodiment, the processor 120 may operate to determine a first designated state in which there is no change in performance of the fingerprint sensor when determining that the protective material is not present. According to one embodiment, when the processor 120 determines that the protective material is present, the processor 120 may set a second designated state in which there is no change in performance of the fingerprint sensor or a second designated state in which there is a performance change in the fingerprint sensor based on the characteristics of the protective material. 3 Can operate to determine a specified state. According to one embodiment, the processor 120 may operate to provide a designated guide related to the protective material in response to the determined designated state.

Hereinafter, methods of operating the electronic device 101 in various embodiments will be described in detail. Operations performed in the electronic device 101 according to various embodiments include a processor 120 including various processing circuitry and/or executable program elements of the electronic device 101. It can be executed by . According to one embodiment, operations performed by the electronic device 101 are stored in the memory 130 and, when executed, may be executed by instructions that cause the processor 120 to operate.

FIG. 7 is a flowchart illustrating a method of operating an electronic device according to an embodiment of the present disclosure.

According to one embodiment, FIG. 7 may show an example of supporting a performance test regarding the recognition performance of the fingerprint sensor 200 in the electronic device 101 according to one embodiment.

In the electronic device 101 according to an embodiment of the present disclosure, a method of supporting a performance test of the fingerprint sensor 200 may be performed, for example, according to the flowchart shown in FIG. 7. The flowchart shown in FIG. 7 is merely a flowchart according to an embodiment of a method for testing the performance of the electronic device 101, and the order of at least some operations may be changed, performed in parallel, performed as independent operations, or at least Some other operations may be performed complementary to at least some of the operations. According to an embodiment of the present disclosure, operations 701 to 711 may be performed by at least one processor 120 of the electronic device 101.

As shown in FIG. 7, the operation method performed by the electronic device 101 according to one embodiment includes an operation 701 of detecting a designated trigger related to a performance test of the fingerprint sensor 200, and the fingerprint sensor 200 An operation of acquiring an image based on (703), an operation of extracting set reference data (705), and an operation of analyzing the state related to the protective material attachable to the display 220 based on the acquired image and reference data ( 707), an operation 709 of determining the state based on the state analysis result, and an operation 711 of performing a designated operation based on the state.

Referring to FIG. 7 , in operation 701, the processor 120 of the electronic device 101 may detect a designated trigger related to a performance test of the fingerprint sensor 200. According to one embodiment, the processor 120 receives an input (e.g., a performance test service (or application) execution request) from a user explicitly requesting a performance test of the fingerprint sensor 200. A designated trigger may be detected based on a user's intended situation) or a second situation (eg, a user's unintended situation) that detects an environment (or condition) designated for performance testing of the fingerprint sensor 200. For example, the first situation may include a case where the electronic device 101 manually performs a performance test by a user. For example, the second situation may include a case where the electronic device 101 automatically performs a performance test.

In one embodiment, the designated trigger may include a first trigger and a second trigger that is different from the first trigger. In one embodiment, the first trigger may represent a trigger for performing a first performance test based on a background image acquired in a designated environment (eg, a darkroom environment). In one embodiment, the second trigger may represent a trigger for performing a second performance test based on a fingerprint image acquired in the use environment of the electronic device 101.

In operation 703, the processor 120 may acquire an image based on the fingerprint sensor 200. According to one embodiment, the processor 120 may control the fingerprint sensor 200 to acquire a designated image. According to one embodiment, the processor 120 may control the fingerprint sensor 200 to obtain a background image or a fingerprint image corresponding to a designated trigger. For example, the fingerprint sensor 200 may perform a photographing operation to obtain a background image or a fingerprint image based on the control of the processor 120.

In one embodiment, the background image may represent an image acquired from an absorbing color home tool or its corresponding environment. In one embodiment, the fingerprint image may represent an image acquired through the fingerprint sensor 200 while the user uses the electronic device 101 (eg, during a fingerprint authentication operation). In one embodiment, the acquired image (e.g., background image and/or fingerprint image) may be characterized according to the retarding characteristics of the protective material, thickness information of the protective material, and/or signal of the image. It can be used to determine characteristic information.

In operation 705, the processor 120 may extract set reference data. According to one embodiment, the processor 120 may obtain reference data related to a designated image from a designated storage area (eg, the memory 130 or the memory of the fingerprint sensor 200). According to one embodiment, when the designated trigger is a first trigger related to a first performance test based on the background image, the processor 120 may extract first reference data related to the background image acquired according to the first trigger. . According to one embodiment, when the designated trigger is a second trigger related to a second performance test based on a fingerprint image, the processor 120 may extract second reference data related to the fingerprint image acquired according to the second trigger. .

In one embodiment, the first reference data may represent reference data for image comparison during the first performance test. In one embodiment, the first reference data may include at least one background image acquired in a designated environment when calibrating the fingerprint sensor 200. In one embodiment, the second reference data may represent reference data for image comparison during a second performance test. In one embodiment, the second reference data may include at least one reference map based on an image pattern extracted through a calculation specified in a calibration operation of the fingerprint sensor 200. In one embodiment, the first reference data and the second reference data may be stored (or set) in the memory 130 during the correction process.

At operation 707, processor 120 determines status related to a protective material attachable on display 220 (e.g., information related to the presence or absence of a protective material and/or characteristics of the protective material) based on the acquired image and reference data. can be analyzed. According to one embodiment, the processor 120 may compare the acquired image (eg, background image or fingerprint image) with corresponding reference data. For example, the processor 120 may compare the acquired background image and first reference data.

For example, the processor 120 may compare the acquired fingerprint image and second reference data. According to one embodiment, the processor 120 may analyze the first state regarding whether a protective material is attached on the display 220 based on the comparison result (e.g., similarity between the acquired image and the reference data). there is. According to one embodiment, when detecting attachment of the protective material, the processor 120 may analyze the second state regarding the change in performance due to the protective material based on the analysis of the characteristics of the protective material.

In operation 709, the processor 120 may determine a specified state related to a change in performance of the fingerprint sensor 200 based on the state analysis result. According to one embodiment, the processor 120 may determine a designated state related to a change in performance of the fingerprint sensor 200 based on analysis results regarding the first state and the second state. According to one embodiment, the processor 120 may determine the designated state based on whether a protective material is detected and analyzing the characteristics of the protective material when the protective material is detected.

In one embodiment, the designated state may include a first designated state, a second designated state, or a third designated state. In one embodiment, the first designated state may represent a state in which no protective material is attached (eg, a state that does not affect the performance of the fingerprint sensor 200). In one embodiment, the second designated state may represent a state in which a protective material is attached but does not substantially affect the performance of the fingerprint sensor 200. In one embodiment, the third designated state may represent a state in which the protective material is attached and affects the performance of the fingerprint sensor 200.

In operation 711, the processor 120 may perform a specified operation based on the specified state. According to one embodiment, the processor 120 may provide a designated guide related to a designated state to the user using visual, auditory, and/or tactile information. According to one embodiment, the processor 120 provides a first designated guide based on a first designated state, a second designated guide based on a second designated state, or a third designated guide based on a third designated state. 3 Can operate to provide designated guidance.

According to one embodiment, the processor 120 may provide a notification that there is no protective material or that there is no problem with the use of the fingerprint sensor 200 in response to determining the first designated state or the second designated state. According to one embodiment, if the performance test is a test that is automatically performed by the electronic device 101 as in the second situation, a notification may not be provided in the first designated state or the second designated state.

According to one embodiment, in response to the third specified state determination, the processor 120 provides guidance (e.g., a description from the perspective of the fingerprint sensor 200) regarding the phenomenon of interaction between the protective material and the fingerprint sensor 200 and/ Alternatively, the removal or replacement of the protective material may be guided based on guidance regarding the phenomenon of the protective material itself (e.g., explanation in terms of the protective material). In one embodiment, the guide in terms of fingerprint sensor 200 may be, for example, “a protective material not suitable for use with a fingerprint sensor.” Or it could include guidance such as “Non-genuine protective materials may degrade fingerprint sensor performance.” In one embodiment, the guidance in terms of protective materials may include, for example, “You are using a non-genuine display protective material.”

According to one embodiment, the processor 120 may provide a guide to induce replacement of the protective material to the user in response to determining the third designated state. In one embodiment, the guide encouraging replacement of the protective material may include, for example, “If the performance of the fingerprint sensor has deteriorated, please replace the attached protective material”, “If the performance of the fingerprint sensor has deteriorated, visit a service center.” Please do so.”, and/or “Using genuine protective materials can improve fingerprint sensor performance.”

According to one embodiment, the processor 120 may store an image (eg, a background image or a fingerprint image) acquired during a performance test process in the memory 130. According to one embodiment, the stored image may be used, for example, as fingerprint performance analysis data of a user visiting a service center. According to one embodiment, the processor 120 may use the stored image for image processing used for fingerprint authentication by the fingerprint sensor 200.

According to one embodiment, processor 120 may provide tuning parameters for touch sensors, proximity sensors, and/or light sensors that may be affected by the protective material, based on the properties of the protective material. there is.

FIG. 8 is a flowchart illustrating a method of operating an electronic device according to an embodiment of the present disclosure.

According to one embodiment, FIG. 8 may show an example of supporting a performance test regarding the recognition performance of the fingerprint sensor 200 in the electronic device 101 according to one embodiment. According to one embodiment, FIG. 8 shows an example of determining and operating a first performance test or a second performance test based on the type of a designated trigger. According to one embodiment, the operation described in FIG. 8 is, for example, performed heuristically in combination with the operations described in FIG. 7, or performed heuristically as a detailed operation of some of the operations described. It can be.

A method of supporting a performance test of the fingerprint sensor 200 in the electronic device 101 according to an embodiment of the present disclosure may be performed, for example, according to the flowchart shown in FIG. 8. The flowchart shown in FIG. 8 is merely a flowchart according to an embodiment of a method for testing the performance of the electronic device 101, and the order of at least some operations may be changed, performed in parallel, performed as independent operations, or at least Some other operations may be performed complementary to at least some of the operations. According to an embodiment of the present disclosure, operations 801 to 811 may be performed by at least one processor 120 of the electronic device 101.

As shown in FIG. 8, the operation method performed by the electronic device 101 according to one embodiment includes an operation 801 for detecting a designated trigger, an operation 803 for identifying the type of the designated trigger, and an operation 803 for identifying the type of the designated trigger. Operation 805 of determining whether it is a first trigger or a second trigger, operation 807 of performing a first performance test based on a background image based on determining the first trigger, operation 807 of determining whether the second trigger is It may include an operation 809 of performing a second performance test based on the fingerprint image, and an operation 811 of providing a guide corresponding to the results of the performance test (eg, the first performance test or the second performance test).

Referring to FIG. 8 , in operation 801, the processor 120 of the electronic device 101 may detect a designated trigger related to a performance test of the fingerprint sensor 200. According to one embodiment, the processor 120 may detect a designated trigger based on the user's intended situation or the user's unintended situation.

At operation 803, processor 120 may identify the type of trigger specified. According to one embodiment, the designated trigger may include a first trigger and a second trigger. In one embodiment, the first trigger may represent a trigger for performing a first performance test based on a background image acquired in a designated environment (eg, a darkroom environment). In one embodiment, the second trigger may represent a trigger for performing a second performance test based on a fingerprint image acquired in the use environment of the electronic device 101.

According to one embodiment, the processor 120 may determine the usage environment of the electronic device 101 through context awareness based on at least one sensor data of the sensor module 260. According to one embodiment, the processor 120 is configured to use the electronic device 101 in a first performance test environment, such as a dark room environment, with no operation of the electronic device 101, and/or upon a user's request to run the first performance test. The type of trigger specified based on the second performance test environment, such as the state in which the fingerprint sensor 200 operates, the state in which a fingerprint image can be acquired by the fingerprint sensor 200, and/or the user's request to execute the second performance test. can be identified.

At operation 805, the processor 120 may determine whether the specified trigger is a first trigger or a second trigger. According to one embodiment, the processor 120 may determine the first trigger based on a situation recognition result related to the first performance test environment. According to one embodiment, the processor 120 may determine the second trigger based on a situation recognition result related to the second performance test environment.

At operation 807, processor 120 may perform a first performance test based on a background image based on determining the first trigger. According to one embodiment, the processor 120 may acquire a background image and perform a first performance test based on comparative analysis between the acquired background image and set first reference data (eg, reference background image). An operation for performing a first performance test according to an embodiment will be described with reference to the drawings described later.

At operation 809, processor 120 may perform a second performance test based on the fingerprint image based on determining the second trigger. According to one embodiment, the processor 120 may acquire a fingerprint image and perform a second performance test based on comparative analysis between the acquired fingerprint image and set second reference data (eg, reference map). An operation for performing a second performance test according to an embodiment will be described with reference to the drawings described later.

In operation 811, the processor 120 may provide a guide corresponding to the performance test results. According to one embodiment, the processor 120 may provide various guides for performance test results as described in the description referring to operation 711 of FIG. 7.

According to an embodiment of the present disclosure, the electronic device 101 is configured to suppress an external light source that comes from outside and operates as noise, such as an image acquired in an environment similar to the environment during correction work in the production stage (e.g., background). It is possible to detect a protective material based on an image) and determine whether the recognition performance of the fingerprint sensor 200 changes accordingly. For example, the electronic device 101 performs a performance test (e.g., the first performance test in FIG. 8) on the fingerprint sensor 200 using an image (e.g., background image) acquired in a designated environment such as a darkroom environment. It can be done. Below, an example of how to perform a first performance test based on a background image will be described.

FIG. 9 is a flowchart illustrating a method of operating an electronic device according to an embodiment of the present disclosure.

According to one embodiment, FIG. 9 may show an example of supporting a performance test regarding the recognition performance of the fingerprint sensor 200 in the electronic device 101 according to one embodiment. According to one embodiment, FIG. 9 may show an example of an operation of performing a first performance test based on a background image. According to one embodiment, the operation described in FIG. 9 is, for example, performed heuristically in combination with the operations described in FIGS. 7 and 8, or performed heuristically as a detailed operation of some of the operations described. It can be.

A method of supporting performance testing of the fingerprint sensor 200 in the electronic device 101 according to an embodiment of the present disclosure may be performed, for example, according to the flowchart shown in FIG. 9. The flowchart shown in FIG. 9 is merely a flowchart according to an embodiment of a method for testing the performance of the electronic device 101, and the order of at least some operations may be changed, performed in parallel, performed as independent operations, or at least Some other operations may be performed complementary to at least some of the operations. According to an embodiment of the present disclosure, operations 901 to 911 may be performed by at least one processor 120 of the electronic device 101.

As shown in FIG. 9, the operation method performed by the electronic device 101 according to one embodiment includes an operation 901 of detecting a designated trigger and an operation of acquiring a background image based on the fingerprint sensor 200 ( 903), an operation of acquiring a set reference background image (905), an operation of comparing background images (907), an operation of determining a specified state based on the comparison result (909), and providing a guide corresponding to the specified state. Operation 911 may be included.

Referring to FIG. 9 , in operation 901, the processor 120 of the electronic device 101 may detect a designated trigger (eg, a first designated trigger). According to one embodiment, the processor 120 receives an input (e.g., a performance test service (or application) execution request) from a user explicitly requesting a performance test of the fingerprint sensor 200. A first designated trigger may be detected based on a second situation (e.g., a user's unintended situation) that detects a designated environment (or condition) for testing the performance of the fingerprint sensor 200 (the user's intended situation) or there is.

According to one embodiment, the first performance test for the fingerprint sensor 200 is maintained in an environment identical or similar to that of the calibration kit (e.g., black, an absorbing color groove tool) used in the manufacturing stage of the electronic device 101. It can be performed under the following conditions. For example, the designated environment for the first performance test may be an environment in which ambient external light is below a certain level (eg, a condition in which the electronic device 101 exists in a dark environment such as a dark room).

According to one embodiment, when the processor 120 detects a first designated trigger based on a first situation (user's intended situation), the sensor module 260 (e.g., illuminance sensor 261) in the first situation. You can use to guide users to configure an environment (e.g., darkroom environment) that checks and responds to surrounding external light conditions.

According to one embodiment, the processor 120 periodically uses the sensor module 260 (e.g., the illumination sensor 261, the acceleration sensor 263, or the gyro sensor 265) to determine a specified condition (e.g., ambient external light). Condition, a condition in which there is no movement of the electronic device 101, a condition in which the electronic device 101 is in a resting state (e.g., sleep mode) without operating, a condition in which the display 220 is off, and /or conditions for using a calibration kit or a similar calibration kit used in the manufacturing stage of the electronic device 101), and when detecting a designated condition, a first designated trigger based on the first situation may be detected. .

According to one embodiment, the first situation (e.g., a situation intended by the user) is when the user starts the process of registering (or re-registering) a fingerprint using the electronic device 101. This may include various situations, such as when accumulated fingerprint recognition performance data (e.g., number of recognition failures) exceeds a specified range, and/or when the user directly selects a fingerprint recognition performance test item created in a menu. According to one embodiment, when the processor 120 detects the first situation, it installs a blocking film in the area of the fingerprint sensor 200 to prevent the user from as much ambient light as possible, or performs a performance test in a dark room environment such as a dark night. We can guide you to do this.

According to one embodiment, the processor 120 measures the illuminance of the surrounding environment using the sensor module 260 and/or the camera module 270 according to the user input for the guide, and measures the illuminance environment below a certain level ( Example: From the first designated condition), the second designated condition can be determined using the sensor module 260. In one embodiment, the second designated condition is a condition in which the electronic device 101 is in an idle state without operating, a condition in which there is no movement of the electronic device 101, and a condition in which the display 220 is off. , and/or may include conditions for using a calibration kit used in the manufacturing step of the electronic device 101 or a calibration kit similar thereto.

According to one embodiment, the second situation (e.g., a situation not intended by the user) is, for example, a case where the electronic device 101 is in a dark room condition, for example, a situation without sunlight or indoor or outdoor lighting. It can be. According to one embodiment, the processor 120 may determine the first specified condition using the sensor module 260. According to one embodiment, the processor 120 may determine the third designated condition using the sensor module 260 from the first designated condition. In one embodiment, the third specified condition is a condition in which the electronic device 101 is in an idle state without operating, a condition in which there is no movement of the electronic device 101, a condition in which the electronic device 101 is charging, and a display. is off condition, a condition in which the electronic device 101 is repeatedly located in a location (e.g., home) more than a certain number of times and/or at a certain time (e.g., a condition in which the electronic device 101 is located in a place that is repeated every day, such as the user's home) ), and/or may include a condition after a certain point in time after the user sufficiently enters a sleep state in conjunction with the sleep function of an application (eg, a health care application) installed on the electronic device 101.

In one embodiment, the second designated condition and the third designated condition may be set the same or different.

In operation 903, the processor 120 may obtain a background image based on the fingerprint sensor 200. According to one embodiment, the processor 120 may detect a first designated trigger based on the first situation or the second situation, and the fingerprint sensor 200 to capture a background image based on detecting the first designated trigger. ) can be controlled. According to one embodiment, the fingerprint sensor 200 may capture at least one background image based on the control of the processor 120. According to one embodiment, the processor 120 may obtain at least one background image from the fingerprint sensor 200.

According to one embodiment, the processor 120 configures an environment identical/similar to the environment in which a calibration kit (e.g., an absorbing home tool) is used in the calibration operation performed in the manufacturing stage of the electronic device 101, and then performs a specified number of A background image can be obtained. According to one embodiment, the processor 120 may control the operation of the fingerprint sensor 200 to acquire a plurality of background images at designated regular time intervals. The operation of acquiring a background image according to an embodiment will be described with reference to FIG. 10, which will be described later.

In operation 905, the processor 120 may obtain a set reference background image. According to one embodiment, the reference background image may include a background image acquired using a specified correction kit (e.g., an absorbing color groove tool (e.g., black)) according to a correction operation at the manufacturing stage of the electronic device 101. You can. According to one embodiment, the reference background image may be set (or stored) in the memory 130 during the manufacturing stage. According to one embodiment, the processor 120 may extract a reference background image (e.g., first reference data) from the memory 130 among various set reference data in response to detection of the first trigger related to the first performance test. there is.

At operation 907, processor 120 may compare background images. According to one embodiment, the processor 120 may compare and analyze the acquired background image and the reference background image for the correction operation. A comparative analysis operation of a background image according to an embodiment will be described with reference to the drawings described later.

In operation 909, the processor 120 may determine the specified state based on the comparison result. According to one embodiment, the designated state is a state in which no protective material is attached (e.g., a first designated state that does not affect the performance of the fingerprint sensor 200), or a state in which the protective material is attached but does not affect the performance of the fingerprint sensor 200. A state that has no influence (eg, a second designated state) and/or a state in which a protective material is attached and affects the performance of the fingerprint sensor 200 (eg, a third designated state) may be determined. According to one embodiment, the processor 120 may determine the designated state based on whether a protective material is detected and analyzing the characteristics of the protective material when the protective material is detected. The operation of determining a designated state according to an embodiment will be described with reference to the drawings described later.

At operation 911, processor 120 may provide a guide corresponding to the specified state. According to one embodiment, the processor 120 may provide various guides for performance test results as described in the description referring to operation 711 of FIG. 7.

FIG. 10 is a flowchart illustrating a method of operating an electronic device according to an embodiment of the present disclosure.

According to one embodiment, FIG. 10 may show an example of supporting a performance test regarding the recognition performance of the fingerprint sensor 200 in the electronic device 101 according to one embodiment. According to one embodiment, FIG. 10 may show an example of an operation for obtaining a background image during a first performance test. According to one embodiment, the operation described in FIG. 10 is, for example, performed heuristically in combination with the operations described in FIGS. 7 to 9, or performed heuristically as a detailed operation of some of the operations described. It can be.

A method of supporting performance testing of the fingerprint sensor 200 in the electronic device 101 according to an embodiment of the present disclosure may be performed, for example, according to the flowchart shown in FIG. 10 . The flowchart shown in FIG. 10 is merely a flowchart according to an embodiment of a method for testing the performance of the electronic device 101, and the order of at least some operations may be changed, performed in parallel, performed as independent operations, or at least Some other operations may be performed complementary to at least some of the operations. According to an embodiment of the present disclosure, operations 1001 to 1009 may be performed by at least one processor 120 of the electronic device 101.

As shown in FIG. 10, an operation method performed by the electronic device 101 according to an embodiment includes an operation 1001 of acquiring a plurality of background images based on the fingerprint sensor 200 and similarity between the background images. An operation of comparing (1003), an operation of determining whether the similarity between the background images satisfies a specified condition (1005), an operation of performing a specified operation if the similarity between the background images does not satisfy the specified condition (1007), and If the similarity between the background images satisfies a specified condition, an operation 1009 of comparing the background images may be included.

Referring to FIG. 10 , in operation 1001, the processor 120 of the electronic device 101 may acquire a plurality of background images based on the fingerprint sensor 200. According to one embodiment, the processor 120 may control the operation of the fingerprint sensor 200 to acquire a background image at designated regular time intervals. According to one embodiment, the fingerprint sensor 200 may operate to capture a background image at certain time intervals designated based on control of the processor 120.

At operation 1003, processor 120 may compare similarity between background images. According to one embodiment, the processor 120 may determine whether there is a change (eg, a difference) between a plurality of background images. For example, the processor 120 may check whether there is a change between background images based at least on shape, brightness, noise, and/or foreign object shape.

At operation 1005, processor 120 may determine whether similarity between background images satisfies a specified condition. According to one embodiment, the processor 120 may determine whether there is a change greater than a specified range between the plurality of background images. According to one embodiment, the processor 120 may determine that a specified condition is not satisfied if there is a change greater than a specified range. According to one embodiment, the processor 120 may determine that a specified condition is satisfied if there is a change below a specified range or if there is no change.

In operation 1005, if the similarity between background images does not satisfy a specified condition (e.g., ‘No’ in operation 1005), the processor 120 may perform the specified operation in operation 1007. According to one embodiment, the processor 120 compares a plurality of background images and stops the performance test when a change in at least one of the shape, brightness, noise, and/or foreign object shape is confirmed between the background images, or You can return to the initial action (e.g. action 1001).

In operation 1005, if the similarity between the background images satisfies a specified condition (e.g., ‘Yes’ in operation 1005), the processor 120 may compare the background images in operation 1009. According to one embodiment, the processor 120 may perform an operation of comparing and analyzing the acquired background image and a set reference background image.

FIGS. 11A and 11B are reference views for explaining an example of detecting a protective material using a background image in an electronic device according to an embodiment of the present disclosure.

According to one embodiment, FIGS. 11A and 11B show a comparative analysis of a background image acquired through the fingerprint sensor 200 and a background image based on a reference background image (or reference data) set during calibration of the fingerprint sensor 200. Examples of actions can be shown. According to one embodiment, FIGS. 11A and 11B may show an example of detecting whether a protective material is attached to the display 220 using a background image during a comparative analysis operation.

According to one embodiment, FIG. 11A may show an example of a reference background image 1100 set when calibrating the fingerprint sensor 200. According to one embodiment, FIG. 11B shows an example of a background image (e.g., the first background image 1110 or the second background image 1120) captured through the fingerprint sensor 200 when performing the first performance test. It can be expressed. According to one embodiment, the first background image 1110 in FIG. 11B is a background image captured through the fingerprint sensor 200 in a state without a protective material (e.g., a protective material not attached to the display 220). An example can be given. According to one embodiment, the second background image 1120 in FIG. 11B is a background image captured through the fingerprint sensor 200 in the presence of a protective material (e.g., with the protective material attached to the display 220). An example can be given.

As illustrated in FIGS. 11A and 11B, it can be confirmed that the first background image 1110 without the protective material is the same as or similar to the reference background image 1100. As illustrated in FIGS. 11A and 11B, it can be confirmed that the second background image 1120 with the protective material is different from the reference background image 1100.

According to one embodiment, when the retarding direction of the protective material is different from the direction of the polarizer 330 in the display 220 of the electronic device 101, there is a change in the background image obtained from the absorbing color groove tool (e.g., black). may occur. For example, the strength of a received signal (e.g., light quantity) may change in a specific direction, thereby changing the characteristics of the output background image. For example, a difference may occur between the reference background image 1100 in a correction operation and the second background image 1120 taken in the presence of a protective material, and the processor 120 provides protection based on the difference between the background images. It is possible to detect whether substances are attached.

According to one embodiment, when a light source passing through a protective material with retarding properties passes through the polarizer 330 of the display 220 again, a change in the image may occur due to signal loss in a specific direction. Such image change determination may be made based on various methods, such as white ratio, pattern shape, number of patterns, pattern position, major axis/minor axis angle, and/or major axis/minor axis ratio. For example, due to the angle of the retarding and the polarizer 330, when a protective material is attached, dark diagonal stripes 1100 may occur, as shown in the example of the second background image 1120.

FIG. 12 is a reference diagram illustrating an example of signal characteristics of a fingerprint sensor depending on the presence or absence of a protective material in an electronic device according to an embodiment of the present disclosure.

According to one embodiment, FIG. 12 may be a graph showing changes in signal characteristics of the fingerprint sensor 200 depending on whether a protective material is attached on the display 220. According to one embodiment, the first graph 1210 may represent a change in signal characteristics of the fingerprint sensor 200 when a protective material is attached. According to one embodiment, the second graph 1220 may represent a change in signal characteristics of the fingerprint sensor 200 in a state in which no protective material is attached.

According to one embodiment, the performance of the fingerprint sensor 200 may be related to signal characteristics of the fingerprint sensor 200. For example, signal characteristics can be obtained by using a home tool with a specific pattern (e.g., apricot color) during the calibration process of the fingerprint sensor 200, based on how well the white and black patterns in the image are distinguished. It can be quantified. For example, signal characteristics may have a clear white/black border in the absence of a protective material. For example, the signal characteristic becomes darker in the presence of a protective material, in the valleys responsible for the location of light reflection from external objects (e.g. home tools with a certain pattern (e.g. apricot color) or fingerprints), and whiter. /The black border may appear relatively blurred. For example, it may act as a factor in deteriorating the image quality of the fingerprint sensor 200. According to one embodiment, if the signal characteristics of an image are profiled, it may appear as shown in the example of FIG. 12.

As shown in FIG. 12, it can be confirmed that the signal characteristics in the state where the protective material is attached appear to be deteriorated. According to one embodiment, the reason why signal characteristics are degraded is because the specific retarding direction of the protective material does not match the polarization direction of the display 220, or the thickness of the protective material is so thick that the focal distance of the fingerprint sensor 200 is reduced. Changes may occur.

FIG. 13 is a reference diagram illustrating an example of an image processing result of an image acquired through a fingerprint sensor in an electronic device according to an embodiment of the present disclosure.

According to one embodiment, FIG. 13 shows an image acquired in a correction operation (e.g., the reference background image 1100 in FIG. 11a) and an image respectively acquired according to the presence or absence of a protective material (e.g., the first background image in FIG. 11b). After calculating the difference between (1110) and the second background image (1120), an example of the result of performing image processing (e.g., the first result (1310) or the second result (1320)) is shown. It can be expressed.

According to one embodiment, the first result 1310 may represent an example of the result of performing image processing after obtaining the difference between the reference background image 1100 and the first background image 1110 in a state without a protective material attached. there is. According to one embodiment, the second result 1320 may represent an example of the result of performing image processing after obtaining the difference between the reference background image 1100 and the second background image 1120 with the protective material attached. .

According to one embodiment, the processor 120 may obtain a pattern without a pattern or with a specific shape through image processing on an image. According to one embodiment, a pattern having a specific shape may appear in various forms depending on an image processing method. For example, when comparing the reference background image 1100 and the second background image 1120, a specific pattern (eg, a peanut shape) may be extracted as the second result 1320.

According to one embodiment, the image acquired in a state without a protective material attached (e.g., the first background image 1110) may be the same as the image acquired in the correction operation (e.g., the reference background image 1100), and the image When image processing is performed to find the difference between the images, as illustrated in the first result 1310 of FIG. 13, the entire image may be expressed in a single color (e.g., black) and there may be no pattern.

According to one embodiment, an image acquired with a protective material attached (e.g., the second background image 1120) may differ from an image acquired during a correction operation (e.g., the reference background image 1100). , when image processing is performed to obtain differences between images, a specific pattern (e.g., peanut shape) may appear, as illustrated in the second result 1320 of FIG. 13. For example, the first result 1310 and the second result 1320 of FIG. 13 include the angle between the retarding direction of the protective material and the polarization direction of the polarizer 330 of the display 220, and the properties of the protective material (e.g. : This may be a result of thickness characteristics).

According to one embodiment, the processor 120 may detect whether a protective material is attached based on the pattern as illustrated in FIG. 13. In one embodiment, detection of attachment of a protective material may be performed by various methods. According to one embodiment, the processor 120 determines the ratio of a specific color (e.g., white, pixel value), the shape of a specific pattern, the angle that the major axis or minor axis of a specific pattern has, the ratio of the minor axis to the major axis of a specific pattern, and the specific pattern. Whether or not the protective material is attached may be detected based at least on the number and/or location of a specific pattern. In one embodiment, the specific pattern may represent an image pattern that includes a combination of the retarding properties of the protective material and the properties of the polarizer 330 inside the display 220.

According to one embodiment, the processor 120 may determine various states using an image on which image processing has been performed (hereinafter, ‘processed image’). According to one embodiment, if the processed image is monochromatic (e.g., entirely black, as in the first result 1310), no protective material is attached because it does not affect the signal quality of the fingerprint sensor 200. , it can be determined that the recognition performance of the fingerprint sensor 200 is not affected even in the attached state. For example, if the retarding direction and the polarizer direction match, no signal loss occurs and SNR deterioration due to the protective material may not occur. According to one embodiment, when the processed image has a specific pattern (e.g., a white pattern on a black background as in the second result 1320), the signal quality of the fingerprint sensor 200 is degraded, and the fingerprint sensor 200 ) can be judged to be able to deteriorate the recognition performance.

According to one embodiment, the processor 120 may identify (or distinguish) additional characteristics (or characteristics) of the protective material based on the image pattern that degrades signal quality. Examples of this are shown in Figures 14 to 16.

FIG. 14 is a reference diagram illustrating an example of a change in the characteristics of a protective material in an electronic device according to an embodiment of the present disclosure.

According to one embodiment, Figure 14 may represent an example of a change in characteristics of a processed image depending on the thickness of the protective material. According to one embodiment, the processor 120 may determine the first characteristic (eg, thickness information) of the protective material based on the processed image.

According to one embodiment, a change in thickness can be confirmed through a change in the middle width of the image pattern in the processed image. According to one embodiment, in Figure 14, example <1401> represents an example where the protective material has a first thickness (e.g., a base thickness), and example <1403> shows an example where the protective material has a second thickness (e.g., a basic thickness x 2). ), example <1405> represents an example where the protective material has a third thickness (e.g., basic thickness x 3), and example <1407> represents an example where the protective material has a fourth thickness (e.g., basic thickness x 4). Example <1409> represents an example where the protective material has a fifth thickness (e.g., basic thickness x 5), and Example <1411> represents an example where the protective material has a sixth thickness (e.g., basic thickness x 6). , and example <1413> may represent an example in which the protective material has a seventh thickness (e.g., basic thickness x7).

As illustrated in FIG. 14 , it can be seen that as the thickness of the protective material increases (e.g., in the direction of progress from example <1401> to example <1413>), the middle width 1400 of the image pattern becomes thinner. For example, due to a change in the thickness of the protective material, a change (or difference) in the physical distance between the retarding layer of the protective material and the polarizer 330 of the display 220 may occur. According to one embodiment, the intermediate width of the image pattern may become thinner due to signal overlap and/or phase change due to a change in physical distance. According to one embodiment, the processor 120 uses the value of the median width of the image pattern in the processed image to infer the characteristics (e.g., thickness information (or characteristics)) of the protective material attached to the display 220. can do. According to one embodiment, the value of the intermediate width of the image pattern may be the width through the absolute number of pixels, or may be a ratio to the length in the long axis direction.

FIG. 15 is a reference diagram illustrating an example of a change in the characteristics of a protective material in an electronic device according to an embodiment of the present disclosure.

According to one embodiment, FIG. 15 may represent an example of a change in the characteristics of a processed image depending on a difference in polarization characteristics (e.g., direction) between the protective material and the display 220. According to one embodiment, in the case of a protective material, the retarding angle may generally be random, and the level of SNR degradation may be different. For example, the output image angle may change due to changes in retarding characteristics and polarizer characteristics. According to one embodiment, the processor 120 may determine a second characteristic of the protective material (eg, an image output angle depending on the attachment angle of the protective material) based on the processed image. For example, the processor 120 may predict the level of SNR degradation according to a change in the attachment angle of the protective material.

According to one embodiment, in FIG. 15 , element (A) may represent an example of the direction 1510 of the polarizer 330 of the display 220 and the retarding direction 1520 of the protective material. According to one embodiment, in FIG. 15 , element (B) may represent an example of an image processed according to the angle difference between the direction 1510 of the polarizer 330 and the retarding direction 1520. For example, example <1501> may represent an example where the direction 1510 of the polarizer 330 and the retarding direction 1520 of the protective material match. For example, example <1503> represents an example where about a first angle difference (e.g., about 22.5 degrees difference) occurs between the direction 1510 of the polarizer 330 and the retarding direction 1520 of the protective material. You can. For example, example <1505> represents an example where about a second angle difference (e.g., about 45 degrees difference) occurs between the direction 1510 of the polarizer 330 and the retarding direction 1520 of the protective material. You can.

According to one embodiment, the processor 120 may check the polarization characteristics (eg, polarization direction) and signal characteristics of the protective material through the entire angle of the processed image. According to one embodiment, when the retarding direction of the protective material and the polarization direction of the display 220 meet at a specific angle, the long axis direction angle of the combined processed image may change. According to one embodiment, in the case of a general protective material, polarization characteristics may appear in a certain direction or retarding characteristics may appear in a random direction depending on the manufacturing method. According to one embodiment, the processor 120 may infer the retarding direction 1520 of the protective material according to the processed image. According to one embodiment, the processor 120 may predict the ultimately reduced signal level (e.g., SNR deterioration level) through the difference between the retarding direction 1520 and the polarization direction 1510 of the polarizer 330.

FIG. 16 is a reference diagram illustrating an example of distinguishing characteristics of a protective material in an electronic device according to an embodiment of the present disclosure.

According to one embodiment, FIG. 16 is a diagram for distinguishing the characteristics of a protective material (e.g., a first type protective material, a second type protective material, or a third type protective material) based on the white area of the image pattern in the processed image. An example can be given.

According to one embodiment, in FIG. 16 , example <1601> may represent an example of a processed image of an image acquired with a first type protective material (eg, a general type protective material) attached. According to one embodiment, in FIG. 16 , example <1603> may represent an example of a processed image of an image acquired with a second type protective material (eg, a privacy type protective material) attached. According to one embodiment, in FIG. 16, example <1605> may represent an example of an image obtained by processing a third type protective material (eg, an AR coating type protective material).

According to one embodiment, the processor 120 may distinguish the characteristics (eg, type) of the protective material based on the white area (eg, number of white pixels) in the processed image. For example, the processor 120 may distinguish between a special protection material such as a privacy type or an AR coating type and a general type protection material based on the white area in the processed image.

According to one embodiment, a specially treated protective material such as a privacy type or AR coating type, rather than a general protective material, may change not only polarization characteristics but also transmission characteristics. Accordingly, in specially treated protective materials, diffuse reflection may occur internally and noise transmitted to the fingerprint sensor 200 may increase. Here, since the diffusely reflected light is incident in a random direction rather than a vertical direction compared to a general-type protective material, a larger area may appear in the processed image, as illustrated in FIG. 16.

According to one embodiment, the processor 120 may classify the protective material as a special protective material when a relatively large area compared to the area appearing in a general type protective material appears in the processed image more than a specified standard.

According to one embodiment, the signal component of a general protective material can be reduced by up to about 50% due to the retarding characteristic, and the SNR can be reduced by up to about 50% because the noise is constant. According to one embodiment, the SNR of a specially treated protective material may be reduced by up to about 90% due to changes in transmission characteristics, increased noise components, decreased signal components, and increased noise.

According to one embodiment, the processor 120 may classify the protective material as a special protective material when a relatively large area compared to the area appearing in a general type protective material appears in the processed image more than a specified standard. For example, the processor 120 may distinguish between a general protective material and a special protective material based on the white area in the processed image.

According to an embodiment of the present disclosure, the electronic device 101 detects a protective material based on an image (e.g., a fingerprint image) acquired while the user is actually using the electronic device 101, rather than a background image, It is possible to determine whether the recognition performance of the fingerprint sensor 200 changes accordingly. For example, in order to acquire a background image, a performance test for the fingerprint sensor 200 (e.g., suppresses an external light source that comes from outside and operates as noise) in an environment similar to the environment used during the correction work in the production stage. The first performance test in FIG. 8) can be performed.

According to an embodiment of the present disclosure, in the image acquired while the user directly uses the electronic device 101, the user's finger may shield the image acquisition area of the fingerprint sensor 200 from an external light source. According to one embodiment, the electronic device 101 performs a performance test on the fingerprint sensor 200 (e.g., the second performance of FIG. 8) using an image (e.g., a fingerprint image) acquired without restrictions on surrounding external light sources. test) can be performed. Below, an example of how to perform a second performance test based on a fingerprint image will be described.

FIG. 17 is a flowchart illustrating a method of operating an electronic device according to an embodiment of the present disclosure.

According to one embodiment, FIG. 17 may show an example of supporting a performance test regarding the recognition performance of the fingerprint sensor 200 in the electronic device 101 according to one embodiment. According to one embodiment, FIG. 17 shows an example of an operation of performing a second performance test based on a fingerprint image. According to one embodiment, the operation described in FIG. 17 is, for example, performed heuristically in combination with the operations described in FIGS. 7 to 10, or performed heuristically as a detailed operation of some of the operations described. It can be.

A method of supporting a performance test of the fingerprint sensor 200 in the electronic device 101 according to an embodiment of the present disclosure may be performed, for example, according to the flowchart shown in FIG. 17. The flowchart shown in FIG. 17 is merely a flowchart according to an embodiment of the performance testing method of the electronic device 101, and the order of at least some operations may be changed, performed in parallel, performed as independent operations, or at least Some other operations may be performed complementary to at least some of the operations. According to an embodiment of the present disclosure, operations 1701 to 1711 may be performed by at least one processor 120 of the electronic device 101.

As shown in FIG. 17, the operation method performed by the electronic device 101 according to one embodiment includes an operation 1701 of detecting a designated trigger and an operation of acquiring a fingerprint image based on the fingerprint sensor 200 ( 1703), an operation to obtain a set reference map (1705), an operation to compare images (1707), an operation to determine a specified state based on the comparison result (1709), and an operation to provide a guide corresponding to the specified state ( 1711).

Referring to FIG. 17 , in operation 1701, the processor 120 of the electronic device 101 may detect a designated trigger (eg, a second designated trigger). According to one embodiment, the processor 120 receives a first situation (e.g., a user's request to execute a performance test service (or application)) where the user explicitly requests a performance test of the fingerprint sensor 200. A second designated trigger may be detected based on an intended situation) or a second situation detecting an environment (or condition) designated for performance testing of the fingerprint sensor 200 (e.g., an unintended situation of the user).

According to one embodiment, the second performance test for the fingerprint sensor 200 may be performed in a designated environment in which the user uses the electronic device 101. For example, the designated environment for the second performance test includes an environment in which the user uses the fingerprint sensor 200 (e.g., a fingerprint-based authentication operation, a fingerprint registration/re-enrollment operation, or a request to run a performance test by the user). can do.

According to one embodiment, the processor 120 generates a second designated trigger based on a first situation (the user's intended situation), such as the user executing a performance test of the fingerprint sensor 200 on the electronic device 101. It can be sensed. According to one embodiment, the processor 120 is configured to perform a performance test in a second situation (a situation unintended by the user), such as when the user performs a fingerprint-based authentication operation or performs a fingerprint registration/re-enrollment operation. Can guide users. According to one embodiment, processor 120 may detect a second designated trigger based on the first situation or the second situation.

According to one embodiment, the first situation (e.g., the situation intended by the user) is when the user directly selects a fingerprint recognition performance test item created in the menu using the electronic device 101. Can include situations where tests are explicitly run. According to one embodiment, the second situation (e.g., a situation unintentional by the user) is when the user initiates the process of registering (or re-enrolling) a fingerprint using the electronic device 101, and/or In situations where the user does not explicitly run a performance test and routinely uses the electronic device 101, such as when the accumulated fingerprint recognition performance data (e.g., number of recognition failures) in 101 exceeds a specified range, It may include situations where a specified environment is detected.

In operation 1703, the processor 120 may acquire a fingerprint image based on the fingerprint sensor 200. According to one embodiment, the processor 120 may detect a second designated trigger based on the first situation or the second situation, and the fingerprint sensor 200 to take a fingerprint image based on detecting the second designated trigger. ) can be controlled. According to one embodiment, the fingerprint sensor 200 may capture at least one fingerprint image based on the control of the processor 120. According to one embodiment, the processor 120 may obtain at least one fingerprint image from the fingerprint sensor 200.

According to one embodiment, the processor 120 may acquire a specified number of fingerprint images. According to one embodiment, the processor 120 may control the operation of the fingerprint sensor 200 to acquire a plurality of fingerprint images at designated time intervals. The operation of acquiring a fingerprint image according to an embodiment may include, for example, an operation of acquiring a plurality of fingerprint images in the same or similar manner as the operation described in the description with reference to FIG. 10 of the drawing above. You can.

In operation 1705, the processor 120 may obtain a set reference map. According to one embodiment, the reference map may include numerical data obtained by quantifying (e.g., binarizing/scaling) an image pattern based on a fingerprint image averaged during the manufacturing stage of the electronic device 101. According to one embodiment, the reference map may be set (or stored) in the memory 130 at the manufacturing stage. According to one embodiment, the processor 120 may extract a reference map (e.g., second reference data) from the memory 130 among various set reference data in response to detection of a second trigger related to the second performance test. . In one embodiment, this is explained with reference to the drawings described below in relation to a reference map.

At operation 1707, processor 120 may compare images. According to one embodiment, the processor 120 may compare and analyze the acquired fingerprint image and a reference map corresponding to a set reference fingerprint image. A comparison and analysis operation of images according to an embodiment will be described with reference to the drawings described later.

In operation 1709, the processor 120 may determine the specified state based on the comparison result. According to one embodiment, the designated state is a state in which no protective material is attached (e.g., a first designated state that does not affect the performance of the fingerprint sensor 200), or a state in which the protective material is attached but does not affect the performance of the fingerprint sensor 200. A state that has no influence (eg, a second designated state) and/or a state in which a protective material is attached and affects the performance of the fingerprint sensor 200 (eg, a third designated state) may be determined. According to one embodiment, the processor 120 may determine the designated state based on whether a protective material is detected and analyzing the characteristics of the protective material when the protective material is detected. The operation of determining the designated state according to one embodiment may be, for example, determined in the same manner as described in the description with reference to FIGS. 11 to 16 described above.

In operation 1711, the processor 120 may provide a guide corresponding to the specified state. According to one embodiment, the processor 120 may provide various guides for performance test results as described in the description referring to operation 711 of FIG. 7.

FIG. 18 is a flowchart illustrating a method of operating an electronic device according to an embodiment of the present disclosure.

According to one embodiment, FIG. 18 may show an example of supporting a performance test regarding the recognition performance of the fingerprint sensor 200 in the electronic device 101 according to one embodiment. According to one embodiment, FIG. 18 may show an example of an operation of comparing a reference map and conversion data (eg, image pattern) of a fingerprint image during a second performance test based on a fingerprint image. According to one embodiment, the operation described in FIG. 18 is performed heuristically, for example, in combination with the operations described in FIGS. 7 to 10 and/or FIG. 17, or the details of some of the described operations are reduced. It can be performed heuristically as an action.

A method of supporting a performance test of the fingerprint sensor 200 in the electronic device 101 according to an embodiment of the present disclosure may be performed, for example, according to the flowchart shown in FIG. 18. The flowchart shown in FIG. 18 is merely a flowchart according to an embodiment of the performance testing method of the electronic device 101, and the order of at least some operations may be changed, performed in parallel, performed as independent operations, or at least Some other operations may be performed complementary to at least some of the operations. According to an embodiment of the present disclosure, operations 1801 to 1807 may be performed by at least one processor 120 of the electronic device 101.

As shown in FIG. 18, the operation method performed by the electronic device 101 according to one embodiment includes an operation 1801 of acquiring a fingerprint image, an operation 1803 of extracting an image pattern based on the fingerprint image, It may include an operation 1805 of acquiring a set reference map, and an operation 1807 of comparing the image pattern and the reference map.

Referring to FIG. 18, in operation 1801, the processor 120 of the electronic device 101 may acquire a fingerprint image. According to one embodiment, the processor 120 may control the fingerprint sensor 200 to capture a fingerprint image based on detecting a second designated trigger. According to one embodiment, the fingerprint sensor 200 may capture at least one fingerprint image based on the control of the processor 120. According to one embodiment, the processor 120 may obtain at least one fingerprint image from the fingerprint sensor 200.

In operation 1803, the processor 120 may extract an image pattern based on the fingerprint image. According to one embodiment, the processor 120 may extract an image pattern through binarization/spanning of the fingerprint image. For example, the image pattern may include numerical data obtained by binarizing/scaling the image pattern based on the acquired fingerprint image. According to one embodiment, in order to compare the acquired fingerprint image with the reference map, the processor 120 may quantify an image area in a range corresponding to the reference map in the fingerprint image. For example, the processor 120 may convert each block into a specific numerical value using characteristics (e.g., mean, median, variance, and/or standard deviation) of pixel values in the corresponding range in each block.

At operation 1805, processor 120 may obtain a set reference map. According to one embodiment, the reference map of the processor 120 may include numerical data obtained by binarizing/scaling an image pattern based on a fingerprint image averaged during the manufacturing stage of the electronic device 101. According to one embodiment, the reference map may be set (or stored) in the memory 130 at the manufacturing stage. According to one embodiment, the processor 120 may extract a reference map (e.g., second reference data) from memory 130 among various preset reference data in response to detection of a second trigger related to the second performance test. there is.

At operation 1807, processor 120 may compare the image pattern and the reference map. According to one embodiment, the processor 120 may perform image comparison through comparison between an image pattern based on an acquired fingerprint image and a set reference map.

According to one embodiment, the processor 120 may determine whether a protective material is attached based on a result of comparing the image pattern with a reference map. According to one embodiment, the processor 120 may determine whether a protective material is attached based on a similarity value calculated between the image pattern of the acquired fingerprint image and the reference map. According to one embodiment, the processor 120 may compare the calculated similarity value with a set threshold value and determine whether a protective material is attached based on the comparison result. According to one embodiment, when determining the attachment of a protective material, the processor 120 may provide a designated guide to the user. According to one embodiment, the processor 120 may provide a guide to the user with visual information through a message or a designated image.

According to one embodiment, the processor 120 may store collected data (e.g., calculated values, extracted image patterns, and/or block patterns) collected while performing the second performance test in the memory 130. . According to one embodiment, the processor 120 may use the stored collected data to provide data related to a performance test of the fingerprint sensor 200 using a protective material. According to one embodiment, the processor 120 may update a reference map and/or a designated threshold value for improved performance using stored collected data.

FIG. 19 is a reference diagram for explaining an example of detecting a protective material using a fingerprint image in an electronic device according to an embodiment of the present disclosure.

According to one embodiment, FIG. 19 may show an example of a fingerprint image acquired through the fingerprint sensor 200. According to one embodiment, FIG. 19 may show an example of detecting whether a protective material is attached to the display 220 using a fingerprint image during a comparative analysis operation.

According to one embodiment, FIG. 19 shows an example of a fingerprint image (e.g., a first fingerprint image 1910 or a second fingerprint image 1920) captured through the fingerprint sensor 200 when performing a second performance test. It can be expressed. According to one embodiment, the example first fingerprint image 1910 in FIG. 19 is a fingerprint captured through the fingerprint sensor 200 in a state without a protective material (e.g., a protective material not attached to the display 220). Examples of images can be shown. According to one embodiment, the second fingerprint image 1920 in FIG. 19 is a fingerprint image captured through the fingerprint sensor 200 in the presence of a protective material (e.g., with the protective material attached to the display 220). An example can be given.

As illustrated in FIG. 19 , it can be seen that there is a difference between the first fingerprint image 1910 without the protective material and the second fingerprint image 1920 with the protective material.

According to one embodiment, the fingerprint image (e.g., the first fingerprint image 1910 or the second fingerprint image 1920) may be an image acquired during fingerprint authentication by the user using the electronic device 101. there is. According to one embodiment, as illustrated in FIG. 19, a specific pattern as previously described in relation to the background image may appear in the fingerprint image depending on whether a protective material is attached. For example, in the case of the first fingerprint image 1910 obtained without a protective material attached, only a pattern corresponding to the fingerprint may exist. For example, in the case of the second fingerprint image 1920 obtained with a protective material attached, a pattern by the protective material may be further included in the pattern corresponding to the fingerprint. For example, as illustrated in FIG. 19 , in the case of the second fingerprint image 1920, it can be seen that a dark pattern 1900 is further included in the middle of the image.

According to one embodiment, a specific pattern corresponding to a protective material may exist in a fingerprint image obtained when a user routinely uses the electronic device 101. For example, a specific pattern may be created by combining the retarding properties of the protective material and the polarization properties of the display 220, as previously described in relation to the background image.

According to one embodiment, the processor 120 may detect whether a protective material is attached based on a specific pattern in a general use environment of the electronic device 101. According to one embodiment, the electronic device 101 may set a reference map including pattern information and characteristics of a protective material for detecting an image pattern from a fingerprint image and store it in the memory 130. According to one embodiment, the processor 120 may make a numerical determination by comparing an image pattern from a fingerprint image and a reference map. For example, the processor 120 may determine whether the protective material is attached based at least on the difference, similarity, and/or pattern difference between the image pattern and the reference map.

According to one embodiment, the processor 120 may extract an image pattern (eg, image for comparison) for comparison with a reference map from a fingerprint image. According to one embodiment, the image pattern may be extracted based on an area that includes a range in which a specific image pattern displayed by the protective material appears within the acquired fingerprint image. According to one embodiment, the processor 120 may block an area including a range where a specific image pattern appears and extract an image pattern of the same form as the reference map using the blocked image. Examples of this are shown in Figures 20A and 20B.

FIGS. 20A and 20B are reference views showing examples of image patterns extracted based on a reference map and a fingerprint image in an electronic device according to an embodiment of the present disclosure.

According to one embodiment, FIG. 20A may represent an example of a reference map 2010. According to one embodiment, FIG. 20B may show an example of an image pattern 2020 (eg, binarization/ranging pattern) extracted from a fingerprint image.

As illustrated in FIGS. 20A and 20B, the reference map and the image pattern may be composed of a set 2000 of blocks in the form of, for example, an n x m matrix. According to one embodiment, each block in the set of blocks 2000 may be composed of a set of acquired fingerprint image pixels. For example, a x b pixels may be gathered to form one block, and n x m blocks may be gathered to form a reference map and/or image pattern. For example, the reference map may be composed of n x m blocks made up of a set of a x b consecutive pixels (e.g., a 6 x 6 block of 30 x 30 pixels). According to one embodiment, there may be only one reference map, or multiple reference maps with one or more various blocks and the number of pixels per block may be stored and compared with the extracted image pattern.

According to one embodiment, as illustrated in FIG. 20A, the reference map 2010 is a numerical data form containing features of a fingerprint image acquired while a protective material is applied on the display 220 of the electronic device 101. Information expressed as can be displayed. According to one embodiment, a specified value may be input during the manufacturing stage of the electronic device 101. According to one embodiment, the input specified value (e.g., numerical value) may be updated based on various data. According to one embodiment, the specified input value (e.g., numerical value) may be optimized by updating the numerical value according to the user's data. For example, the electronic device 101 may learn user data through machine learning and update numerical values based on the learned user data.

According to one embodiment, the reference map 2010 may have characteristics of a protective material. Accordingly, the more similar the obtained image pattern 2020 is to the image pattern of the reference map 2010, the more likely it is that a protective material is attached.

According to one embodiment, as illustrated in FIG. 20B, the extracted image pattern 2020 may be the same or smaller than the original fingerprint image. According to one embodiment, one block in the extracted image pattern 2020 may be composed of a set of one or more pixels.

According to one embodiment, the value of each block may be binarized (e.g., 0/1, -1,0,1) or a ranged value (e.g., x to y) may be input. According to one embodiment, the value of a block may be determined in various ways, such as the following example. According to one embodiment, the average of all pixels in a block may be obtained and then compared with the average of the block to binarize/range the decision. According to one embodiment, each block can be grouped into a specific number of blocks, the average of the pixels in the corresponding range is calculated, and then the blocks belonging to the range can be binarized/ranged. According to one embodiment, it can be determined by specifying a fixed threshold value and then comparing the average of the pixels and the threshold value for each block, or by comparing each pixel and then binarizing/ranging the block. there is. According to one embodiment, the square of the average value and the difference value of the entire block or some blocks may be determined and binarized/ranged. It is not limited thereto, and according to one embodiment, various other methods may be used in addition to the above examples.

For example, when blocking an image pattern extracted from an original fingerprint image, each pixel in the block is It can be expressed as one number using numerical information and determined by binarizing/scaling the number.

According to one embodiment, the processor 120 may compare the block value with the reference map based on the block value determined by binarizing/scoping as described above. An example of this is shown in Figure 21.

FIG. 21 is a reference diagram illustrating an example of comparing an image pattern based on a fingerprint image with a reference map in an electronic device according to an embodiment of the present disclosure.

According to one embodiment, example <2101> may represent an example of an image pattern extracted from a fingerprint image. According to one embodiment, example <2103> may represent an example of a reference map.

According to one embodiment, the processor 120 may compare the extracted image pattern and the reference map, as illustrated in FIG. 21. According to one embodiment, the processor 120 may determine similarity by comparing an image pattern and a reference map using, for example, a difference value or correlation.

According to one embodiment, the similarity judgment using the difference value may use the difference value of each block value between the reference map and the extracted image pattern. For example, the comparison value can be calculated by combining formulas such as the average, absolute value, total sum, and/or square of the difference values between the corresponding blocks. According to one embodiment, the processor 120 may determine the degree of similarity between the extracted image pattern and the reference map by determining the similarity of the values of each block through the corresponding formula.

According to one embodiment, similarity determination using correlation may use a specified correlation formula. For example, correlation can represent a way to determine the extent to which two populations (e.g., extracted image patterns and reference maps) have similar characteristics. According to one embodiment, the definition of general correlation can be expressed as follows.

As illustrated in <Equation 1>, the correlation r is the standard deviation of each group (e.g. , ) can be obtained by dividing. According to one embodiment, the correlation r may be obtained as a value ranging from -1 to 1. For example, “1” may indicate the same image, and “-1” may indicate an inverted image. According to one embodiment, the processor 120 may determine that the image pattern and the reference map are the same image if the correlation r is "1", and if the correlation r is "-1", the image pattern and the reference map may be determined. It can be judged that these are different images. According to one embodiment, the processor 120 may determine whether a protective material is attached based on the correlation r.

According to one embodiment, a specific threshold may be set, and the processor 120 may determine whether or not the protective material is attached through the calculated correlation r and the specified threshold. For example, in general, if the correlation r is about 0.5 or more, it can be judged to have a high correlation with a similar pattern. For an example of this, please refer to <Table 1>.

Fingerprint1 Fingerprint2 Fingerprint3 Fingerprint4 Fingerprint5 average No Film -0.11 0.27 0.00 -0.23 0.12 0.01 Film No. One 0.89 0.89 0.84 0.84 0.89 0.87 Film No. 2 0.84 0.89 0.95 0.84 0.89 0.88 Film No. 3 0.63 0.61 0.73 0.5 0.5 0.59

According to one embodiment, <Table 1> shows the correlation of five fingerprint images acquired through the fingerprint sensor 200 for each state without a protective material (e.g., a protective film) attached and with another protective material attached. An example can be given. For example, <Table 1> may represent values obtained by binarizing example fingerprint images and then calculating correlation with a specific reference map. As shown in <Table 1>, it can be seen that the three states with the protective material attached have higher correlation values compared to the state without the protective material attached.

According to one embodiment, the correlation value may change depending on the binarization/ranging method of the extracted blocks, as well as the size or number of blocks for each image pattern.

An operation method performed by the electronic device 101 according to an embodiment of the present disclosure includes detecting a designated trigger related to a performance test of a fingerprint sensor formed below the display, and based on detecting the designated trigger, An operation of controlling the fingerprint sensor to acquire an image, an operation of extracting reference data set in the memory of the electronic device, and a protection attachable to the display based on the reference data and a designated image acquired through the fingerprint sensor. An operation of determining a state related to a material, an operation of determining whether the performance of the fingerprint sensor changes based on the state related to the protective material, and an operation of performing a designated operation based on whether the performance of the fingerprint sensor changes. can do.

According to one embodiment, the designated trigger may include a first trigger for performing a first performance test based on a background image acquired in a designated environment. According to one embodiment, the designated trigger may include a second trigger for performing a second performance test based on a fingerprint image acquired in a usage environment of the electronic device.

According to one embodiment, an operation method performed by the electronic device 101 includes identifying a type of a designated trigger based on a performance test environment, and determining that the designated trigger is the first trigger. The method may include performing a first performance test using a background image and performing a second performance test using the fingerprint image based on determining that the specified trigger is the second trigger.

According to one embodiment, the operation of performing the first performance test includes obtaining the background image through the fingerprint sensor in a designated environment, obtaining first reference data set in the memory, the background image and It may include detecting whether the protective material is attached based on comparison between the first reference data. According to one embodiment, the first reference data may include at least one reference background image acquired using a designated calibration kit when calibrating the fingerprint sensor. According to one embodiment, the designated environment may include a similar environment corresponding to an environment during calibration of the fingerprint sensor.

According to one embodiment, the operation of acquiring the background image may include an operation of acquiring a plurality of background images at designated regular time intervals and an operation of determining similarity between the plurality of background images.

According to one embodiment, the operation of performing the second performance test includes acquiring the fingerprint image through the fingerprint sensor, extracting an image pattern based on the fingerprint image, and applying a second standard set in the memory. It may include an operation of acquiring data and an operation of detecting whether the protective material is attached based on comparison between the image pattern and the second reference data. According to one embodiment, the second reference data may include at least one reference map that quantifies an image pattern extracted based on an averaged fingerprint image during calibration of the fingerprint sensor.

According to one embodiment, the operation of extracting the image pattern may include extracting the image pattern by quantifying an image area in a range corresponding to the reference map in the fingerprint image.

According to one embodiment, the operation of determining whether the performance has changed includes determining the presence or absence of a protective material on the display based on the designated image and the reference data. When determining the presence of the protective material, the protective material is determined. It may include an operation of analyzing the characteristics of a material and an operation of determining whether the performance of the fingerprint sensor changes based on the characteristics of the protective material.

The various embodiments of the present disclosure disclosed in the specification and drawings are merely provided as specific examples to easily explain the technical content of the present disclosure and aid understanding of the present disclosure, and are not intended to limit the scope of the present disclosure. Therefore, the scope of the present disclosure should be construed as including all changes or modified forms derived based on the technical idea of the present disclosure in addition to the embodiments disclosed herein.

101: Electronic devices
120: processor
130: memory
160, 220: Display
176, 260: sensor module
200: Fingerprint sensor

Claims (20)

In the electronic device 101,
display(160, 220);
a fingerprint sensor 200 located below the displays 160 and 220;
memory (130); and
and a processor 120 operatively connected to the displays 160 and 220, the fingerprint sensor 200, and the memory 130, wherein the processor 120 includes,
Detect a designated trigger related to a performance test of the fingerprint sensor 200,
Based on detecting the designated trigger, control the fingerprint sensor 200 to acquire a designated image,
Extracting the reference data set in the memory 130,
Based on the designated image acquired through the fingerprint sensor 200 and the reference data, determine a state related to a protective material attachable to the display 160, 220,
Based on the state related to the protective material, determine whether the performance of the fingerprint sensor 200 changes, and
An electronic device configured to perform a designated operation based on whether the performance of the fingerprint sensor 200 changes.
The method of claim 1, wherein the processor:
An electronic device configured to detect designated triggers associated with said performance test in a user's intended situation or a user's unintended situation.
The method of claim 1, wherein the designated trigger is:
A first trigger for performing a first performance test based on a background image acquired in a specified environment, and
An electronic device comprising a second trigger for performing a second performance test based on a fingerprint image acquired in a usage environment of the electronic device.
The method of claim 3, wherein the processor:
Identify the type of trigger specified based on the performance test environment,
Based on determining that the designated trigger is the first trigger, perform a first performance test using the background image,
An electronic device configured to perform a second performance test using the fingerprint image based on determining that the designated trigger is the second trigger.
The method of any one of claims 1 to 4, wherein the processor:
Obtaining the background image through the fingerprint sensor in a designated environment,
Acquire first reference data set in the memory,
is set to detect whether the protective material is attached based on comparison between the background image and the first reference data,
The first reference data includes at least one reference background image obtained using a specified correction kit when calibrating the fingerprint sensor.
The method of claim 5, wherein the processor:
Acquire multiple background images at specified time intervals,
An electronic device configured to determine similarity between the plurality of background images.
The method of claim 5, wherein the processor:
An electronic device configured to acquire a background image in a similar environment corresponding to an environment during calibration of the fingerprint sensor, based on the sensor module of the electronic device.
The method of any one of claims 1 to 4, wherein the processor:
Obtaining the fingerprint image through the fingerprint sensor,
Extracting an image pattern based on the fingerprint image,
Acquire second reference data set in the memory,
is set to detect whether the protective material is attached based on comparison between the image pattern and the second reference data,
The second reference data includes at least one reference map that quantifies an image pattern extracted based on an averaged fingerprint image during calibration of the fingerprint sensor.
The method of claim 8, wherein the processor:
An electronic device configured to extract the image pattern from the fingerprint image by quantifying an image area in a range corresponding to the reference map.
The method of claim 8, wherein the processor:
Calculate similarity between the image pattern and the second reference data,
Compare the calculated similarity with the specified threshold,
An electronic device configured to determine that a protective material is present on the display when the similarity is greater than or equal to the specified threshold.
The method of any one of claims 1 to 4, wherein the processor:
Determine the presence or absence of a protective material on the display based on the designated image and the reference data,
Upon determining that the protective material is present, the properties of the protective material are analyzed,
An electronic device configured to determine whether performance of the fingerprint sensor changes based on the characteristics of the protective material.
The method of claim 11, wherein the processor:
Upon determining the absence of the protective material, determine a first designated state in which there is no change in performance of the fingerprint sensor,
When determining that the protective material is present, determine a second designated state in which there is no change in performance of the fingerprint sensor or a third designated state in which there is a performance change in the fingerprint sensor based on the characteristics of the protective material,
An electronic device configured to provide specified guidance regarding a protective substance in response to a determined specified state.
In a method of operating an electronic device,
An operation 701 of detecting a designated trigger related to a performance test of the fingerprint sensor 200 formed below the display 160, 220;
An operation 703 of controlling the fingerprint sensor 200 to acquire a designated image based on detecting the designated trigger;
An operation 705 of extracting preset reference data from the memory 130 of the electronic device 101;
An operation 707 of determining a state related to a protective material attachable to the display 160 or 220 based on a designated image acquired through the fingerprint sensor 200 and the reference data;
An operation 709 of determining whether the performance of the fingerprint sensor 200 changes based on the state related to the protective material; and
A method including an operation 711 of performing a designated operation based on whether the performance of the fingerprint sensor 200 changes.
The method of claim 13, wherein the designated trigger is:
A first trigger for performing a first performance test based on a background image acquired in a specified environment, and
A method including a second trigger for performing a second performance test based on a fingerprint image acquired in a usage environment of the electronic device.
According to clause 13,
Actions that identify the type of trigger specified based on the performance test environment;
Based on determining that the specified trigger is the first trigger, performing a first performance test using the background image;
A method comprising performing a second performance test using the fingerprint image based on determining that the designated trigger is the second trigger.
The method of any one of claims 13 to 15, wherein the operation of performing the first performance test includes:
An operation of acquiring the background image through the fingerprint sensor in a designated environment,
An operation of acquiring first reference data set in the memory,
Comprising the operation of detecting whether the protective material is attached based on comparison between the background image and the first reference data,
The first reference data includes at least one reference background image obtained using a specified calibration kit during calibration of the fingerprint sensor,
A method wherein the designated environment includes a similar environment corresponding to an environment during a calibration operation of the fingerprint sensor.
The method of claim 16, wherein the operation of acquiring the background image includes:
An operation of acquiring a plurality of background images at specified regular time intervals,
A method comprising determining similarity between the plurality of background images.
The method of any one of claims 13 to 15, wherein the operation of performing the second performance test is:
An operation of acquiring the fingerprint image through the fingerprint sensor,
An operation of extracting an image pattern based on the fingerprint image,
An operation of acquiring second reference data set in the memory,
Comprising the operation of detecting whether the protective material is attached based on comparison between the image pattern and the second reference data,
The second reference data includes at least one reference map that quantifies an image pattern extracted based on an averaged fingerprint image during calibration of the fingerprint sensor.
The method of claim 18, wherein the operation of extracting the image pattern includes:
A method comprising extracting the image pattern from the fingerprint image by quantifying an image area in a range corresponding to the reference map.
The method of any one of claims 13 to 15, wherein the operation of determining whether the performance has changed includes:
An operation of determining the presence or absence of a protective material on the display based on the designated image and the reference data;
upon determining that the protective material is present, analyzing the properties of the protective material;
A method comprising determining whether the performance of the fingerprint sensor changes based on the characteristics of the protective material.

Images