KR102115546B1 - Electronic device, user authentication method thereof, and user biometric information registration method thereof - Google Patents

Abstract

An electronic device, a method for authenticating its user, and a method for registering user biometric information thereof, the composite sensor generates a signal representing an image of an object from light reflected from the surface of the object by infrared rays irradiated on the surface of the object. After the infrared light irradiated on the surface of the object penetrates into the object, it generates a signal representing the spectrum of the object from the light emitted from the surface of the object, and the processor performs image authentication for the object using the signal representing the image of the object. Spoofing using a forged image can be prevented by performing biometric authentication on an object using a signal representing the spectrum of the object.

Description

Electronic device, its user authentication method, and its user biometric information registration method {Electronic device, user authentication method thereof, and user biometric information registration method thereof}

It relates to an electronic device for authenticating a user using the biometrics of the user, a user authentication method thereof, and a method for registering user biometric information thereof.

Conventional portable electronic devices such as smart phones and tablet PCs provide a function of authenticating a user using biometrics such as a user's fingerprint, iris, and face. Recently, a full screen type smartphone having a whole front screen has been generalized to maximize a smartphone screen without increasing the size of the smartphone. Most of the conventional portable electronic devices employ a structure in which a fingerprint sensor in the form of a button is exposed to the outside under the display window on the front. This conventional fingerprint sensor acted as the biggest obstacle in the implementation of the full screen, but the current full screen type smart phone is equipped with a fingerprint sensor on the back of the smart phone or other types of biometrics such as iris recognition, face recognition, etc. The problem is being solved by replacing it with cognitive technology.

When the fingerprint sensor is installed on the back of the smartphone, it is not easy to place the finger on the fingerprint sensor when the user holds the smartphone by hand compared to the case where it is installed on the front. There was a problem that unlocking was impossible. Accordingly, a method of replacing fingerprint recognition with other types of biometric recognition technologies such as iris recognition and face recognition, or mounting fingerprint recognition, iris recognition, and face recognition functions has been gradually generalized. However, the user's iris image or face image can be easily copied to a medium such as paper, silicone, clay, rubber, and the like, and user authentication can be relatively easily passed when attempting user authentication using a forged biometric image. There was. Accordingly, spoofing, such as unlocking a smartphone using a forged biometric image, can be a social problem.

The present invention is to provide electronic devices capable of preventing spoofing using biological images copied onto media such as paper, silicone, clay, and rubber. In addition, it is to provide a user authentication method of the electronic device. In addition, it is to provide a method for registering user biometric information of the electronic device. It is not limited to the technical problems as described above, and another technical problem may be derived from the following description.

An electronic device according to an aspect of the present invention is a display panel for displaying a screen; A light emitting module that irradiates infrared light on the surface of the object; The infrared light irradiated on the surface of the object generates a signal representing the image of the object from the light reflected from the surface of the object, and after the irradiated infrared light penetrates into the object, the light emitted from the surface of the object A composite sensor generating a signal representing the spectrum of the object; And a processor that performs image authentication on the object using a signal representing the image of the object and biometric authentication on the object using a signal representing the spectrum of the object.

The composite sensor uses a plurality of spectroscopic filters to generate a signal representing the spectrum of the object from light emitted from the surface of the object after the infrared light irradiated on the surface of the object penetrates into the object, and the plurality of The spectroscopic filter may transmit or block a plurality of wavelength bands belonging to the wavelength band absorbed in the process of being emitted from the surface of the living body after penetrating the human body in the wavelength band of infrared rays irradiated on the surface of the object. The living body may be a human eye cornea or facial skin.

The composite sensor includes an image sensor that generates a signal representing an image of the object from light reflected from the surface of the object using a plurality of infrared filters; And a spectroscopic sensor that generates a signal representing the spectrum of the object from light emitted from the surface of the object after penetrating into the object using a plurality of spectral filters.

The plurality of infrared filters and the plurality of spectral filters may be arranged in a matrix structure of a two-dimensional plane, and the array surfaces of the plurality of infrared filters and the plurality of spectral filters may form a light receiving surface.

The image sensor generates a signal representing the image of the object from the light reflected from the surface of the object by using a plurality of infrared filters arranged in a matrix structure corresponding to a part of the matrix structure of the two-dimensional plane, and the spectroscopy The sensor signals the spectrum of the object from light emitted from the surface of the object after penetrating into the object using a plurality of spectral filters arranged in a matrix structure corresponding to different parts of the matrix structure of the two-dimensional plane Can generate

The image sensor includes a plurality of infrared filters arranged in a matrix structure corresponding to a portion of the matrix structure of the two-dimensional plane to filter light reflected from the surface of the object; And a plurality of photoelectric elements disposed under the plurality of infrared filters to convert light filtered by the plurality of infrared filters into electrical signals.

The spectroscopic sensor includes a plurality of spectroscopic filters arranged in a matrix structure corresponding to different parts of the matrix of the two-dimensional plane to filter light emitted from the surface of the object after penetrating into the object; And a plurality of photoelectric elements disposed under the plurality of spectral filters to convert light filtered by the plurality of spectral filters into electrical signals.

A plurality of color filters, the plurality of infrared filters, and the plurality of spectral filters are arranged in a matrix structure of a two-dimensional plane, and the arrangement surface of the plurality of color filters, the plurality of infrared filters, and the plurality of spectral filters is received Can form a face.

Each of the spectroscopic filters may be formed in a form in which metal patterns having a certain shape are periodically arranged to block a wavelength band determined according to the period in light emitted from the surface of the object after penetrating into the object.

The metal patterns of one of the plurality of spectroscopic filters and the metal patterns of the other of the spectroscopic filters are arranged at different periods, and the one and the other of the spectroscopic filters have different wavelength bands. Can be blocked.

Each of the spectroscopic filters includes an upper reflection layer and a lower reflection layer separated from each other; A dielectric layer interposed between the upper reflection layer and the lower reflection layer, wherein at least two materials having different refractive indices are alternately disposed; And a buffer layer disposed between at least one of the upper reflection layer and the lower reflection layer and the dielectric layer, wherein each spectral filter has a relative volume between the two materials in light emitted from the surface of the object after penetrating into the object. The wavelength band determined by the ratio can be passed.

In the dielectric layer, at least two regions having different relative volume ratios between the two materials may exist, and regions having different relative volume ratios of the dielectric layer may pass different wavelength bands.

The spectroscopic sensor includes the plurality of spectroscopic filters; A plurality of band-pass filters arranged one-to-many to the plurality of spectral filters and disposed under the plurality of spectral filters; And a plurality of photoelectric elements disposed one-to-one corresponding to the plurality of band-pass filters and disposed under the plurality of band-pass filters, wherein the plurality of band-pass filters corresponding to the respective spectral filters are filtered wavelength bands of the respective spectral filters. Transmitting different wavelength bands in the, the plurality of infrared filters are arranged in a matrix structure corresponding to a part of the matrix structure of the two-dimensional plane, the plurality of band-pass filters are different parts of the matrix structure of the two-dimensional plane It can be arranged in a matrix structure corresponding to.

The plurality of band-pass filters may be at least two types of infrared filters and infrared filters of the same type.

The electronic device further includes a rear panel disposed under the display panel and having at least one light passing hole, and the composite sensor passes through the display panel and passes through at least one light passing hole of the rear panel. The signal representing the image of the object and the signal representing the spectrum of the object may be generated from the incident light.

The electronic device further includes at least one collimator lens disposed under the display panel to convert light passing through the display panel into parallel light, and the composite sensor passes through the display panel to pass through the at least one collimator lens. Through it, a signal representing the image of the object and a signal representing the spectrum of the object may be generated from the incident light.

The composite sensor is disposed under the display panel, and the composite sensor generates a signal indicative of the image of the object from light passing through the display panel when infrared light irradiated on the surface of the object is reflected from the surface of the object, , After the irradiated infrared light penetrates into the object, it is emitted from the surface of the object to generate a signal representing the spectrum of the object from light passing through the display panel.

A user authentication method for an electronic device according to another aspect of the present invention includes the steps of irradiating infrared rays on the surface of an object; Generating a signal representing an image of the object from light reflected from the surface of the object by infrared rays irradiated on the surface of the object; Performing image authentication on the object using a signal representing the image of the object; Generating a signal representing a spectrum of the object from light emitted from the surface of the object after infrared rays irradiated on the surface of the object penetrate the object; Performing biometric authentication on the object using a signal representing the spectrum of the object; And authenticating a user according to the result of performing the image authentication and the result of performing the biometric authentication.

The step of generating a signal representing the spectrum of the object generates a signal representing the spectrum of the object from light emitted from the surface of the object after penetrating into the object using a plurality of spectroscopic filters, and the plurality of spectra The filter may transmit or block a plurality of wavelength bands belonging to the wavelength band absorbed in the process of being emitted from the surface of the living body after penetrating into a human body among the wavelength bands of infrared rays emitted from the light emitting module.

The user authentication method further includes detecting whether the object is closer than a critical distance, and the step of irradiating the infrared light irradiates infrared light on the surface of the object when the object is closer than a critical distance. You can.

The user authentication method further includes checking whether there is a user authentication request, and the step of irradiating the infrared light may irradiate infrared light on the surface of the object when it is determined that the user authentication request is made.

The user authentication method may include displaying a graphic for guiding the user's eyes or face to be accurately positioned; And displaying an image of the object on the displayed graphic.

The user authentication method further includes a step of checking whether a certain time has elapsed from the start of the step of displaying the graphic, and generating a signal representing the image of the object has been confirmed as having passed. In a case, generating a signal representing an image of the object from light reflected from the surface of the object, and generating a signal representing the spectrum of the object may be performed on the surface of the object when it is determined that the predetermined time has elapsed. The irradiated infrared light can generate a signal representing the spectrum of the object from light emitted from the surface of the object.

According to another aspect of the present invention, there is provided a computer-readable recording medium recording a program for executing the user authentication method on a computer.

According to another aspect of the present invention, a method for registering biometric information of a user of an electronic device includes irradiating infrared rays on the surface of an object; Generating a signal representing an image of the object from light reflected from the surface of the object by infrared rays irradiated on the surface of the object; Generating an image for the object from a signal representing the image of the object; Generating a signal representing a spectrum of the object from light emitted from the surface of the object after infrared rays irradiated on the surface of the object penetrate the object; Generating a spectrum of the object from a signal representing the spectrum of the object; And registering the generated image and the generated spectrum as biometric information of a user of the electronic device.

Generating a signal representing the spectrum of the object generates a signal representing the spectrum of the object from light emitted from the surface of the object after penetrating into the object using a plurality of spectroscopic filters, and the plurality of spectra The filter may transmit or block a plurality of wavelength bands belonging to the wavelength band absorbed in the process of being emitted from the surface of the living body after penetrating the human body in the wavelength band of light irradiated on the object.

The user biometric information registration method includes displaying a graphic for guiding the user's eyes or face to be accurately positioned; And displaying an image of the object on the displayed graphic.

The method for registering user biometric information further includes checking whether a certain period of time has elapsed from the start of the step of displaying the graphic, and generating the signal representing the image of the object is said to have passed. Generating a signal representing the image of the object from light reflected from the surface of the object when it is confirmed, and generating a signal representing the spectrum of the object is performed when the predetermined time has elapsed. A signal representing the spectrum of the object can be generated from light emitted from the surface.

According to another aspect of the present invention, there is provided a computer-readable recording medium recording a program for executing the user biometric information registration method on a computer.

The complex sensor of the electronic device generates a signal representing the image of the object from the light reflected from the surface of the object by the infrared light emitted from the surface of the object. By generating a signal representing the spectrum of the object from the emitted light, it is impossible to authenticate the user using the forged image by image-based image authentication and spectrum-based biometric authentication. As such, spoofing using a forged image copied to a medium such as paper, silicone, clay, rubber, etc. can be prevented by combining image-based image authentication and spectrum-based biometric authentication.

In particular, the composite sensor of the present invention is irradiated on the surface of an object by using a plurality of spectral filters that can be implemented as nanostructures smaller than the size of an infrared filter of a complementary metal oxide semiconductor (CMOS) image sensor instead of a prism or diffraction grating. It is very easy to apply to small electronic devices such as smartphones by generating a signal representing the spectrum of the object from the light emitted from the surface of the object after infrared rays penetrate into the object. Through integration, it is possible to implement a complex sensor very easily.

An object using a plurality of spectral filters that transmit or block a plurality of wavelength bands belonging to a wavelength band absorbed in the process of being emitted from the surface of the body after penetrating into a human body among the wavelength bands of infrared rays irradiated to the object A signal representing the spectrum of is generated and biometric authentication of the user is performed using this signal, so that spoofing using a forged image can be fundamentally blocked. In other words, inanimate media such as paper, silicone, clay, rubber, etc., which are used to forge human biological images, do not have the components of human organism, and the internal tissue of such inanimate media is completely different from the internal manipulation of human organism. As the spectrum generated by irradiating infrared light onto the surface of the biological body and the spectrum generated by irradiating infrared light onto the counterfeit medium have a completely different shape, spoofing using a forged image may be fundamentally blocked.

In addition, the spectrum of infrared radiation emitted back to the surface of the human body after light penetrates into the human body appears differently, so spectrum-based biometric authentication checks whether the object located close to the electronic device is biological or not. It can also provide the ability to check whether an object located close to the device is a true user of the electronic device. In addition to preventing spoofing using media such as paper, silicone, clay, rubber, etc., in which user images are copied when performing image-based image authentication and spectrum-based biometric authentication according to the present invention, spectrum-based biometric authentication is image-based authentication The security of user authentication can be greatly strengthened.

1 is a view showing an example of an electronic device according to an embodiment of the present invention.
FIG. 2 is an example of biometric recognition of the electronic device illustrated in FIG. 1.
3 is an exploded view of the electronic device shown in FIG. 1.
4 is a view showing an example of a cross-section of the electronic device shown in FIG. 2.
5 is a view showing another example of a cross-section of the electronic device shown in FIG. 2.
6 is a view showing another example of a cross-section of the electronic device shown in FIG. 2.
7 is a view showing another example of a cross-section of the electronic device shown in FIG. 2.
8 is a view showing another example of a cross-section of the electronic device shown in FIG. 2.
9 is a view showing another example of a cross-section of the electronic device shown in FIG. 2.
10 is a view showing a cross-section of a human body and a part of the cross-section shown in FIG.
11 is a diagram schematically showing an example of the structure of the composite sensor 5 shown in FIGS. 3-7.
12 is a view schematically showing another example of the structure of the complex sensor 5 shown in FIGS. 3-7.
13 is a diagram schematically showing another example of the structure of the composite sensor 5 shown in FIGS. 3-7.
14 is a cross-sectional view of one embodiment of the composite sensor 5 shown in FIG. 11.
15 is a cross-sectional view of another embodiment of the composite sensor 5 shown in FIG. 11.
16 is a view showing an example of the spectral filter 521 shown in FIGS. 14 and 15.
17 is a graph showing simulation results of transmission characteristics of the spectral filter 521 shown in FIG. 16.
18 is a view showing another example of the spectral filter 521 shown in FIGS. 14 and 15.
19 is a graph showing simulation results of transmission characteristics of the spectral filter 521 shown in FIG. 18.
20 is a diagram schematically showing another example of the structure of the spectroscopic sensor 52 shown in FIGS. 3-7.
21 is a cross-sectional view of another embodiment of the composite sensor 5 shown in FIG. 11.
22 is a cross-sectional view of another embodiment of the composite sensor 5 shown in FIG. 11.
23 is a block diagram of an electronic circuit of the electronic device shown in FIG. 3.
24 is a flowchart of a user authentication method according to an embodiment of the present invention.
25 is a flowchart of a user authentication method according to another embodiment of the present invention.
26 is a flowchart of a method for registering user biometric information according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Embodiments of the present invention to be described below are electronic devices capable of preventing spoofing using biometric images copied to media such as paper, silicone, clay, rubber, etc., a user authentication method of the electronic devices, and users of the electronic devices It relates to a method of registering biometric information. In particular, examples of biometric authentication used in embodiments of the present invention include human iris authentication, face authentication, iris authentication, and the like. The embodiments described below are electronic devices capable of preventing spoofing by simultaneously performing image authentication and biometric authentication for a user's iris, face, etc., and user authentication methods for the electronic devices, and registering user biometric information of the electronic devices It's about how. Hereinafter, such an electronic device may be simply referred to as an “electronic device”.

1 is a view showing an example of an electronic device according to an embodiment of the present invention. Referring to Figure 1, a typical example of an electronic device according to an embodiment of the present invention is a full screen (full screen) type of a smart phone that is mostly the front screen. In recent years, a full-screen type of smartphone has been generally generalized by minimizing the bezel to maximize the screen of the smartphone without increasing the size of the smartphone. Most of the conventional smart phones employ a structure in which a fingerprint sensor in the form of a button is exposed to the outside under the display window on the front. This conventional fingerprint sensor acted as the biggest obstacle in the implementation of the full screen, but the current full screen type smart phone is equipped with a fingerprint sensor on the back of the smart phone or other types of biometrics such as iris recognition, face recognition, etc. The problem is being solved by replacing it with cognitive technology.

When the fingerprint sensor is installed on the back of the smartphone, it is not easy to place the finger on the fingerprint sensor when the user holds the smartphone by hand compared to the case where it is installed on the front. There was a problem that unlocking was impossible. Accordingly, a method of replacing fingerprint recognition with other types of biometric recognition technologies such as iris recognition and face recognition, or mounting fingerprint recognition, iris recognition, and face recognition functions has been gradually generalized. As shown in FIG. 1, the current full-screen smartphone does not have the entire front screen as various sensors, such as a camera, an illumination sensor, and a proximity sensor, are installed at the top of the front. Particularly, in the case of a smart phone that performs biometric recognition such as iris recognition and face recognition using infrared rays, such as the electronic device according to the present embodiment, in addition to the various sensors described above on the front, infrared LEDs (Light Emitting Diode), infrared rays The camera is installed.

FIG. 2 is an example of biometric recognition of the electronic device shown in FIG. 1. 2 (a) shows a state in which the electronic device according to the present embodiment recognizes a user's iris, and FIG. 2 (b) shows a state in which the user's face is recognized. Referring to (a) of FIG. 2, the electronic device according to the present embodiment irradiates infrared rays to the iris of a user, and generates an iris image from light reflected from the iris due to infrared irradiation. Referring to (b) of FIG. 2, the electronic device according to the present embodiment irradiates infrared light to the user's face, and generates a face image from light reflected from the irradiated face. The electronic device has been introduced through various patent literatures, papers, etc. to authenticate the user by checking whether the generated iris image or face image matches the actual iris image or face image of the user stored in the smartphone.

However, since the user's iris image or face image can be easily copied to media such as paper, silicone, clay, rubber, etc., when the user is authenticated using only the iris image or face image, the user's authentication is performed using the forged image. It is easy to pass. In fact, there have been reports of cases where the user's authentication has been reported using a fake eye made by placing a contact lens on the paper after the iris image is printed on the paper. Cases that have passed are also reported. Accordingly, in the embodiments of the present invention described below, spoofing using a forged image becomes impossible by performing image-based image authentication and spectrum-based biometric authentication in parallel.

3 is an exploded view of the electronic devices shown in FIGS. 1 and 2. Referring to FIG. 3, the electronic device shown in FIG. 1 includes a cover glass 1, a touch panel 2, a display panel 3, a rear panel 4, a composite sensor 5, a bracket 6, printing It is composed of a circuit board 7, a light emitting module 8, a battery 9, and a housing 10. 3 is an exploded view of a smart phone corresponding to a typical example of an electronic device according to the present embodiment. According to the embodiment shown in FIG. 3, a composite sensor serving as a spectrometer together with the role of a conventional infrared camera is installed under the display panel 3. As the infrared camera previously installed on the front top disappears from the front top, the ratio of the screen to the front of the electronic device may increase. Unlike the embodiment shown in Figure 3, the composite sensor 5 may be installed on the front top of the electronic device.

In order to prevent this embodiment from being easily understood and to prevent blurring of the features of the present embodiment, FIG. 3 shows a main configuration of a smartphone necessary for understanding the present embodiment. Those of ordinary skill in the art to which this embodiment belongs may understand that another configuration may be added in addition to the configuration illustrated in FIG. 3. For example, a pressure sensitive adhesive (PSA) layer for adhesion between the cover glass 1 and the touch panel 2 may be inserted. The cover glass 1, the touch panel 2, the display panel 3, and the rear panel 4 are shown in a simple square plate shape to reduce the complexity of the drawings, but may be formed in various shapes such as a curved plate shape. On the other hand, some of the components of the electronic device shown in Figure 2 may be manufactured integrally. For example, the touch panel 2 and the display panel 3 may be integrally manufactured. In this case, a pressure sensitive adhesive (PSA) layer may be inserted between the cover glass 1 and the polarizing plate of the display panel 3.

The cover glass 1 is a flat plate made of transparent material such as glass and polymer, and serves to protect the touch panel 2 and the like located therein. The touch panel 2 is disposed under the cover glass 1 to detect a user's touch position using a plurality of touch sensors arranged in a matrix form. The touch panel 2 may be formed of a plurality of touch sensors, a touch driver IC (TDI) for driving them. Touch panels are categorized into capacitive, resistive, optical, infrared, and SAW (Surface Acoustic Wave) according to their touch sensing method. Capacitive touch panels are mainly used in smartphones or tablet PCs (Personal Computers). Is being used. The capacitive touch panel is further classified into an attachment type, an integrated cover window, and an integrated display. The touch panel 2 has a very high transparency so that the screen of the display panel 3 below it can be clearly seen through it.

The display panel 3 is disposed under the touch panel 2 to display a screen using a plurality of pixels arranged in a matrix form. An example of a pixel of the display panel 3 is an OLED (Organic Light Emitting Diode). The display panel 3 includes a plurality of OLEDs arranged in a matrix form, a polarizer disposed on a plurality of OLEDs to polarize light generated from each OLED, a display driver IC (DDI) for driving the plurality of OLEDs, and a plurality of It may be composed of wires that electrically connect the two OLEDs and the DDI, a substrate on which a plurality of OLEDs and wires are placed, and the like. The area between the pixels of the display panel 3 can be transparently formed so that light reflected by an object located close to the cover glass 1 can pass through the display panel 3 and reach the composite sensor 5. You can.

The rear panel 4 is disposed under the display panel 3 to protect the display panel 3 and to dissipate heat of the display panel 3. As such, since the rear panel 4 protects the display panel 3 and serves to dissipate heat of the display panel 3, it is generally made of an opaque material such as a metal material. As described below, at least one light-passing hole for transmitting light passing through the display panel 3 to the composite sensor 5 may be formed on the rear panel 4. The light irradiated to the cover glass 1 from the outside passes through the transparent cover glass 1 and the touch panel 2, and then the transparent area between the pixels of the display panel 3 and at least one of the back panel 4 The composite sensor 5 can be reached by sequentially passing through the light passing hole. When the rear panel 4 is made of a transparent material, such a light passing hole may not be necessary.

The composite sensor 5 is disposed under the display panel 3 and the spectrum of the signal and the object representing the image of the object from the light passing through the cover glass 1, the touch panel 2, and the display panel 3 in sequence. To generate a signal. If the user's eye is placed in the sensing area of the composite sensor 5, the object will be the user's eye, and if the user's iris image is positioned on the copied medium, the object will be such a copy medium. Likewise, if the user's face is placed in the sensing area of the composite sensor 5, the object will be the user's face, and if the user's face image is placed on the copied medium, the object will be such a copy medium. As shown in FIG. 2, the detection area of the composite sensor 5 is the front side of the electronic device in which the user should position the eye or face area so that the composite sensor 5 can accurately detect the user's eyes or face. It means the realm.

The display panel 3 may display an image of the object so that it can be confirmed whether the object is accurately positioned in the sensing area of the composite sensor 5. For example, the display panel 3 may display the user's eye image so that the user can confirm whether the eye is accurately positioned in the sensing area of the composite sensor 5. For example, the display panel 3 may display the user's face image so that the user can confirm whether the face is accurately positioned in the detection area of the complex sensor 5. As described below, the composite sensor 5 may be disposed under the display panel 3 in a structure that is inserted into the rear panel 4, or under the display panel 3 in a structure that is inserted into the bracket 6 It can also be placed on.

The bracket 6 is disposed under the rear panel 4 and supports the rear panel 4 positioned thereon, and serves to fix the printed circuit board 7, battery 9, and the like. The bracket 6 may be made of a metal material, or may be made of a synthetic resin material, and some may be made of a metal material and the rest may be made of a synthetic resin material. Although shown in FIG. 2 in a simple form, it may be formed in various forms according to the external shape and internal structure of the smartphone, or may be integrally formed with the housing 10. As described below, the bracket 6 may be formed with a groove on which the composite sensor 5 can be seated.

The printed circuit board 7 is disposed under the bracket 6, and a number of electronic components such as a processor and a memory are mounted thereon. The printed circuit board 7 may be divided into a plurality and formed. The physical form of the electronic components of the printed circuit board 7 is not related to the features of this embodiment, and thus illustration of these is omitted. The processor of the printed circuit board 7 performs image authentication for the object using a signal representing the image of the object and biometric authentication for the object using a signal representing the spectrum of the object. The image authentication and biometric authentication processor may be an application processor (AP) or a central processing unit (CPU) of a smartphone, or may be a dedicated processor for image authentication and biometric authentication. The image generation and spectrum generation described below may be processed by separate processors called a graphics processing unit (GPU). The processor of the present embodiment may be a processor in which the above-described processors are integrated into one, or may mean a set of the above-described processors.

The light emitting module 8 is composed of an infrared LED and electronic devices for controlling its brightness and the like, and irradiates infrared light on the surface of the object. The light emitting module 8 may be mounted on the printed circuit board 7 or may be connected to the printed circuit board 7 by wires. A hole in which the light emitting module 8 is inserted may be formed in the bracket 6. The area of the touch panel 2, the display panel 3, and the rear panel 4 so that the infrared light generated from the light emitting module 8 passes through the cover glass 1 and can be irradiated to the surface of the object is the cover glass 1 ). The battery 9 supplies power to the touch panel 2, the display panel 3, the composite sensor 5, the printed circuit board 7, and the like. The housing 10 includes a cover glass 1, a touch panel 2, a display panel 3, a rear panel 4, a composite sensor 5, a bracket 6, a printed circuit board 7, By accommodating all or part of the battery 9, it serves to protect while maintaining the coupling structure between them.

When the composite sensor 5 is disposed under the display panel 3, the composite sensor 5 reflects infrared rays irradiated on the surface of the object from the surface of the object, thereby covering the cover glass 1, the touch panel 2, and A signal representing the image of the object is generated from the light that has passed through the display panel 3 in turn, and the infrared light irradiated on the object penetrates into the object and is emitted from the surface of the object to cover glass 1 and touch panel 2 ), And a signal representing the spectrum of the object is generated from light passing through the display panel 3 in sequence. As described above, the composite sensor 5 may be installed on the front top of the electronic device. In this case, light reflected from the surface of the object among the infrared rays irradiated to the object is directly incident on the composite sensor 5, and the composite sensor 5 generates a signal representing an image of the object from the incident light. Similarly, the light emitted from the surface of the object after penetrating the object among the infrared rays irradiated to the object is directly incident on the composite sensor 5, and the composite sensor 5 is a signal representing the spectrum of the object from the incident light Produces Hereinafter, embodiments of the present invention will be described based on an example in which the composite sensor 5 is disposed under the display panel 3. The following description can also be applied to the complex sensor 5 installed on the front top of the electronic device.

4 is a view showing an example of a cross-section of the electronic device shown in FIG. 2. Referring to FIG. 3, the rear panel 4 includes a protective layer 41 protecting the display panel 3 and a heat dissipation layer 42 dissipating heat of the display panel 3. The protective layer 41 may be made of an elastic material such as sponge or rubber to protect the display panel 3. The heat dissipation layer 42 may be made of a metal material such as aluminum or copper having high thermal conductivity in order to dissipate heat of the display panel 3. As such, since the protective layer 41 is generally made of an opaque material, the protective layer corresponding to the upper region of the composite sensor 5 so that light passing through the display panel 3 can reach the composite sensor 5 A light-passing hole is formed in a partial region of (41). The light-passing hole of the protective layer 41 may be empty or may be filled with a transparent material. The light-passing hole may have the same shape as the light-receiving surface of the composite sensor 5 of the composite sensor, that is, the top surface, and the opening area of the light-passing hole for fixing the composite sensor 5 is the top surface of the composite sensor 5 It may be slightly smaller than the area. A hole into which the composite sensor 5 is inserted is formed in the heat dissipation layer 42. The composite sensor 5 can be fixed by being inserted into the hole of the heat dissipation layer 42 and seated on the upper surface of the bracket 6.

5 is a view showing another example of a cross-section of the electronic device shown in FIG. 3. Referring to FIG. 5, unlike the example shown in FIG. 4, the rear panel 4 is formed with a light through hole in the entire thickness, that is, the protective layer 41 and the heat dissipation layer 42, and the bracket 6 ) Is formed with a groove in which the composite sensor 5 can be seated and fixed. As shown in FIG. 5, the composite sensor 5 can be fixed by being inserted into the groove of the bracket 6 and seated on the bottom surface of the groove. 6 is a view showing another example of a cross-section of the electronic device shown in FIG. 4. Referring to FIG. 6, unlike the example illustrated in FIG. 4, the protective layer 41 of the rear panel 4 has pores corresponding to the pixels of the display panel 3 and a plurality of transparent areas between pixels vertically one to one. A plurality of light passing holes of the form are formed.

When the electronic device detects that an object has entered the detection area of the composite sensor 5 in the screen lock mode, it irradiates infrared light to the object. For example, when the electronic device detects that the user's eye or face has entered the detection area of the composite sensor 5 in the screen lock mode, the user's eye or face is irradiated with infrared light. At this time, in addition to infrared rays, visible light by natural light or the like is irradiated to the user's eyes or face. A part of the light irradiated to the user's eyes or face is reflected and is directed toward the cover glass 1 again, and passes through the cover glass 1, the touch panel 2, and the display panel 3 in sequence. The light passing through the display panel 3 reaches the composite sensor 5 through the light passing hole of the rear panel 4. According to the example shown in FIGS. 3-5, the composite sensor 5 passes through the cover glass 1, the touch panel 2, and the display panel 3 in sequence, and at least one light passes through the rear panel 4. From the light incident through the hole, a signal representing an image of the object and a signal representing the spectrum of the object are generated.

4-6, each pixel of the display panel 3 is represented by a black square point. The light through these black blind spots, that is, between pixels of the display panel 3, etc., reaches the composite sensor 5. Due to the limitations of the fine pixel size, the size of each pixel is exaggerated in FIG. 3, and in reality, rather than three paths as shown in FIG. Light reaches the composite sensor 5 through the path. 4-6, the wires of the display panel 3 are omitted. Since such a wire is also an opaque material, light reaches the composite sensor 5 through a transparent area between the pixel and the wire and between the wire and the pixel. Those of ordinary skill in the art to which this embodiment belongs may understand that the light reflected from the object surface may reach the composite sensor 5 in various other modified forms of the example shown in FIGS. 4-6. .

7 is a view showing another example of a cross-section of the electronic device shown in FIG. 3. Referring to FIG. 7, a collimator lens of a convex lens type is inserted into the light passing hole of the rear panel 4. The rear panel 4 shown in FIG. 7 is formed with an optical through hole formed in the entire thickness, that is, the protective layer 41 and the heat dissipation layer 42, and the complex sensor 5 is seated and fixed to the bracket 6 Grooved grooves are formed. The collimator lens 50 is a structure that is inserted into the light passing hole of the rear panel 4 and is disposed under the display panel 3 to vertically pass the light passing through the display panel 3 to the upper surface of the composite sensor 5. It is converted to incident parallel light. As described below, the plurality of infrared filters 511 and the plurality of spectral filters 521 of the composite sensor 5 are arranged in a matrix structure. When light is vertically incident on each of the plurality of infrared filters 511 and the plurality of spectral filters 521, the transmittance or cutoff rate of each filter is maximized, so that an image and spectrum of an object can be more accurately generated. The collimator lens 50 serves to allow light to be vertically incident on each filter.

8 is a view showing another example of a cross-section of the electronic device shown in FIG. 2. Referring to FIG. 8, a plurality of collimator lenses 50 are inserted into the light passing holes of the rear panel 4. FIG. 8 is an enlarged view of a part of a cross-section of the electronic device shown in FIG. 3 so that a change in a light path by each of the plurality of collimator lenses 50 can be seen. The rear panel 4 shown in FIG. 8 is formed with a light-through hole formed in the entire thickness as in the example shown in FIG. 7, and the complex sensor 5 is mounted on the bracket 6 to be fixed. Is formed. However, in the example shown in FIG. 7, one collimator lens 50 is inserted into the light passing hole of the rear panel 4, but in the example shown in FIG. 8, a plurality of light passing holes of the rear panel 4 are provided. The collimator lens 50 is inserted. As shown in FIG. 8, the plurality of collimator lenses 50 may be manufactured as an integral type in which lenses adjacent to each other are connected.

As in the example shown in FIG. 7, when one collimator lens 50 is inserted into the light passing hole of the rear panel 4, of the light passing hole of the rear panel 4 among the light passing through the display panel 3 As the light traveling along the center line enters the central part of the collimator lens 50, the efficiency of parallel light conversion is high, but the light beam far away from the center line of the light passing hole of the rear panel 4 enters the edge part of the collimator lens. Therefore, the parallel light conversion efficiency is low. Accordingly, in the example shown in FIG. 8, a plurality of collimator lenses 50 are inserted into the light passing holes of the rear panel 4 so as to correspond one to one vertically to a plurality of transparent areas between the pixels of the display panel 3 and the pixels. It is. Accordingly, light rays passing through a plurality of pixels and transparent areas between the pixels of the display panel 3 may be incident on the center of each collimator lens.

9 is a view showing another example of a cross-section of the electronic device shown in FIG. 3. Referring to FIG. 9, in the protective layer 41 of the rear panel 4, a plurality of light-transmitting holes having a porous shape corresponding one to one vertically to a plurality of transparent areas between pixels of the display panel 3 and pixels are formed. In addition, the light-passing hole of the heat dissipation layer 42 is formed with one light-passing hole in the form of opening between a plurality of light-passing holes of the protective layer 41. A plurality of collimator lenses 50 are inserted into the light passing holes of the heat dissipation layer 42. FIG. 8 is an enlarged view of a part of a cross section of the electronic device shown in FIG. 2 so that a change in a light path by each of the plurality of collimator lenses 50 can be seen. A plurality of light-passing holes in the protective layer 41 of the rear panel 4 pass through the transparent area between the pixels of the display panel 3 and the pixels in order to improve the efficiency of the parallel light conversion of each collimator lens 50. Each of the collimator lens 50 serves to guide to be incident on the center.

As shown in FIGS. 7-9, when the user positions the eye in the sensing area of the composite sensor 5, infrared light generated in the light emitting module 8 is irradiated to the user's eye. Some of the infrared light irradiated to the user's eye is reflected from the iris surface of the eye, and some penetrate the cornea of the eye and then return to the corneal surface and are emitted from the corneal surface. When the user places the face in the detection area of the composite sensor 5, infrared light generated in the light emitting module 8 is irradiated to the user's face. Some of the infrared light irradiated to the user's face is reflected from the face surface, and some penetrate into the skin of the face and then return to the surface of the face skin and are emitted from the surface of the face skin. In addition to infrared rays, the user's eyes or face are irradiated with natural or artificial light sources, and like infrared rays, some are reflected and some are emitted after penetration. As described above, the light directed toward the cover glass 1 passes through the cover glass 1, the touch panel 2, and the display panel 3 in this order. The light passing through the display panel 3 reaches the composite sensor 5 through at least one collimator lens 50 inserted into the light passing hole of the rear panel 4.

According to the example shown in FIGS. 7-9, the composite sensor 5 passes through the cover glass 1, the touch panel 2, and the display panel 3 in turn, and passes at least one light through the rear panel 4. A signal representing an image of the object and a signal representing the spectrum of the object are generated from light incident through the hole and at least one collimator lens 50. In the example shown in FIGS. 7-9, the collimator lens 50 is shown to cover the entire surface of the composite sensor 5, that is, the light receiving surface, but the collimator lens 50 is a spectroscopic sensor that is a part of the upper surface of the stake sensor 5 The upper surface of 52, that is, only the light receiving surface of the spectroscopic sensor 52 may be covered. In this case, the composite sensor 5 passes through the cover glass 1, the touch panel 2, and the display panel 3 in sequence, and at least one light passing hole of the rear panel 4 and at least one collimator lens ( 50) to generate a signal representing the spectrum of the object from the light incident.

FIG. 10 is a view showing a cross section of a part of a human body and a cross section shown in FIG. 4. 10 (a) shows a part of the cross-section shown in FIG. 4 and a cross-section of the human skin. Due to the size limitation of one drawing, part (a) of FIG. 10 is shown as a cross section of the human skin and a part of the cross section shown in FIG. 4, but in reality, it is far more apart than that shown. Referring to Figure 10 (a), the human skin consists of two layers of the epidermis (epidermis) and the dermis (dermis), and underneath the dermis (blood vessel) passes. Some of the infrared light emitted from the light emitting module 8 to the user's face is reflected from the face surface, and the rest penetrates into the face skin. Infrared rays that penetrate into the skin of the face move through the body and repeat reflections and refractions at different refractive index boundaries. Some of the infrared light that has penetrated into the skin of the face returns to the face surface and is emitted from the face surface. Because the tissue inside a person's skin is different for each individual, the light emitted from the face surface transmits unique and unique spectrum information for each person.

Some of the infrared rays irradiated to the face from the light emitting module 8 are reflected from the surface of the epidermis of the face, some penetrate into the skin, some are absorbed in the two layers of the epidermis and dermis, and the others are scattered. This absorption is mainly due to components in the blood, such as hemoglobin, bilirubin, beta-carotene, and the like. Human face forgery is achieved by copying a user's face image onto an inanimate medium such as paper or silicone. Since the inanimate medium does not have components of human skin such as blood, and the internal tissue of the inanimate medium is completely different from the internal tissue of human skin, the spectrum of light emitted after returning to the surface after it penetrates into the human skin After penetrating into the skin, it returns to the surface of the skin and has a completely different shape from the emitted light spectrum. Fig. 10 (b) shows a cross section of the cornea of the human eye. The cornea consists of six layers: the epitherium, the Bowman's membrane, the stroma, the descemet's membrane, and the endothelium. As with human skin, the components and internal tissues of the cornea are completely different from the components and internal tissues of the inanimate medium, so the spectrum of light emitted back into the surface after penetration into the inanimate medium returns to the surface and then again after penetration into the cornea. It has a completely different shape from the spectrum of emitted light.

The complex sensor 5 generates a signal representing the image of the object from the light passing through the display panel 3 when the infrared light irradiated on the surface of the object located in its detection area is reflected from the surface of the object, and the surface of the object After the infrared ray irradiated on the object penetrates into the object, it returns to the surface of the object and is emitted from the surface of the object to generate a signal representing the spectral characteristics of the object from light passing through the display panel 3. In this way, the composite sensor 5 of this embodiment is reflected from the surface of the object and penetrates into the object in addition to image authentication using the object image generated from the light passing through the display panel 3, and then returns to the surface of the object. Since the body is authenticated by using the characteristics of the spectrum generated from the light emitted from the surface of the display panel 3 and emitted from the surface of the iris, a medium such as paper, silicone, clay, rubber, etc. The spoofing used may be blocked by nature.

A counterfeit image can be created by someone other than the user by unlocking the user's smartphone and copying the user's biometric image to media such as paper, silicone, clay, rubber, etc. to sneak the contents of the smartphone or download a file. . As described above, since the tissue of this medium is completely different from the tissue in the human skin and the corneal tissue, the spectrum generated when the forged image is located in the sensing area of the complex sensor 5 shows that the user's eye or face is a complex sensor ( When it is located in the sensing area of 5), it has a completely different shape from the spectrum generated. Accordingly, even if the forged image passes the image authentication process, it cannot pass the biometric authentication process. Therefore, it is impossible to unlock the smartphone using the forged image.

In addition, as described above, since the tissue inside the human cornea or the tissue inside the skin is different for each individual, the spectrum of infrared rays emitted back to the surface of the body after light penetrates into the human body appears differently for each individual and is spectrum-based. In addition to verifying the presence of an object located close to the electronic device, the biometric authentication can provide a function to confirm whether the object located close to the electronic device is a true user of the electronic device. In addition to preventing spoofing using media such as paper, silicone, clay, rubber, etc., in which user images are copied when performing image-based image authentication and spectrum-based biometric authentication according to this embodiment, spectrum-based biometric authentication is image-based As authentication can be supplemented, the security of user authentication can be greatly enhanced.

The complex sensor 5 of the present embodiment is reflected from the surface of the object, and the image sensor 51 which generates a signal representing the image of the object from light passing through the display panel 3 and the surface of the object after penetration into the object It is composed of a spectroscopic sensor 52 that generates a signal representing a spectrum of an object from light emitted and passed through the display panel 3. When the infrared wavelength is 700 to 900 nm, the image sensor 51 may generate a clear face image as the reflectivity of the human skin is good. Particularly, in this wavelength band, since the contrast pattern changes clearly according to the wavelength change by the pigment cells in the iris, the image sensor 51 can generate a signal representing a clear iris image. In the case of 810 ~ 850nm of the infrared wavelength band 700 ~ 900nm, the penetration depth of the human body is large. Accordingly, when infrared rays of 810 to 850 nm are irradiated to the user's eyes or face, the spectroscopic sensor 52 may generate a signal representing the spectrum of the tissue characteristics of the user's cornea and face skin. In this embodiment, the light emitting module 8 is preferably irradiated with infrared rays of 810 ~ 850nm on the surface of the object.

FIG. 11 is a diagram schematically showing an example of the structure of the composite sensor 5 shown in FIGS. 3-7. In FIG. 11, the abbreviation "I" represents the infrared filter 511, the abbreviation "S" represents the spectroscopic filter 521, and the abbreviation "P" represents the photoelectric elements 512 and 522. The image sensor 51 of this embodiment uses a plurality of infrared filters 511 and a plurality of photoelectric elements 512 to reflect the infrared light irradiated on the surface of the object from the surface of the object and pass through the display panel 3 It generates a signal representing the image of the object. The spectroscopic sensor 52 uses a plurality of spectroscopic filters 521 and a plurality of photoelectric elements 522 to emit infrared light irradiated onto the surface of the object, and then is emitted from the surface of the object to emit the display panel 3. A signal representing the spectrum of an object is generated from the light that has passed. A photodiode is a typical example of each photoelectric element 512 and 522. Accordingly, the abbreviation of the photoelectric element is indicated as "P" in FIG. 10.

The same type of photodiode may be used as the photoelectric element 512 of the image sensor 51 and the photoelectric element 522 of the spectroscopic sensor 52, or different types of photodiodes may be used. The signal representing the image of the object may be a signal converted into a digital signal after the electrical signals output from the plurality of photoelectric elements 512, that is, the plurality of photodiodes 512 are amplified. Similarly, the signal representing the spectrum of the object may be a signal converted to a digital signal after the electrical signals output from the plurality of photodiodes 522 are amplified. Amplification of the electrical signals output from the plurality of photodiodes 512 and 522 and conversion to digital signals may be performed on the integrated sensor and the integrated amplifier and converter, or with separate amplifiers other than the combined sensor 5 It can also be performed by a converter.

Referring to FIG. 11, a plurality of infrared filters 511 of the image sensor 51 and a plurality of spectroscopic filters 521 of the spectroscopic sensor 52 are arranged in a matrix structure of a two-dimensional plane. The array surface of the plurality of infrared filters 511 and the plurality of spectroscopic filters 521 forms a light receiving surface of the composite sensor 5 for light passing through the display panel 3. That is, the plurality of infrared filters 511 are arranged in a matrix structure corresponding to a part of the matrix structure of the two-dimensional plane, and the plurality of spectral filters 521 are of a matrix structure corresponding to other parts of the matrix structure of the two-dimensional plane. Are arranged. Therefore, the light passing through the display panel 3 is a plurality of infrared filters 511 so that light passing through the display panel 3 is vertically incident on each of the plurality of infrared filters 511 and the plurality of spectral filters 521. And it is preferable to be vertically incident on the array surface of the plurality of spectroscopic filters (521). As described above, the technical means for vertically entering the array surfaces of the plurality of infrared filters 511 and the plurality of spectroscopic filters 521 has been described above.

That is, the image sensor 51 of the present embodiment is reflected by the surface of the object using a plurality of infrared filters 511 arranged in a matrix structure corresponding to a part of the matrix structure of the two-dimensional plane passes through the display panel 3 Generates a signal representing an image of an object from one light. The spectroscopic sensor 52 of this embodiment penetrates into an object using a plurality of spectroscopic filters 521 arranged in a matrix structure corresponding to different parts of a two-dimensional planar matrix structure, and then is emitted from the surface of the object to display panel (3) A signal representing the spectrum of the object is generated from the light passing through it.

The plurality of infrared filters 511 of the image sensor 51 are arranged in a matrix structure corresponding to a part of the matrix structure of the two-dimensional plane forming the light-receiving surface of the composite sensor 5, and the light passing through the display panel 3 To filter. The plurality of infrared filters 511 is composed of various types of infrared filters 511 that transmit infrared rays of different wavelength bands from light passing through the display panel. For example, the plurality of infrared filters 511 transmits infrared rays in the 810 to 820 nm bands, and a plurality of infrared filters that block light in the remaining bands, and transmits infrared rays in the 820 to 830 nm bands and blocks the light in the remaining bands. It can be composed of two infrared filters, a plurality of infrared filters that transmit infrared rays in the 830 to 840 nm band and block light in the remaining band, and a plurality of infrared filters that transmit infrared rays in the 840 to 850 nm band and block the light in the remaining band. have.

The plurality of photoelectric elements 512 are arranged in the same matrix structure as the matrix structure of the plurality of infrared filters 511 so as to correspond one-to-one up and down to the plurality of infrared filters 511 and disposed under the plurality of infrared filters 511. The plurality of photoelectric elements 512 converts light filtered by the plurality of infrared filters 511 into electrical signals to generate a signal representing an image of the object from light that is reflected from the surface of the object and passes through the display panel 3. do. Each photoelectric element 512 converts light filtered by each infrared filter 511 that is one-to-one up and down to this, into electrical signals of different levels according to its intensity. An image of an object may be generated from a combination of electrical signals generated by each of the plurality of photoelectric elements 512.

The plurality of spectroscopic filters 521 of the spectroscopic sensor 52 are arranged in a matrix structure corresponding to other parts of the matrix structure of the two-dimensional plane forming the light-receiving surface of the complex sensor 5 and passed through the display panel 3 Filter light. The plurality of spectroscopic filters 521 of this embodiment return to the human biological surface after penetrating into the human biological body in the wavelength band of infrared rays irradiated from the light emitting module 8 to the human biological surface for biometric authentication of the object. Transmitting or blocking a plurality of wavelength bands belonging to a wavelength band absorbed in the process of being emitted from a human biological surface. Here, the human body may be a human eye cornea or facial skin. The plurality of spectral filters 521 are various types of spectral filters 521 that filter the light passing through the display panel 3 by transmitting or blocking infrared rays of different wavelength bands from the light passing through the display panel 3 It consists of, and various types of spectral filters 521 are alternately arranged in a matrix structure as described above.

For example, the various types of spectral filters 521 are filled in one row or one column of a matrix belonging to different parts of a matrix structure of a two-dimensional plane, one for each type, and when the arrangement of all kinds of spectral filters 521 is completed Again, they can be arranged alternately in a manner that they are filled one by one. As described above, the spectroscopic sensor 52 of the present embodiment can detect a spectrum in a band of 810 to 850 nm for biometric authentication of an object located close to an electronic device. In this case, the plurality of spectral filters 521 may be composed of various types of spectral filters 521 that divide and filter the bands of 810 to 850 nm in units of several nm. For example, the plurality of spectral filters 521 may be composed of various types of spectral filters 521 for dividing and filtering 810 to 850 nm bands in 5 nm units.

That is, the plurality of spectral filters 521 is composed of several types of spectral filters 521 that transmit or block different wavelength bands of the 810 ~ 815nm band, 815 ~ 820nm band, ..., 845 ~ 850nm band Can be. The smaller the unit for dividing the band of light passing through the display panel 3, that is, the more types of the plurality of spectroscopic filters 521, the higher the resolution of the spectrum detected by the spectroscopic sensor 52 may be. However, as the number of times that one type of spectral filter 521 that transmits or blocks the same wavelength band is overlapped and arranged is reduced, the accuracy of the spectrum detected by the spectral sensor 52 may be lowered. Therefore, it is preferable that the size of the band division unit for the light passing through the display panel 3 is designed by bridge the resolution and accuracy of the spectrum suitable for biometric authentication of an object located close to the electronic device.

 The plurality of photoelectric elements 522 are arranged in the same matrix structure as the matrix structures of the plurality of spectroscopic filters 521 so as to correspond one to one vertically to the plurality of spectroscopic filters 521 and disposed under the plurality of spectroscopic filters 521. The plurality of photoelectric elements 522 converts light filtered by the plurality of spectral filters 521 into electrical signals, and then penetrates into the object and is emitted from the surface of the object to indicate the spectrum of the object from the light passing through the display panel. Generate a signal. Each photoelectric element 522 converts light filtered by each spectral filter 521 corresponding to it up and down one to one to electrical signals of different levels according to its intensity. One of the plurality of photoelectric elements 522 penetrates into the object and then is emitted from the surface of the object to generate an electrical signal corresponding to a certain wavelength band of light passing through the display panel, and the other photoelectric elements have different wavelength bands. Generates an electrical signal corresponding to. Spectra required for biometric authentication of an object may be generated from a combination of electrical signals generated by each of the plurality of photoelectric elements 522.

The plurality of photoelectric elements 522 may be disposed under the plurality of spectroscopic filters 521 so as to correspond one-to-many to the plurality of spectroscopic filters 521. In this case, filtering light by one spectroscopic filter 521 is incident on a plurality of optoelectronic devices 522 disposed under the spectroscopic filter 521 among the plurality of optoelectronic devices 522. Each of the plurality of photoelectric elements 522 disposed under the spectral filter 521 converts the filtered light into an electrical signal by the spectral filter 521. For example, when the size of each spectral filter 521 is made larger than the size of each photoelectric element 522, the plurality of photoelectric elements 522 are plural such that they correspond one-to-many to the plurality of spectral filters 521. It may be disposed under the spectral filter (521).

Conventional biometric authentication technology uses a prism, a diffraction grating, and the like to generate a spectrum of an object, so it is difficult to miniaturize and thus cannot be applied to a smartphone. Accordingly, the composite sensor 5 of the present embodiment uses a plurality of spectral filters that can be implemented as nanostructures smaller than the size of an infrared filter of a complementary metal oxide semiconductor (CMOS) image sensor instead of a prism or diffraction grating. After passing through, it is emitted from the surface of the object and generates a signal representing the spectrum of the object from light passing through the display panel 3. The miniaturization means of the spectroscopic filter of this embodiment will be described in detail below. That is, the plurality of spectral filters 521 of this embodiment penetrates into a human body in the wavelength band of light irradiated onto an object from the light emitting module 8 and then returns to the human biological surface and is emitted from the human biological surface. It can be said to be a combination of spectral filters that transmit or block different wavelength bands by dividing the wavelength bands absorbed in the process.

For example, the plurality of spectral filters 521 of the present embodiment is a process of returning to the surface of the cornea and emitting from the corneal surface after penetrating into the cornea of the human eye in the wavelength band of light irradiated to the object from the light emitting module 8 It can be said to be a combination of spectral filters that transmit or block different wavelength bands by dividing the wavelength band absorbed at. Otherwise, the plurality of spectroscopic filters 521 of the present embodiment return to the surface of the face skin after being penetrated into the face skin of the person in the wavelength band of light irradiated from the light emitting module 8 to the object, and are emitted from the face skin surface It can be said to be a combination of spectral filters that transmit or block different wavelength bands by dividing the wavelength band absorbed at.

In the example shown in FIG. 11, the plurality of infrared filters 511 are arranged in a matrix structure corresponding to the right part of the matrix structure of the two-dimensional plane forming the light-receiving surface of the complex sensor 5, and the plurality of spectral filters ( 521) is arranged in a matrix structure corresponding to the left part of the matrix structure of the two-dimensional plane forming the light-receiving surface of the composite sensor 5. 12 is a view schematically showing another example of the structure of the complex sensor 5 shown in FIGS. 3-7. In the example shown in FIG. 12, the plurality of infrared filters 511 are arranged in a matrix structure corresponding to a cross section where four corners are removed from a matrix structure of a two-dimensional plane forming a light receiving surface of the composite sensor 5 The plurality of spectroscopic filters 521 are arranged in a matrix structure corresponding to four corner portions of a two-dimensional planar matrix structure forming the light-receiving surface of the composite sensor 5. 13 is a diagram schematically showing another example of the structure of the composite sensor 5 shown in FIGS. 3-7. In the example shown in FIG. 13, a plurality of infrared filters 511 are arranged in a matrix structure corresponding to a center portion from which an edge portion is removed from a matrix structure of a two-dimensional plane forming a light-receiving surface of the composite sensor 5, The plurality of spectroscopic filters 521 are arranged in a matrix structure corresponding to the edge portion of the matrix structure of the two-dimensional plane forming the light-receiving surface of the composite sensor 5.

In the example shown in FIG. 11, as a plurality of spectroscopic filters 521 are concentrated and arranged in one region of the light-receiving surface of the composite sensor 5, a spectrum for light irradiated to the region can be generated relatively accurately. On the other hand, there is an advantage that the spectrum for some of the light irradiated on the light-receiving surface of the composite sensor 5 may be omitted. Meanwhile, in the example shown in FIGS. 12-13, the spectrum of the entire light irradiated on the light receiving surface of the complex sensor 5 is scattered and arranged without being concentrated on any one area of the light receiving surface of the complex sensor 5 While there is an advantage that it can be generated, there is a disadvantage that the accuracy of the spectrum may be slightly lowered due to the small number of spectral filters 521 arranged in each region. In consideration of the performance of each spectral filter 521, it is preferable that an arrangement structure of a plurality of infrared filters 511 and a plurality of spectral filters 521 is designed. Those of ordinary skill in the art to which this embodiment belongs understand that a plurality of infrared filters 511 and a plurality of spectral filters 521 may be arranged in various structures in addition to the example shown in FIGS. 11-13. You can.

14 is a cross-sectional view of one embodiment of the composite sensor 5 shown in FIG. 11. 14 illustrates an example in which the composite sensor 5 is implemented by integrating the image sensor 51 and the spectroscopic sensor 52 on one substrate. Those of ordinary skill in the art to which this embodiment belongs can understand that the image sensor 51 and the spectroscopic sensor 52 may be implemented on separate substrates. However, a method of integrating a plurality of infrared filters 511 and a plurality of spectroscopic filters 521 on one substrate may be advantageous for simplifying the manufacturing process of the composite sensor 5. In FIG. 14, the abbreviation "L" represents a microlens, the abbreviation "I" represents an infrared filter, the abbreviation "S" represents a spectral filter, and the abbreviation "P" represents a photoelectric device. Since the composite sensor 5 of the example shown in FIGS. 14 and 12 and 13 can be easily understood, the illustration of the implementation of the composite sensor 5 shown in FIGS. 12 and 13 is omitted. FIG. 14 (a) shows an embodiment of the composite sensor 5 employing a front side illumination (FSI) image sensor 51, and FIG. 14 (b) shows a back illumination type (BSI, Shown is an embodiment of the composite sensor 5 employing the Back Side Illumination) image sensor 51.

Looking at the portion corresponding to the image sensor 51 of Figure 14 (a), a plurality of photoelectric elements 512 on the composite sensor substrate 500 of the two-dimensional plane forming the light-receiving surface of the composite sensor (5) They are arranged and stacked in a matrix structure corresponding to a part of the matrix structure. An insulating layer 503 in which wires for supplying power to each infrared filter 511 and each photoelectric element 512 are embedded is stacked on the plurality of photoelectric elements 512. Accordingly, the insulating layer 503 is also referred to as a wiring layer. The insulating layer 503 may be made of a transparent dielectric material such as SiO ₂ . On the insulating layer 503, a plurality of infrared filters 511 are arranged in the same matrix structure as the matrix structures of the plurality of spectroscopic filters 521 so as to correspond one-to-one to the plurality of photoelectric elements 512. The light filtered by each infrared filter 511 passes between the wires of the insulating layer 503 to reach each photoelectric element 512. A planarization layer 502 for planarizing an array surface of the plurality of infrared filters 511 is stacked on the plurality of infrared filters 511. The planarization layer 502 is made of a transparent dielectric material. On the planarization layer 502, a plurality of microlenses 501 are stacked by being arranged in the same matrix structure as the matrix structure of the plurality of infrared filters 511 so as to correspond one-to-one to the plurality of infrared filters 511. Each micro-lens 501 serves to ensure that light incident on it is focused on each infrared filter 511 below it.

Looking at the portion corresponding to the spectroscopic sensor 52 of Figure 14 (a), a plurality of photoelectric elements 522 on the composite sensor substrate 500 of a two-dimensional plane forming the light-receiving surface of the composite sensor (5) They are arranged and stacked in a matrix structure corresponding to other parts of the matrix structure. An insulating layer 503 in which wires for supplying power to each spectral filter 521 and each photoelectric element 522 are embedded is stacked on the plurality of photoelectric elements 522. On the insulating layer 503, a plurality of spectral filters 521 are stacked by being arranged in the same matrix structure as the matrix structure of the plurality of photoelectric elements 522 so as to correspond one-to-one to the plurality of photoelectric elements 522. The light filtered by each spectroscopic filter 521 passes between the wires of the insulating layer 503 to reach each photoelectric element 522. A planarization layer 502 for planarizing an array surface of the plurality of spectroscopic filters 521 is stacked on the plurality of spectroscopic filters 521. On the planarization layer 502, a plurality of microlenses 501 are stacked by being arranged in the same matrix structure as the matrix structure of the plurality of spectroscopic filters 521 so as to correspond one-to-one to the plurality of spectroscopic filters 521. Each micro-lens 501 serves to focus light incident on it into each spectral filter 521 below it.

Looking at the portion corresponding to the image sensor 51 of Figure 14 (b), a wire for supplying power to each infrared filter 511 and each photoelectric element 512 is embedded on the composite sensor substrate 500 The insulating layer 503 is stacked. On the insulating layer 503, a plurality of photoelectric elements 512 are arranged in a matrix structure corresponding to a part of a matrix structure of a two-dimensional plane forming a light-receiving surface of the composite sensor 5 and stacked. A plurality of infrared filters 511 are stacked on the plurality of photoelectric elements 512 in the same matrix structure as the matrix structures of the plurality of spectroscopic filters 521 so as to correspond one-to-one to the plurality of photoelectric elements 512. As described above, in the image sensor 51 of FIG. 14 (a), the insulating layer 503 is disposed on the plurality of photoelectric elements 512, whereas in the image sensor 51 of FIG. 14 (b), the plurality of photoelectrics The image sensor 51 of FIG. 14B has a stacked structure similar to that of the image sensor 51 of FIG. 14A, except that the insulating layer 503 is disposed under the device 512. . Accordingly, the remaining configuration of the image sensor 51 of FIG. 14 (b) will be replaced with the description of the image sensor 51 of FIG. 14 (a).

 Looking at the portion corresponding to the spectroscopic sensor 52 of FIG. 14B, a wire for supplying power to each spectroscopic filter 521 and each photoelectric element 522 is embedded on the composite sensor substrate 500. The insulating layer 503 is stacked. On the insulating layer 503, a plurality of photoelectric elements 522 are stacked in a matrix structure corresponding to other parts of the matrix structure of the two-dimensional plane forming the light-receiving surface of the composite sensor 5. On the plurality of photoelectric elements 522, a plurality of spectroscopic filters 521 are stacked in the same matrix structure as the matrix structure of the plurality of photoelectric elements 522 so as to correspond one-to-one to the plurality of photoelectric elements 522. As described above, in the spectroscopic sensor 52 of FIG. 14 (a), the insulating layer 503 is disposed on the plurality of photoelectric elements 512, whereas in the spectroscopic sensor 52 of FIG. 14 (b), the plurality of photoelectric The spectroscopic sensor 52 of FIG. 14B has a stacked structure similar to the spectroscopic sensor 52 of FIG. 14A except that the insulating layer 503 is disposed under the device 512. . Accordingly, the rest of the configuration of the spectroscopic sensor 52 of FIG. 14B will be replaced with a description of the spectroscopic sensor 52 of FIG. 14A.

14 (a) and 14 (b), since the insulating layer 503 is disposed on the plurality of photoelectric elements 512 in the front-illuminated composite sensor 5, it is incident on the plurality of photoelectric elements 512. There is a disadvantage that there is a lot of light loss. On the other hand, in the back-illuminated composite sensor 5, since the insulating layer 503 is disposed under the plurality of photoelectric elements 512, there is little loss of light incident on the plurality of photoelectric elements 512, but between infrared filters adjacent to each other. The disadvantage is that light mixing in different bands can occur. FIG. 14 is a simplified conceptual diagram of a composite sensor structure according to an embodiment of the present invention, and those skilled in the art to which this embodiment belongs may understand that other layers may be added in addition to the layer illustrated in FIG. 14. have. For example, an anti-reflection layer that prevents reflection of light incident on the plurality of photoelectric devices 512 may be stacked on the plurality of photoelectric devices 522.

15 is a cross-sectional view of another embodiment of the composite sensor 5 shown in FIG. 11. The example shown in FIG. 15 is different from the example shown in FIG. 14 in that three-quarters of the plurality of infrared filters 511 shown in FIG. 14 are replaced with a color filter 5110. Hereinafter, the example illustrated in FIG. 15 will be mainly described as a difference from the example illustrated in FIG. 14, and the omitted contents will be replaced by description of the example illustrated in FIG. 14. 15, a plurality of color filters 5110, a plurality of infrared filters 511, and a plurality of spectral filters 521 of the image sensor 51 are arranged in a matrix structure of a two-dimensional plane. The arrangement surface of the plurality of color filters 5110, the plurality of infrared filters 511, and the plurality of spectral filters 521 forms a light receiving surface of the composite sensor 5 for light passing through the display panel 3 do. That is, the plurality of color filters 5110 and the plurality of infrared filters 511 are arranged in a matrix structure corresponding to a part of the matrix structure of the two-dimensional plane, and the plurality of spectral filters 521 of the matrix structure of the two-dimensional plane They are arranged in a matrix structure corresponding to other parts.

The plurality of color filters 5110 transmits red band light and blocks red band light, a plurality of green filters that transmit green light and block light of the remaining band, and blue band light It is composed of a plurality of blue filters that transmit and block light in the rest of the band. In the unit filter composed of four color filters 5100, one red filter, two green filters, and one blue filter are alternately arranged in a matrix structure of a Bayer mosaic pattern, as shown in FIG. Likewise, one of the two green filters is replaced with an infrared filter 511. That is, the plurality of red filters, the plurality of green filters, the plurality of blue filters, and the plurality of infrared filters 511 are patterns in which the unit filters, each of which are combined one by one, are repeated, and the matrix corresponding to a part of the matrix structure of the two-dimensional plane Arranged in a structure, the plurality of spectral filters 521 are arranged in a matrix structure corresponding to different parts of the matrix structure of the two-dimensional plane.

In addition to the role of the infrared camera and the role of the spectrometer, the complex sensor 5 shown in FIG. 15 also plays a role of a visible light camera capable of generating a signal representing a color image, and the visible light camera and infrared light installed on the top of the front of the smartphone. As the camera disappears from the front top, the proportion of the screen on the front of the electronic device can be almost maximized. Since the composite sensor 5 shown in FIG. 15 has a lower infrared-based black-and-white image resolution than the composite sensor 5 shown in FIG. 14, when the matrix resolution of the composite sensor 5 is low, reliability of image authentication is deteriorated. Can be. Considering the matrix resolution of the composite sensor 5, it is necessary to properly select from the example shown in FIG. 14 and the example shown in FIG. In the example shown in FIGS. 11-13, it can be seen that the three-quarters of the plurality of infrared filters 511 is replaced by the color filter 5110, so that the complex sensor 5 shown in FIG. 15 can be implemented in various arrangements. have.

The structure of the image sensor 51 shown in FIGS. 14 and 15 is a structure of a typical CMOS image sensor, and a part of the pixels of the CMOS image sensor is replaced with the spectral filter of this embodiment, so that the complex sensor 5 according to this embodiment is easy It can be seen that can be manufactured. If the person skilled in the art to which this embodiment belongs belongs, the composite sensor 5 according to this embodiment can be easily manufactured by replacing some of the pixels of the CCD (Charge Coupled Device) image sensor with the spectral filter of this embodiment. Can understand.

16 is a view showing an example of the spectral filter 521 shown in FIGS. 14 and 15. The plane of the spectroscopic filter 521 shown in FIGS. 14 and 15 is illustrated in FIG. 16A, and the cross-section of the spectroscopic filter 521 is illustrated in FIG. 16B. Referring to FIG. 16, each spectroscopic filter 521 is formed in a form in which metal patterns having a certain shape are periodically arranged to determine a wavelength band determined according to an arrangement period of metal patterns in light passing through the display panel 3. It can cut off and pass the rest of the wavelength band. Each of the spectroscopic filters S1 and S2 is implemented in a form in which the disk-shaped metal patterns 162 are periodically arranged in a two-dimensional grid structure on the transparent substrate 161. The transparent substrate 161 may be made of various materials such as glass, polymer, Ge, GeSe, ZnS, ZnSe, sapphire, CaF ₂ and MgF ₂ . The insulating layers 503 of FIGS. 14 and 15 may serve as the transparent substrate 161. In this case, each of the spectroscopic filters S1 and S2 is implemented in a form in which the disk-shaped metal patterns 162 are periodically arranged in a two-dimensional lattice structure on the insulating layer 503. The metal pattern 162 may be a plasmonic metal Au, Ag, Al, Cu, or an alloy containing at least two of them, or an alloy containing at least one of them and other elements, Cr, which is a light absorbing metal, Ni, Ti, Pt, Sn, Sb, Mo, W, V, Ta, Te, Ge, Si, or alloys of at least two of them or alloys comprising at least one of them and containing other elements.

Each metal pattern may be a nano-structure in various forms, such as a square disk, a polygon, a rod, and a crossbar, in addition to a circular disk. These metal patterns may be periodically arranged in a linear lattice structure or a 2D lattice structure, and a 2D lattice structure may be a square lattice structure or a hexagonal lattice structure. The metal pattern of the nanostructure has a property of absorbing light in a specific wavelength band and transmitting light in the remaining wavelength band by its surface plasmon. The surface plasmon resonance phenomenon refers to a phenomenon in which light of a specific wavelength and free electrons of a metal surface propagate along the surface when light enters the metal surface. Plasmon refers to similar particles in which free electrons induced on a metal surface collectively vibrate by an electric field of incident light, and is present locally on the metal surface. In the case of a metal nanostructure having a surface plasmon characteristic locally, it exhibits a characteristic of absorbing light in a specific wavelength band among incident light when light is incident.

Conventional biometric authentication technology uses a prism, a diffraction grating, and the like to generate a spectrum of an object, so it is difficult to miniaturize and thus cannot be applied to a smartphone. As described above, the present embodiment is smart by adopting a kind of plasmonic filter that enables light filtering by a form in which metal patterns corresponding to nanostructures are periodically arranged as the spectral filter 521 of the spectroscopic sensor 52. It is possible to implement the spectroscopic sensor 52 which is very easy to integrate with the existing CMOS image sensor, which is widely used as an image sensor of the phone. Accordingly, the image sensor and the biometric authentication of the user can be simultaneously performed by only bringing the user's eyes or face close to the electronic device, thereby realizing the complex sensor 5 capable of preventing spoofing using a forged image.

As shown in FIG. 16, the metal patterns of the spectroscopic filter 521 form an array of metal nanostructures having a periodic lattice structure, and specific light absorption enhanced in a specific wavelength band by coupling of the surface plasmon and the lattice mode To the light reflection phenomenon. As a result, the spectrum of light passing through the array of metal nanostructures forms a dip curve in which the transmittance is rapidly lowered in a selective wavelength band in which specific light absorption or light reflection phenomenon is enhanced. "D1, D2" of the spectroscopic filters S1, S2 represents the length of the metal pattern, and "P1, P2" represents the spacing between the metal patterns. The duty cycle D1 / P1 of the spectroscopic filter S1 and the duty cycle D2 / P2 of the spectroscopic filter S2 are preferably maintained the same, but may be different. That is, it is preferable that the duty cycles of all spectroscopic filters remain the same, but they may be different. The duty cycle is preferably 30 to 80%. If the duty cycle is less than 30%, the dip size is very small, and when it exceeds 80%, a dip curve that is too wide tends to be generated.

The spectral shape formed by the metal patterns of the spectroscopic filter 521 of this embodiment depends on the geometric structure such as the type of the metal material, the arrangement period of the metal patterns, the size of the metal pattern, etc., in particular by the spectroscopic filter 521 The central wavelength of the blocked light is dominantly determined by the period of arrangement of the metal patterns. According to this embodiment, the center wavelength of light blocked by the spectroscopic filter 521 can be easily changed by changing the arrangement period of the metal patterns. If the metal patterns of one of the plurality of spectroscopic filters 521 and the metal patterns of the other of the spectroscopic filters are arranged at different periods, one of the spectroscopic filters and the other of the spectroscopic filters have different wavelength bands. Will be blocked.

As described above, the spectroscopic sensor 52 of this embodiment aims to detect a spectrum in the band 810 to 850 nm for biometric authentication of an object located close to an electronic device. For the spectrum detection in the band of 430 to 700 nm, it is preferable that the arrangement period of the metal patterns is designed between 0.1 and 1.5 μm. For example, various types of spectral filters 521 that divide and filter the bands of 810 to 850 nm in 5 nm increments may be implemented by gradually changing the period of the metal patterns at intervals corresponding to 5 nm units in the range of 0.1 to 1.5 μm. Can. In addition, the thickness of each metal pattern is preferably 5 ~ 500nm. If it is smaller than 5 nm, the proportion of free electrons scattered on the surface increases, which acts as a large factor for plasmon attenuation.

Conventionally, a metal nanohole array structure having a transmissive characteristic for a target band has been utilized as a plasmonic filter. The metal nanohole array structure exhibits a specific light transmission phenomenon in a specific wavelength band by coupling the surface plasmon wave and the lattice mode along the metal surface. In addition, since the metal nanohole array structure is based on coupling between traveling waves unlike the case of the metal pattern array structure of the present embodiment, various modes exist and there is a feature that is not defined as a single transmission band. The presence of such a multimode may cause signal distortion in the process of processing the signal wavelength incident on each photoelectric device. In addition, the spectral filter 521 of the present embodiment blocks a wavelength band determined according to an arrangement period of metal patterns in light passing through the display panel 3, so that a free spectral range depending on the period is relatively Because it is wide, it can cover the entire band of 810 ~ 850nm compared to the conventional transmission type filter.

17 is a graph showing simulation results of transmission characteristics of the spectral filter 521 shown in FIG. 16. Fabricated 50nm-thick nanodiscs using aluminum and arranged in a hexagonal lattice structure with 50% duty cycle. The dip curve of the transmittance was calculated by computer simulation while changing the arrangement period from 200 nm to 1500 nm at 100 nm intervals. The spectral filter 521 shown in FIGS. 17 to 16 has a characteristic in which a deep curve in which transmittance rapidly decreases from 0.35 um to 2 um is continuously varied, that is, a single stop band whose center wavelength is continuously varied. It can be seen that it exhibits characteristics.

As described above, the spectroscopic filter 521 illustrated in FIG. 16 blocks a wavelength band determined according to an arrangement cycle of metal patterns in light passing through the display panel 3. The plurality of photoelectric elements 522 convert a light filtered by the plurality of spectroscopic filters 521 according to the embodiment shown in FIG. 16 into an electrical signal to generate a signal representing a spectrum of an object. The spectral filter 521 illustrated in FIG. 16 is a blocking plasmonic filter, and the spectrum generated using light filtered by the plurality of spectral filters 521 according to the embodiment illustrated in FIG. 16 is a conventional transmission plasmonic. It is inversely related to the spectrum generated from the light filtered by the filter. The processor may perform biometric authentication on the object using a signal representing the inverse spectrum, or after restoring the normal spectrum from the inverse spectrum, biometric authentication on the object using the normal spectrum.

18 is a view showing another example of the spectral filter 521 shown in FIGS. 14 and 15. Fig. 18 (a) shows a cross section of the spectroscopic filter 521 shown in Figs. 14 and 15, and Fig. 18 (b) shows a plane of the spectroscopic filter 521. Referring to FIG. 18, each spectroscopic filter 521 includes an upper reflection layer 181, a lower reflection layer 182, a dielectric layer 183, and a buffer layer 184. The upper reflection layer 181 and the lower reflection layer 182 are separated from each other to reflect some of the light incident on it and transmit the rest. Each of the upper reflective layer 181 and the lower reflective layer 182 is formed of a metal thin film such as Ag, Au, Al, Cr, Mo, or the like having semi-transmissive properties, or a dispersion brag composed of a periodic multi-layer structure of a high refractive index dielectric layer and a low refractive index dielectric layer. It may be a reflective film (Distributed Bragg Reflector). The upper reflection layer 181 and the lower reflection layer 182 form upper and lower surfaces of each spectroscopic filter 521.

The dielectric layer 183 is inserted between the upper reflective layer 181 and the lower reflective layer 182, and at least two materials 1831 and 1832 having different refractive indices are alternately arranged. At least two regions of the dielectric layer 183 have different relative volume ratios of the two materials 1831 and 1832. As illustrated in FIG. 18B, at least two regions of the dielectric layer 183 have different relative width ratios of the two materials 1831 and 1832 along one direction. Here, one direction refers to a horizontal direction extending to the left and right, which is the longitudinal direction of the dielectric layer 183.

One of the two materials 1831 and 1832 of the dielectric layer 183 may be a relatively low refractive index dielectric, and the other material 1832 may be a relatively high refractive index dielectric. Conversely, one material 1831 may be a relatively high refractive index dielectric, and the other material 1832 may be a relatively low refractive index dielectric. Examples of low-refractive-index dielectric materials include fluorine-based ultraviolet resins, spin on glass, hydrogen silsesquioxane, magnesium fluoride (MgF ₂ ), and calcium fluoride (CaF ₂ ). And silicon dioxide (SiO ₂ ). The dielectric material having a high refractive index is, for example, a metal oxide such as titanium dioxide (TiO ₂ ).

The Fabry-Perot interferometer was first devised by Charles Fabry and Alfred Perot, and is generally constructed by inserting a cavity between two high-reflectivity mirrors. When light enters either of the two mirrors, multiple interference occurs in the cavity, transmitting only specific wavelengths and reflecting other wavelengths. The conventional linear variable filter is an optical filter of a Fabry-Perot resonator structure, and has a mirror layer above and below the dielectric cavity, and has a structure in which the thickness of the dielectric cavity is linearly varied in the longitudinal direction. Since the resolution of the conventional linear variable filter is determined by the ratio of the height to the length of the linear variable filter, there is a difficulty in miniaturization.

18, regions having different center wavelengths of light transmitted by the spectroscopic filter 521 are illustrated as three regions A, B, and C. In this case, in the three regions A, B, and C, the two materials 1831 and 1832 are alternately arranged in the same cycle, and the relative width ratios of the two substances 1831 and 1832 are different in the three regions, respectively. Due to this structure, the wavelengths of light transmitted through the three regions A, B, and C are different. The relative volume ratio between the two materials 1831 and 1832 may be defined as a duty cycle. The duty cycle in this embodiment is a relative volume ratio occupied by any one component 1831 of the two substances 1831 and 1832. By gradually changing the duty cycle between the two materials 1831 and 1832 of the dielectric layer 183, the refractive index of the dielectric layer 183 can be varied in one direction.

The wavelength of light passing through the spectral filter 521 shown in FIG. 18 is determined by the optical thickness of the dielectric layer 183. The optical thickness may be defined as a value obtained by multiplying the physical thickness by a refractive index. In the present embodiment, the optical wavelength of the dielectric region controls the central wavelength of the transmission band of the spectroscopic filter 521 by changing the refractive index in addition to the physical thickness. On the other hand, the width of a pair of two materials 1831 and 1832 is related to the wavelength of the filter to pass. For example, the width of a pair of two materials 1831 and 1832 may be designed to be sufficiently smaller than the wavelength of light, in which case the light cannot distinguish the two materials 1831 and 1832 as separate objects. It is recognized as an effective medium defined by a certain effective dielectric constant.

At this time, the optical constant of the effective medium is determined by the geometric distribution and relative volume fraction of the two objects. Therefore, each spectral filter 521 of the present embodiment passes a wavelength band determined according to a relative volume ratio between two materials 1831 and 1832 in light passing through the display panel 3, and can block the remaining wavelength bands. have. As described above, at least two regions in the dielectric layer 183 have different relative volume ratios between the two alternately arranged materials 1831 and 1832. Regions having different relative volume ratios of the dielectric layer 183 pass different wavelength bands. For example, various types of spectral filters 521 for dividing and filtering the bands of 810 to 850 nm in 5 nm units divide the duty cycle between the two materials 1831 and 1832 of the dielectric layer 183 at intervals corresponding to 5 nm units. It can be implemented by gradually changing.

The buffer layer 184 is inserted between at least one of the upper reflection layer 181 and the lower reflection layer 182 and the dielectric layer 183. That is, the buffer layer 184 may be inserted between the upper reflection layer 181 and the dielectric layer 183, or may be inserted between the lower reflection layer 182 and the dielectric layer 183, and the upper reflection layer 181 and the dielectric layer 183 ) And between the lower reflective layer 182 and the dielectric layer 183. The buffer layer 184 works with the dielectric layer 183 as an optical resonance cavity. The presence of the buffer layer 140 increases the effective thickness of the optical resonant cavity, thereby maintaining the thickness of the dielectric layer 183 small, while bringing the effect of moving the center wavelength of the transmission band of the spectroscopic filter 521 to the long wavelength region. When an optical resonant cavity is constructed using only the dielectric layer 183, an aspect ratio for filling any one material in the dielectric layer 183 made of a combination of two materials having a spacing period sufficiently smaller than the wavelength of the incident light and having different refractive indices from each other There is a process difficulty due to the excessively increasing condition.

Therefore, the existence of the buffer layer 184 has an advantage that the variable range of the transmission wavelength can be effectively increased while keeping the thickness of the dielectric layer 183 small. Accordingly, the miniaturization of the spectral filter 521 is possible, and thus it is possible to implement the spectral sensor 52 which is very easy to integrate with the existing CMOS image sensor which is widely used as an image sensor of a smartphone. Accordingly, the image sensor and the biometric authentication for the user can be performed simultaneously, so that the composite sensor 5 capable of preventing spoofing using a forged image can be realized. The buffer layer 184 may be the same material as any one of the two materials 1831 and 1832. 18 illustrates a case where the material of the buffer layer 184 and the second material 1832 are the same. Meanwhile, the buffer layer 184 and the two materials 1831 and 1832 may be different materials.

19 is a graph showing simulation results of transmission characteristics of the spectral filter 521 shown in FIG. 18. The alternating arrangement period of the two materials 1831 and 1832 is 200 nm, the thickness of the dielectric layer 183 is 60 nm, and among the two materials 1831 and 1832, the refractive index of the low refractive index material is 1.38, and the refractive index of the high refractive index material is 2.7. Was made. As the upper and lower reflective layers 181 and 182, an Ag layer having a thickness of 20 nm having a semi-transmissive property was used. When the width of the low-refractive-index nanostructure was increased from 50 nm to 150 nm at 20 nm intervals, the change pattern of the transmission band was calculated by computer simulation. It can be seen from FIG. 19 to FIG. 18 that the spectral filter 521 illustrated in FIG. 18 exhibits a characteristic that the center wavelength of the transmission band moves to the long wavelength region as the filling rate of the high refractive index nanostructure increases.

20 is a diagram schematically showing another example of the structure of the spectroscopic sensor 52 shown in FIGS. 3-7. In FIG. 20, the abbreviation "S" represents the spectral filter 521, the abbreviation "F" represents the band-pass filter 504, and the abbreviation "P" represents the photoelectric device 522. In order for the features of the embodiment shown in FIG. 20 to be well represented, the plurality of spectral filters 521, the plurality of band-pass filters 504, and the plurality of photoelectric elements 522 are briefly expressed in a simple grid pattern. Since the embodiment shown in FIG. 20 is not related to the image sensor 51 among the complex sensors 5 shown in FIGS. 1-6, only the spectroscopic sensor 52 is shown. The portion of the image sensor 51 will be replaced with the description of the embodiments shown in FIGS. 11-13.

20, unlike the embodiments shown in FIGS. 11-13, a plurality of infrared filters 511 are arranged in a matrix structure corresponding to a part of a matrix structure of a two-dimensional plane, and a plurality of band-pass filters Reference numeral 504 is arranged in a matrix structure corresponding to other parts of the matrix structure of the two-dimensional plane, and the plurality of spectral filters 521 are different parts of the matrix structure of the two-dimensional plane forming the light-receiving surface of the complex sensor 5 It is arranged in a reduced matrix structure of the corresponding matrix structure. The plurality of band-pass filters 504 are arranged in a matrix structure corresponding to other parts of the matrix structure of the two-dimensional plane forming the light-receiving surface of the composite sensor 5 so as to correspond one-to-many to the plurality of spectral filters 521. And is disposed under the plurality of spectroscopic filters 521. Accordingly, a plurality of band-pass filters 504 correspond to each spectral filter 521 vertically.

20 shows an example in which four band-pass filters 504 correspond to one spectral filter 521 vertically. The plurality of band-pass filters 504 corresponding to each spectral filter 521 transmits different wavelength bands in the filtering wavelength band of each spectral filter 521. That is, any one of the plurality of band-pass filters 504 transmits only a portion of the wavelength bands of the filtering wavelength bands of each spectral filter 521, and the other band-pass filters 504 are each Among the filtering wavelength bands of the spectroscopic filter 521, only other wavelength bands are transmitted. The plurality of optoelectronic devices 522 are arranged in a matrix structure corresponding to other parts of the matrix structure of the two-dimensional plane so as to correspond one-to-one up and down to the plurality of band-pass filters 504 and disposed under the plurality of band-pass filters 504. do.

According to this embodiment, that is, the narrower the filtering band of each spectral filter 521 is, the higher the resolution of the spectrum of the object is, so that the accuracy of biometric authentication can be improved. When each spectral filter 521 is implemented with a plasmonic filter as shown in FIG. 16, or a Fabry-Perot filter as shown in FIG. 18, the period of arrangement of the metal patterns of the plasmonic filter or two materials of the dielectric layer Manufacturing the difference in the relative volume ratio of the microscopically has a limitation in narrowing the filtering bandwidth of each spectral filter 521 because there are limitations in the manufacturing process. This acts as a limitation of the resolution of an object's spectrum. Accordingly, the plurality of band-pass filters 504 corresponding to each spectral filter 521 transmits different wavelength bands among the filtering wavelength bands of each spectral filter 521 to limit the filtering bandwidth of each spectral filter 521. By overcoming, a high-resolution spectrum of the object is possible, and the accuracy of biometric authentication can be improved.

21 is a cross-sectional view of another embodiment of the composite sensor 5 shown in FIG. 11. 21 (a) shows an example in which the composite sensor 5 is implemented by integrating the image sensor 51 and the spectroscopic sensor 52 on a single substrate as in FIG. 21 (a) shows an embodiment of the complex sensor 5 employing the front-illuminated image sensor 51, and FIG. 21 (b) shows the complex sensor employing the back-illuminated image sensor 51. An embodiment of (5) is shown. 21C illustrates an example in which the spectroscopic filter 521 shown in FIG. 16 is applied to the spectroscopic sensors 52 of FIGS. 21A and 21B. In FIG. 21, the abbreviation "L" represents a microlens, the abbreviation "I" represents an infrared filter, the abbreviation "S" represents a spectral filter, the abbreviation "P" represents an optoelectronic device, and the abbreviation "F" represents a band transmission. Represents a filter (band pass filter). The embodiment of FIG. 21 is the same as the embodiment of FIG. 14, except that the band-pass filter 504 is disposed under each spectral filter 521 of the spectroscopic sensor 52. Differences from and will be mainly described, and the contents omitted below, for example, the parts corresponding to the image sensor 51 will be replaced by the description of the embodiment of FIG. 14.

Looking at the portion corresponding to the spectroscopic sensor 52 of Figure 21 (a), a plurality of photoelectric elements 522 on the composite sensor substrate 500 of the two-dimensional plane forming the light-receiving surface of the composite sensor (5) They are arranged and stacked in a matrix structure corresponding to other parts of the matrix structure. An insulating layer 503 in which wires for supplying power to each spectral filter 521 and each photoelectric element 522 are embedded is stacked on the plurality of photoelectric elements 522. On the insulating layer 503, a plurality of band-pass filters 504 are stacked by being arranged in the same matrix structure as the matrix structure of the plurality of photoelectric elements 522 so as to correspond one-to-one to the plurality of photoelectric elements 522. Light filtered by each band-pass filter 504 passes between the wires of the insulating layer 503 to reach each photoelectric element 522.

Looking at a portion corresponding to the spectroscopic sensor 52 of FIG. 21B, a wire for supplying power to each spectroscopic filter 521 and each photoelectric element 522 is embedded on the composite sensor substrate 500. The insulating layer 503 is stacked. On the insulating layer 503, a plurality of photoelectric elements 522 are stacked in a matrix structure corresponding to other parts of the matrix structure of the two-dimensional plane forming the light-receiving surface of the composite sensor 5. A plurality of band-pass filters 504 are stacked on the plurality of photoelectric elements 522 in the same matrix structure as those of the plurality of photoelectric elements 522 so as to correspond one-to-one to the plurality of photoelectric elements 522 vertically. The light filtered by each band-pass filter 504 directly reaches each photoelectric element 522.

As shown in (a) and (b) of FIG. 21, a plurality of spectral filters 521 on the plurality of band-pass filters 504 are vertically and one-to-one corresponding to the plurality of band-pass filters 504. The spectral filter 521 is arranged in a reduced matrix structure and stacked. A planarization layer 502 for planarizing an array surface of the plurality of band-pass filters 504 is stacked on the plurality of spectroscopic filters 521. The planarization layer 502 is made of a transparent dielectric material. On the planarization layer 502, a plurality of microlenses 501 are stacked by being arranged in the same matrix structure as the matrix structure of the plurality of spectroscopic filters 521 so as to correspond one-to-one to the plurality of spectroscopic filters 521. When it is difficult to manufacture the apertures of the plurality of microlenses 501 of the composite sensor differently, as shown in FIG. 20, the plurality of microlenses 501 are one-to-one up and down on the plurality of spectroscopic filters 521. To correspond, the matrix structures of the plurality of spectroscopic filters 521 may be arranged and stacked in the same matrix structure. Each micro-lens 501 serves to focus light incident on it into each spectral filter 521 below it.

 Referring to (c) of FIG. 21, the three spectroscopic filters 521 are formed in such a manner that their respective metal patterns are arranged at different periods. Two different color filters 5110 are disposed under each spectral filter 521. A separate planarization layer may be formed on the color filter 5110. The meaning that a certain band-pass filter 504 transmits only a part of the wavelength bands of the filtering wavelength bands of each spectral filter 521 means that the filtering band of the band-pass filter 504 coincides with some of the wavelength bands. In addition, it means that a wavelength band different from the filtering wavelength band of each spectral filter 521 may be included in addition to some of the wavelength bands. That is, if a part of the filtering band of each spectral filter 521 is included in the filtering band of the red filter, and the rest of the filtering band of each spectral filter 521 is included in the filtering band of the green filter, the red filter and the green filter Using it, a high resolution spectrum of an object can be made possible.

As described above, since at least two kinds of color filters and the same color filters of the plurality of color filters 5110 of the image sensor 51 can be used as the plurality of band-pass filters 504, the entire CMOS image sensor 51 can be used. Only by stacking the spectral filter 521 of this embodiment on some of the color filters, the implementation of the composite sensor 5 according to this embodiment may be possible. In addition, since each spectroscopic filter 521 may be manufactured to a size larger than that of each color filter 5110, manufacturing of each spectroscopic filter 521 may be greatly facilitated. 19A and 19B, a plurality of microlenses 501 is illustrated to correspond to the plurality of band-pass filters 504 one-to-one. As shown in (a) and (b) of FIG. 19, in the manufacturing process of the microlens 501, it may be difficult to integrally manufacture microlenses of different sizes, for focusing light to the color filter 5110. Microlenses of the same type as the microlenses can be produced for the spectral filter, or the microlens array layer can be omitted.

In the above, examples in which a plurality of band-pass filters 504 correspond one-to-many to a plurality of spectral filters 521 and disposed under the plurality of spectral filters 521 have been described. The plurality of band-pass filters 504 may be one-to-one corresponding to the plurality of spectroscopic filters 521, and may be disposed under the plurality of spectroscopic filters 521, and the plurality of band-pass filters 504 may include a plurality of spectroscopic filters 521. It may be disposed under the plurality of spectral filters 521 in many-to-one correspondence. In the former case, one band-pass filter 504 corresponding to one-to-one spectral filter 521 transmits the filtering wavelength band of the spectral filter 521. In the latter case, one band-pass filter 504 corresponding to the plurality of spectral filters 521 corresponding to a part of the plurality of spectral filters 521 is one-to-one, and the entire filtering wavelength band of the plurality of spectral filters 521 Permeate. The plurality of band-pass filters 504 serves to improve the overall filtering performance by filtering the filtered light again by the plurality of spectroscopic filters 521.

In addition, a plurality of band-pass filters 504 may be disposed on the plurality of spectroscopic filters 521 in one-to-many, one-to-one, or many-to-one correspondence with the plurality of spectroscopic filters 521. In the example in which the plurality of band-pass filters 504 are disposed under the plurality of spectroscopic filters 521, the plurality of photoelectric elements 522 convert light filtered by the plurality of band-pass filters 504 into electrical signals, In the example in which the plurality of band-pass filters 504 are disposed on the plurality of spectroscopic filters 521, the plurality of photoelectric elements 522 convert light filtered by the plurality of spectroscopic filters 521 into electrical signals. In the latter example, the plurality of band-pass filters 504 may improve the filtering performance of each spectral filter 521 by preventing light from unnecessary wavelength bands other than its filtering wavelength band from entering the plurality of spectral filters 521. It plays a role to help. In various examples as described above, one type of color filter and the same color filters of the plurality of color filters 5110 of the image sensor 51 may be used as the plurality of band-pass filters 504.

22 is a cross-sectional view of another embodiment of the composite sensor 5 shown in FIG. 11. According to the above, the example shown in FIG. 15 is different from the example shown in FIG. 14 in that three-quarters of the plurality of infrared filters 511 shown in FIG. 14 are replaced with a color filter 5110. . Likewise, the example shown in FIG. 22 differs from the example shown in FIG. 21 in that three-quarters of the plurality of infrared filters 511 shown in FIG. 21 are replaced by the color filter 5110. The example illustrated in FIG. 22 can be understood from the description of the example illustrated in FIG. 21 and the description of the example illustrated in FIG. 15. Therefore, the description of the example shown in FIG. 22 will be replaced with the description of the example shown in FIG. 21 and the example of FIG. 15.

23 is a block diagram of the electronic circuit of the electronic device shown in FIG. 3. Referring to FIG. 22, the electronic circuit of the electronic device shown in FIG. 3 includes a touch panel 2, a display panel 3, a composite sensor 5, a light emitting module 8, a sensor module 11, and a power supply module It consists of (12), processor (71), communication module (72), input / output interface module (73), memory (74), and bus (75). In order to prevent this embodiment from being easily understood and to prevent blurring of the features of the present embodiment, FIG. 23 shows a main configuration of an electronic circuit of a smartphone necessary for understanding the present embodiment. Those of ordinary skill in the art to which this embodiment belongs may understand that another configuration may be added in addition to the configuration illustrated in FIG. 23. For example, the electronic circuit of the electronic device shown in FIG. 3 may further include a Global Positioning System (GPS) module, an audio module, and the like.

 The sensor module 11 may be composed of sensors built into a smartphone in addition to the complex sensor 5, for example, a pressure sensor, a proximity sensor, an illuminance sensor, a gyro sensor, and the like. The power supply module 12 converts the power of the battery 9 into driving power of the touch panel 2, the display panel 3, the composite sensor 5, the sensor module 11, and the printed circuit board 7 It serves to supply and consists of a charging circuit and a voltage conversion circuit. The path for supplying power from the power supply module 12 to other components is obvious to a person skilled in the art to which this embodiment belongs and is omitted to reduce the complexity of the drawings. The processor 71, the communication module 72, the input / output interface module 73, the memory 74, and the bus 75 are implemented with electronic components mounted on the printed circuit board 7, and are generally separated into several. It is mounted on the printed circuit board (7).

The processor 71 controls other components of the electronic device, for example, the touch panel 2, the display panel 3, the composite sensor 5, and the sensor module 11, and the touch panel 2 and the communication module (72), processing data input from the input / output interface module 73, reading data from the memory 74 or storing data in the memory 74. In particular, the processor 71 of the present embodiment receives a signal representing an image of an object from the complex sensor 5, generates an image of the object from the input signal, and stores the generated object image and the user stored in the memory 74. User authentication of an object is performed by comparing an iris image or a face image. In addition, the processor 71 receives a signal representing the spectrum of the object from the composite sensor 5, generates a spectrum of the object from the input signal, and the generated spectrum and the iris spectrum or face of the user stored in the memory 74 Biometric authentication of the object is performed by comparing the spectra. The operation of the processor 71 will be described in detail below with reference to the flowchart of FIGS. 24-26.

The communication module 72 communicates with other electronic devices. For example, the communication module 72 may perform communication with other electronic devices according to various communication methods such as Long Term Evolution (LTE), Wi-Fi, and Bluetooth. The input / output interface module 73 may output commands or data input from a user or other external device to other components of the electronic device, or output commands or data received from other components of the electronic device to the user or other external devices. have. The input / output interface module 73 is composed of a USB (Universal Serial Bus) interface, an audio interface, and the like. Various data are stored in the memory 74. In particular, the user's biometric image and spectrum data are stored in the memory 74 of this embodiment. The bus 75 serves to transfer data between the processor 71 and various components.

24 is a flowchart of a user authentication method according to an embodiment of the present invention. FIG. 24 shows a flow of a user authentication method when the electronic device shown in FIG. 23 is in a screen lock mode. Referring to FIG. 24, the user authentication method according to the present embodiment is an electronic device shown in FIGS. 1-3 and 23, and a watch in an electronic device including the composite sensor 5 according to the embodiment shown in FIG. 4-22. It consists of thermally processed steps. Therefore, the contents described above with respect to the electronic device illustrated in FIGS. 1-3 and 23 and the complex sensor 5 illustrated in FIG. 4-22 are applied to the user authentication method described below, even if omitted below. .

In step 2401, the sensor module 11 of the electronic device detects which object is close to the cover glass 1 below the critical distance in the screen lock mode. For example, the critical distance may be 5 cm and may be designed with various values. The proximity detection may be performed using a proximity sensor of the sensor module 11. If it is determined in step 2301 that the object is closer than the critical distance, the object is considered to have entered the sensing area of the complex sensor 5 and proceeds to step 2402, otherwise the screen lock mode is maintained. Here, the screen lock mode refers to a mode in which power supply to the components of the electronic device except for the processor 71 and the sensor module 11 is stopped, for example, the touch panel 2 and the display panel 3 are stopped. .

In step 2402, the processor 71 of the electronic device switches from the screen lock mode to the low power mode, and the display panel 3 of the electronic device displays graphics for guiding the user's eyes or face to be accurately positioned. For example, the display panel 3 may display a graphic representing a human eye or face outline. When the actual outline of the user's eyes or face is matched with the displayed outline, the user's eye or face is accurately positioned in the detection area of the composite sensor 5. Here, the low power mode is a part of the components of the electronic device for user authentication, for example, a touch panel 2, a display panel 3, a composite sensor 5, a light emitting module 8, a processor 71 Refers to a mode in which power is supplied to the back. In step 2403, the light emitting module 8 of the electronic device irradiates infrared rays on the surface of the object under the control of the processor 71.

In step 2404, the complex sensor 5 of the electronic device uses a plurality of infrared filters 511 to reflect the infrared light irradiated on the surface of the object in step 2403 from the light passing through the display panel 3 after being reflected from the surface of the object. Generates a signal that represents the image. The image sensor 51 of the composite sensor 5 generates a signal representing an image of an object from light passing through the display panel 3 when infrared rays irradiated on the surface of the object are reflected from the surface of the object. Since the signal generation in step 2404 is only for showing the object image to the user, the color filter 5110 may be used. In step 2405, the processor 71 of the electronic device receives a signal representing an image of the object from the composite sensor 5, and generates an image of the object from the input signal. The signal representing the image of the object is continuously input serially from the composite sensor 5 to the processor 71. The processor 71 may generate an image of an object by converting a serially input signal into two-dimensional image data. In step 2406, the display panel 3 of the electronic device displays the object image thus generated on the graphic displayed in step 2402 so that the object image generated in step 2405 overlaps with the graphic displayed in step 2402.

In step 2407, the processor 71 of the electronic device checks whether a certain time has elapsed since the start of step 2402. For example, the predetermined time may be 5 seconds and may be designed with various values. If it is determined in step 2407 that a certain time has elapsed, the process proceeds to steps 2408 and 2411, otherwise it returns to step 2404. Whenever steps 2404-2406 are repeated, an image corresponding to the position of the user's eyes or face at each execution time of steps 2404-2406 is repeatedly displayed. Accordingly, the user may adjust his / her eye or face position while checking the eye or face image that overlaps with the graphic displayed in step 2402 for a certain period of time so that his or her eye or face is accurately positioned in the detection area of the composite sensor 5. Can be.

In step 2408, the complex sensor 5 of the electronic device uses the plurality of infrared filters 511 to reflect the infrared light irradiated on the surface of the object from the surface of the object and reflects the image of the object from the light passing through the display panel 3. Generate a signal to indicate. In step 2409, the processor 71 of the electronic device receives a signal representing an image of the object from the complex sensor 5, and generates an image of the object from the input signal. The signal representing the image of the object is continuously input serially from the composite sensor 5 to the processor 71. The processor 71 may convert a serially input signal into 2D image data or 3D image data for the user's image authentication. In operation 2410, the processor 71 of the electronic device performs image authentication for the object using the object image generated in operation 2409. The processor 71 may perform image authentication for the object by comparing the object image generated in operation 2306 with the iris or face image of the user stored in the memory 74.

In step 2411, the complex sensor 5 of the electronic device uses a plurality of spectral filters 521 to irradiate the surface of the object, and the infrared ray penetrates into the object and then is emitted from the surface of the object and passes through the display panel 3 Generate a signal representing the spectrum of an object from light. In step 2412, the processor 71 of the electronic device receives a signal representing the spectrum of the object from the composite sensor 5, and generates a spectrum of the object from the input signal. A signal representing the spectrum of the object is input serially from the composite sensor 5 to the processor 71. The processor 71 may generate a spectrum of an object by converting a serially input signal into a two-dimensional curve graph. In operation 2413, the processor 71 of the electronic device performs biometric authentication on the object using the spectrum generated in operation 2412. The processor 71 may perform biometric authentication on the object by comparing the spectrum generated in operation 2412 with the user's eye or face spectrum stored in the memory 74. Steps 2408-2410 and steps 2411-2413 are performed simultaneously.

In step 2414, the processor 71 of the electronic device authenticates the user according to the result of performing image authentication in step 2410 and the result of performing biometric authentication in step 2413. The processor 71 indicates that the result of performing the image authentication in step 2410 matches the object image generated in step 2409 and the iris or face image of the user stored in the memory 74, and the result of performing the biometric authentication in step 2413 If the spectrum generated in step 2412 indicates that the spectrum of the user stored in the memory 74 is consistent, the user authentication is determined to be successful, and the screen lock mode is switched to the screen unlock mode. If the object image generated in step 2409 and the iris or face image of the user stored in the memory 74 do not match or the spectrum generated in step 2412 does not match the user's spectrum stored in the memory 74, the processor 71 It is determined that authentication has failed, and the screen lock mode is maintained.

25 is a flowchart of a user authentication method according to another embodiment of the present invention. FIG. 25 shows a flow of a user authentication method when the electronic device shown in FIG. 23 is in a screen unlock mode. Referring to FIG. 25, the user authentication method according to the present embodiment is an electronic device shown in FIGS. 1-3 and 23 and a watch in an electronic device including the composite sensor 5 according to the embodiment shown in FIG. 4-22. It consists of thermally processed steps. Therefore, the contents described above with respect to the electronic device illustrated in FIGS. 1-3 and 23 and the complex sensor 5 illustrated in FIG. 4-22 are applied to the user authentication method described below, even if omitted below. . In addition, steps 2502-2514 of the steps of the present embodiment are duplicated with steps 2402-2414 shown in FIG. 23, and thus will be briefly described.

In step 2501, the processor 71 of the electronic device determines whether there is a user authentication request from which application is being executed by the processor 71 in the screen unlock mode. If it is determined in step 2501 that there is a user authentication request, the process proceeds to step 2502, otherwise, the current state is maintained. In operation 2502, the display panel 3 of the electronic device displays graphics for guiding the user's eyes or face to be accurately positioned under the control of the processor 71. In step 2503, the light emitting module 8 of the electronic device irradiates infrared rays on the surface of the object under the control of the processor 71. Unlike the embodiment shown in FIG. 24, since this embodiment is executed in the screen unlock mode, abnormal proximity of any object other than the user rarely occurs. Accordingly, this embodiment does not consider whether any object is in proximity.

In step 2504, the composite sensor 5 of the electronic device generates a signal representing an image of the object from light passing through the display panel 3 by reflecting infrared light irradiated on the surface of the object in step 2503. In operation 2505, the processor 71 of the electronic device receives a signal representing an image of the object from the composite sensor 5, and generates an image of the object from the input signal. In operation 2506, the display panel 3 of the electronic device displays the object image thus generated on the graphic displayed in operation 2502 such that the object image generated in operation 2505 overlaps with the graphic displayed in operation 2502.

In step 2507, the processor 71 of the electronic device checks whether a certain time has elapsed since the start of step 2502. If it is determined in step 2507 that a certain time has elapsed, the process proceeds to steps 2508 and 2511, otherwise it returns to step 2504. In step 2508, the complex sensor 5 of the electronic device generates a signal representing an image of the object from light passing through the display panel 3 by reflecting infrared light irradiated on the surface of the object. In operation 2509, the processor 71 of the electronic device receives a signal representing an image of the object from the complex sensor 5, and generates an image of the object from the input signal. In operation 2510, the processor 71 of the electronic device performs image authentication for the object using the object image generated in operation 2509.

In step 2511, the complex sensor 5 of the electronic device generates a signal representing the spectrum of the object from light passing through the display panel 3, which is emitted from the surface of the object after the infrared light irradiated on the surface of the object penetrates the object. do. In step 2512, the processor 71 of the electronic device receives a signal representing the spectrum of the object from the complex sensor 5, and generates a spectrum of the object from the input signal. In step 2513, the processor 71 of the electronic device performs biometric authentication on the object using the spectrum generated in step 2512. The processor 71 may perform biometric authentication on the object by comparing the spectrum generated in operation 2512 with the user's eye or face spectrum stored in the memory 74. Steps 2508-2510 and steps 2511-2513 are run simultaneously.

In step 2514, the processor 71 of the electronic device authenticates the user according to the result of performing image authentication in step 2510 and the result of performing biometric authentication in step 2513. The processor 71 indicates that the result of performing the image authentication in step 2510 matches the object image generated in step 2509 and the iris or face image of the user stored in the memory 74, and the result of performing biometric authentication in step 2513 If the spectrum generated in step 2512 indicates that the spectrum of the user stored in the memory 74 is consistent, the user authentication is determined to be successful, and the application process after the user authentication is completed is processed.

If the object image generated in step 2509 and the iris or face image of the user stored in the memory 74 do not match or the spectrum generated in step 2512 does not match the user's spectrum stored in the memory 74, the processor 71 It is determined that authentication has failed, and the process proceeds to step 2515. In operation 2515, the display panel 3 of the electronic device displays a message indicating that user authentication has failed under the control of the processor 71. After completion of step 2515, return to step 2502. Steps 2502-2515 are repeatedly executed until the user authentication in step 2514 is successful or the application is executed by the user or the user authentication process is cancelled.

26 is a flowchart of a method for registering user biometric information according to an embodiment of the present invention. The user biometric information method illustrated in FIG. 26 starts when the biometric information registration application is executed by the user in the screen unlock mode. Referring to FIG. 26, the method for registering user biometric information according to the present embodiment is an electronic device shown in FIGS. 1-3 and 23, and an electronic device including the composite sensor 5 according to the embodiment shown in FIG. 4-22. It consists of steps processed in time series. Therefore, even if omitted below, the contents described above with respect to the electronic device shown in FIGS. 1-3 and 23 and the complex sensor 5 shown in FIG. 4-22 are also applied to the method for registering user biometric information to be described below. Applies. In addition, steps 2601-2606 and 2608-2611 among the steps of the present embodiment are overlapped with steps 2402-2407, 2408-2409, and 2411-2412 shown in FIG. 23, and thus will be briefly described.

In operation 2601, the display panel 3 of the electronic device displays a graphic for guiding the user's eyes or face to be accurately positioned under the control of the processor 71. In step 2602, the light emitting module 8 of the electronic device irradiates infrared rays on the surface of the object under the control of the processor 71. In step 2603, the complex sensor 5 of the electronic device generates a signal representing an image of the object from light passing through the display panel 3 when infrared rays irradiated to the surface of the object are reflected from the surface of the object in step 2602. In operation 2604, the processor 71 of the electronic device receives a signal representing an image of the object from the complex sensor 5, and generates an image of the object from the input signal. In step 2605, the display panel 3 of the electronic device displays the object image thus generated on the graphic displayed in step 2502 so that the object image generated in step 2604 overlaps with the graphic displayed in step 2601.

In step 2606, the processor 71 of the electronic device checks whether a certain time has elapsed since the start of step 2601. If it is determined in step 2606 that a certain time has elapsed, the process proceeds to steps 2608 and 2610, otherwise proceeds to step 2607. In step 2607, the processor 71 of the electronic device determines whether biometric information registration has been canceled by the user. If it is determined in step 2607 that the biometric information registration is canceled, execution of the biometric information registration application is terminated, otherwise, the process returns to step 2603. The user may cancel registration of his biometric information if the user determines that the contour of his or her eyes or face is not exactly aligned with the contour of the graphic displayed in step 2601.

In step 2608, the composite sensor 5 of the electronic device generates a signal representing an image of the object from light passing through the display panel 3 by reflecting infrared light irradiated on the surface of the object. In step 2609, the processor 71 of the electronic device receives a signal representing an image of the object from the complex sensor 5, and generates an image of the object from the input signal. In step 2610, the composite sensor 5 of the electronic device generates a signal representing the spectrum of the object from light passing through the display panel 3, emitted from the surface of the object after the infrared light irradiated on the surface of the object penetrates the object. do. In step 2611, the processor 71 of the electronic device receives a signal representing the spectrum of the object from the complex sensor 5 and generates a spectrum of the object from the input signal.

In operation 2612, the processor 71 of the electronic device stores the object image generated in operation 2609 and the spectrum generated in operation 2611 as a set, and stores the biometric image in the memory 74 as the user's biometric information. The spectrum generated in step 2611 is registered as biometric information of the user. The object image generated in step 2609 and the spectrum generated in step 2611 may be registered as user biometric information together with the user ID. Several sets of biometric images and spectra may be stored in the memory 74. If any one of these multiple sets matches the biometric image and spectrum generated from the output signal of the composite sensor 5, user authentication may be designed to be satisfied. Steps 2608-2609 and 2610-2611 are performed simultaneously, and steps 2608-2609 and 2610-2611 may be repeated several times to increase the accuracy of the user's biological image and spectrum. Steps 2608-2609 and 2610-2611 may be set as an average of a plurality of images as a user's final biometric image when repeated several times, and an average of a plurality of spectrums may be set as a user's final biometric spectrum.

On the other hand, the user authentication method and the user biometric information registration method according to the embodiments of the present invention as described above can be written as a program executable on a computer processor, and recorded and executed on a computer-readable recording medium. Prescribing can be implemented in a computer. A computer includes any type of computer capable of executing a program, such as a desktop computer, a notebook computer, a smart phone, or an embedded type computer. In addition, the structure of the data used in the above-described embodiment of the present invention can be recorded on a computer-readable recording medium through various means. The computer-readable recording medium is a storage device such as RAM, ROM, magnetic storage medium (eg, floppy disk, hard disk, etc.), optical reading medium (eg, CD-ROM, DVD, etc.). Includes media.

So far, the present invention has been focused on the preferred embodiments. Those skilled in the art to which the present invention pertains will appreciate that the present invention can be implemented in a modified shape without departing from the essential characteristics of the present invention. Therefore, the disclosed embodiments should be considered in terms of explanation, not limitation. The scope of the present invention is shown in the claims rather than the foregoing description, and all differences within the equivalent range should be interpreted as being included in the present invention.

1 ... cover glass
2 ... touch panel
3 ... display panel
4 ... rear panel
41 ... protective layer 42 ... heat dissipation layer
5 ... complex sensor
51 ... Image sensor 511 ... Infrared filter
5110 ... color filter
52 ... spectral sensor 521 ... spectral filter
512, 522 ... photoelectric element
500 ... composite sensor substrate 501 ... micro lens
502 ... flattening layer 503 ... insulating layer
504 ... Bandpass filter
161 ... transparent substrate 162 ... metal pattern
181 ... Upper reflective layer 182 ... Lower reflective layer
183 ... dielectric layer 184 ... buffer layer
50 ... collimator lens
6 ... bracket
7 ... Printed Circuit Board
71 ... processor 72 ... communication module
73 ... I / O interface module 74 ... Memory
75 ... bus
8 ... Light emitting module
9 ... battery
10 ... housing
11 ... sensor module
12 ... Power supply module

Claims (30)

A display panel for displaying a screen;
A light emitting module that irradiates infrared light on the surface of the object;
An image sensor that generates a signal representing an image of the object from light reflected from the surface of the object by using infrared filters to irradiate the surface of the object;
A spectroscopic sensor that generates a signal representing the spectrum of the object from light emitted from the surface of the object after the irradiated infrared light penetrates the object using a plurality of spectral filters; And
And a processor that performs image authentication on the object using a signal representing the image of the object and biometric authentication on the object using a signal representing the spectrum of the object. According to claim 1,
The plurality of spectral filters transmit or block a plurality of wavelength bands belonging to the wavelength band absorbed in the process of being emitted from the surface of the living body after penetrating into the human body among the wavelength bands of infrared rays irradiated on the surface of the object Electronic equipment. According to claim 2,
The living body is an electronic device that is a human eye cornea or facial skin. delete According to claim 1,
The plurality of infrared filters and the plurality of spectral filters are arranged in a matrix structure of a two-dimensional plane, the array surface of the plurality of infrared filters and the plurality of spectral filters to form a light receiving surface. The method of claim 5,
The image sensor generates a signal representing the image of the object from light reflected from the surface of the object by using a plurality of infrared filters arranged in a matrix structure corresponding to a part of the matrix structure of the two-dimensional plane,
The spectroscopic sensor uses a plurality of spectroscopic filters arranged in a matrix structure corresponding to different parts of the matrix structure of the two-dimensional plane to infiltrate the object and then spectrum of the object from light emitted from the surface of the object. An electronic device that generates a signal to indicate. The method of claim 5,
The image sensor
A plurality of infrared filters arranged in a matrix structure corresponding to a part of the matrix structure of the two-dimensional plane to filter light reflected from the surface of the object; And
An electronic device including a plurality of photoelectric elements disposed under the plurality of infrared filters to convert light filtered by the plurality of infrared filters into electrical signals. The method of claim 7,
The spectroscopic sensor
A plurality of spectral filters arranged in a matrix structure corresponding to different parts of the matrix of the two-dimensional plane and filtering light emitted from the surface of the object after penetrating into the object; And
An electronic device including a plurality of photoelectric elements disposed under the plurality of spectroscopic filters to convert light filtered by the plurality of spectroscopic filters into electrical signals. According to claim 1,
The plurality of color filters, the plurality of infrared filters, and the plurality of spectral filters are arranged in a matrix structure of a two-dimensional plane, and the arrangement surfaces of the plurality of color filters, the plurality of infrared filters, and the plurality of spectral filters are received. An electronic device forming a surface. According to claim 2,
Each of the spectroscopic filters is an electronic device that blocks wavelength bands determined according to the period in light emitted from the surface of the object after it penetrates into the object by periodically forming metal patterns having a constant shape. The method of claim 10,
The metal patterns of one of the plurality of spectroscopic filters and the metal patterns of the other of the spectroscopic filters are arranged at different periods,
The one spectral filter and the other spectral filter are electronic devices that block different wavelength bands. According to claim 2,
Each spectral filter is
An upper reflective layer and a lower reflective layer separated from each other;
A dielectric layer interposed between the upper reflective layer and the lower reflective layer, wherein at least two materials having different refractive indices are alternately disposed; And
And a buffer layer disposed between at least one of the upper reflection layer and the lower reflection layer and the dielectric layer,
Each spectral filter is an electronic device that passes through a wavelength band determined by a relative volume ratio between the two materials in light emitted from the surface of the object after penetrating into the object. The method of claim 12,
In the dielectric layer, there are at least two areas having different relative volume ratios between the two materials,
An electronic device having regions having different relative volume ratios of the dielectric layer pass through different wavelength bands. The method of claim 5,
The spectroscopic sensor
The plurality of spectral filters;
A plurality of band-pass filters arranged one-to-many to the plurality of spectral filters and disposed under the plurality of spectral filters; And
It includes a plurality of photoelectric elements disposed under the plurality of band-pass filters in one-to-one correspondence to the plurality of band-pass filters,
The plurality of band-pass filters corresponding to the respective spectral filters transmit different wavelength bands in the filtering wavelength band of each spectral filter,
The plurality of infrared filters are arranged in a matrix structure corresponding to a part of the matrix structure of the two-dimensional plane, and the plurality of band-pass filters are arranged in a matrix structure corresponding to other parts of the matrix structure of the two-dimensional plane device. The method of claim 14,
The plurality of band-pass filters are electronic devices that are at least two types of infrared filters and infrared filters of the same type. According to claim 1,
It is disposed below the display panel and further includes a rear panel having at least one light passing hole,
The image sensor generates a signal representing an image of the object from light incident through at least one light passing hole of the rear panel passing through the display panel,
The spectroscopic sensor is an electronic device that passes through the display panel and generates a signal representing the spectrum of the object from light incident through at least one light passing hole of the rear panel. According to claim 1,
And disposed under the display panel further comprises at least one collimator lens for converting light passing through the display panel to parallel light,
The image sensor generates a signal representing an image of the object from light incident through the at least one collimator lens through the display panel,
The spectroscopic sensor is an electronic device that passes through the display panel and generates a signal representing the spectrum of the object from light incident through the at least one collimator lens. According to claim 1,
The image sensor and the spectroscopic sensor are disposed under the display panel,
The image sensor generates a signal indicative of the image of the object from light passing through the display panel when infrared light irradiated on the surface of the object is reflected from the surface of the object,
The spectroscopic sensor is an electronic device that generates a signal representing the spectrum of the object from light passing through the display panel after being emitted from the surface of the object after the irradiated infrared light penetrates the object. In the user authentication method of the electronic device,
The light emitting module is irradiated with infrared rays on the surface of the object;
The image sensor may include generating a signal representing an image of the object from light reflected from the surface of the object by using a plurality of infrared filters to irradiate infrared light onto the surface of the object;
The processor performs image authentication on the object using a signal representing the image of the object;
The spectroscopic sensor generates a signal representing the spectrum of the object from light emitted from the surface of the object after the infrared light irradiated on the surface of the object penetrates the object using a plurality of spectral filters;
The processor performing biometric authentication on the object using a signal representing the spectrum of the object; And
And the processor authenticating a user according to the result of performing the image authentication and the result of performing the biometric authentication. The method of claim 19,
The plurality of spectroscopic filters transmit or block a plurality of wavelength bands belonging to a wavelength band absorbed in the process of being emitted from the surface of the living body after penetrating into a human body from among the wavelength bands of infrared rays emitted from the light emitting module. User authentication method. The method of claim 19,
Further comprising the step of detecting whether the object is closer than the critical distance,
In the step of irradiating the infrared rays, a user authentication method of irradiating infrared rays on the surface of the object when the objects are closer than a critical distance. The method of claim 19,
Further comprising the step of checking whether there is a user authentication request,
The step of irradiating the infrared ray is a user authentication method of irradiating infrared rays on the surface of the object when it is determined that the user authentication request is made. The method of claim 19,
Displaying a graphic for guiding the user's eyes or face to be accurately positioned; And
And displaying an image of the object on the displayed graphic. The method of claim 23,
Further comprising the step of checking whether a certain time has elapsed from the start of the step of displaying the graphic,
Generating a signal representing the image of the object generates a signal representing the image of the object from light reflected from the surface of the object when it is determined that the predetermined time has elapsed,
The step of generating a signal representing the spectrum of the object generates a signal representing the spectrum of the object from light emitted from the surface of the object when it is determined that the predetermined time has elapsed. A computer-readable recording medium recording a computer program for executing the method of any one of claims 19 to 24 on a computer. In the method of registering user biometric information of an electronic device,
The light emitting module is irradiated with infrared rays on the surface of the object;
The image sensor may include generating a signal representing an image of the object from light reflected from the surface of the object by using a plurality of infrared filters to irradiate infrared light onto the surface of the object;
The processor includes generating an image of the object from a signal representing the image of the object;
The spectroscopic sensor generates a signal representing the spectrum of the object from light emitted from the surface of the object after the infrared light irradiated on the surface of the object penetrates the object using a plurality of spectral filters;
The processor generating a spectrum of the object from a signal representing the spectrum of the object; And
The processor comprises registering the generated image and the generated spectrum as biometric information of a user of the electronic device. The method of claim 26,
The plurality of spectroscopic filters transmit or block a plurality of wavelength bands belonging to a wavelength band absorbed in the process of being emitted from the surface of the living body after penetrating the human body in the wavelength band of light irradiated on the object. Biometric information registration method. The method of claim 26,
Displaying a graphic for guiding the user's eyes or face to be accurately positioned; And
And displaying the image of the object on the displayed graphic. The method of claim 28,
Further comprising the step of checking whether a certain time has elapsed from the start of the step of displaying the graphic,
Generating a signal representing the image of the object generates a signal representing the image of the object from light reflected from the surface of the object when it is determined that the predetermined time has elapsed,
The step of generating a signal representing the spectrum of the object is a method of registering user biometric information to generate a signal representing the spectrum of the object from light emitted from the surface of the object when it is determined that the predetermined time has elapsed. A computer-readable recording medium recording a computer program for executing the method of any one of claims 26 to 29 on a computer.

Images