KR20190100838A - Electronic device, user authentication method thereof, and user biometric information registration method thereof - Google Patents

Abstract

The present invention relates to an electronic device, to a user authentication method thereof, and to a user biometric information registration method thereof. A composite sensor generates a signal representing an image of an object from the light reflected from a surface of an infrared radiation irradiated to the surface of the object, and generates a signal representing a spectrum of the object from the light emitted from the surface of the object after the infrared radiation irradiated to the surface of the object penetrates into the object. The processor performs image authentication on the object using the signal representing the image of the object, and uses the signal representing the spectrum of the object to prevent spoofing using a fake image by performing biometric authentication on the object.

Description

Electronic device, its user authentication method, and its user biometric information registration method

An electronic device for authenticating a user using biometrics of a user, a method for authenticating a user thereof, and a method for registering user biometric information thereof.

Conventional portable electronic devices, such as smart phones and tablet PCs (Personal Computers), provide a function for authenticating a user using biometrics such as a user's fingerprint, iris, and face. In recent years, a full-screen smartphone having a full screen has become common in order to maximize the smartphone screen without increasing the size of the smartphone. Most of the conventional portable electronic devices employ a structure in which a fingerprint sensor in the form of a button is exposed to the outside under the front display window. Such a conventional fingerprint sensor has been the biggest obstacle in the implementation of the full screen, the current full-screen type smart phone has a fingerprint sensor installed on the back of the smartphone or other types of biometrics such as fingerprint recognition, iris recognition, face recognition We solve this problem by replacing it with recognition technology.

When the fingerprint sensor is installed on the back of the smartphone, it is not easy to place the finger on the fingerprint sensor while the user is holding the smartphone with the hand, and when the smartphone is placed on a table, There was a problem that unlocking was impossible. Accordingly, a method of replacing fingerprint recognition with other types of biometric technologies such as iris recognition and face recognition or mounting all of fingerprint recognition, iris recognition, and face recognition functions is gradually becoming common. However, the user's iris image or face image can be easily copied to a medium such as paper, silicone, clay, rubber, etc., and user authentication can be passed relatively easily when attempting user authentication using a forged biometric image. There was this. Accordingly, spoofing such as smartphone unlocking using a forged biometric image may become a social problem.

An object of the present invention is to provide an electronic device capable of preventing spoofing using a biological image copied to a medium such as paper, silicon, clay, rubber, or the like. Moreover, it is providing the user authentication method of the electronic device. Another object of the present invention is to provide a method for registering user biometric information of the electronic device. The present invention is not limited to the above-described technical problems, and other technical problems may be derived from the following description.

An electronic device according to an aspect of the present invention includes a display panel for displaying a screen; A light emitting module irradiating infrared rays to the surface of the object; Infrared radiation irradiated onto the surface of the object generates a signal representing an image of the object from light reflected off the surface of the object, and from the light emitted from the surface of the object after the irradiated infrared light penetrates into the object A composite sensor for generating a signal indicative of the spectrum of the object; And a processor configured to perform image authentication on the object using a signal representing the image of the object and perform biometric authentication on the object using a signal representing the spectrum of the object.

The complex sensor generates a signal representing the spectrum of the object from the light emitted from the surface of the object after the infrared rays irradiated onto the surface of the object using a plurality of spectral filters, The spectral filter may transmit or block a plurality of wavelength bands belonging to wavelength bands absorbed in the process of being emitted from the surface of the living body after penetrating the human body in the infrared wavelength band irradiated on the surface of the object. The living body may be human eye cornea or facial skin.

The composite sensor may include an image sensor generating a signal representing an image of the object from light reflected from the surface of the object using a plurality of infrared filters; And a spectroscopic sensor generating a signal representing the spectrum of the object from light emitted from the surface of the object after penetrating the object using a plurality of spectroscopic filters.

The plurality of infrared filters and the plurality of spectroscopic filters may be arranged in a matrix structure of a two-dimensional plane, so that the plurality of infrared filters and the arrangement surface of the plurality of spectroscopic filters may form a light receiving surface.

The image sensor generates a signal representing an image of the object from the light reflected from the surface of the object using a plurality of infrared filters arranged in a matrix structure corresponding to a portion of the matrix structure of the two-dimensional plane, the spectroscopic The sensor is a signal representing the spectrum of the object from the light emitted from the surface of the object after penetrating the object using a plurality of spectral filters arranged in a matrix structure corresponding to the other part of the matrix structure of the two-dimensional plane Can be generated.

The image sensor includes a plurality of infrared filters arranged in a matrix structure corresponding to a portion of the matrix structure of the two-dimensional plane to filter the light reflected from the surface of the object; And a plurality of optoelectronic devices disposed under the plurality of infrared filters to convert light filtered by the plurality of infrared filters into electrical signals.

The spectroscopic sensor includes a plurality of spectroscopic filters arranged in a matrix structure corresponding to another part of the matrix of the two-dimensional plane and filtering light emitted from the surface of the object after penetrating into the object; And a plurality of optoelectronic devices disposed under the plurality of spectral filters to convert light filtered by the plurality of spectral filters into electrical signals.

A plurality of color filters, the plurality of infrared filters, and the plurality of spectroscopic filters are arranged in a matrix structure of a two-dimensional plane, so that the arrangement surface of the plurality of color filters, the plurality of infrared filters, and the plurality of spectroscopic filters is received. It can form a face.

Each spectral filter may be formed in a shape in which metal patterns having a predetermined shape are periodically arranged to block a wavelength band determined according to the period in light emitted from the surface of the object after penetrating into the object.

The metal patterns of one of the spectral filters and the metal patterns of the other spectral filter are arranged at different periods, and the one and the other spectral filters have different wavelength bands. You can block.

Each spectroscopic filter includes an upper reflection layer and a lower reflection layer separated from each other; A dielectric layer interposed between the upper reflection layer and the lower reflection layer and alternately arranged with at least two materials having different refractive indices; And a buffer layer disposed between at least one of the upper and lower reflection layers and the dielectric layer, each spectroscopic filter having a relative volume between the two materials in light emitted from the surface of the object after penetrating into the object. The wavelength band determined by the ratio can be passed.

At least two regions having different relative volume ratios between the two materials may be present in the dielectric layer, and regions having different relative volume ratios of the dielectric layer may pass through different wavelength bands.

The spectroscopic sensor includes the plurality of spectroscopic filters; A plurality of band pass filters disposed under the plurality of spectral filters in one-to-many correspondence with the plurality of spectral filters; And a plurality of optoelectronic devices disposed under the plurality of bandpass filters in a one-to-one correspondence with the plurality of bandpass filters, wherein the plurality of bandpass filters corresponding to each of the spectral filters are filtering wavelength bands of the respective spectral filters. Transmit different wavelength bands, and the plurality of infrared filters are arranged in a matrix structure corresponding to a portion of the matrix structure of the two-dimensional plane, and the plurality of band pass filters are different portions of the matrix structure of the two-dimensional plane. It may be arranged in a matrix structure corresponding to.

The plurality of bandpass filters may be infrared filters of the same type as at least two types of infrared filters among the plurality of infrared filters.

The electronic device further includes a rear panel disposed below the display panel and having at least one light passing hole formed therein, and the composite sensor passes through the display panel and through at least one light passing hole of the rear panel. A signal representing an image of the object and a signal representing the spectrum of the object may be generated from the incident light.

The electronic device further includes at least one collimator lens disposed under the display panel to convert light passing through the display panel into parallel light, and the composite sensor passes through the display panel to the at least one collimator lens. A signal representing the image of the object and a signal representing the spectrum of the object may be generated from the light incident through the light source.

The composite sensor is disposed under the display panel, and the composite sensor generates a signal representing an image of the object from light transmitted through the display panel by reflecting infrared rays emitted from the surface of the object. After the irradiated infrared rays penetrate the object, the emitted infrared light may be emitted from the surface of the object to generate a signal representing the spectrum of the object from the light passing through the display panel.

According to another aspect of the present invention, a method for authenticating a user of an electronic device includes irradiating infrared rays to a surface of an object; Generating a signal representing an image of the object from light reflected from the surface of the object by infrared radiation irradiated onto the surface of the object; Performing image authentication on the object using a signal representing the image of the object; Generating a signal representing the spectrum of the object from light emitted from the surface of the object after infrared radiation irradiated onto the surface of the object penetrates into the object; Performing biometric authentication on the object using a signal representative of the spectrum of the object; And authenticating a user according to a result of performing the image authentication and a result of performing the biometric authentication.

Generating a signal indicative of the spectrum of the object generates a signal indicative of the spectrum of the object from light emitted from the surface of the object after penetrating into the object using a plurality of spectroscopic filters, wherein the plurality of spectroscopy The filter may transmit or block a plurality of wavelength bands belonging to the wavelength band absorbed in the process of being emitted from the surface of the living body after penetrating the human body of the infrared wavelength band irradiated from the light emitting module.

The user authentication method may further include detecting whether the object is closer than the threshold distance, and irradiating the infrared light may irradiate infrared rays on the surface of the object when the object is closer than the threshold distance. Can be.

The user authentication method may further include checking whether there is a user authentication request, and irradiating the infrared light may irradiate infrared light on the surface of the object when it is determined that the user authentication request is received.

The user authentication method includes displaying a graphic for guiding a user's eyes or face to be accurately positioned; And displaying an image of the object on the displayed graphic.

The user authentication method may further include checking whether a predetermined time has elapsed since the start of displaying the graphic, and generating the signal representing the image of the object is determined that the predetermined time has elapsed. And generating a signal representing an image of the object from the light reflected from the surface of the object, and generating a signal representing the spectrum of the object to the surface of the object when it is determined that the predetermined time has elapsed. The irradiated infrared light may generate a signal representing the spectrum of the object from the light emitted from the surface of the object.

According to still another aspect of the present invention, there is provided a computer-readable recording medium having recorded thereon a program for executing the user authentication method on a computer.

According to another aspect of the present invention, there is provided a method of registering user biometric information of an electronic device, the method comprising: irradiating infrared rays to a surface of an object; Generating a signal representing an image of the object from light reflected from the surface of the object by infrared radiation irradiated onto the surface of the object; Generating an image for the object from a signal representing the image of the object; Generating a signal representing the spectrum of the object from light emitted from the surface of the object after infrared radiation irradiated onto the surface of the object penetrates into the object; Generating a spectrum of the object from a signal representing the spectrum of the object; And registering the generated image and the generated spectrum as biometric information of a user of the electronic device.

Generating a signal indicative of the spectrum of the object generates a signal indicative of the spectrum of the object from light emitted from the surface of the object after penetrating into the object using a plurality of spectroscopic filters, wherein the plurality of spectroscopy The filter may transmit or block a plurality of wavelength bands belonging to wavelength bands absorbed in the process of being emitted from the surface of the living body after penetrating the human body of the wavelength band of the light irradiated to the object.

The user biometric information registration method includes displaying a graphic for guiding the user's eyes or face to be accurately positioned; And displaying an image of the object on the displayed graphic.

The user biometric information registration method further includes a step of checking whether a predetermined time has elapsed since the start of displaying the graphic, and generating the signal representing the image of the object is that the predetermined time has elapsed. If it is confirmed that generating a signal representing the image of the object from the light reflected from the surface of the object, and generating a signal representing the spectrum of the object is determined that the predetermined time has elapsed From the light emitted from the surface, a signal representing the spectrum of the object can be generated.

According to still another aspect of the present invention, there is provided a computer-readable recording medium having recorded thereon a program for executing the user biometric information registration method on a computer.

The composite sensor of the electronic device generates a signal representing an image of the object from the light reflected from the surface of the object, and the infrared light emitted from the surface of the object is separated from the surface of the object after the infrared light penetrates into the object. By generating a signal representing the spectrum of the object from the emitted light, user authentication using a fake image is impossible due to image-based image authentication and spectrum-based biometric authentication. As such, spoofing using forged images copied to a medium such as paper, silicon, clay, rubber, etc. may be prevented by performing image-based image authentication and spectrum-based biometric authentication.

In particular, the composite sensor of the present invention is irradiated to the surface of the object using a plurality of spectral filter that can be implemented as a nanostructure smaller than the size of the infrared filter of the CMOS (Complementary Metal Oxide Semiconductor) image sensor instead of the prism, the diffraction grating After infrared ray penetrates into the object, it generates a signal representing the spectrum of the object from the light emitted from the surface of the object, so it is very easy to apply to small electronic devices such as smartphones. Integration makes it possible to implement a complex sensor very easily.

After penetrating the human body in the infrared wavelength band irradiated to the object, the object using a plurality of spectral filters that transmit or block a plurality of wavelength bands belonging to the wavelength band absorbed in the process of being emitted from the surface of the living body By generating a signal representing the spectrum of the user and biometric authentication of the user using the signal, spoofing using a fake image can be blocked at source. That is, inanimate media such as paper, silicone, clay, and rubber used for forgery of a living body image of humans do not have constituents of human living organisms. Since the spectrum generated by irradiating infrared rays onto a biological surface of the living body and the spectrum generated by irradiating infrared rays onto a counterfeit medium have completely different shapes, spoofing using a fake image may be blocked.

In addition, the spectrum of infrared radiation emitted and returned back to the surface of the living body after light has penetrated into the living body is different for each individual, so spectrum-based biometric authentication is used to check whether or not the object is located near an electronic device. It can also provide the ability to determine if an object located close to the device is a true user of the device. When image-based image authentication and spectrum-based biometric authentication according to the present invention are combined, spectrum-based biometric authentication is performed based on spectrum-based biometric authentication in addition to spoofing using media such as paper, silicon, clay, and rubber, to which user images are copied. The security of user authentication can be greatly enhanced.

1 is a view showing an example of an electronic device according to an embodiment of the present invention.
2 is a diagram illustrating biometric recognition of the electronic device illustrated in FIG. 1.
3 is an exploded view of the electronic device shown in FIG. 1.
4 is a diagram illustrating an example of a cross section of the electronic device illustrated in FIG. 2.
FIG. 5 is a diagram illustrating another example of a cross section of the electronic device illustrated in FIG. 2.
FIG. 6 is a diagram illustrating another example of a cross section of the electronic device illustrated in FIG. 2.
FIG. 7 is a diagram illustrating another example of a cross section of the electronic device illustrated in FIG. 2.
FIG. 8 is a diagram illustrating another example of a cross section of the electronic device illustrated in FIG. 2.
FIG. 9 is a diagram illustrating another example of a cross section of the electronic device illustrated in FIG. 2.
FIG. 10 is a view showing a portion of the cross section and a cross section of a human living body shown in FIG.
11 is a diagram schematically showing an example of the structure of the composite sensor 5 shown in FIGS. 3-7.
FIG. 12 is a view schematically showing another example of the structure of the composite sensor 5 shown in FIGS. 3-7.
FIG. 13 is a view schematically showing another example of the structure of the composite sensor 5 shown in FIGS. 3-7.
FIG. 14 is a sectional view of an embodiment of the composite sensor 5 shown in FIG.
15 is a cross-sectional view of another embodiment of the composite sensor 5 shown in FIG.
FIG. 16 is a diagram showing an example of the spectral filter 521 shown in FIGS. 14 and 15.
FIG. 17 is a graph showing simulation results of transmission characteristics of the spectral filter 521 of FIG. 16.
FIG. 18 is a diagram illustrating another example of the spectral filter 521 illustrated in FIGS. 14 and 15.
FIG. 19 is a graph illustrating simulation results of transmission characteristics of the spectral filter 521 of FIG. 18.
20 is a view schematically showing another example of the structure of the spectroscopy sensor 52 shown in FIGS. 3-7.
FIG. 21 is a sectional view of another embodiment of the composite sensor 5 shown in FIG.
22 is a cross-sectional view of another embodiment of the composite sensor 5 shown in FIG.
FIG. 23 is a block diagram illustrating an electronic circuit of the electronic device illustrated in FIG. 3.
24 is a flowchart illustrating a user authentication method according to an embodiment of the present invention.
25 is a flowchart illustrating a user authentication method according to another embodiment of the present invention.
26 is a flowchart illustrating a user biometric information registration method according to an embodiment of the present invention.

Hereinafter, with reference to the drawings will be described embodiments of the present invention; Embodiments of the present invention to be described below are electronic devices capable of preventing spoofing using a biological image copied to a medium such as paper, silicon, clay, rubber, etc., a user authentication method of the electronic device, and a user of the electronic device. It relates to a biometric information registration method. In particular, examples of biometric authentication used in embodiments of the present invention include human iris authentication, face authentication, iris authentication, and the like. Embodiments described below are electronic devices capable of preventing spoofing by performing image authentication and biometric authentication on a living body such as an iris or a face of a user, a user authentication method of the electronic device, and user biometric information registration of the electronic device. It is about a method. Hereinafter, such electronic devices may be referred to simply as "electronic devices."

1 is a view showing an example of an electronic device according to an embodiment of the present invention. Referring to FIG. 1, a representative example of an electronic device according to an embodiment of the present invention may be a full screen smart phone having a front surface of a screen. In recent years, full screen smartphones have been generally used in which a front bezel is minimized to minimize the bezel in order to maximize the smartphone screen without increasing the size of the smartphone. Most conventional smartphones employ a structure in which a fingerprint sensor in the form of a button is exposed to the outside under a display window on the front. Such a conventional fingerprint sensor has been the biggest obstacle in the implementation of the full screen, the current full-screen type smart phone has a fingerprint sensor installed on the back of the smartphone or other types of biometrics such as fingerprint recognition, iris recognition, face recognition We solve this problem by replacing it with recognition technology.

When the fingerprint sensor is installed on the back of the smartphone, it is not easy to place the finger on the fingerprint sensor while the user is holding the smartphone with the hand, and when the smartphone is placed on a table, There was a problem that unlocking was impossible. Accordingly, a method of replacing fingerprint recognition with other types of biometric technologies such as iris recognition and face recognition or mounting all of fingerprint recognition, iris recognition, and face recognition functions is gradually becoming common. As shown in FIG. 1, the current full-screen type smart phone does not have the entire front of the screen as various sensors such as a camera, an illumination sensor and a proximity sensor are installed on the top of the front. In particular, in the case of a smartphone that performs biometrics such as iris recognition and face recognition using infrared rays, such as the electronic device according to the present embodiment, an infrared LED (Light Emitting Diode) and an infrared ray, in addition to the various sensors described above, The camera is installed.

2 is a diagram illustrating biometric recognition of the electronic device illustrated in FIG. 1. FIG. 2A illustrates a state in which an electronic device recognizes an iris of a user, and FIG. 2B illustrates a state of recognizing a user's face. Referring to FIG. 2A, the electronic device irradiates infrared rays to the iris of the user, and generates an iris image from light reflected from the iris due to the infrared irradiation. Referring to FIG. 2B, the electronic device irradiates infrared rays to a face of a user, and generates a face image from light reflected from the irradiated face. Electronic devices have been introduced through a number of patent documents, papers, and the like to authenticate the user by checking whether the iris image or face image generated in this way and the actual iris image or face image of the user stored in the smartphone.

However, since the user's iris image or face image can be easily copied to a medium such as paper, silicon, clay, or rubber, if the user is authenticated using only the iris image or face image, the user's authentication is performed using a fake image. There is a possibility to pass easily. In fact, there have been reports of cases where a user's authentication was made using fake eyes made by putting an iris image on a paper after printing an iris image on the paper. Passing cases are also reported. Accordingly, in the embodiments of the present invention described below, spoofing using a fake image is impossible by using both image-based image authentication and spectrum-based biometric authentication.

3 is an exploded view of the electronic device illustrated in FIGS. 1 and 2. Referring to FIG. 3, the electronic device illustrated in FIG. 1 includes a cover glass 1, a touch panel 2, a display panel 3, a rear panel 4, a composite sensor 5, a bracket 6, and printing. It consists of a circuit board 7, a light emitting module 8, a battery 9, and a housing 10. 3 is an exploded view of a smartphone corresponding to a representative example of the electronic device according to the present embodiment. According to the embodiment shown in FIG. 3, a combined sensor acting as a spectrometer together with the role of a conventional infrared camera is installed under the display panel 3. As the infrared camera, which was previously installed on the top of the front, disappears from the top of the front, the ratio of the screen to the front of the electronic device may increase. Unlike the embodiment shown in FIG. 3, the composite sensor 5 may be installed on the upper front of the electronic device.

In order to prevent the present embodiment from being blurred while the present embodiment can be easily understood, the main configuration of the smartphone required for understanding the present embodiment is shown in FIG. 3. Those skilled in the art to which the present embodiment pertains may understand that another configuration may be added in addition to the configuration shown in FIG. 3. For example, a pressure sensitive adhesive (PSA) layer may be inserted between the cover glass 1 and the touch panel 2 for adhesion therebetween. The cover glass 1, the touch panel 2, the display panel 3, the rear panel 4, etc. are shown in a simple rectangular plate shape to reduce the complexity of the drawing, but may be formed in various shapes such as a curved plate shape. On the other hand, some of the components of the electronic device shown in Figure 2 may be manufactured integrally. For example, the touch panel 2 and the display panel 3 may be manufactured in one piece. In this case, a pressure sensitive adhesive (PSA) layer may be inserted between the cover glass 1 and the polarizing plate of the display panel 3.

The cover glass 1 serves to protect the touch panel 2 and the like located inside the transparent glass plate such as glass and polymer. The touch panel 2 detects a user's touch position using a plurality of touch sensors disposed under the cover glass 1 in a matrix form. The touch panel 2 may be composed of a plurality of touch sensors, a TDI (Touch Driver IC) for driving them, and the like. Touch panels are classified into capacitive, resistive, optical, infrared, and surface acoustic wave (SAW) according to their touch sensing method. Capacitive touch panels are mainly used in smartphones and tablet PCs. It is used. Capacitive touch panels are again classified into attachable, integrated cover windows, integrated displays, and the like. The touch panel 2 has a very high transparency so that the screen of the display panel 3 below it can be clearly seen through it.

The display panel 3 displays a screen using a plurality of pixels disposed under the touch panel 2 and arranged in a matrix form. An example of the pixel of the display panel 3 may be an organic light emitting diode (OLED). The display panel 3 includes a plurality of OLEDs arranged in a matrix form, a polarizer disposed on the plurality of OLEDs to polarize light generated from each OLED, a display driver IC (DDI) for driving the plurality of OLEDs, Wires electrically connecting the two OLEDs and the DDI, and a substrate on which the plurality of OLEDs and the wires are placed. The area between the pixels of the display panel 3 may be transparent so that light reflected by an object located near the cover glass 1 may pass through the display panel 3 and reach the composite sensor 5. Can be.

The rear panel 4 is disposed below the display panel 3 to protect the display panel 3 and to dissipate heat of the display panel 3. As such, since the rear panel 4 serves to protect the display panel 3 and dissipate heat of the display panel 3, the rear panel 4 is generally made of an opaque material such as a metal material. As described below, at least one light passing hole may be formed in the rear panel 4 to transmit the light passing through the display panel 3 to the complex sensor 5. The light irradiated to the cover glass 1 from the outside passes through the transparent cover glass 1 and the touch panel 2, and then the transparent area between the pixels of the display panel 3 and at least one of the rear panel 4. By passing through the light passing hole one by one, the composite sensor 5 can be reached. If the rear panel 4 is made of a transparent material, such a light passing hole may not be necessary.

The composite sensor 5 is disposed below the display panel 3 to represent a signal and spectrum of the object representing an image of the object from the light passing through the cover glass 1, the touch panel 2, and the display panel 3 in order. Generates a signal representing. If the user's eye is placed in the detection area of the composite sensor 5, the object will be the user's eye, and if the user's iris image is located with the copied medium, the object will be such a copy medium. Similarly, if the user's face is placed in the detection area of the complex sensor 5, the object will be the user's face, and if the user's face image is placed on the copied medium, the object will be such a copy medium. As shown in FIG. 2, the detection area of the composite sensor 5 is the front of the electronic device where the user should position the eye or face area so that the composite sensor 5 can accurately detect the user's eyes or face. It means an area.

The display panel 3 may display an image of the object to confirm whether the object is correctly positioned in the detection area of the complex sensor 5. For example, the display panel 3 may display an eye image of the user so that the user can confirm whether the eye is accurately positioned in the detection area of the complex sensor 5. For example, the display panel 3 may display an image of the face of the user so that the user can confirm whether the face is accurately positioned in the detection area of the complex sensor 5. As described below, the composite sensor 5 may be disposed below the display panel 3 in a structure inserted into the rear panel 4, or beneath the display panel 3 in a structure inserted into the bracket 6. It may be arranged in.

The bracket 6 is disposed below the rear panel 4 to support the rear panel 4 positioned thereon, and serves to fix the printed circuit board 7 and the battery 9 positioned thereunder. The bracket 6 may be made of metal and may be made of synthetic resin, and some may be made of metal and others may be made of synthetic resin. Although shown in a simple form in FIG. 2, it may be formed in various forms according to the external shape and internal structure of the smart phone, and may be integrally formed with the housing 10. As described below, the bracket 6 may be formed with a groove in which the complex sensor 5 may be seated.

The printed circuit board 7 is disposed under the bracket 6, and a number of electronic components such as a processor and a memory are mounted thereon. The printed circuit board 7 may be divided into a plurality of pieces. Since the physical form of the electronic components of the printed circuit board 7 is not related to the features of the present embodiment, their illustration is omitted. The processor of the printed circuit board 7 performs image authentication on the object using a signal representing an image of the object and performs biometric authentication on the object using a signal representing the spectrum of the object. The processor for image authentication and biometric authentication may be an application processor (AP) or a central processing unit (CPU) of a smartphone or a dedicated processor for image authentication and biometric authentication. Image generation and spectrum generation described below may be processed by a separate processor called a graphics processing unit (GPU). The processor of the present embodiment may be a processor having a structure in which the above-described processors are integrated into one, or may mean a set of the above-described processors.

The light emitting module 8 is composed of an infrared LED and electronic devices for adjusting the brightness thereof, and irradiates infrared rays onto the surface of the object. The light emitting module 8 may be mounted on the printed circuit board 7 or connected to the printed circuit board 7 by an electric wire. The bracket 6 may have a hole in which the light emitting module 8 is inserted. The area of the touch panel 2, the display panel 3, and the rear panel 4 may cover the area of the cover glass 1 so that infrared rays generated from the light emitting module 8 may pass through the cover glass 1 and irradiate the surface of the object. May be slightly smaller than). The battery 9 supplies power to the touch panel 2, the display panel 3, the composite sensor 5, and the printed circuit board 7. The housing 10 has a cover glass 1, a touch panel 2, a display panel 3, a rear panel 4, a composite sensor 5, a bracket 6, a printed circuit board 7, By accommodating all or part of the battery 9, it serves to protect while maintaining the coupling structure between them.

When the composite sensor 5 is disposed below the display panel 3, the composite sensor 5 reflects the infrared rays emitted from the surface of the object to the cover glass 1, the touch panel 2, and A signal representing an image of the object is generated from light passing through the display panel 3 in sequence, and infrared rays emitted from the object penetrate into the object and are emitted from the surface of the object to cover the glass cover 1 and the touch panel 2. ) And a signal representing the spectrum of the object from the light passing through the display panel 3 in sequence. As described above, the composite sensor 5 may be installed on the front top of the electronic device. In this case, light reflected from the surface of the object among the infrared rays irradiated to the object is incident directly to the composite sensor 5, the composite sensor 5 generates a signal representing the image of the object from the incident light. Similarly, the light emitted from the surface of the object after penetrating into the object among the infrared rays irradiated to the object is incident directly on the composite sensor 5, and the composite sensor 5 is a signal representing the spectrum of the object from the incident light. Create Hereinafter, embodiments of the present invention will be described based on an example in which the composite sensor 5 is disposed below the display panel 3. The following description may be applied to the complex sensor 5 installed on the front upper side of the electronic device as it is.

4 is a diagram illustrating an example of a cross section of the electronic device illustrated in FIG. 2. Referring to FIG. 3, the rear panel 4 includes a protective layer 41 for protecting the display panel 3 and a heat dissipation layer 42 for dissipating heat of the display panel 3. The protective layer 41 may be made of an elastic material such as sponge or rubber to protect the display panel 3. The heat dissipation layer 42 may be made of metal such as aluminum or copper having high thermal conductivity in order to dissipate heat of the display panel 3. As described above, since the protective layer 41 is generally made of an opaque material, a protective layer corresponding to an upper region of the composite sensor 5 so that light passing through the display panel 3 can reach the composite sensor 5. A light passing hole is formed in a part of 41. The light passing hole of the protective layer 41 may be empty or may be filled with a transparent material. The light passing hole may have the same shape as that of the light receiving surface of the compound sensor 5, that is, the upper surface of the compound sensor. It may be slightly smaller than the area. The heat dissipation layer 42 is formed with a hole into which the composite sensor 5 is inserted. The composite sensor 5 may be inserted into the hole of the heat dissipation layer 42 and fixed to the upper surface of the bracket 6.

FIG. 5 is a diagram illustrating another example of a cross section of the electronic device illustrated in FIG. 3. Referring to FIG. 5, unlike the example illustrated in FIG. 4, a light passing hole is formed in the rear panel 4 in its entirety, that is, the protective layer 41 and the heat dissipating layer 42, and the bracket 6 ) Has a groove in which the composite sensor 5 can be seated and fixed. As shown in FIG. 5, the complex sensor 5 may be fixed by being inserted into a groove of the bracket 6 and seated on the bottom surface of the groove. FIG. 6 is a diagram illustrating another example of a cross section of the electronic device illustrated in FIG. 4. Referring to FIG. 6, unlike the example illustrated in FIG. 4, the protective layer 41 of the rear panel 4 has a one-to-one porosity corresponding to a plurality of transparent regions between pixels of the display panel 3 up and down. A plurality of light passing holes of the form are formed.

When the electronic device detects that an object enters the detection area of the composite sensor 5 in the screen lock mode, the electronic device irradiates infrared rays to the object. For example, when the electronic device detects that the user's eyes or face enters the detection area of the complex sensor 5 in the screen lock mode, the electronic device irradiates infrared rays to the user's eyes or face. At this time, the user's eyes or face is also irradiated with visible light by natural light in addition to infrared. Some of the light irradiated to the user's eyes or face is reflected and directed back to the cover glass 1, and passes through the cover glass 1, the touch panel 2, and the display panel 3 in order. Light passing through the display panel 3 reaches the complex sensor 5 through the light passing hole of the rear panel 4. According to the example shown in FIGS. 3-5, the composite sensor 5 passes through the cover glass 1, the touch panel 2, and the display panel 3 in order to pass through at least one light passing through the rear panel 4. From the light incident through the hole, a signal representing the image of the object and a signal representing the spectrum of the object are generated.

4-6, each pixel of the display panel 3 is represented by a black square point. Light between these black blind spots, that is, pixels, etc. of the display panel 3 reaches the composite sensor 5. Due to the limitations of the finely sized pixel illustration, the size of each pixel is exaggerated in FIG. Through the path, the light reaches the composite sensor 5. 4-6, wires of the display panel 3 are omitted. Since the wire is also an opaque material, light reaches the composite sensor 5 through the transparent region between the pixel and the wire and the wire and the pixel. Those skilled in the art can understand that the light reflected from the surface of the object in the various modified forms of the example shown in FIGS. 4-6 can reach the composite sensor 5. .

FIG. 7 is a diagram illustrating another example of a cross section of the electronic device illustrated in FIG. 3. Referring to FIG. 7, a convex lens-type collimator lens is inserted into the light passing hole of the rear panel 4. In the rear panel 4 illustrated in FIG. 7, light passing holes are formed in the entire thickness, that is, the protective layer 41 and the heat dissipating layer 42, and the composite sensor 5 is seated and fixed to the bracket 6. Grooves that can be formed are formed. The collimator lens 50 is inserted into the light passing hole of the rear panel 4 and is disposed below the display panel 3 so that light passing through the display panel 3 is perpendicular to the upper surface of the composite sensor 5. Convert into incident parallel light. As described below, the plurality of infrared filters 511 and the plurality of spectral filters 521 of the composite sensor 5 are arranged in a matrix structure. When light is vertically incident on each of the plurality of infrared filter 511 and the plurality of spectral filters 521, the transmittance or blocking rate of each filter may be maximized to more accurately generate an image and spectrum of an object. The collimator lens 50 serves to allow light to enter the filter vertically.

FIG. 8 is a diagram illustrating another example of a cross section of the electronic device illustrated in FIG. 2. Referring to FIG. 8, a plurality of collimator lenses 50 are inserted into the light passing holes of the rear panel 4. In FIG. 8, a portion of the cross section of the electronic device illustrated in FIG. 3 is enlarged so that the change in the light path by each of the plurality of collimator lenses 50 can be seen. In the rear panel 4 illustrated in FIG. 8, a light passing hole is formed in the entire thickness thereof, as in the example illustrated in FIG. 7, and the groove in which the complex sensor 5 is seated and fixed to the bracket 6 is provided. Is formed. However, in the example shown in FIG. 7, one collimator lens 50 is inserted into the light passing hole of the rear panel 4. In the example shown in FIG. 8, a plurality of collimator lenses 50 are inserted into the light passing hole of the rear panel 4. The collimator lens 50 is inserted. As shown in FIG. 8, the plurality of collimator lenses 50 may be manufactured in a single body in which neighboring lenses are connected to each other.

As shown in FIG. 7, when one collimator lens 50 is inserted into the light passing hole of the rear panel 4, the light passing hole of the back panel 4 is formed of the light passing through the display panel 3. Since the light rays traveling along the center line are incident on the center portion of the collimator lens 50, the parallel light conversion efficiency is high, but the light rays far from the center line of the light passing hole of the rear panel 4 are incident on the edge portion of the collimator lens. Therefore, the parallel light conversion efficiency is low. Accordingly, in the example shown in FIG. 8, the plurality of collimator lenses 50 are inserted into the light passing holes of the rear panel 4 so as to correspond one-to-one to the plurality of transparent regions between the pixels of the display panel 3 up and down. It is. Accordingly, light rays passing through a plurality of transparent regions between the pixels of the display panel 3 may be incident on the center of each collimator lens.

FIG. 9 is a diagram illustrating another example of a cross section of the electronic device illustrated in FIG. 3. Referring to FIG. 9, the protective layer 41 of the rear panel 4 is formed with a plurality of light passing holes having a porous shape corresponding one to one vertically to a plurality of transparent regions between pixels of the display panel 3. The light passing hole of the heat dissipating layer 42 is formed with one light passing hole having an opening formed between the plurality of light passing holes of the protective layer 41. A plurality of collimator lenses 50 are inserted into the light passing holes of the heat dissipation layer 42. In FIG. 8, a portion of the cross section of the electronic device illustrated in FIG. 2 is enlarged so that the change in the light path by each of the plurality of collimator lenses 50 can be seen. The plurality of light passing holes of the protective layer 41 of the rear panel 4 pass through the transparent region between the pixels of the display panel 3 to improve the parallel light conversion efficiency of each collimator lens 50. It serves to guide the incident to the center of each collimator lens 50.

As shown in FIGS. 7-9, when the user places the eye in the detection area of the complex sensor 5, the infrared rays generated in the light emitting module 8 are irradiated to the user's eyes. Some of the infrared radiation irradiated to the user's eye is reflected at the eye's iris surface, and some penetrates into the cornea of the eye and then returns to the corneal surface and is emitted from the corneal surface. When the user places the face in the detection area of the composite sensor 5, the infrared rays generated in the light emitting module 8 is irradiated to the user's face. Some of the infrared radiation irradiated on the user's face is reflected at the face surface, and some penetrates into the face skin and then returns to the surface of the face skin and is emitted from the surface of the face skin. In addition to infrared rays, the user's eyes or face are irradiated with natural light or artificial light, and, like infrared rays, some are reflected and some are emitted after penetration. As such, the light directed toward the cover glass 1 passes through the cover glass 1, the touch panel 2, and the display panel 3 in order. The light passing through the display panel 3 reaches the complex sensor 5 through at least one collimator lens 50 inserted into the light passing hole of the rear panel 4.

According to the example illustrated in FIGS. 7-9, the composite sensor 5 passes through the cover glass 1, the touch panel 2, and the display panel 3 in order to pass through at least one light passing through the rear panel 4. A signal representing the image of the object and a signal representing the spectrum of the object are generated from the light incident through the hole and the at least one collimator lens 50. In the example shown in FIGS. 7-9, the collimator lens 50 is illustrated as a structure covering the entire surface of the composite sensor 5, that is, the entire light receiving surface. However, the collimator lens 50 is a part of the upper surface of the equity sensor 5. The upper surface of 52, that is, only the light receiving surface of the spectroscopy sensor 52 may be covered. In this case, the composite sensor 5 passes through the cover glass 1, the touch panel 2, and the display panel 3 in order, and at least one light passing hole of the rear panel 4 and at least one collimator lens ( A signal representing the spectrum of the object is generated from the light incident through 50).

FIG. 10 is a view showing a portion of the cross section and a cross section of a human living body shown in FIG. 4. FIG. 10A illustrates a portion of the cross section shown in FIG. 4 and a cross section of the human skin. Due to the size limitation of one figure, a portion of the cross section shown in FIG. 4 and a cross section of the human skin are shown in close proximity to each other in FIG. 4 but are actually far more apart than shown. Referring to FIG. 10A, the human skin is composed of two layers of epidermis and dermis, and blood vessels pass under the dermis. Some of the infrared rays irradiated to the user's face from the light emitting module 8 are reflected from the face surface and the rest penetrates into the face skin. Infrared rays that penetrate into the facial skin move through the body, repeating reflections and refractions at the boundaries of different refractive indices. Some of the infrared light that penetrates into the facial skin returns to the face surface and is emitted from the face surface. Because the tissue inside a person's skin varies from person to person, the light emitted from the face's surface conveys unique spectral information.

Some of the infrared rays irradiated on the face from the light emitting module 8 are reflected on the epidermal surface of the face, some penetrate into the skin, some are absorbed within the two layers of the epidermis and dermis, and the rest are scattered. This absorption is mainly due to components in the blood such as hemoglobin, bilirubin, beta-carotene and the like. Forgery of a person's face is achieved by copying a user's face image onto an inanimate medium such as paper or silicon. Not only does the inanimate medium contain no components of human skin, such as blood, but since the internal tissue of the inanimate medium is completely different from the internal tissue of the human skin, the spectrum of light that penetrates into it and then returns to the surface is emitted. After penetrating into it, it returns to the skin surface and has a completely different shape from the spectrum of light emitted. 10 (b) shows a human eye corneal cross section. The cornea consists of six layers: epitherium, bowman's membrane, stroma, decemet's membrane, and endothelial. Like human skin, the components and internal tissues of the cornea are completely different from the components and internal tissues of the inanimate medium, so the spectrum of light emitted and returned to the surface after infiltrating into the inanimate medium again penetrates into the cornea and then back to the corneal surface. This results in a completely different shape from the spectrum of the emitted light.

The composite sensor 5 generates a signal representing an image of the object from the light transmitted through the display panel 3 by the infrared rays reflected from the surface of the object located in the sensing area of the object, and the surface of the object The infrared rays irradiated into the object return to the surface of the object and are emitted from the surface of the object to generate a signal representing the spectral characteristics of the object from the light passing through the display panel 3. As described above, the composite sensor 5 of the present embodiment returns to the surface of the object after penetrating into the object in addition to image authentication using the object image generated from the light reflected from the surface of the object and passing through the display panel 3. The spectral characteristics generated from the light emitted from the surface of the light and passed through the display panel 3 are used to authenticate the living body, so that media such as paper, silicon, clay, rubber Spoofing used may be blocked at source.

Someone other than the user can create a fake image by copying the user's biometric image to media such as paper, silicon, clay, rubber, etc. to unlock the user's smartphone, sneak up on the smartphone's contents, or download a file. . As described above, since the tissue of the medium is completely different from the tissue in the human skin and the corneal tissue, the spectrum generated when the fake image is located in the detection area of the composite sensor 5 is characterized by the fact that the eye or face of the user has a complex sensor ( When it is located in the sensing area of 5), it has a completely different shape from the generated spectrum. Accordingly, even if the fake image has passed the image authentication process, it cannot pass through the biometric authentication process. Therefore, it is impossible to unlock the smartphone using the fake image.

In addition, as described above, since the tissue inside the human cornea or the tissue inside the skin is different for each individual, the spectrum of infrared rays emitted back to the surface of the living body after light penetrates into the human living body is different from person to person. In addition to verifying the biometrics of an object located in proximity to an electronic device, the biometric authentication may also provide the ability to verify that an object located near the electronic device is a true user of the electronic device. In case of combining image-based image authentication and spectrum-based biometric authentication according to the present embodiment, spectrum-based biometric authentication is performed based on image-based authentication in addition to spoofing using media such as paper, silicon, clay, and rubber, to which user images are copied. Authentication can be supplemented, which greatly enhances the security of user authentication.

The composite sensor 5 of the present embodiment is reflected from the surface of the object and penetrates into the object and the image sensor 51 generating a signal representing the image of the object from the light passing through the display panel 3 from the surface of the object. It consists of a spectroscopic sensor 52 that generates a signal representing the spectrum of the object from the light emitted and passed through the display panel 3. When the wavelength of the infrared light is 700 to 900 nm, the image sensor 51 may generate a clear face image as the reflectance of the human skin is well represented. In particular, in this wavelength band, since the change in the contrast pattern is apparent by the change in wavelength by the pigment cells in the iris, the image sensor 51 can generate a signal representing a clear iris image. In the infrared wavelength band of 700 ~ 900nm 810 ~ 850nm the depth of human body appears large. Therefore, when infrared rays of 810 to 850 nm are irradiated to the eyes or face of the user, the spectroscopic sensor 52 may generate a signal representing the spectrum of tissue characteristics of the eye cornea and face skin of the user. In the present embodiment, the light emitting module 8 preferably irradiates infrared rays of 810 ~ 850nm to the surface of the object.

11 is a diagram schematically showing an example of the structure of the composite sensor 5 shown in FIGS. 3-7. In FIG. 11, the abbreviation "I" represents an infrared filter 511, the abbreviation "S" represents a spectral filter 521, and the abbreviation "P" represents an optoelectronic device 512 and 522. In the image sensor 51 according to the present exemplary embodiment, the infrared rays irradiated onto the surface of the object by using the plurality of infrared filters 511 and the plurality of photoelectric elements 512 are reflected from the surface of the object to pass through the display panel 3. Generates a signal from the image representing the object. The spectroscopic sensor 52 uses a plurality of spectral filters 521 and a plurality of photoelectric elements 522 to radiate the display panel 3 after being emitted from the surface of the object after the infrared rays irradiated into the surface of the object. Generates a signal representing the spectrum of the object from the light passing through it. Representative examples of the respective photoelectric devices 512 and 522 include a photodiode. Accordingly, the abbreviation of the optoelectronic device is indicated by "P" in FIG.

The same type of photodiode may be used as the photoelectric element 512 of the image sensor 51 and the photoelectric element 522 of the spectroscopic sensor 52, or different types of photodiodes may be used. The signal representing the image of the object may be a signal converted into a digital signal after amplifying the electrical signals output from the plurality of photoelectric elements 512, that is, the plurality of photodiodes 512. Similarly, the signal representing the spectrum of the object may be a signal converted into a digital signal after the electrical signals output from the plurality of photodiodes 522 are amplified. Amplification of the electrical signals output from the plurality of photodiodes 512 and 522 and conversion into digital signals may be performed by an amplifier and a converter integrated with the composite sensor 5, It may be performed by a converter.

Referring to FIG. 11, the plurality of infrared filters 511 of the image sensor 51 and the plurality of spectral filters 521 of the spectroscopic sensor 52 are arranged in a matrix structure of a two-dimensional plane. An array surface of the plurality of infrared filters 511 and the plurality of spectroscopic filters 521 forms a light receiving surface of the composite sensor 5 with respect to light passing through the display panel 3. That is, the plurality of infrared filter 511 is arranged in a matrix structure corresponding to a portion of the matrix structure of the two-dimensional plane, the plurality of spectral filters 521 is a matrix structure corresponding to another portion of the matrix structure of the two-dimensional plane Are arranged. Accordingly, the light passing through the display panel 3 is incident to the plurality of infrared filter 511 such that the light passing through the display panel 3 is incident on the plurality of infrared filter 511 and the plurality of spectral filters 521, respectively. And it is preferable to make the incident perpendicular to the array surface of the plurality of spectral filter 521. As described above, the technical means for allowing the plurality of infrared filters 511 and the plurality of spectral filters 521 to be incident perpendicularly to each other are perpendicularly described.

That is, the image sensor 51 of the present embodiment is reflected from the surface of the object and passed through the display panel 3 using the plurality of infrared filters 511 arranged in a matrix structure corresponding to a part of the matrix structure of the two-dimensional plane. Generates a signal representing an image of an object from a light. The spectroscopic sensor 52 of this embodiment penetrates into an object using a plurality of spectral filters 521 arranged in a matrix structure corresponding to another part of a matrix structure of a two-dimensional plane, and then is emitted from the surface of the object to display the display panel. From the light passing through (3), a signal representing the spectrum of the object is generated.

The plurality of infrared filters 511 of the image sensor 51 are arranged in a matrix structure corresponding to a part of a matrix structure of a two-dimensional plane that forms a light receiving surface of the composite sensor 5, and passes through the display panel 3. To filter. The plurality of infrared filter 511 is composed of several types of infrared filter 511 for transmitting infrared rays of different wavelength bands from the light passing through the display panel. For example, the plurality of infrared filters 511 may transmit a plurality of infrared filters for transmitting infrared rays in the 810 to 820 nm band and block light in the remaining bands, and a plurality of infrared filters for transmitting infrared rays in the 820 to 830 nm band and block light in the remaining bands. Infrared filter, a plurality of infrared filters that transmit infrared rays of the 830 ~ 840nm band and block the light of the remaining bands, and a plurality of infrared filters that transmit infrared rays of the 840 ~ 850nm band and blocks the light of the remaining bands have.

The photoelectric elements 512 are arranged under the plurality of infrared filters 511 in the same matrix structure as the matrix structure of the plurality of infrared filters 511 so as to correspond one-to-one to the plurality of infrared filters 511. The plurality of optoelectronic devices 512 converts the light filtered by the plurality of infrared filters 511 into electrical signals to generate a signal representing an image of the object from the light reflected from the surface of the object and passing through the display panel 3. do. Each photoelectric device 512 converts the light filtered by the infrared filter 511 corresponding to one up and down one by one into an electrical signal having a different level according to its intensity. An image of the object may be generated from a combination of electrical signals generated by each of the plurality of optoelectronic devices 512.

The plurality of spectral filters 521 of the spectroscopy sensor 52 are arranged in a matrix structure corresponding to another part of the matrix structure of the two-dimensional plane forming the light receiving surface of the complex sensor 5, and pass through the display panel 3. Filter the light. The plurality of spectral filters 521 of the present exemplary embodiment return to the surface of the human body after penetrating the human body in the wavelength band of infrared rays irradiated from the light emitting module 8 to the surface of the human body for biometric authentication of the object. It transmits or blocks a plurality of wavelength bands belonging to the wavelength band absorbed in the process of being emitted from the surface of the human body. Here, the human living body may be human eye cornea or facial skin. The plurality of spectral filters 521 filter various types of spectral filters 521 for filtering light passing through the display panel 3 by blocking or transmitting infrared rays of different wavelength bands from the light passing through the display panel 3. The spectroscopic filters 521 of various types are alternately arranged in a matrix structure as described above.

For example, the various types of spectral filters 521 are filled in one row or one column of the matrix belonging to different parts of the matrix structure of the two-dimensional plane one by one, and when the arrangement of all kinds of spectral filters 521 is completed. They can be arranged alternately by filling in one by one again. As described above, the spectroscopic sensor 52 of the present embodiment may detect a spectrum in the 810 to 850 nm band for biometric authentication of an object located near the electronic device. In this case, the plurality of spectral filters 521 may be composed of various types of spectral filters 521 that filter by dividing the 810 to 850 nm band by several nm units. For example, the plurality of spectral filters 521 may be composed of various types of spectral filters 521 that filter by dividing the 810 to 850 nm band by 5 nm units.

That is, the plurality of spectral filters 521 is composed of various types of spectral filters 521 that transmit or block different wavelength bands in the 810 to 815 nm band, the 815 to 820 nm band, and the 845 to 850 nm band. Can be. The smaller the unit for dividing the band of the light passing through the display panel 3, that is, the more types of the plurality of spectral filters 521, the higher the resolution of the spectrum detected by the spectral sensor 52. However, as the number of overlapping arrangements of one type of spectral filter 521 that transmits or blocks the same wavelength band is reduced by that much, the accuracy of the spectrum detected by the spectroscopic sensor 52 may be lowered. Therefore, it is desirable to design the size of the band division unit for the light passing through the display panel 3 by bridge the resolution and accuracy of the spectrum suitable for biometric authentication of an object located in proximity to the electronic device.

 The plurality of photoelectric elements 522 are arranged under the plurality of spectral filters 521 in the same matrix structure as the matrix structure of the plurality of spectral filters 521 so as to correspond one-to-one to the plurality of spectral filters 521. The photoelectric elements 522 represent the spectrum of the object from the light emitted from the surface of the object after passing through the display panel after penetrating the object by converting the light filtered by the plurality of spectral filters 521 into an electrical signal. Generate a signal. Each photoelectric device 522 converts the light filtered by each spectral filter 521 corresponding one to one up and down to the electric signal of different levels according to its intensity. Some of the plurality of optoelectronic devices 522 penetrates into the object and then emits from the surface of the object to produce an electrical signal corresponding to a certain wavelength band of light passing through the display panel, while the other optoelectronic device has a different wavelength band. Generates an electrical signal corresponding to A spectrum required for biometric authentication of an object may be generated from a combination of electrical signals generated by each of the plurality of optoelectronic devices 522.

The plurality of photoelectric elements 522 may be disposed under the plurality of spectral filters 521 so as to correspond to the plurality of spectral filters 521 one to many. In this case, the light filtered by the one spectral filter 521 is incident on the plurality of photoelectric elements 522 disposed under the spectral filter 521 among the plurality of photoelectric elements 522. Each of the photoelectric elements 522 disposed under the spectral filter 521 converts the filtered light into an electrical signal by the spectral filter 521. For example, when the size of each spectral filter 521 is manufactured to be larger than the size of each photoelectric element 522, the plurality of photoelectric elements 522 may be plural in one-to-many correspondence with the plurality of spectral filters 521. May be disposed below the two spectral filters 521.

Conventional biometrics technology is difficult to miniaturize because the spectrum of the object is generated using a prism, a diffraction grating, etc., it is impossible to apply to a smartphone. Accordingly, the composite sensor 5 of the present embodiment uses a plurality of spectral filters that may be implemented as nanostructures smaller than the size of the infrared filter of the CMOS (Complementary Metal Oxide Semiconductor) image sensor instead of the prism and the diffraction grating. After penetrating, light emitted from the surface of the object and passing through the display panel 3 generates a signal representing the spectrum of the object. The miniaturization means of the spectral filter of the present embodiment will be described in detail below. That is, the plurality of spectral filters 521 of the present embodiment penetrates into the living body of the human among the wavelength band of the light irradiated to the object from the light emitting module 8 and then returns to the living body surface of the human and is emitted from the living body surface of the human. It can be said to be a combination of spectral filters that transmit or block different wavelength bands by dividing the wavelength band absorbed in the process.

For example, the plurality of spectral filters 521 of the present embodiment penetrates into the human cornea of the wavelength band of light irradiated to the object from the light emitting module 8 and then returns to the corneal surface to be emitted from the corneal surface. It can be said to be a combination of spectral filters that transmit or block different wavelength bands by dividing the wavelength band absorbed by. Alternatively, the plurality of spectral filters 521 of the present embodiment penetrates into the human skin of the wavelength band of light emitted from the light emitting module 8 and then returns to the facial skin surface and is emitted from the facial skin surface. It can be said to be a combination of spectral filters that transmit or block different wavelength bands by dividing the wavelength band absorbed by.

In the example shown in FIG. 11, the plurality of infrared filters 511 are arranged in a matrix structure corresponding to the right part of the matrix structure of the two-dimensional plane forming the light receiving surface of the composite sensor 5, and the plurality of spectral filters ( 521 is arranged in a matrix structure corresponding to the left portion of the two-dimensional planar matrix structure forming the light receiving surface of the composite sensor 5. FIG. 12 is a view schematically showing another example of the structure of the composite sensor 5 shown in FIGS. 3-7. In the example shown in FIG. 12, the plurality of infrared filter 511 is arranged in a matrix structure corresponding to the cross portion from which four corner portions are removed from the two-dimensional planar matrix structure forming the light receiving surface of the composite sensor 5. The plurality of spectral filters 521 are arranged in a matrix structure corresponding to four corner portions of the matrix structure of the two-dimensional plane forming the light receiving surface of the complex sensor 5. FIG. 13 is a view schematically showing another example of the structure of the composite sensor 5 shown in FIGS. 3-7. In the example shown in FIG. 13, the plurality of infrared filter 511 is arranged in a matrix structure corresponding to the center portion from which the edge portion is removed from the matrix structure of the two-dimensional plane forming the light receiving surface of the composite sensor 5, The plurality of spectral filters 521 are arranged in a matrix structure corresponding to the edge portion of the matrix structure of the two-dimensional plane forming the light receiving surface of the complex sensor 5.

In the example shown in FIG. 11, as the plurality of spectral filters 521 are concentrated and arranged in one region of the light receiving surface of the complex sensor 5, a spectrum of light irradiated to the region can be generated relatively accurately. On the other hand, there is a disadvantage in that the spectrum of some of the light irradiated to the light receiving surface of the complex sensor 5 may be missing. On the other hand, in the example shown in Figs. 12-13, the spectrum of the entire light irradiated to the light receiving surface of the composite sensor 5 is arranged as scattered and arranged without being concentrated in any one area of the light receiving surface of the composite sensor 5 On the other hand, the number of the spectral filters 521 arranged in each region is small, and thus the accuracy of the spectrum may be slightly reduced. In consideration of the performance of each spectral filter 521, it is preferable that an arrangement structure of the plurality of infrared filters 511 and the plurality of spectral filters 521 is designed. Those skilled in the art will understand that a plurality of infrared filter 511 and a plurality of spectral filters 521 may be arranged in various structures other than the examples shown in FIGS. 11-13. Can be.

FIG. 14 is a sectional view of an embodiment of the composite sensor 5 shown in FIG. 14 shows an example in which the composite sensor 5 is implemented by integrating the image sensor 51 and the spectroscopic sensor 52 on one substrate. Those skilled in the art can understand that the image sensor 51 and the spectroscopic sensor 52 can be implemented on a separate substrate. However, a method of integrating the plurality of infrared filters 511 and the plurality of spectral filters 521 on one substrate may be advantageous for simplifying the manufacturing process of the composite sensor 5. In FIG. 14, the abbreviation "L" represents a microlens, the abbreviation "I" represents an infrared filter, the abbreviation "S" represents a spectral filter, and the abbreviation "P" represents an optoelectronic device. The illustration of the embodiment of the composite sensor 5 shown in FIGS. 12 and 13 is omitted since it can be easily understood how the composite sensor 5 of the example shown in FIGS. FIG. 14A illustrates an embodiment of the composite sensor 5 employing a front side illumination (FSI) image sensor 51, and FIG. 14B illustrates a backside irradiation type (BSI). Back Side Illumination) An embodiment of the composite sensor 5 employing the image sensor 51 is shown.

Referring to a part corresponding to the image sensor 51 of FIG. 14A, a plurality of photoelectric elements 512 on the composite sensor substrate 500 form a light receiving surface of the composite sensor 5. Stacked and arranged in a matrix structure corresponding to a portion of the matrix structure. On the plurality of optoelectronic devices 512, an insulating layer 503 in which wires for supplying power to each infrared filter 511 and each optoelectronic device 512 is embedded is stacked. Accordingly, the insulating layer 503 may be referred to as a wiring layer. The insulating layer 503 may be made of a transparent dielectric material such as SiO ₂ . On the insulating layer 503, a plurality of infrared filter 511 are arranged and stacked in the same matrix structure as that of the plurality of spectral filters 521 so as to correspond one to one up and down with the plurality of photoelectric elements 512. Light filtered by each infrared filter 511 passes through wires of the insulating layer 503 to reach each photoelectric device 512. The planarization layer 502 for flattening the arrangement surface of the plurality of infrared filter 511 is stacked on the plurality of infrared filter 511. The planarization layer 502 is made of a transparent dielectric material. On the planarization layer 502, a plurality of microlenses 501 are stacked and arranged in the same matrix structure as that of the plurality of infrared filter 511 so as to correspond one to one up and down with the plurality of infrared filters 511. Each microlens 501 serves to focus light incident on it to each infrared filter 511 beneath it.

Referring to the portion corresponding to the spectroscopic sensor 52 of FIG. 14A, a plurality of photoelectric elements 522 on the composite sensor substrate 500 form a light receiving surface of the composite sensor 5. Stacked and arranged in a matrix structure corresponding to other parts of the matrix structure. An insulating layer 503 having a plurality of spectral filters 521 and wires embedded with wires for supplying power to the photoelectric elements 522 is stacked on the photoelectric elements 522. On the insulating layer 503, a plurality of spectral filters 521 are stacked and stacked in the same matrix structure as that of the plurality of optoelectronic devices 522 so as to correspond to the plurality of optoelectronic devices 522 one to one. Light filtered by each spectroscopic filter 521 passes through wires of the insulating layer 503 to reach each photoelectric device 522. On the plurality of spectral filters 521, a planarization layer 502 for flattening the arrangement surface of the plurality of spectral filters 521 is stacked. On the planarization layer 502, a plurality of microlenses 501 are arranged and stacked in the same matrix structure as that of the plurality of spectral filters 521 so as to correspond to the plurality of spectral filters 521 one-to-one. Each microlens 501 serves to focus light incident on it to each spectral filter 521 below it.

Looking at the portion corresponding to the image sensor 51 of Figure 14 (b), a wire for supplying power to each infrared filter 511 and each photoelectric device 512 is embedded on the composite sensor substrate 500. The insulating layer 503 is laminated. A plurality of optoelectronic devices 512 are arranged and stacked on the insulating layer 503 in a matrix structure corresponding to a part of a two-dimensional planar matrix structure forming a light receiving surface of the complex sensor 5. The infrared filters 511 are arranged on the plurality of optoelectronic devices 512 in the same matrix structure as the matrix structure of the plurality of spectral filters 521 so as to be one-to-one corresponding to the plurality of photoelectric devices 512. As described above, in the image sensor 51 of FIG. 14A, the insulating layer 503 is disposed on the plurality of photoelectric elements 512, whereas in the image sensor 51 of FIG. 14B, the plurality of photoelectric cells is disposed. The image sensor 51 of FIG. 14B has a laminated structure similar to that of the image sensor 51 of FIG. 14A except that an insulating layer 503 is disposed below the element 512. . Accordingly, the remaining configuration of the image sensor 51 of FIG. 14B will be replaced with the description of the image sensor 51 of FIG. 14A.

 Referring to the portion corresponding to the spectroscopy sensor 52 of FIG. 14B, wires for supplying power to each spectroscopic filter 521 and each photoelectric device 522 are embedded on the composite sensor substrate 500. The insulating layer 503 is laminated. A plurality of optoelectronic devices 522 are arranged on the insulating layer 503 in a matrix structure corresponding to other parts of the two-dimensional planar matrix structure forming the light receiving surface of the complex sensor 5. A plurality of spectral filters 521 are arranged on the plurality of optoelectronic devices 522 in the same matrix structure as the matrix structure of the plurality of photoelectric devices 522 so as to be one-to-one corresponding to the plurality of photoelectric devices 522. As described above, in the spectroscopic sensor 52 of FIG. 14A, the insulating layer 503 is disposed on the plurality of photoelectric elements 512, while in the spectroscopic sensor 52 of FIG. 14B, the plurality of photoelectric cells is disposed. Except that the insulating layer 503 is disposed under the element 512, the spectroscopy sensor 52 of FIG. 14B has a stacked structure similar to that of the spectroscopy sensor 52 of FIG. 14A. . Accordingly, the remaining configuration of the spectroscopic sensor 52 of FIG. 14B will be replaced with the description of the spectroscopic sensor 52 of FIG. 14A.

14A and 14B, the insulating layer 503 is disposed on the plurality of optoelectronic devices 512 in the front-illuminated composite sensor 5, so that the plurality of optoelectronic devices 512 are incident on the plurality of optoelectronic devices 512. The disadvantage is that there is a lot of light loss. On the other hand, in the back-illumination composite sensor 5, since the insulating layer 503 is disposed under the plurality of photoelectric elements 512, there is almost no loss of light incident on the plurality of photoelectric elements 512, but between adjacent infrared filters. There is a disadvantage in that different bands of light mixing can occur. FIG. 14 is a schematic conceptual view of a structure of a composite sensor according to an embodiment of the present invention. Those skilled in the art may understand that other layers may be added in addition to the layer shown in FIG. 14. have. For example, an anti-reflection layer for preventing reflection of light incident on the plurality of optoelectronic devices 512 may be stacked on the plurality of optoelectronic devices 522.

15 is a cross-sectional view of another embodiment of the composite sensor 5 shown in FIG. The example illustrated in FIG. 15 is different from the example illustrated in FIG. 14 in that three quarters of the plurality of infrared filter 511 illustrated in FIG. 14 are replaced with a color filter 5110. Hereinafter, the example illustrated in FIG. 15 will be described based on differences from the example illustrated in FIG. 14, and the following description will be replaced with the description of the example illustrated in FIG. 14. Referring to FIG. 15, a plurality of color filters 5110, a plurality of infrared filters 511, and a plurality of spectral filters 521 of the image sensor 51 are arranged in a matrix structure of a two-dimensional plane. Arranged surfaces of the plurality of color filters 5110, the plurality of infrared filters 511, and the plurality of spectroscopic filters 521 form a light receiving surface of the composite sensor 5 with respect to light passing through the display panel 3. do. That is, the plurality of color filters 5110 and the plurality of infrared filters 511 are arranged in a matrix structure corresponding to a part of the matrix structure of the two-dimensional plane, and the plurality of spectral filters 521 are of the matrix structure of the two-dimensional plane. It is arranged in a matrix structure corresponding to other parts.

The plurality of color filters 5110 may include a plurality of red filters that transmit light in a red band and block light in the remaining bands, a plurality of green filters that transmit green light and block light in a remaining band, and light in a blue band. It consists of a plurality of blue filters for transmitting the light and blocking the light of the remaining band. In the unit filter consisting of four color filters 5100, one red filter, two green filters, and one blue filter are alternately arranged in a matrix structure of a Bayer mosaic pattern, as shown in FIG. Likewise, one of the two green filters is replaced with the infrared filter 511. That is, the plurality of red filters, the plurality of green filters, the plurality of blue filters, and the plurality of infrared filters 511 are matrixes corresponding to a part of the matrix structure of the two-dimensional plane in a pattern in which unit filters each of which is combined one by one are repeated. The plurality of spectral filters 521 are arranged in a matrix structure corresponding to another part of the matrix structure of the two-dimensional plane.

The composite sensor 5 shown in FIG. 15 has a role of a visible light camera that can generate a signal representing a color image in addition to a role of an infrared camera and a spectroscope, and also a visible light camera and an infrared light installed at the top of the smartphone. As the camera disappears from the top of the front, the proportion of the screen to the front of the electronic device can be maximized. Since the composite sensor 5 shown in FIG. 15 has a lower resolution of the infrared based black and white image than the composite sensor 5 shown in FIG. 14, the reliability of image authentication may be deteriorated when the matrix resolution of the composite sensor 5 is low. Can be. In consideration of the matrix resolution of the composite sensor 5, it is necessary to select appropriately from the example shown in FIG. 14 and the example shown in FIG. In the example illustrated in FIGS. 11-13, three-quarters of the plurality of infrared filters 511 are replaced by the color filters 5110, and thus, the composite sensor 5 illustrated in FIG. 15 may be realized in various arrangements. have.

The structure of the image sensor 51 shown in FIGS. 14 and 15 is a structure of a typical CMOS image sensor, which replaces some of the pixels of the CMOS image sensor with the spectral filter of this embodiment, thereby facilitating the composite sensor 5 according to the present embodiment. It can be seen that it can be prepared. Those skilled in the art to which this embodiment belongs will replace the part of the pixels of the CCD (Charge Coupled Device) image sensor with the spectral filter of this embodiment so that the composite sensor 5 according to this embodiment can be easily manufactured. Can be understood.

FIG. 16 is a diagram showing an example of the spectral filter 521 shown in FIGS. 14 and 15. FIG. 16A illustrates a plane of the spectral filter 521 illustrated in FIGS. 14 and 15, and a cross section of the spectral filter 521 illustrated in FIG. 16B. Referring to FIG. 16, each spectral filter 521 is formed in a shape in which metal patterns having a predetermined shape are arranged periodically so that a wavelength band determined according to the arrangement period of the metal patterns in the light passing through the display panel 3 is shown. Block and pass the rest of the wavelength band. Each of the spectral filters S1 and S2 is implemented in a form in which disc-shaped metal patterns 162 are periodically arranged in a two-dimensional lattice structure on the transparent substrate 161. The transparent substrate 161 may be made of various materials such as glass, polymer, Ge, GeSe, ZnS, ZnSe, sapphire, CaF ₂ , MgF _2, and the like. The insulating layer 503 of FIGS. 14 and 15 may serve as the transparent substrate 161. In this case, each of the spectral filters S1 and S2 is implemented in a form in which the disk-shaped metal patterns 162 are periodically arranged in a two-dimensional lattice structure on the insulating layer 503. The metal pattern 162 may be Au, Ag, Al, Cu, which is a plasmonic metal, or an alloy containing at least two of these, or at least one of them, and other elements, and may include Cr, which is a light absorbing metal, Ni, Ti, Pt, Sn, Sb, Mo, W, V, Ta, Te, Ge, Si, or at least two alloys thereof or alloys containing at least one of these and other elements.

Each metal pattern may be a nanostructure of various forms such as a square disk, a polygon, a rod, and a crossbar in addition to a circular disk. Such metal patterns may be periodically arranged in a linear lattice structure or a two-dimensional lattice structure. The two-dimensional lattice structure may be a square lattice structure or a hexagonal lattice structure. The metal pattern of the nanostructure has a property of absorbing light of a specific wavelength band and transmitting light of the remaining wavelength band by its surface plasmon. Surface plasmon resonance refers to a phenomenon in which light of a specific wavelength and free electrons of a metal surface resonate when light enters a metal surface, thereby propagating light of a specific wavelength along the surface. Plasmon refers to similar particles in which free electrons induced on a metal surface collectively vibrate by an electric field of incident light, and are present locally on the metal surface. In the case of the metal nanostructure having the surface plasmon property locally, when light is incident, the metal nanostructure exhibits a characteristic of absorbing light of a specific wavelength band among the incident light.

Conventional biometrics technology is difficult to miniaturize because the spectrum of the object is generated using a prism, a diffraction grating, etc., it is impossible to apply to a smartphone. As described above, the present embodiment adopts a kind of plasmonic filter that allows light filtering by the form in which the metal patterns corresponding to the nanostructures are periodically arranged as the spectral filter 521 of the spectral sensor 52. It becomes possible to implement the spectroscopy sensor 52 which is very easy to integrate with the conventional CMOS image sensor widely used as the image sensor of the phone. Accordingly, the image sensor and the biometric authentication of the user can be simultaneously performed only by bringing the user's eyes or face close to the electronic device, thereby realizing the composite sensor 5 capable of preventing spoofing using a fake image.

As shown in FIG. 16, the metal patterns of the spectral filter 521 form a metal nanostructure array having a periodic lattice structure, and specific light absorption enhanced in a specific wavelength band by coupling of the surface plasmon and the lattice mode. To light reflection phenomenon. As a result, the spectrum of light passing through the array of metal nanostructures forms a dip curve in which the transmittance is sharply lowered in the selective wavelength band where specific light absorption or light reflection is enhanced. "D1, D2" of the spectral filters S1, S2 represents the length of the metal pattern, and "P1, P2" represents the interval between the metal patterns. The duty cycle D1 / P1 of the spectral filter S1 and the duty cycle D2 / P2 of the spectral filter S2 are preferably kept the same but may be different. That is, the duty cycle of all spectroscopic filters is preferably kept the same, but may be different. The duty cycle is preferably 30 to 80%. If the duty cycle is less than 30%, the dip size is very small, and if it exceeds 80%, there is a tendency to produce too wide dip curves.

The spectral shape formed by the metal patterns of the spectral filter 521 of this embodiment depends on the geometry such as the type of metal material, the arrangement period of the metal patterns, the size of the metal pattern, and the like, and in particular by the spectral filter 521 The central wavelength of the light to be blocked is dominantly determined by the arrangement period of the metal patterns. According to the present embodiment, the center wavelength of the light blocked by the spectral filter 521 may be easily changed by changing the arrangement period of the metal patterns. If the metal patterns of one of the spectral filters 521 and the metal patterns of the other spectral filter are arranged at different periods, one of the spectral filters and the other of the spectral filters may have different wavelength bands. Will be blocked.

As described above, the spectroscopic sensor 52 of the present embodiment aims to detect a spectrum in the 810 to 850 nm band for biometric authentication of an object located in proximity to the electronic device. For spectral detection in the 430 to 700 nm band, the arrangement period of the metal patterns is preferably designed between 0.1 and 1.5 μm. For example, various types of spectroscopic filters 521 for dividing the 810 to 850 nm band by 5 nm units may be implemented by gradually changing the period of the metal patterns at intervals corresponding to 5 nm units within the range of 0.1 to 1.5 μm. Can be. In addition, the thickness of each metal pattern is preferably 5 to 500 nm. If it is smaller than 5 nm, the ratio of free electrons scattered on the surface is increased to act as a large factor of plasmon attenuation.

Conventionally, a metal nanohole array structure having a transmission type characteristic for a target band has been utilized as a plasmonic filter. The metal nanohole array structure exhibits a specific light transmission phenomenon in a specific wavelength band by coupling between the surface plasmon wave and the lattice mode that travels along the metal surface. In addition, since the metal nanohole array structure is based on coupling between traveling waves, unlike the metal pattern array structure of the present embodiment, various modes exist and are not defined as a single transmission band. The presence of such a multi-mode may cause signal distortion in the process of processing the signal wavelength incident on each photoelectric device. In addition, since the spectral filter 521 of the present embodiment blocks the wavelength band determined according to the arrangement period of the metal patterns in the light passing through the display panel 3, the free spectral range depending on the period is relatively low. It is wide enough to cover the entire 810 ~ 850nm band compared to the conventional transmission filter.

FIG. 17 is a graph showing simulation results of transmission characteristics of the spectral filter 521 of FIG. 16. 50 nm thick nanodisks were fabricated using aluminum and arranged in a hexagonal lattice structure of 50% duty cycle. The dip curve of the transmittance was calculated by computer simulation while changing the arrangement period at 200 nm to 1500 nm at 100 nm intervals. The spectroscopic filter 521 illustrated in FIGS. 17 to 16 has a characteristic in which a dip curve whose transmittance is sharply lowered from 0.35um to 2um is continuously variable, that is, a single stop band in which the center wavelength is continuously changed. It can be seen that the characteristics are shown.

As described above, the spectral filter 521 illustrated in FIG. 16 blocks the wavelength band determined by the arrangement period of the metal patterns in the light passing through the display panel 3. The plurality of optoelectronic devices 522 generates a signal representing the spectrum of the object by converting light filtered by the plurality of spectral filters 521 according to the embodiment shown in FIG. 16 into an electrical signal. The spectral filter 521 shown in FIG. 16 is a cut-off plasmonic filter, and the spectrum generated by the light filtered by the plurality of spectral filters 521 according to the embodiment shown in FIG. 16 is a conventional transmission plasmonic. It is inversely related to the spectrum produced from the light filtered by the filter. The processor may perform biometric authentication on the object using the signal representing the reversed phase spectrum, or may perform biometric authentication on the object using the normal spectrum after restoring the normal spectrum from the reversed phase spectrum.

FIG. 18 is a diagram illustrating another example of the spectral filter 521 illustrated in FIGS. 14 and 15. FIG. 18A illustrates a cross section of the spectral filter 521 illustrated in FIGS. 14 and 15, and a plane of the spectral filter 521 is illustrated in FIG. 18B. Referring to FIG. 18, each spectral filter 521 includes an upper reflection layer 181, a lower reflection layer 182, a dielectric layer 183, and a buffer layer 184. The upper reflection layer 181 and the lower reflection layer 182 are separated from each other to reflect some of the light incident thereon and transmit the rest. Each of the upper reflection layer 181 and the lower reflection layer 182 is formed in the form of a metal thin film such as Ag, Au, Al, Cr, Mo, etc., having semi-transmissive properties, or a distributed Bragg formed of a periodic multilayer structure of a high refractive index dielectric layer and a low refractive index dielectric layer. It may be a Reflective Bragg Reflector. The upper reflection layer 181 and the lower reflection layer 182 form upper and lower surfaces of each spectral filter 521.

The dielectric layer 183 is interposed between the upper reflection layer 181 and the lower reflection layer 182, and at least two materials 1831 and 1832 having different refractive indices are alternately disposed. At least two regions of the dielectric layer 183 having different relative volume ratios of the two materials 1831 and 1832 are present. As shown in FIG. 18B, there are at least two regions in the dielectric layer 183 having different relative width ratios of the two materials 1831 and 1832 along one direction. Here, one direction refers to a horizontal direction extending left and right, which is a length direction of the dielectric layer 183.

One of the two materials 1831 and 1832 of the dielectric layer 183 may be a relatively low refractive index dielectric, and the other material 1832 may be a relatively high refractive index dielectric. Conversely, either material 1831 may be a relatively high refractive index dielectric and the other material 1832 may be a relatively low refractive index dielectric. Examples of low refractive index dielectrics include fluorine-based ultraviolet resin, spin on glass, hydrogen silsesquioxane, magnesium fluoride (MgF ₂ ) and calcium fluoride (CaF ₂ ). , Silicon dioxide (SiO ₂ ), and the like. Examples of the high refractive index dielectrics include metal oxides such as titanium dioxide (TiO ₂ ) and the like.

The Fabry-Perot interferometer was first designed by Charles Fabry and Alfred Perot, and is generally constructed by inserting a cavity between two high reflectivity mirrors. When light enters either of the two mirrors, multiple interference occurs in the cavity, transmitting only a specific wavelength and reflecting other wavelengths. The conventional linear variable filter is a kind of optical filter of a Fabry-Perot resonator structure, and has a mirror layer above and below the dielectric cavity, and has a structure in which the thickness of the dielectric cavity varies linearly in the longitudinal direction. Since the resolution of the conventional linear variable filter is determined by the ratio of the height to the length of the linear variable filter, it is difficult to miniaturize.

In FIG. 18, regions where the center wavelengths of light transmitted by the spectral filter 521 are different from each other are illustrated as three regions A, B, and C. In FIG. In this case, two materials 1831 and 1832 are alternately arranged in three regions A, B, and C with the same period, and the relative width ratios of the two materials 1831 and 1832 are different in the three regions, respectively. Due to this structure, the wavelengths of light transmitted through the three regions A, B, and C are different. The relative volume ratio between the two materials 1831 and 1832 may be defined as a duty cycle. The duty cycle in this embodiment is the relative volume fraction of one of the materials 1831 of the two materials 1831 and 1832. The refractive index of the dielectric layer 183 may be varied in one direction by gradually changing the duty cycle between the two materials 1831 and 1832 of the dielectric layer 183.

The wavelength of light passing through the spectral filter 521 illustrated in FIG. 18 is determined by the optical thickness of the dielectric layer 183. The optical thickness may be defined as a value obtained by multiplying the physical thickness by the refractive index. In this embodiment, the optical thickness of the dielectric region controls the center wavelength of the transmission band of the spectral filter 521 by changing the refractive index in addition to the physical thickness. On the other hand, the width of a pair of two materials (1831, 1832) has a relationship with the wavelength of the filter to pass through. For example, the width of a pair of two materials 1831 and 1832 can be designed to be sufficiently smaller than the wavelength of light, in which case the light cannot separate the two materials 1831 and 1832 into individual objects. One effective medium defined by a certain effective dielectric constant.

In this case, the optical constant of the effective medium is determined by the geometric distribution and relative volume fraction of the two objects. Therefore, each spectral filter 521 of the present embodiment can pass the wavelength band determined according to the relative volume ratio between the two materials 1831 and 1832 in the light passing through the display panel 3, and block the remaining wavelength band. have. As described above, the dielectric layer 183 includes at least two regions having different relative volume ratios between two materials 1831 and 1832 disposed alternately. Regions with different relative volume ratios of the dielectric layer 183 pass different wavelength bands. For example, various types of spectroscopic filters 521 that filter by dividing the 810 to 850 nm band by 5 nm units have a duty cycle between two materials 1831 and 1832 of the dielectric layer 183 at intervals corresponding to 5 nm units. It can be implemented by gradual change.

The buffer layer 184 is inserted between at least one of the upper reflection layer 181 and the lower reflection layer 182 and the dielectric layer 183. That is, the buffer layer 184 may be inserted between the upper reflection layer 181 and the dielectric layer 183, may be inserted between the lower reflection layer 182 and the dielectric layer 183, and the upper reflection layer 181 and the dielectric layer 183. And between the bottom reflection layer 182 and the dielectric layer 183, respectively. The buffer layer 184 acts as an optical resonance cavity with the dielectric layer 183. The presence of the buffer layer 140 increases the effective thickness of the optical resonant cavity, thereby bringing the effect of shifting the center wavelength of the transmission band of the spectral filter 521 to the long wavelength region while keeping the thickness of the dielectric layer 183 small. In the case of configuring the optical resonant cavity using only the dielectric layer 183, the aspect ratio for filling any one of the materials in the dielectric layer 183 composed of a combination of two materials having different intervals of refraction and a size sufficiently smaller than the wavelength of the incident light An excessively increasing condition occurs and there are process difficulties.

Therefore, the presence of the buffer layer 184 has the advantage that it is possible to effectively increase the variable range of the transmission wavelength while keeping the thickness of the dielectric layer 183 small. Accordingly, miniaturization of the spectral filter 521 is possible, thereby enabling the implementation of the spectral sensor 52 which is very easy to integrate with an existing CMOS image sensor widely used as an image sensor of a smartphone. Accordingly, the image sensor and the biometric authentication for the user can be carried out at the same time to realize the composite sensor 5 that can prevent spoofing using a fake image. The buffer layer 184 may be the same material as one of two materials 1831 and 1832. FIG. 18 illustrates a case where the material of the buffer layer 184 and the second material 1832 are the same. Meanwhile, the buffer layer 184 and the two materials 1831 and 1832 may be different materials.

FIG. 19 is a graph illustrating simulation results of transmission characteristics of the spectral filter 521 of FIG. 18. The alternating placement period of the two materials 1831 and 1832 is 200 nm, the thickness of the dielectric layer 183 is 60 nm, the refractive index of the low refractive index material is 1.38 and the refractive index of the high refractive index material is 2.7 among the two materials 1831 and 1832. It was set as. The upper and lower reflective layers 181 and 182 used a 20 nm thick Ag layer having semi-transmissive properties. The variation of the transmission band when the width of the low refractive index nanostructure was increased by 50 nm to 150 nm at 20 nm intervals was calculated by computer simulation. 19 to 18, it can be seen that the spectral filter 521 illustrated in FIG. 18 exhibits a characteristic in which the center wavelength of the transmission band moves to the long wavelength region as the filling rate of the high refractive index nanostructure increases.

20 is a view schematically showing another example of the structure of the spectroscopy sensor 52 shown in FIGS. 3-7. In FIG. 20, the abbreviation "S" represents the spectral filter 521, the abbreviation "F" represents the bandpass filter 504, and the abbreviation "P" represents the photoelectric element 522. In order to clearly show the features of the embodiment illustrated in FIG. 20, the plurality of spectral filters 521, the plurality of bandpass filters 504, and the plurality of photoelectric elements 522 are briefly represented by a simple grid pattern. The embodiment shown in FIG. 20 is not related to the image sensor 51 of the composite sensor 5 shown in FIGS. 1-6, so only the spectroscopic sensor 52 portion is shown. The image sensor 51 portion will be replaced with the description of the embodiments shown in FIGS. 11-13.

Unlike the embodiments illustrated in FIGS. 11-13, the embodiment illustrated in FIG. 20 is arranged in a matrix structure corresponding to a portion of a matrix structure in a two-dimensional plane, and the plurality of bandpass filters 504 is arranged in a matrix structure corresponding to another portion of the matrix structure of the two-dimensional plane, and the plurality of spectral filters 521 are different portions of the matrix structure of the two-dimensional plane forming the light receiving surface of the complex sensor 5. The matrix structure corresponds to the reduced matrix structure. The plurality of bandpass filters 504 are arranged in a matrix structure corresponding to other parts of the matrix structure of the two-dimensional plane that forms the light receiving surface of the complex sensor 5 so as to correspond to the plurality of spectral filters 521 one to many. And are disposed below the plurality of spectral filters 521. As a result, the plurality of bandpass filters 504 correspond to each of the spectral filters 521 up and down.

20 shows an example in which four band pass filters 504 correspond to one spectroscopic filter 521 up and down. The plurality of bandpass filters 504 corresponding to the spectral filters 521 transmit different wavelength bands in the filtering wavelength bands of the spectral filters 521. That is, one of the bandpass filters 504 of the plurality of bandpass filters 504 transmits only a portion of the wavelength bands of the filtering wavelength bands of the spectral filters 521, and the other bandpass filter 504 is each of the bandpass filters 504. Only the other wavelength band of the filtering wavelength band of the spectral filter 521 is transmitted. The plurality of optoelectronic devices 522 are arranged in a matrix structure corresponding to other parts of the matrix structure of the two-dimensional plane so as to correspond one-to-one to the plurality of bandpass filters 504 and disposed below the plurality of bandpass filters 504. do.

According to the present embodiment, that is, the narrower the filtering band of each spectral filter 521, the higher the resolution of the spectrum of the object, so that the accuracy of biometric authentication can be improved. When each spectral filter 521 is implemented as a plasmonic filter as shown in FIG. 16, or a Fabry-Perot filter as shown in FIG. Since manufacturing of the relative volume ratio of the microscopically different fractions has a limitation in the manufacturing process, there is a limit in narrowing the filtering bandwidth of each spectral filter 521. This acts as a limitation of the resolution of the spectrum of the object. Accordingly, the plurality of bandpass filters 504 corresponding to each of the spectral filters 521 transmits different wavelength bands among the filtering wavelength bands of the spectral filters 521, thereby limiting the filtering bandwidth limit of each spectral filter 521. By overcoming the high resolution spectrum of the object, the accuracy of biometric authentication can be improved.

FIG. 21 is a sectional view of another embodiment of the composite sensor 5 shown in FIG. FIG. 21A illustrates an example in which the composite sensor 5 is implemented by integrating the image sensor 51 and the spectroscopic sensor 52 on one substrate as in FIG. 14. An embodiment of the composite sensor 5 employing the front-illuminated image sensor 51 is shown in FIG. 21A, and the composite sensor employing the back-illuminated image sensor 51 in FIG. 21B. An embodiment of (5) is shown. FIG. 21C illustrates an example in which the spectral filter 521 illustrated in FIG. 16 is applied to the spectroscopic sensors 52 of FIGS. 21A and 21B. In Fig. 21, the abbreviation "L" represents a microlens, the abbreviation "I" represents an infrared filter, the abbreviation "S" represents a spectral filter, the abbreviation "P" represents an optoelectronic device, and the abbreviation "F" stands for band transmission. Represents a band pass filter. The embodiment of FIG. 21 is the same as the embodiment of FIG. 14 except that a bandpass filter 504 is disposed below each spectral filter 521 of the spectroscopy sensor 52. Descriptions will be made focusing on differences from and, and portions corresponding to the contents omitted below, for example, the image sensor 51 will be replaced with the description of the embodiment of FIG. 14.

Referring to the portion corresponding to the spectroscopic sensor 52 of FIG. 21A, a plurality of photoelectric elements 522 on the composite sensor substrate 500 form a light receiving surface of the composite sensor 5. Stacked and arranged in a matrix structure corresponding to other parts of the matrix structure. On the photoelectric elements 522, an insulating layer 503 in which wires for supplying power to each of the spectral filters 521 and 522 is embedded is stacked. On the insulating layer 503, a plurality of bandpass filters 504 are stacked and arranged in the same matrix structure as that of the plurality of optoelectronic devices 522 so as to correspond one-to-one to the plurality of optoelectronic devices 522. Light filtered by each bandpass filter 504 passes through wires of the insulating layer 503 to reach each photoelectric device 522.

Referring to the part corresponding to the spectroscopy sensor 52 of FIG. 21B, wires for supplying power to each spectroscopic filter 521 and each photoelectric device 522 are embedded on the composite sensor substrate 500. The insulating layer 503 is laminated. A plurality of optoelectronic devices 522 are arranged on the insulating layer 503 in a matrix structure corresponding to other parts of the two-dimensional planar matrix structure forming the light receiving surface of the complex sensor 5. The plurality of bandpass filters 504 are arranged on the plurality of optoelectronic devices 522 in the same matrix structure as the matrix structure of the plurality of optoelectronic devices 522 so as to correspond one-to-one to the plurality of optoelectronic devices 522. Light filtered by each bandpass filter 504 immediately reaches each photoelectric device 522.

As shown in (a) and (b) of FIG. 21, a plurality of spectral filters 521 may be arranged on the plurality of bandpass filters 504 so as to correspond to the plurality of bandpass filters 504 up and down one-to-one. The matrix structure of the spectral filter 521 is arranged in a reduced matrix structure and stacked. On the plurality of spectral filters 521, a planarization layer 502 is formed to planarize an arrangement surface of the plurality of bandpass filters 504. The planarization layer 502 is made of a transparent dielectric material. On the planarization layer 502, a plurality of microlenses 501 are arranged and stacked in the same matrix structure as that of the plurality of spectral filters 521 so as to correspond to the plurality of spectral filters 521 one-to-one. When it is difficult to manufacture different diameters of the plurality of microlenses 501 of the composite sensor (1), as shown in FIG. 20, the plurality of microlenses 501 is face-to-face with the plurality of spectral filters 521. Correspondingly, the matrix structures of the plurality of spectral filters 521 may be arranged and stacked in the same matrix structure. Each microlens 501 serves to focus light incident on it to each spectral filter 521 below it.

 Referring to (c) of FIG. 21, three spectroscopic filters 521 are formed in such a manner that their metal patterns are arranged at different periods. Two different color filters 5110 are disposed below each spectral filter 521. A separate planarization layer may be formed on the color filter 5110. The fact that a bandpass filter 504 transmits only a portion of the wavelength bands of each spectral filter 521 means that the filtering band of the bandpass filter 504 coincides with the part of the wavelength band. In addition, it means that it may include a wavelength band different from the filtering wavelength band of each spectral filter 521 in addition to the partial wavelength band. That is, if a part of the filtering band of each spectral filter 521 is included in the filtering band of the red filter, and the remainder of the filtering band of each spectral filter 521 is included in the filtering band of the green filter, the red filter and the green filter Using high resolution spectra of the object can be enabled.

As described above, at least two kinds of color filters and the same color filters of the plurality of color filters 5110 of the image sensor 51 may be used as the plurality of band pass filters 504. Only by stacking the spectroscopic filter 521 of the present embodiment on some of the color filters, it is possible to implement the composite sensor 5 according to the present embodiment. In addition, since each spectral filter 521 may be manufactured in a larger size than each color filter 5110, the manufacture of each spectral filter 521 may be very easy. 19A and 19B, the plurality of microlenses 501 are illustrated to correspond one-to-one to the plurality of bandpass filters 504. As shown in (a) and (b) of FIG. 19, microlenses of different sizes may be difficult to be integrally manufactured in the manufacturing process of the microlens 501. Microlenses of the same kind as the microlenses may be manufactured for the spectral filter, or the microlens array layer may be omitted.

In the above, examples in which the plurality of bandpass filters 504 correspond to the plurality of spectral filters 521 one-to-many are disposed below the plurality of spectral filters 521. The plurality of bandpass filters 504 may be disposed below the plurality of spectral filters 521 in a one-to-one correspondence with the plurality of spectral filters 521, and the plurality of bandpass filters 504 may be a plurality of spectral filters 521. It may be arranged under a plurality of spectral filter 521 corresponding to many to one. In the former case, one bandpass filter 504 corresponding to one spectral filter 521 transmits the filtering wavelength band of the spectral filter 521. In the latter case, one bandpass filter 504 that corresponds to a plurality of spectral filters 521 corresponding to a part of the plurality of spectral filters 521 has a total filtering wavelength band of the plurality of spectral filters 521. It penetrates. The plurality of bandpass filters 504 serves to improve the overall filtering performance by re-filtering the filtered light by the plurality of spectral filters 521.

In addition, the plurality of bandpass filters 504 may be disposed on the plurality of spectral filters 521 in one-to-one, one-to-one, or many-to-one correspondence with the plurality of spectral filters 521. In the example in which the plurality of bandpass filters 504 are disposed under the plurality of spectral filters 521, the plurality of optoelectronic devices 522 converts the light filtered by the plurality of bandpass filters 504 into electrical signals. In the example in which the plurality of bandpass filters 504 are disposed on the plurality of spectral filters 521, the plurality of photoelectric elements 522 converts the light filtered by the plurality of spectral filters 521 into electrical signals. In the latter example, the plurality of bandpass filters 504 prevent the incidence of unnecessary wavelength bands other than its filtering wavelength band to the plurality of spectral filters 521 so that the filtering performance of each spectral filter 521 can be improved. To play a role. In various examples as described above, color filters of the same type as color filters of the plurality of color filters 5110 of the image sensor 51 may be used as the plurality of band pass filters 504.

22 is a cross-sectional view of another embodiment of the composite sensor 5 shown in FIG. As described above, the example illustrated in FIG. 15 is different from the example illustrated in FIG. 14 in that three quarters of the plurality of infrared filter 511 illustrated in FIG. 14 are replaced by the color filter 5110. . Likewise, the example shown in FIG. 22 differs from the example shown in FIG. 21 in that three quarters of the plurality of infrared filters 511 shown in FIG. 21 are replaced with a color filter 5110. The example shown in FIG. 22 may be understood from the description of the example shown in FIG. 21 and the description of the example shown in FIG. 15. Therefore, the description of the example illustrated in FIG. 22 will be replaced with the description of the example illustrated in FIG. 21 and the description of the example illustrated in FIG. 15.

FIG. 23 is a block diagram illustrating an electronic circuit of the electronic device illustrated in FIG. 3. Referring to FIG. 22, the electronic circuit of the electronic device illustrated in FIG. 3 includes a touch panel 2, a display panel 3, a complex sensor 5, a light emitting module 8, a sensor module 11, and a power supply module. 12, a processor 71, a communication module 72, an input / output interface module 73, a memory 74, and a bus 75. In order to prevent the present embodiment from being blurred while the present embodiment can be easily understood, FIG. 23 shows the main configuration of the electronic circuit of the smartphone required for understanding the present embodiment. Those skilled in the art to which the present embodiment pertains may understand that another configuration may be added in addition to the configuration shown in FIG. 23. For example, the electronic circuit of the electronic device illustrated in FIG. 3 may further include a Global Positioning System (GPS) module, an audio module, and the like.

 The sensor module 11 may be configured of sensors embedded in a smartphone, for example, a pressure sensor, a proximity sensor, an illuminance sensor, a gyro sensor, etc. in addition to the complex sensor 5. The power supply module 12 converts the power of the battery 9 into driving power of the touch panel 2, the display panel 3, the complex sensor 5, the sensor module 11, and the printed circuit board 7. It serves to supply and consists of charging circuit, voltage conversion circuit and so on. Paths through which power is supplied from the power supply module 12 to other components are obvious to those skilled in the art to which the present embodiment belongs, and thus are omitted to reduce the complexity of the drawings. The processor 71, the communication module 72, the input / output interface module 73, the memory 74, and the bus 75 are implemented by electronic components mounted on the printed circuit board 7 and generally have several separations. It is mounted on the printed circuit board 7.

The processor 71 controls other components of the electronic device, for example, the touch panel 2, the display panel 3, the complex sensor 5, and the sensor module 11, and the touch panel 2 and the communication module. 72, the data inputted from the input / output interface module 73 is processed, and data is read from the memory 74 or stored in the memory 74. In particular, the processor 71 of the present embodiment receives a signal representing an image of an object from the composite sensor 5, generates an image of the object from the input signal, and generates a user image stored in the memory 74 and the generated object image. User authentication is performed on the object by comparing an iris image or a face image. In addition, the processor 71 receives a signal representing the spectrum of an object from the complex sensor 5, generates a spectrum of the object from the input signal, and generates the generated spectrum and an iris spectrum or face of the user stored in the memory 74. The biometric authentication of the object is performed by comparing the spectra. The operation of the processor 71 will be described in detail below with reference to the flowcharts of FIGS. 24-26.

The communication module 72 communicates with other electronic devices. For example, the communication module 72 may perform communication with other electronic devices according to various communication methods such as Long Term Evolution (LTE), Wi-Fi, and Bluetooth. The input / output interface module 73 may output a command or data input from a user or another external device to another component of the electronic device, or output a command or data received from another component of the electronic device to the user or another external device. have. The input / output interface module 73 includes a USB (Universal Serial Bus) interface, an audio interface, and the like. Various data is stored in the memory 74. In particular, the memory 74 of the present embodiment stores the biometric image and spectral data of the user. The bus 75 serves to transfer data between the processor 71 and various components.

24 is a flowchart illustrating a user authentication method according to an embodiment of the present invention. FIG. 24 illustrates a flow of a user authentication method when the electronic device shown in FIG. 23 is in the screen lock mode. Referring to FIG. 24, the user authentication method according to the present embodiment is an electronic device illustrated in FIGS. 1-3 and 23, and a watch in an electronic device including the composite sensor 5 according to the embodiment illustrated in FIGS. 4-22. It consists of steps that are thermally processed. Therefore, even if omitted below, the above descriptions regarding the electronic devices shown in FIGS. 1-3 and 23 and the composite sensor 5 shown in FIGS. 4-22 also apply to the user authentication method described below. .

In step 2401, the sensor module 11 of the electronic device detects which object is closer than the critical distance on the cover glass 1 in the screen lock mode. For example, the critical distance may be 5 cm and may be designed to various values. Such proximity detection may be performed using a proximity sensor or the like of the sensor module 11. If it is determined in step 2301 that the object is close to the threshold distance, the object enters the sensing area of the composite sensor 5 and proceeds to step 2402. Otherwise, the screen lock mode is maintained. Here, the screen lock mode refers to a mode in which power supply is interrupted to most of the components of the electronic device except for the processor 71 and the sensor module 11, for example, the touch panel 2 and the display panel 3. .

In operation 2402, the processor 71 of the electronic device switches from the screen lock mode to the low power mode, and the display panel 3 of the electronic device displays a graphic for guiding the user's eyes or face accurately. For example, the display panel 3 may display a graphic representing a human eye or face contour. When the actual outline of the user's eyes or face is aligned with the outline displayed as described above, the eyes or face of the user are accurately positioned in the detection area of the composite sensor 5. Here, the low power mode is a part of the components of the electronic device for user authentication, for example, the touch panel 2, the display panel 3, the composite sensor 5, the light emitting module 8, the processor 71 It refers to the mode in which power is supplied to the lamp. In operation 2403, the light emitting module 8 of the electronic device irradiates infrared rays to the surface of the object under the control of the processor 71.

In operation 2404, the composite sensor 5 of the electronic device uses a plurality of infrared filters 511 to irradiate the surface of the object with infrared light reflected from the surface of the object and pass through the display panel 3 in step 2403. Generates a signal representing the image of. The image sensor 51 of the composite sensor 5 generates a signal representing an image of the object from the light transmitted through the display panel 3 by the infrared rays irradiated onto the surface of the object. Since the signal generation in operation 2404 is merely for showing the object image to the user, the color filter 5110 may be used. In operation 2405, the processor 71 of the electronic device receives a signal representing an image of the object from the complex sensor 5, and generates an image of the object from the input signal. A signal representing an image of an object is serially input from the composite sensor 5 to the processor 71. The processor 71 may generate an image of an object by converting serially input signals into two-dimensional image data. In operation 2406, the display panel 3 of the electronic device displays the object image generated as described above on the graphic displayed in operation 2402 so that the object image generated in operation 2405 overlaps with the graphic displayed in operation 2402.

In operation 2407, the processor 71 of the electronic device determines whether a predetermined time has elapsed since the start of operation 2402. For example, the constant time may be 5 seconds and may be designed with various values. If it is determined in step 2407 that the predetermined time has elapsed, the process proceeds to steps 2408 and 2411, and otherwise returns to step 2404. Each time steps 2404-2406 are repeated, an image corresponding to the eye or face position of the user at each execution time of steps 2404-2406 is repeatedly displayed. Accordingly, the user may adjust his or her eye or face position while checking the eye or face image superimposed on the graphic displayed in step 2402 for a certain time so that his or her eye or face is accurately positioned in the detection area of the composite sensor 5. Can be.

In operation 2408, the composite sensor 5 of the electronic device uses the plurality of infrared filters 511 to reflect an image of the object from light passing through the display panel 3 by reflecting infrared rays irradiated onto the surface of the object. Generates a signal that represents. In operation 2409, the processor 71 of the electronic device receives a signal representing the image of the object from the complex sensor 5, and generates an image of the object from the input signal. A signal representing an image of an object is serially input from the composite sensor 5 to the processor 71. The processor 71 may convert the serially input signal into 2D image data or 3D image data for image authentication of the user. In operation 2410, the processor 71 of the electronic device performs image authentication on the object by using the object image generated in operation 2409. The processor 71 may perform image authentication on the object by comparing the object image generated in operation 2306 with the iris or face image of the user stored in the memory 74.

In step 2411, the composite sensor 5 of the electronic device uses a plurality of spectral filters 521 to emit infrared rays emitted from the surface of the object and then passes from the surface of the object to pass through the display panel 3. Generates a signal representing the spectrum of an object from light. In operation 2412, the processor 71 of the electronic device receives a signal representing the spectrum of the object from the complex sensor 5 and generates the spectrum of the object from the input signal. A signal representing the spectrum of the object is serially input from the compound sensor 5 to the processor 71. The processor 71 may generate a spectrum of an object by converting a serially input signal into a two-dimensional curve graph. In operation 2413, the processor 71 of the electronic device performs biometric authentication on the object using the spectrum generated in operation 2412. The processor 71 may perform biometric authentication on the object by comparing the spectrum generated in operation 2412 with the eye or face spectrum of the user stored in the memory 74. Steps 2408-2410 and 2411-2413 proceed simultaneously.

In step 2414, the processor 71 of the electronic device authenticates the user according to the result of performing image authentication in step 2410 and the result of performing biometric authentication in step 2413. The processor 71 indicates that the result of performing image authentication in step 2410 matches that of the object image generated in step 2409 and the iris or face image of the user stored in the memory 74, and the result of performing biometric authentication in step 2413 is determined. When the spectrum generated in step 2412 and the spectrum of the user stored in the memory 74 match, it is determined that the user authentication is successful, and the screen lock mode is switched to the screen lock release mode. If the object image generated in step 2409 and the iris or face image of the user stored in the memory 74 do not match, or the spectrum generated in step 2412 and the spectrum of the user stored in the memory 74 do not match, It determines that the authentication has failed and maintains the screen lock mode.

25 is a flowchart illustrating a user authentication method according to another embodiment of the present invention. FIG. 25 illustrates a flow of a user authentication method when the electronic device shown in FIG. 23 is in the screen lock release mode. Referring to FIG. 25, the user authentication method according to the present embodiment is an electronic device illustrated in FIGS. 1-3 and 23, and the electronic device includes a complex sensor 5 according to the embodiment illustrated in FIGS. 4-22. It consists of steps that are thermally processed. Therefore, even if omitted below, the above descriptions regarding the electronic devices shown in FIGS. 1-3 and 23 and the composite sensor 5 shown in FIGS. 4-22 also apply to the user authentication method described below. . In addition, since steps 2502-2514 of the steps of the present embodiment overlap with steps 2402-2414 shown in FIG. 23, it will be briefly described.

In operation 2501, the processor 71 of the electronic device checks whether there is a user authentication request from an application running by the processor 71 in the screen unlock mode. If it is determined in step 2501 that there is a user authentication request, the flow proceeds to step 2502, otherwise the current state is maintained. In operation 2502, the display panel 3 of the electronic device displays a graphic for guiding the user's eyes or face to be accurately positioned under the control of the processor 71. In operation 2503, the light emitting module 8 of the electronic device irradiates infrared rays to the surface of the object under the control of the processor 71. Unlike the embodiment illustrated in FIG. 24, since the present embodiment is executed in the screen lock release mode, abnormal proximity of any object other than the user rarely occurs. Accordingly, this embodiment does not consider the proximity of any object.

In operation 2504, the complex sensor 5 of the electronic device generates a signal representing an image of the object from the light that passes through the display panel 3 by reflecting infrared rays irradiated onto the surface of the object in step 2503. In operation 2505, the processor 71 of the electronic device receives a signal representing an image of the object from the complex sensor 5, and generates an image of the object from the input signal. In operation 2506, the display panel 3 of the electronic device displays the object image thus generated on the graphic displayed in operation 2502 so that the object image generated in operation 2505 overlaps with the graphic displayed in operation 2502.

In operation 2507, the processor 71 of the electronic device determines whether a predetermined time has elapsed since the start of operation 2502. If it is determined in step 2507 that the predetermined time has elapsed, the process proceeds to steps 2508 and 2511, and otherwise, returns to step 2504. In operation 2508, the complex sensor 5 of the electronic device generates a signal representing an image of the object from the light transmitted through the display panel 3 by reflecting infrared rays emitted from the surface of the object. In operation 2509, the processor 71 of the electronic device receives a signal representing the image of the object from the complex sensor 5, and generates an image of the object from the input signal. In operation 2510, the processor 71 of the electronic device performs image authentication on the object by using the object image generated in operation 2509.

In step 2511, the composite sensor 5 of the electronic device generates a signal representing the spectrum of the object from the light transmitted through the display panel 3 after being emitted from the surface of the object after infrared rays irradiated into the object are penetrated into the object. do. In operation 2512, the processor 71 of the electronic device receives a signal representing the spectrum of the object from the complex sensor 5, and generates the spectrum of the object from the input signal. In operation 2513, the processor 71 of the electronic device performs biometric authentication on the object by using the spectrum generated in operation 2512. The processor 71 may perform biometric authentication on the object by comparing the spectrum generated in operation 2512 with the eye or face spectrum of the user stored in the memory 74. Steps 2508-2510 and 2511-2513 proceed simultaneously.

In step 2514, the processor 71 of the electronic device authenticates the user according to the result of performing image authentication in step 2510 and the result of performing biometric authentication in step 2513. The processor 71 indicates that the result of performing image authentication in step 2510 matches that of the object image generated in step 2509 and the iris or face image of the user stored in the memory 74, and the result of performing biometric authentication in step 2513 is determined. If the spectrum generated in step 2512 and the spectrum of the user stored in the memory 74 match, it is determined that the user authentication is successful, and the application process after the user authentication is completed is processed.

If the object image generated in step 2509 and the iris or face image of the user stored in the memory 74 do not match or the spectrum generated in step 2512 does not match the spectrum of the user stored in the memory 74, If it is determined that authentication has failed, the process proceeds to step 2515. In operation 2515, the display panel 3 of the electronic device displays a message indicating that user authentication has failed under the control of the processor 71. After completion of step 2515, the process returns to step 2502. Steps 2502-2515 are repeatedly executed until the user authentication in step 2514 succeeds or the execution of the application or the user authentication process is canceled by the user.

26 is a flowchart illustrating a user biometric information registration method according to an embodiment of the present invention. The user biometric information method shown in FIG. 26 is started when the biometric information registration application is executed by the user in the screen lock release mode. Referring to FIG. 26, the method for registering user biometric information according to the present embodiment is an electronic device illustrated in FIGS. 1-3 and 23 and includes an electronic device including the composite sensor 5 according to the embodiment illustrated in FIGS. 4-22. Consists of steps processed in time series. Therefore, even if omitted below, the contents described above with respect to the electronic device shown in FIGS. 1-3 and 23 and the composite sensor 5 shown in FIGS. 4-22 also apply to the user biometric information registration method described below. Apply. In addition, steps 2601-2606 and 2608-2611 of the present exemplary embodiment overlap with steps 2402-2407, 2408-2409, and 2411-2412 shown in FIG.

In operation 2601, the display panel 3 of the electronic device displays a graphic for guiding the user's eyes or face to be accurately positioned under the control of the processor 71. In operation 2602, the light emitting module 8 of the electronic device irradiates infrared rays to the surface of the object under the control of the processor 71. In operation 2603, the complex sensor 5 of the electronic device generates a signal representing an image of the object from the light transmitted through the display panel 3 by reflecting infrared rays irradiated onto the surface of the object. In operation 2604, the processor 71 of the electronic device receives a signal representing an image of the object from the complex sensor 5, and generates an image of the object from the input signal. In operation 2605, the display panel 3 of the electronic device displays the object image thus generated on the graphic displayed in operation 2502 so that the object image generated in operation 2604 overlaps with the graphic displayed in operation 2601.

In operation 2606, the processor 71 of the electronic device determines whether a predetermined time has elapsed since the start of operation 2601. If it is determined in step 2606 that the predetermined time has elapsed, the process proceeds to steps 2608 and 2610, and otherwise proceeds to step 2607. In operation 2607, the processor 71 of the electronic device determines whether the biometric information registration has been canceled by the user. If it is confirmed in step 2607 that the biometric information registration is canceled, the execution of the biometric information registration application is terminated, otherwise, return to step 2603. If the user determines that the contour of his or her eye or face is not exactly aligned with the contour of the graphic displayed in step 2601, the user may cancel his or her biometric information registration.

In operation 2608, the composite sensor 5 of the electronic device generates a signal representing an image of the object from the light transmitted through the display panel 3 by reflecting infrared rays emitted from the surface of the object. In operation 2609, the processor 71 of the electronic device receives a signal representing an image of the object from the complex sensor 5, and generates an image of the object from the input signal. In step 2610, the composite sensor 5 of the electronic device generates a signal representing the spectrum of the object from the light emitted through the display panel 3 after the infrared rays irradiated into the object penetrate the object. do. In operation 2611, the processor 71 of the electronic device receives a signal representing the spectrum of the object from the complex sensor 5, and generates the spectrum of the object from the input signal.

In step 2612, the processor 71 of the electronic device stores the biometric information of the user in the memory 74 by storing the object image generated in step 2609 and the spectrum generated in step 2611 as a user's biometric information. The spectrum generated in step 2611 is registered as the user's biometric information. The object image generated in operation 2609 and the spectrum generated in operation 2611 may be registered as user biometric information together with the user's ID. The memory 74 may store several sets of biometric images and spectra. Any one of these sets may be designed to satisfy user authentication if it matches the biometric image and spectrum generated from the output signal of the composite sensor 5. Steps 2608-2609 and 2610-2611 may be performed at the same time, and steps 2608-2609 and 2610-2611 may be repeated several times to increase the accuracy of the user's biological image and spectrum. In steps 2608-2609 and 2610-2611, the average of the plurality of images may be set as the user's final biometric image when repeated several times, and the average of the plurality of spectra may be set as the user's final biospectral.

On the other hand, the user authentication method and the user biometric information registration method according to the embodiments of the present invention as described above can be written as a program executable in the processor of the computer, the program is recorded and executed in a computer-readable recording medium Can be implemented in a computer. Computers include all types of computers capable of executing programs, such as desktop computers, notebook computers, smart phones, and embedded type computers. In addition, the structure of the data used in the above-described embodiment of the present invention can be recorded on the computer-readable recording medium through various means. The computer-readable recording medium may be a storage medium such as a RAM, a ROM, a magnetic storage medium (for example, a floppy disk, a hard disk, etc.), an optical reading medium (for example, a CD-ROM, DVD, etc.). Media.

So far I looked at the center of the preferred embodiment for the present invention. Those skilled in the art will appreciate that the present invention can be implemented in a modified shape without departing from the essential characteristics of the present invention. Therefore, the disclosed embodiments should be considered in descriptive sense only and not for purposes of limitation. The scope of the present invention is shown in the claims rather than the foregoing description, and all differences within the scope will be construed as being included in the present invention.

1 ... cover glass
2 ... touch panel
3 ... display panel
4 ... rear panel
41 ... protective layer 42 ... heat-radiating layer
5 ... composite sensor
51 ... image sensor 511 ... infrared filter
5110 ... color filters
52 ... spectral sensor 521 ... spectral filter
512, 522 ... Optoelectronics
500 ... composite sensor substrate 501 ... microlens
502 ... flattening layer 503 ... insulating layer
504 ... bandpass filter
161 ... transparent substrate 162 ... metal pattern
181 ... Upper Reflective Layer 182 ... Lower Reflective Layer
183 ... dielectric layer 184 ... buffer layer
50 ... collimator lens
6 ... Bracket
7 ... printed circuit board
71 ... Processor 72 ... Communication Module
73 ... I / O interface module 74 ... Memory
75 ... bus
8 ... light emitting module
9 ... battery
10 ... housing
11 ... sensor module
12 ... power supply module

Claims (30)

A display panel displaying a screen;
A light emitting module irradiating infrared rays to the surface of the object;
Infrared radiation irradiated onto the surface of the object generates a signal representing an image of the object from light reflected off the surface of the object, and from the light emitted from the surface of the object after the irradiated infrared light penetrates into the object A composite sensor for generating a signal indicative of the spectrum of the object; And
And a processor configured to perform image authentication on the object using a signal representing the image of the object and perform biometric authentication on the object using a signal representing the spectrum of the object. The method of claim 1,
The composite sensor generates a signal representing the spectrum of the object from the light emitted from the surface of the object after the infrared rays irradiated onto the surface of the object using a plurality of spectral filters,
The plurality of spectral filters transmits or blocks a plurality of wavelength bands belonging to a wavelength band absorbed in the process of being emitted from the surface of the living body after penetrating into the human body of the infrared wavelength band irradiated onto the surface of the object. Electronics. The method of claim 2,
The living body is an electronic device that is the human cornea or facial skin. The method of claim 1,
The composite sensor
An image sensor using a plurality of infrared filters to generate a signal representing an image of the object from light reflected from the surface of the object; And
And a spectroscopic sensor for generating a signal representing the spectrum of the object from light emitted from the surface of the object after penetrating the object using a plurality of spectroscopic filters. The method of claim 4, wherein
The plurality of infrared filters and the plurality of spectral filters are arranged in a matrix structure of a two-dimensional plane so that the array surface of the plurality of infrared filters and the plurality of spectral filters to form a light receiving surface. The method of claim 5,
The image sensor generates a signal representing the image of the object from the light reflected from the surface of the object using a plurality of infrared filters arranged in a matrix structure corresponding to a portion of the matrix structure of the two-dimensional plane,
The spectroscopic sensor penetrates the object using a plurality of spectroscopic filters arranged in a matrix structure corresponding to another part of the matrix structure of the two-dimensional plane, and then collects the spectrum of the object from light emitted from the surface of the object. An electronic device that generates a signal that represents. The method of claim 5,
The image sensor
A plurality of infrared filters arranged in a matrix structure corresponding to a portion of the matrix structure of the two-dimensional plane to filter light reflected from the surface of the object; And
And a plurality of optoelectronic devices disposed under the plurality of infrared filters to convert light filtered by the plurality of infrared filters into electrical signals. The method of claim 7, wherein
The spectroscopy sensor
A plurality of spectral filters arranged in a matrix structure corresponding to another part of the matrix of the two-dimensional plane and filtering light emitted from the surface of the object after penetrating into the object; And
And a plurality of optoelectronic devices disposed under the plurality of spectral filters to convert light filtered by the plurality of spectral filters into electrical signals. The method of claim 4, wherein
A plurality of color filters, the plurality of infrared filters, and the plurality of spectroscopic filters are arranged in a matrix structure in a two-dimensional plane, so that the arrangement surface of the plurality of color filters, the plurality of infrared filters, and the plurality of spectroscopic filters are received. Electronic device forming a face. The method of claim 2,
Each spectral filter is formed of a metal pattern having a predetermined shape periodically arranged to block the wavelength band determined by the period in the light emitted from the surface of the object after penetrating the object. The method of claim 10,
The metal patterns of one of the spectral filters and the metal patterns of the other spectral filter are arranged at different periods,
The one of the spectral filter and the other one of the spectral filter cuts different wavelength bands. The method of claim 2,
Each spectroscopic filter
Upper and lower reflection layers spaced apart from each other;
A dielectric layer interposed between the upper reflection layer and the lower reflection layer and alternately arranged with at least two materials having different refractive indices; And
A buffer layer disposed between at least one of the upper and lower reflection layers and the dielectric layer,
Each of the spectral filters passes through a wavelength band determined by the relative volume ratio between the two materials in light emitted from the surface of the object after penetrating into the object. The method of claim 11,
The dielectric layer has at least two regions having different relative volume ratios between the two materials,
The region of the dielectric layer having different relative volume ratios passes different wavelength bands. The method of claim 5,
The spectroscopy sensor
The plurality of spectral filters;
A plurality of band pass filters disposed under the plurality of spectral filters in one-to-many correspondence with the plurality of spectral filters; And
And a plurality of optoelectronic devices disposed under the plurality of bandpass filters in a one-to-one correspondence with the plurality of bandpass filters.
The plurality of band pass filters corresponding to the spectral filters transmit different wavelength bands in the filtering wavelength bands of the spectral filters,
The plurality of infrared filters are arranged in a matrix structure corresponding to a portion of the matrix structure of the two-dimensional plane, the plurality of bandpass filters are electrons arranged in a matrix structure corresponding to another portion of the matrix structure of the two-dimensional plane device. The method of claim 14,
The plurality of band pass filters are at least two kinds of infrared filters of the plurality of infrared filters, and the same kind of infrared filters. The method of claim 1,
A rear panel disposed below the display panel and having at least one light passing hole formed therein;
The composite sensor is configured to generate a signal representing an image of the object and a signal representing the spectrum of the object from light incident through the display panel through at least one light passing hole of the rear panel. The method of claim 1,
At least one collimator lens disposed under the display panel to convert light passing through the display panel into parallel light;
The composite sensor is configured to generate a signal representing an image of the object and a signal representing the spectrum of the object from light incident through the display panel through the at least one collimator lens. The method of claim 1,
The composite sensor is disposed below the display panel,
The composite sensor generates a signal representing an image of the object from the light reflected from the surface of the object is reflected from the surface of the object and passed through the display panel, and the irradiated infrared rays penetrate the object Electronic devices that later generate signals representing the spectrum of the object from light emitted from the surface of the object and passing through the display panel. In the user authentication method of the electronic device,
Irradiating infrared rays onto the surface of the object;
Generating a signal representing an image of the object from light reflected from the surface of the object by infrared radiation irradiated onto the surface of the object;
Performing image authentication on the object using a signal representing the image of the object;
Generating a signal representative of the spectrum of the object from light emitted from the surface of the object after infrared radiation irradiated onto the surface of the object penetrates into the object;
Performing biometric authentication on the object using a signal representative of the spectrum of the object; And
Authenticating a user according to a result of performing the image authentication and a result of performing the biometric authentication. The method of claim 19,
Generating a signal representing the spectrum of the object generates a signal representing the spectrum of the object from light emitted from the surface of the object after penetrating into the object using a plurality of spectral filters,
The plurality of spectral filters transmits or blocks a plurality of wavelength bands belonging to a wavelength band absorbed in the process of being emitted from the surface of the living body after penetrating into the living body of the human body of the infrared wavelength band irradiated from the light emitting module. User authentication method. The method of claim 19,
Detecting whether the object is close to less than a threshold distance;
The irradiating the infrared rays is a user authentication method for irradiating infrared rays on the surface of the object when the object is closer than the threshold distance. The method of claim 19,
Checking whether there is a user authentication request;
The irradiating the infrared rays is a user authentication method for irradiating infrared rays on the surface of the object when it is confirmed that the user authentication request. The method of claim 19,
Displaying a graphic for guiding the user's eyes or face to be positioned accurately; And
And displaying an image of the object on the displayed graphic. The method of claim 23,
Checking whether a predetermined time has elapsed since the start of the step of displaying the graphic;
The generating of the signal representing the image of the object may include generating a signal representing the image of the object from light reflected from the surface of the object when it is determined that the predetermined time has elapsed.
The generating of the signal representing the spectrum of the object may include generating a signal representing the spectrum of the object from light emitted from the surface of the object when it is determined that the predetermined time has elapsed. A computer-readable recording medium having recorded thereon a program for executing the method of any one of claims 19 to 24 on a computer. In the user biometric information registration method of the electronic device,
Irradiating infrared rays onto the surface of the object;
Generating a signal representing an image of the object from light reflected from the surface of the object by infrared radiation irradiated onto the surface of the object;
Generating an image for the object from a signal representing the image of the object;
Generating a signal representative of the spectrum of the object from light emitted from the surface of the object after infrared radiation irradiated onto the surface of the object penetrates into the object;
Generating a spectrum of the object from a signal representing the spectrum of the object; And
And registering the generated image and the generated spectrum as biometric information of a user of the electronic device. The method of claim 26,
Generating a signal representing the spectrum of the object generates a signal representing the spectrum of the object from light emitted from the surface of the object after penetrating into the object using a plurality of spectral filters,
The plurality of spectral filters penetrate or block a plurality of wavelength bands belonging to a wavelength band absorbed in the process of being emitted from the surface of the living body after penetrating the human body of the wavelength band of the light irradiated to the object. How to register biometric information. The method of claim 26,
Displaying a graphic for guiding the user's eyes or face to be positioned accurately; And
And displaying an image of the object on the displayed graphic. The method of claim 28,
Checking whether a predetermined time has elapsed since the start of the step of displaying the graphic;
The generating of the signal representing the image of the object may include generating a signal representing the image of the object from light reflected from the surface of the object when it is determined that the predetermined time has elapsed.
The generating of the signal representing the spectrum of the object may include generating a signal representing the spectrum of the object from light emitted from the surface of the object when it is determined that the predetermined time has elapsed. A computer-readable recording medium having recorded thereon a program for executing the method of any one of claims 26 to 29.

Images