KR102455248B1 - Learning method using fingers and recording medium recording the same - Google Patents

Abstract

The present invention relates to a learning method using a finger and a recording medium recording the same, in which learning suitable for a learner's level or content learning matched to a finger may be automatically performed when a student's finger or parent's finger touches a learning application running on a learner terminal. A learning method using a finger according to the present invention comprises the steps of: (a) providing a learning service through a learner terminal; (b) matching the current learning menu and the learner's fingerprint in the learner terminal and registering the matched information; (c) providing a fingerprint input screen when a learning service is executed in the learner terminal; (d) receiving the learner's fingerprint from the learner terminal and comparing the learner's fingerprint with the registered fingerprint; and (e) executing a learning menu corresponding to the registered fingerprint when the fingerprint input from the learner terminal is consistent with the registered fingerprint.

Description

Learning method using fingers and recording medium recording the same

The present invention relates to a learning method using a finger and a recording medium recording the same, and more particularly, when a learner's finger or a parent's finger is in contact with a learning application executed on a learner's terminal, learning or finger according to the learner's level It relates to a learning method using a finger, which enables automatic execution of matched content learning, and a recording medium recording the same.

Milk-T elementary school learning service provided by the applicant through a tablet terminal or web page provides learning menus such as milk-T school study, math by level, English by level, and creative thinking.

In order for elementary school students to use the Milk T learning service through a tablet terminal, they must go through a multi-step process of running the Milk T application, finding and selecting a desired learning menu from among a number of menus.

Therefore, in the case of low-grade learners, there was a problem in that the learning motivation was lowered because it was difficult or cumbersome to find the desired learning menu from a plurality of learning menus and to execute the learning.

As a related prior patent document, there is Republic of Korea Patent Publication No. 10-2020-0137755 (published on Dec. 9, 2020), which describes a learning content system in which young children participate.

It is an object of the present invention to automatically execute learning suitable for the learner's level or content learning matched to the finger when the learner's finger or the parent's finger is in contact with a learning application executed in a learner terminal, learning using a finger To provide a method and a recording medium recording the same.

A learning system using a finger according to an embodiment of the present invention for achieving the above object includes a touch input/output unit for receiving a finger touch input and providing a learning menu on a screen; a storage unit for storing one or more learning contents corresponding to the learning menu; a content matching unit for matching the fingerprint image acquired through the touch input/output unit with the learning content according to the learning menu; a learning execution unit providing the learning menu on the screen of the touch input/output unit and executing learning content corresponding to the selected learning menu; And the learning content corresponding to the learning menu selected through the touch input/output unit and the fingerprint image obtained through the touch input/output unit are matched and registered through the content matching unit, the learning execution unit is executed, and a finger touch is performed through the touch input/output unit When the input and the same fingerprint image as the registered fingerprint image is obtained through the touch input/output unit, the control unit for controlling the learning execution unit to execute the learning content matched to the registered fingerprint image.

The learning menu includes milk T school study, mathematics by level, English by level, creative thinking ability, and milk T middle school general class menu, and the milk T school study menu includes study by subject, special lecture for test, correctional tutoring, middle school special lecture menu And, the math menu for each level includes storytelling concept learning, math by domain, FunFun calculation game, type breaking, and skill charging menu, and the English menu for each level is Speaking, Ssok Words, Pang Pang Phonics, Line Reading, Tantan Grammer, Including English Songs and English Library menu, the creative thinking menu includes special learning, homework solving, genius book club, elementary life menu, and the Milk T middle school comprehensive menu includes study manager, multi-learning player, self-directed learning, It may include a math English correction service, a math Q&A community, and a 10-minute math English menu.

The touch input/output unit includes light-emitting elements emitting light, and at least one light sensor that senses light emitted from the light-emitting elements and reflected by the fingerprint in contact with the fingerprint contact surface to generate an image corresponding to the fingerprint. a fingerprint sensor including a light sensing array unit and a selective light transmitting unit including a plurality of pinholes forming a propagation path of light incident to the light sensor; and receiving the image from the fingerprint sensor, extracting a first image corresponding to the first region of the fingerprint and a second image corresponding to the second region of the fingerprint, and comparing the first and second images to determine the degree of similarity It may include a fingerprint detection unit that determines and performs fingerprint authentication according to the result.

The first area of the fingerprint is one area of the fingerprint in contact with the fingerprint contact surface, and the second area of the fingerprint is located outside the one area of the fingerprint and is another area of the fingerprint that is not in contact with the fingerprint contact surface. can

The fingerprint detector may include: a fingerprint image extractor configured to receive the image from the photodetector and extract the first image and the second image; an image processing unit performing image preprocessing on the first and second images; a similarity determining unit that compares the first image and the second image, which have been pre-processed by the image processing unit, to determine whether there is a similarity; and an authentication performing unit configured to perform fingerprint authentication according to a result of the similarity determining unit.

The similarity determining unit compares the luminance of the first image with the luminance of the second image, checks whether the first and second images have a similar pattern to each other, and determines whether the fingerprint in contact with the fingerprint contact surface is a biometric fingerprint. can be judged

On the other hand, the learning method using a finger according to an embodiment of the present invention for achieving the above object, (a) providing a learning service through a learner terminal; (b) matching and registering the current learning menu and the learner's fingerprint in the learner terminal; (c) providing a fingerprint input screen when the learning service is executed in the learner terminal; (d) receiving the learner's fingerprint from the learner terminal and comparing it with a registered fingerprint; and (e) executing a learning menu corresponding to the registered fingerprint when the fingerprint input from the learner terminal matches the registered fingerprint.

In step (b), the learner terminal may match the fingerprint image acquired through the touch input/output unit with the learning content according to the learning menu, and store it in the storage unit for registration.

The touch input/output unit may include a fingerprint sensor and a fingerprint detection unit.

The fingerprint sensor includes light-emitting elements emitting light, and at least one light sensor that generates an image corresponding to the fingerprint by sensing light emitted from the light-emitting elements and reflected by the fingerprint in contact with the fingerprint contact surface. It may include a sensing array unit and a selective light transmitting unit including a plurality of pinholes forming a propagation path of light incident to the optical sensor.

The fingerprint detector receives the image from the fingerprint sensor, extracts a first image corresponding to the first region of the fingerprint and a second image corresponding to the second region of the fingerprint, and compares the first and second images to determine the degree of similarity, and perform fingerprint authentication according to the result.

The first area of the fingerprint is one area of the fingerprint in contact with the fingerprint contact surface, and the second area of the fingerprint is located outside the one area of the fingerprint and is another area of the fingerprint that is not in contact with the fingerprint contact surface. can

The fingerprint detector receives an image from light sensing through an image extraction unit, extracts a first image and a second image, performs image preprocessing on the first and second images through an image processing unit, and determines similarity By comparing the first image and the second image pre-processed by the image processing unit through the unit, the degree of similarity is determined, and whether or not the fingerprint is authenticated according to the result of the similarity determining unit through the authentication performing unit.

The similarity determining unit compares the luminance of the first image with the luminance of the second image, checks whether the first and second images have a similar pattern to each other, and determines whether the fingerprint in contact with the fingerprint contact surface is a biometric fingerprint. to judge

On the other hand, the learning method using a finger according to an embodiment of the present invention for achieving the above object, (a) providing a learning service through a learner terminal; (b) matching and registering the current learning menu and the learner's fingerprint in the learner terminal; (c) providing a fingerprint input screen when the learning service is executed in the learner terminal; (d) receiving the learner's fingerprint from the learner terminal and comparing it with a registered fingerprint; and (e) executing a learning menu corresponding to the registered fingerprint when the fingerprint input from the learner terminal matches the registered fingerprint. can be programmed in

According to the present invention, school study, math, English, and creative thinking skills menus are matched and registered with the fingerprint of each finger of the learner, and the learner can immediately execute the desired learning with a finger touch at any time.

In addition, according to the present invention, by touching the thumb, milk T school study is executed, it is possible to provide a function of increasing the ability to study by oneself and holding a self-directed learning habit.

In addition, according to the present invention, by touching the index finger to execute math learning for each level, it is possible to provide a function for mastering math from basic to deep.

In addition, according to the present invention, by touching the middle finger to execute English learning for each level, it is possible to provide a function of improving English skills while interacting with various multimedia contents.

In addition, according to the present invention, as the creative thinking ability learning is executed by touching the ring finger, it is possible to easily and conveniently acquire culture and information in various fields, and various indirect experiences for each field according to the standards presented in the revised curriculum It can provide the function of cultivating creative thinking ability.

In addition, according to the present invention, as the middle school comprehensive class learning is executed by touching the little finger, take a career education special lecture, design a career path that enhances your own competitiveness through a career aptitude test, and find a career path that suits you It can provide the ability to find and strengthen basic concepts and develop skills that are not shaken by any difficulty.

1 is a configuration diagram schematically showing the overall configuration of a learning induction system according to an embodiment of the present invention.
2 is a configuration diagram schematically showing the internal configuration of a learning chatbot according to an embodiment of the present invention.
3 is a diagram illustrating an operation process of a conversion unit of a learning chatbot according to an embodiment of the present invention.
4 is a diagram illustrating an operation flowchart of a learning chatbot according to an embodiment of the present invention.
5 is a diagram illustrating a detailed operation flowchart of a learning chatbot according to an embodiment of the present invention.
6 is a view showing today's learning screen provided by the learning chatbot according to an embodiment of the present invention.
7 is a view showing a full menu screen provided by a learning chatbot according to an embodiment of the present invention.
8 is a diagram illustrating a question board screen provided by a learning chatbot according to an embodiment of the present invention.
9 is a diagram illustrating a learning window screen provided by a learning chatbot according to an embodiment of the present invention.
10 is a diagram illustrating an example of providing today's learning for 1st to 6th graders in the learning chatbot according to an embodiment of the present invention.
11 is a diagram illustrating an example of providing today's learning for kids in a learning chatbot according to an embodiment of the present invention.
12 is a diagram illustrating an example of providing a full menu in a learning chatbot according to an embodiment of the present invention.
13 is a schematic cross-sectional view of a learner terminal according to an embodiment of the present invention.
14 is a block diagram schematically illustrating the fingerprint detection unit shown in FIG. 6 or FIG. 7 .
15 is an operation flowchart illustrating a learning method using a finger according to an embodiment of the present invention.
16 is a view showing an example of a screen for the Milk T elementary learning service provided through the learner terminal according to an embodiment of the present invention.
17 is a diagram illustrating a fingerprint registration icon for registering a learner's fingerprint by matching the learning menu provided through the learner terminal according to an embodiment of the present invention.
18 is a diagram illustrating an example of receiving a learner's fingerprint for matching with the currently executed learning menu in the learning terminal according to an embodiment of the present invention.
19 is a diagram illustrating a state in which a user's finger print contacts a fingerprint contact surface of a learner terminal according to an embodiment of the present invention.

Advantages and features of the present invention, and methods for achieving them, will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be embodied in various different forms, only this embodiment allows the disclosure of the present invention to be complete, and common knowledge in the technical field to which the present invention belongs It is provided to fully inform the possessor of the scope of the invention, and the present invention is only defined by the scope of the claims. Accordingly, in some embodiments, well-known process steps, well-known device structures, and well-known techniques have not been specifically described in order to avoid obscuring the present invention. Like reference numerals refer to like elements throughout.

In order to clearly express various layers and regions in the drawings, the thicknesses are enlarged. Throughout the specification, like reference numerals are assigned to similar parts. When a part of a layer, film, region, plate, etc. is said to be “on” another part, it includes not only cases where it is “directly on” another part, but also cases where there is another part in between. Conversely, when we say that a part is "just above" another part, we mean that there is no other part in the middle. Also, when a part of a layer, film, region, plate, etc. is said to be “under” another part, it includes not only the case where the other part is “directly under” but also the case where there is another part in the middle. Conversely, when we say that a part is "just below" another part, it means that there is no other part in the middle.

Spatially relative terms "below", "beneath", "lower", "above", "upper", etc. It can be used to easily describe a correlation between an element or components and other elements or components. The spatially relative terms should be understood as terms including different orientations of the device during use or operation in addition to the orientation shown in the drawings. For example, when an element shown in the figures is turned over, an element described as "beneath" or "beneath" another element may be placed "above" the other element. Accordingly, the exemplary term “below” may include both directions below and above. The device may also be oriented in other orientations, and thus spatially relative terms may be interpreted according to orientation.

In the present specification, when a part is said to be connected to another part, it includes not only a case in which it is directly connected, but also a case in which it is electrically connected with another element interposed therebetween. In addition, when it is said that a part includes a certain component, this means that other components may be further included, rather than excluding other components, unless otherwise stated.

In this specification, terms such as first, second, third, etc. may be used to describe various components, but these components are not limited by the terms. The above terms are used for the purpose of distinguishing one component from other components. For example, without departing from the scope of the present invention, a first component may be referred to as a second or third component, and similarly, the second or third component may be alternately named.

Unless otherwise defined, all terms (including technical and scientific terms) used herein may be used with the meaning commonly understood by those of ordinary skill in the art to which the present invention belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless clearly defined in particular.

Hereinafter, a learning induction system and method using a learning chatbot according to a preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1 is a configuration diagram schematically showing the overall configuration of a learning system using a finger according to an embodiment of the present invention.

Referring to FIG. 1 , a learning system 100 using a finger according to an embodiment of the present invention includes a plurality of web servers 102 , a learner terminal 104 , a first server ( Server 1 ) 110 , and a second It includes a server (Server 2) (120).

The web server 102 is a server that provides a learning service through a web page in cooperation with the first server 110 .

The learner terminal 104 is a terminal carried by the learner, and accesses the web server 102 to receive a learning service from the web server 102 . The learner terminal 104 may be, for example, a tablet terminal or a smart phone.

The first server 110 is a server that provides a service for executing or inducing learning through the learning execution unit 114 .

The first server 110 includes a touch input/output unit 112 and a learning execution unit 114 . Here, the learning execution unit 114 may be, for example, a Learning Chot Bot.

The touch input/output unit 112 provides a learning menu on the screen, receives a touch input for menu selection, or receives a voice or text input for a user's question.

The learning menu may include, for example, a milk T elementary learning menu. Milk T elementary school learning menu may include milk T school study, math by level, English by level, creative thinking ability, and milk T middle school general class menu.

The learning execution unit 114 processes the voice or text input through the touch input/output unit 112 and induces learning by one of keyword recommendation, simple response, progress check, associative learning, simple game, and emotional coaching according to the processing result. can do.

The learning execution unit 114 may provide a learning induction service using the chatbot service API and the WE model.

The learning execution unit 114 provides today's learning, the entire menu, and homework questions for the learning menu, the user selects a subject, executes the learning of the selected subject, and answers the homework question to Q&A data and a learning encyclopedia. It is possible to provide a learning answer by using the data for

The learning execution unit 114 may provide a milk T elementary school learning menu with respect to the learning menu, and a milk T school study, math by level, English by level, creative thinking ability, and milk T middle school general class menu.

The learning execution unit 114 provides a learning window for the learning menu, automatically selects a subject based on the learning window subject code, and provides a learning answer to the homework question by using the Q&A data and the learning encyclopedia data. can provide

The second server 120 is a server that interworks with the first server 110 to provide a chatbot management API, WE model, database, elastic searcher, and the like.

The database is used to store project/intent/sentence information and token weight for each project.

The elastic searcher is used to compare sentence information and sentence vector for similarity using index vector field.

Here, the web server 102 , the first server 110 , and the second server 120 may be implemented by, for example, a computer device or the like.

The learner terminal 14 accesses the web server 102, and as a milk T elementary learning menu for the learning menu, the milk T school study, mathematics by level, English by level, creative thinking ability, and milk T middle school general class menu can be provided. .

2 is a configuration diagram schematically illustrating an internal configuration of a learning execution unit according to an embodiment of the present invention.

Referring to FIG. 2 , the learning execution unit 114 according to an embodiment of the present invention may include a transforming unit 210 , an embedding processing unit 220 , and a learning inducing unit 230 .

The conversion unit 210 converts the voice input through the touch input/output unit 112 into text or converts the text into speech, recognizes it and generates it as a language model.

The embedding processing unit 220 embeds the converted character.

The learning induction unit 230 may induce learning by one of keyword recommendation, simple response, progress check, associative learning, simple game, and emotional coaching according to the embedding process result.

Speech recognition in which the conversion unit 210 converts voice into text (Speech To Text) is a technology required to receive voice input for learners who are not familiar with keypad input.

Speech recognition refers to processing in which a computer interprets the speech language spoken by a person and converts the content into text data. Also known as Speech-to-Text (STT) technology. It is attracting attention as a method of entering characters instead of a keyboard. It is applied when device control and information retrieval are required by voice, such as robots and telematics. A representative algorithm is HMM (Hidden Markov Model). As shown in FIG. 3 , voices uttered by various speakers are preprocessed and statistically modeled to construct an acoustic model, and a language model is constructed through corpus collection. 3 is a diagram illustrating an operation process of a conversion unit of a learning execution unit according to an embodiment of the present invention. As shown in FIG. 3 , in the learning execution unit 114 , the converting unit 220 converts voice into text data through a preprocessing process, an acoustic model process, a language model, and a postprocessing process.

In addition, the conversion unit 210 may also perform a text-to-speech (TTS) function for converting text input by the user into speech. Of course, the converter 210 may perform a Speech to Text (STT) function for converting speech input from the user into text.

The embedding processing unit 220 performs a natural language processing (NLP) process when pre-processing and embedding the converted characters, and outputs character data by post-processing.

National Language Processing (NLP) technology is a field that combines computer science, artificial intelligence, and linguistics. It is an AI technology that deals with the understanding, generation, and analysis of human language using computers.

Natural language processing or natural language processing is one of the main fields of artificial intelligence that studies and implements human language phenomena to be described using machines such as computers. Natural language processing is naturally closely related to linguistics, which studies language itself, and linguistic cognitive science, which explores the internal mechanisms of language phenomena, since the subject of study is language. It is a representative field that uses a lot of mathematical and statistical tools for implementation, especially machine learning tools. Various applications such as information retrieval, QA system, automatic document classification, newspaper article clustering, and interactive agent are being made. Although research on natural language has been going on for a long time, even in 2018, computers still cannot understand natural language like humans. Instead, the technology for processing a large amount of information using superficial probability and statistics without a deep understanding of language has been greatly developed.

The present invention may utilize a text embedding technique for encoding words and sentences into numeric vectors. These vector representations are designed to capture the linguistic content of text and can be used to evaluate the similarity between a query and a document.

A method of expressing a word in the form of a dense vector is called word embedding. And since this dense vector is a result obtained through the word embedding process, it is also called an embedding vector.

Word embedding methods include LSA, Word2Vec, FastText, and Glove. Embedding(), a tool provided by Keras, does not use the aforementioned methods, but converts words into dense vectors with random values and then learns word vectors in the same way as learning the weights of an artificial neural network. use the method

The (true) embedding, which indicates the characteristics and similarity of words, is a prediction-based method through learning, such as Word2Vec.

At this point, the distributed hypothesis emerges. The distribution hypothesis states that words in the same context, that is, words that appear in similar positions, have similar meanings. Therefore, words that exist in similar positions in a certain text will have a high degree of similarity between words.

Meanwhile, in FIG. 2 , the learning induction unit 230 may induce learning by one of keyword recommendation, simple response, progress check, associative learning, simple game, and emotional coaching according to the embedding process result.

The learning execution unit 114 solves the student's curiosity in real time through the chatbot, and recommends related keywords that you would like to know additionally. Through this, students can feel curiosity and find interest in learning.

The learning induction unit 230 checks the learning progress and informs when a conversation is created about what learning is required.

When there is a climate change, the learning induction unit 230 asks for greetings related to the weather, and enables emotional communication with the learner. For example, the learning induction unit 230 is raining today, so make sure to bring an umbrella!, the sun is shining. How are you feeling today? It can induce emotional communication with learners, such as

The learning induction unit 230 recommends associative learning through a simple game in the chatbot. It is a game to inspire interest and to induce learning later. By recommending keywords or associative learning that have the same vector value as the answer to the question asked by the student, it makes students feel interested in in-depth learning.

4 is a diagram illustrating an operation flowchart of a learning execution unit according to an embodiment of the present invention.

Referring to FIG. 4 , the learning execution unit 114 according to an embodiment of the present invention may be divided into a front operation and a back operation.

In the case of the front operation, one service executes the learning chatbot on the tablet PC for Milk T Elementary School carried by the user (S410), the learning chatbot provides a menu for today's learning, the full menu, and homework questions. . Accordingly, the user selects a subject (S420) and executes a homework question (S430). Here, homework questions are provided using Q&A data or learning encyclopedia data.

In the case of the front operation, when the other service executes the learning chatbot on the tablet PC for Milk T elementary school carried by the user (S440), the learning chatbot provides a learning window and automatically selects a subject based on the learning window subject code provides (S450). Accordingly, the learning chatbot provides homework questions using Q&A data or learning encyclopedia data (S460).

In the case of the back operation, it provides the ADMIN function for the chatbot manager utilization and operation, and also provides the CRM function for the user chatbot usage history.

5 is a configuration diagram schematically showing the configuration of a learning system using a finger according to an embodiment of the present invention.

That is, FIG. 5 is a diagram showing the internal configuration of the learner terminal 104 to which the finger learning system according to an embodiment of the present invention is applied.

Referring to FIG. 5 , the learner terminal 104 to which the present invention is applied may include a touch input/output unit 502 , a storage unit 504 , a matching unit 506 , a learning execution unit 508 , and a control unit 510 . can

The touch input/output unit 502 receives a finger touch and provides a learning menu on the screen.

The storage unit 504 stores one or more learning contents corresponding to the learning menu. Also, the storage 504 may store a fingerprint image acquired through the touch input/output unit or a fingerprint image registered through the touch input/output unit.

The matching unit 506 matches the fingerprint image obtained through the touch input/output unit with the learning content according to the learning menu.

The learning execution unit 508 provides the learning menu on the screen of the touch input/output unit, and executes the learning content corresponding to the selected learning menu.

The control unit 510 matches and registers the learning content corresponding to the learning menu selected through the touch input/output unit and the fingerprint image obtained through the touch input/output unit through the matching unit, the learning execution unit is executed, and a finger touch is input through the touch input/output unit. , when the same fingerprint image as the registered fingerprint image is obtained through the touch input/output unit, the learning execution unit may be controlled to execute the learning content matched to the registered fingerprint image.

The learning menu provided to the learner terminal 104 includes milk T school study, math by level, English by level, creative thinking ability, and milk T middle school general class menu.

Milk T school study menu includes study by subject, special lecture for exam preparation, correctional tutoring, and special lecture menu for middle school. The mathematics menu by level includes storytelling concept learning, mathematics by area, FunFun calculation game, type breaking, and skill charging menu. .

The English menu for each level includes Speaking, Words, Phonics, Reading, Tantan Grammer, English Songs, and English Library menus.

Milk T middle school general class menu may include study manager, multi-learning player, self-directed learning, math English correction service, math question and answer community, and 10-minute math English menu.

6 and 7 are plan views schematically illustrating a learner terminal according to an embodiment of the present invention.

6 and 7 , the learner terminal 104 according to an embodiment of the present invention may include a display panel PN and a driving circuit DCP for driving the display panel PN.

The display panel PN may include a display area DA and a non-display area NDA. The display area DA may display an image by providing the pixels PXL, and the non-display area NDA may be positioned at at least one side of the display area DA. For example, the non-display area NDA may be provided to surround the display area DA.

A plurality of pixels PXL may be provided in the display area DA. According to an embodiment, each of the pixels PXL may include at least one light emitting device. According to an embodiment, the light emitting device may be an organic light emitting diode or a light emitting unit including ultra-small inorganic light emitting diodes having a size in a micro to nano scale range, but the present invention is not limited thereto. The display device DD may display an image in the display area DA by driving the pixels PXL in response to input image data.

The non-display area NDA is an area surrounding at least one side of the display area DA, and may be the remaining area except for the display area DA. According to an embodiment, the non-display area NDA may include a wiring area, a pad area, and/or various dummy areas.

The display area DA may include a touch input/output unit 502 . The touch input/output unit 502 may include a fingerprint sensor FPS and a fingerprint detection unit FPDP.

In the display panel PN of the learner terminal 104 according to an embodiment of the present invention, one area may be set as a sensing area SA capable of detecting a user's fingerprint or the like. That is, at least a portion of the display area DA may be the sensing area SA. The sensing area SA may include at least some of the pixels PXL provided in the display area DA.

Here, the detection area SA may include a fingerprint sensor FPS that detects a user's fingerprint.

As shown in FIG. 6 , only a portion of the display area DA may be set as the sensing area SA. Also, according to another exemplary embodiment, the entire display area DA may be set as the sensing area SA as shown in FIG. 7 . When the entire display area DA is set as the sensing area SA, the non-display area NDA surrounding the display area DA may become the non-sensing area NSA. A plurality of light sensors PSR may be disposed in the sensing area SA together with a plurality of pixels PXL.

The optical sensors PSR may be disposed on the other side of the display device DD facing one surface (eg, an image display surface) on which an image is displayed. That is, the optical sensors PSR may be disposed on a surface on which an image is not displayed on the display device DD. The light sensors PSR may use a light emitting device provided in the sensing area SA or at least one pixel PXL disposed around the sensing area SA as a light source for detecting a fingerprint or the like. To this end, the optical sensors PSR may overlap at least some of the pixels PXL disposed in the sensing area SA or may be disposed around the pixels PXL.

The optical sensors PSR may constitute a photo-sensing fingerprint sensor together with the pixels PXL of the sensing area SA, in particular, light emitting devices provided in the pixels PXL. That is, the optical sensors PSR may detect that the light emitted from the light emitting device is reflected by the user, and detect the reflected light to detect the user's fingerprint. Although it has been described that the optical sensors PSR are used for fingerprint detection in the above-described embodiment, the present invention is not limited thereto. According to an embodiment, the optical sensors PSR may be used to perform various functions, such as a touch sensor or a scanner, in addition to detecting a fingerprint.

The inventor's terminal 104 according to the above-described embodiment of the present invention is displayed on the display panel PN using the optical sensors PSR disposed in the sensing area SA included in the display area DA. It can detect the shape or pattern of a positioned object. For example, the inventor's terminal 104 may detect a user's fingerprint. In addition, the inventor's terminal 104 according to an embodiment of the present invention may detect a user's fingerprint using light emitted from the pixels PXL. As such, when a fingerprint sensor-embedded display device is implemented by using the pixels PXL as a light source without a separate external light source, the thickness of the inventor's terminal 104 can be reduced and the manufacturing cost thereof can be reduced. However, the present invention is not limited thereto, and a separate external light source for fingerprint sensing may be employed.

The driving circuit DCP may drive the display panel PN. For example, the driving circuit DCP outputs a data signal corresponding to image data to the display panel PN or outputs a driving signal for the optical sensors PSR and receives electrical signals from the optical sensors PSR. A signal (eg, a sensing signal) may be received. The driving circuit DCP may detect the user's fingerprint shape using electrical signals.

In some embodiments, the driving circuit DCP may include a panel driving unit PNDP and a fingerprint detecting unit FPDP. For convenience, although the panel driver PNDP and the fingerprint detector FPDP are illustrated separately in FIGS. 6 and 7 , the present invention is not limited thereto. For example, at least a portion of the fingerprint detector FPDP may be integrated together with the panel driver PNDP or may operate in conjunction with the panel driver PNDP.

The panel driver PNDP may supply a data signal corresponding to the image data signal to the pixels PXL while sequentially watching the pixels PXL of the display area DA. In this case, the display panel PN may display an image corresponding to the image data.

In some embodiments, the panel driver PNDP may supply a driving signal for fingerprint sensing to the pixels PXL. Such a driving signal may be provided to cause the pixels PXL to emit light to operate as a light source for the light sensors PSR. The driving signal for fingerprint sensing may be transmitted to the pixels PXL provided in a specific area of the display panel PN, for example, the pixels PXL provided in the sensing area SA. In various embodiments, a driving signal for fingerprint sensing may be transmitted to the pixels PXL of the sensing area SA by the fingerprint detector FPDP.

The fingerprint detector FPDP transmits a driving signal (eg, a driving voltage) for driving the optical sensors PSR to the optical sensors PSR, and based on the electrical signals received from the optical sensors PSR Thus, the user's fingerprint can be detected.

FIG. 8 is a cross-sectional view schematically illustrating the learner terminal shown in FIG. 6 or FIG. 7 , and FIG. 9 is a plan view schematically illustrating the selective light transmitting part of FIG. 8 .

6, 7, 8 and 9 , the learner terminal 104 according to an embodiment of the present invention may include a display panel PN, a touch sensor TS, and a window WD. have.

The display panel PN may include a fingerprint sensor SA, and the fingerprint sensor SA may include a light sensing array unit LSL and a selective light transmitting unit LBL.

The light sensing array unit LSL includes at least one light emitting element LD that emits light, and at least one of the light emitting elements that sense the light emitted from the light emitting elements and reflected by the fingerprint in contact with the fingerprint contact surface to generate an image corresponding to the fingerprint. of the optical sensor (PSR).

The selective light transmitting part LBL includes a plurality of pinholes forming a propagation path of light incident to the optical sensor PSR.

The learner terminal 104 according to an embodiment of the present invention includes a first surface SF1 on which an image is displayed and a second surface SF2 facing the first surface SF1, and includes a light sensing array unit LSL. ) may be located closer to the second surface SF2 than to the first surface SF1 . In an embodiment of the present invention, the first surface SF1 of the learner terminal 104 may be a display surface on which an image is displayed. One area of the first surface SF1 of the learner terminal 104 may be a fingerprint contact surface to which a user's finger fingerprint is in contact.

The display panel PN may display an image. The type of the display panel PN is not particularly limited as it displays an image. A display panel capable of self-emission such as an organic light emitting display panel (OLED panel) may be used as the display panel PN. In addition, as the display panel (PN), a liquid crystal display panel (LCD panel), an electro-phoretic display panel (EPD panel), and an electro-wetting display panel (EWD panel) ) may be used. When the non-luminous display panel is used as the display panel PN of the learner terminal 104 according to an embodiment of the present invention, the learner terminal 104 may include a backlight unit for supplying light to the display panel PN. have.

The display panel PN may include a substrate SUB, a selective light transmitting part LBL, a display module DM, and a thin film encapsulation layer TFE. In an embodiment of the present invention, the display module DM includes a display element unit DPL including a light emitting element LD that emits light and a pixel circuit unit PCL including circuit elements driving the light emitting element LD. may include

The substrate SUB is a base substrate of the display panel PN and may be a substantially transparent light-transmitting substrate. According to an embodiment, the substrate SUB may be a rigid substrate including glass or tempered glass, or a flexible substrate made of a plastic material. In one embodiment of the present invention, the substrate SUB may be a flexible substrate.

The pixel circuit unit PCL is disposed on one surface (eg, an upper surface) of the substrate SUB, and may include at least one conductive layer. For example, the pixel circuit unit PCL is formed in each of the pixel areas PXA and includes a plurality of circuit elements constituting the pixel circuit of the corresponding pixel PXL, and various power sources and signals for driving the pixels PXL. It may include wires for supplying In this case, the pixel circuit unit PCL may include various circuit elements such as at least one transistor and a capacitor, and a plurality of conductive layers for forming wirings connected thereto. Also, the pixel circuit unit PCL may include at least one insulating layer provided between the plurality of conductive layers. Also, the pixel circuit unit PCL may include a fan-out line unit disposed in the non-display area NDA of the display panel PN to supply power and signals corresponding to lines connected to the pixels PXL.

The display element part DPL may be disposed on one surface of the substrate SUB including the pixel circuit part PCL. In some embodiments, the display element unit DPL may include a plurality of light emitting elements LD connected to circuit elements and/or wirings of the pixel circuit unit PCL through a contact hole or the like. For example, the display element unit DPL may include at least one light emitting element LD disposed in each of the pixel areas PXA. That is, the display element unit DPL may include a plurality of light emitting elements LD. In one embodiment of the present invention, the light emitting devices LD may be organic light emitting diodes or micro light emitting devices using a structure in which an inorganic crystal structure is grown.

The thin film encapsulation layer TFE may be disposed on the display element part DPL to cover at least the display area DA.

The selective light transmitting part LBL may be positioned between the substrate SUB and the pixel circuit part PCL at least in the sensing area SA. For example, the selective light transmitting part LBL may be disposed on one surface of the substrate SUB to be positioned between the substrate SUB and the pixel circuit part PCL. The selective light transmitting part LBL may include a plurality of pinholes PIH and a light blocking pattern LBP.

The light blocking pattern LBP may be formed of a light blocking material and/or a light absorption material. For example, the light blocking pattern LBP may be formed of an opaque metal layer between adjacent pinholes PIH. The pinholes PIH may be through holes passing through at least one region of the light blocking pattern LBP.

The selective light transmitting part LBL may include a plurality of pinholes PIH uniformly provided in the light blocking pattern LBP to have a predetermined size and spacing. However, the present invention is not limited thereto, and the size, shape, number, resolution, and/or arrangement of the pinholes PIH may be variously changed. For example, the pinholes PIH may be irregularly provided in the light blocking pattern LBP.

The pinholes PIH may be formed with an appropriate size and spacing sufficient to detect a clearer fingerprint shape while preventing diffraction of incident light. For example, a width of each of the pinholes PIH may be set to be approximately 10 times or more with respect to a wavelength of incident light in order to prevent light diffraction. The pinholes PIH may be spaced apart from each other at regular intervals. The pinholes PIH may form a grid arrangement as shown in FIG. 8 . The distance between the pinholes PIH is a distance between the light blocking layer LBL and the light sensing array unit LSL, a wavelength of incident light, and an observation field of view (field of view: FOV) required for the pinholes PIH. ) can be determined based on For example, 3 to 15 pixels PXL may be positioned between two adjacent (or adjacent) pinholes PIH in order to detect a relatively clear fingerprint shape, but the present invention is not limited thereto. it is not According to an embodiment, 15 or more pixels PXL or 3 or less pixels PXL may be positioned between two adjacent (or adjacent) pinholes PIH.

Each of the pinholes PIH may have a circular shape in plan view, but the present invention is not limited thereto. According to an embodiment, the pinholes PIH may have various shapes including an equilateral triangle, a square, and a regular hexagon. The density (or resolution) of the pinholes PIH may be uniform throughout the sensing area SA, but the present invention is not limited thereto. It may be high in , and low in other regions of the sensing area SA. Each of the pinholes PIH may be an effective hole in which a focal point F in which the reflected light RL incident to the light sensing array unit LSL is converged is formed.

Some of the light incident to the selective light transmitting part LBL may be blocked by the light blocking pattern LBP, and the remaining part may pass through the pinholes PIH to reach the light sensing array part LSL. The selective light transmitting part LBL may selectively transmit the reflected light RL reflected from an object located on the first surface SF1 of the display device DD, for example, a user's fingerprint. The above-described pinholes PIH may mean an optical hole, and each of the pinholes PIH may be a type of a through hole or a light-transmitting hole.

In the above-described display panel PN, the selective light transmitting unit LBL is disposed between the display element unit DPL and the light sensing array unit LSL to selectively transmit only a portion of light (eg, the reflected light RL). An optical system for controlling a light path or the like can be configured.

The above-described selective light transmitting portion LBL may correspond to the display area DA of the display panel PN, and may have a size (or area) greater than or equal to that of the display area DA. For example, when the sensing area SA is the entire display area DA, the selective light transmitting part LBL may have a size (or area) that is greater than or equal to that of the display area DA. As another example, when the sensing area SA is a part of the display area DA, the selective light transmitting part LBL has a size (or area) greater than or equal to the sensing area SA and smaller than or equal to the display area DA, or the sensing area SA. ) can have the same size (or area) as As another example, when the sensing area SA is a part of the display area DA, the selective light transmitting part LBL may have the same size (or area) as the sensing area SA.

The light sensing array unit LSL may be attached to the other surface (or lower surface) of the substrate SUB to overlap at least one area of the display panel PN. For example, the light sensing array unit LSL may be disposed to overlap the display panel PN at least in the sensing area SA. The light sensing array unit LSL may include a plurality of light sensors PSR dispersed at a predetermined density (or resolution) and/or intervals.

The optical sensors PSR may be implemented as a semiconductor layer or a semiconductor chip in which a plurality of photoelectric conversion elements (eg, photodiodes, phototransistors, photogates, pinned photodiodes, etc.) are formed. According to an embodiment, the optical sensors PSR may be a semiconductor layer in which an image sensor such as a CMOS image sensor (CIS) or a charge coupled device (CCD) is implemented.

The light sensors PSR of the light sensing array unit LSL may output an electrical signal corresponding to the reflected light RL received through the pinholes PIH as a sensing signal. The reflected light RL received by each optical sensor PSR has different optical characteristics (eg, frequency, wavelength, size, etc.). Accordingly, each of the optical sensors PSR may output a sensing signal (or electrical signal) having different electrical characteristics corresponding to the optical characteristics of the reflected light RL. The sensing signal output by the optical sensors PSR may be converted into image data and used to identify a user's fingerprint.

In an embodiment of the present invention, the light sensing array unit LSL including the light sensors PSR and the selective light transmitting unit LBL including the pinholes PIH are the sensing area SA of the learner terminal 104 . It is possible to configure the optical system of the fingerprint sensor in the

In an embodiment of the present invention, in aligning the pixels PXL, the pinholes PIH, and the optical sensors PSR of the sensing area SA, the diameter d of each of the pinholes PIH ) and the thickness T may correspond to parameters that determine angles of view (θ) for the upper and lower surfaces based on each pinhole PIH. By controlling the size, spacing, and/or resolution of the light sensing array unit LSL including the light sensors PSR and the selective light transmitting unit LBL including the pinholes PIH, the above-described parameters may be determined. have.

The touch sensor TS and the window WD may be disposed on the display panel PN including the above-described configurations.

The touch sensor TS may be disposed on a surface on which an image is displayed on the display panel PN to receive a user's touch input and/or a hover input. The touch sensor TS may recognize a touch input and/or a hover input of the learner terminal 104 by detecting a touch capacitance by contact and/or proximity of a separate input means such as a user's hand or a similar conductor. have. Here, the touch input means a direct touch (or contact) by the user's hand or a separate input means, and the hover input means that the user's hand or a separate input means is the learner terminal 104 including a touch sensor (TS). It can mean being near but not touching.

In addition, the touch sensor TS may detect a user's touch action and move an object displayed on the learner terminal 104 from an originally displayed location to another location in response to the touch action. Here, the touch operation may include at least one of a single single touch, a multi-touch, and a touch gesture. For example, there may be various touch operations including a specific gesture such as enlarging or reducing text or an image by moving a predetermined distance while touching the user's finger on the touch surface of the touch sensor TS.

The window WD is a member disposed at the top of the learner terminal 104 including the display panel PN, and may be a substantially transparent light-transmitting substrate. The window WD transmits an image from the display panel PN and at the same time relieves an external shock, thereby preventing the display panel PN from being damaged or malfunctioning due to an external shock. Here, the external impact is an external force that can be expressed as pressure, stress, or the like, and may mean a force that may cause a defect in the display panel PN. The window WD may include a rigid or flexible substrate, and the material of the window WD is not particularly limited.

A method of detecting a fingerprint of the learner terminal 104 according to the above-described embodiment will be briefly described as follows. During the fingerprint sensing period in which the optical sensors PSR are activated, in a state in which a user's finger (eg, fingerprint area) is in contact with or close to the sensing area SA, the pixels PXL of the sensing area SA The light emitting devices LD provided in the light emitting device may emit light. For example, during the fingerprint sensing period, the light emitting devices LD provided in all the pixels PXL of the sensing area SA may emit light simultaneously or sequentially. Alternatively, only the light emitting devices LD included in some pixels PXL among the pixels PXL of the sensing area SA emit light at a predetermined interval, or light of a specific color (eg, blue light in a short wavelength band) may be emitted. Only the light emitting devices LD included in some pixels PXL emitting light) may selectively emit light.

Some of the light EL emitted from the pixels PXL may be reflected from the user's finger and may pass through the pinholes PIH formed in each layer of the learner terminal 104 to be incident on the optical sensors PSR. . In this case, the user's fingerprint shape (fingerprint pattern) may be detected based on the difference in light quantity and/or the waveform of the reflected light RL reflected from the ridges and valleys of the fingerprint, respectively.

10 is a perspective view schematically illustrating an example of a detection area of a learner terminal including a fingerprint sensor according to an embodiment of the present invention.

6 to 10 , the learner terminal 104 may include a display panel PN, a window WD, and a fingerprint sensor FPS. Although not shown directly in the drawing, the touch sensor TS may be positioned between the display panel PN and the window WD.

The fingerprint sensor (FPS) recognizes the user's fingerprint by sensing the light (or light) reflected by the valley between the ridge and the ridge of the user's fingerprint through the optical sensors (PSR). It may be an optical fingerprint sensor. In an embodiment, the fingerprint sensor FPS senses the reflected light RL passing through the selective light transmitting unit LBL and the selective light transmitting unit LBL passing the reflected light RL reflected by the fingerprint to receive an electrical signal. The generated light sensing array unit LSL may be included.

In FIG. 10 , the selective light transmitting unit LBL is configured independently from the display panel PN in order to directly explain that the selective light transmitting unit LBL constitutes the fingerprint sensor FPS together with the light sensing array unit LSL. Although illustrated, the present invention is not limited thereto.

According to an embodiment, in implementing the fingerprint sensor FPS, the selective light transmitting unit LBL may be stacked on the light sensing array unit LSL during a packaging process for the light sensing array unit LSL. Alternatively, in a process for implementing the light sensing array unit LSL, a selective light transmitting unit LBL is laminated on the light sensing array unit LSL in the form of a layer in one or more layers constituting the light sensing array unit LSL. can be That is, the fingerprint sensor FPS may be implemented in a form in which the selective light transmitting unit LBL is embedded in the light sensing array unit LSL, and with respect to the light sensing array unit LSL in which the selective light transmitting unit LBL is built. A packaging process may be performed. That is, according to an embodiment, the selective light transmitting unit LBL and the light sensing array unit LSL may be integrally formed. However, the present invention is not limited thereto. In some embodiments, the selective light transmitting part LBL may be disposed between the substrate SUB and the pixel circuit part PCL of the display panel PN as shown in FIG. 8 , with the substrate SUB interposed therebetween. The fingerprint sensor (FPS) of the learner terminal 104 may be configured together with the light sensing array unit (LSL).

The selective light transmitting part LBL may be implemented by various methods using a material having low light transmittance and low reflectance. For example, the selective light transmitting part LBL has a low reflectance (or high absorption) characteristic while blocking light (or light), and a material that can maintain its hardness even with changes in temperature or humidity can be implemented using

In an embodiment of the present invention, the display module DM and the selective light transmitting portion LBL of the display panel PN may be disposed substantially parallel to each other. Accordingly, the light (or light, EL) from the plurality of light emitting elements LD in the display module DM may be transmitted in the direction of the fingerprint located in the window WD, and the reflected light RL reflected by the fingerprint may be transmitted. The light may be transmitted to the selective light transmitting unit LBL within an angle of view θ formed by the pinholes PIH of the selective light transmitting unit LBL.

The fingerprint sensor FPS may detect a fingerprint in contact with or close to the learner terminal 104 . When the user's fingerprint is placed on the first surface SF1 of the learner terminal 104, light emitted from the light emitting elements LD in the display module DM becomes a light source and is transmitted and reflected to the user's fingerprint, The reflected reflected light RL may pass through (or pass) the display module DM and the substrate SUB and may be transmitted to the light sensing array unit LBL through the pinholes PIH of the selective light transmitting unit LBL. have.

The light sensing array unit LSL includes a plurality of light sensors PSR, and each light sensor PSR senses the reflected light RL reflected by different regions of the fingerprint, and the sensed reflected light RL. ) to generate an electrical signal corresponding to Each of the optical sensors PSR generates an electrical signal corresponding to the light reflected by the ridges of the fingerprint or to generate an electrical signal corresponding to the light reflected by the valleys between the ridges. can The amount (or intensity) of the light sensed by each optical sensor PSR may vary according to the shape of the fingerprint on which the light is reflected, and electrical signals having different levels may be generated according to the amount of the sensed light. That is, the electrical signals from the plurality of optical sensors PSR may each include contrast information (or image information), and a region corresponding to each optical sensor PSR is ridged through a processing operation on the electrical signal. Whether it is a ridge or a valley may be determined, and an overall fingerprint image may be constructed by combining the determined information.

Regions of the fingerprint that are optically sampled at the learner terminal 104 may be defined. As an example, a plurality of fingerprint pixels FPXL may be defined to correspond to the plurality of optical sensors PSR of the light sensing array unit LSL, and each fingerprint pixel FPXL has one pinhole PIH. ) and a subject area shown by one optical sensor PSR. In one embodiment of the present invention, each fingerprint pixel FPXL may refer to one pixel PXL included in an area of the sensing area SA where the user's fingerprint is sensed by the user's contact or proximity. have. However, the present invention is not limited thereto, and according to embodiments, a plurality of pixels PXL included in one area of the sensing area SA may form a group to constitute each fingerprint pixel FPXL. have.

A distance O between the display panel PN and the selectively light transmitting portion LBL, a distance f between the selective light transmitting portion LBL and the light sensing array unit LSL, and a thickness T of the selective light transmitting portion LBL , the shape and size of the fingerprint pixel FPXL corresponding to each pinhole PIH may be determined according to the diameter d and the shape of each pinhole PIH.

A region reflecting light that may pass through one pin hole PIH may be included in each fingerprint pixel FPXL, and the region may be defined as an optical sampling region OSA. According to the optical sampling area OSA, a corresponding light sensing area LSA may also be defined in the light sensing array unit LSL. The light sensing area LSA corresponds to the optical sampling area OSA, and the reflected light RL reflected from the optical sampling area OSA passes through one pinhole PIH, and thus the optical sensor of the light sensing array unit LSL. It may mean an area incident to the PSRs.

As described above, as an optical fingerprint sensor (FPS) is easily implemented using some components included in the learner terminal 104, for example, a selective light transmitting unit (LBL) and a light sensing array unit (LSL). A large-area fingerprint sensor (FPS) capable of detecting the entire user's finger may be implemented. That is, a large-area fingerprint sensor (FPS) having a larger size (or area) than a user's finger may be implemented.

11A to 11D are plan views for explaining the arrangement of fingerprint pixels, pinholes, and optical sensors according to embodiments of the present invention. Specifically, FIGS. 11A to 11D show the relative sizes, densities (or resolutions) of the fingerprint pixels FPXL, the pinholes PIH, and the optical sensors PSR located in the sensing area SA of FIG. 10 , and / or plan views for explaining the arrangement relationship according to various embodiments.

First, referring to FIG. 11A , the sensing area SA may include a smaller number of pinholes PIH and light sensors PSR than the fingerprint pixels FPXL and PXL. For example, the pinholes PIH and the light sensors PSR have a smaller size than the fingerprint pixels FPXL and PXL, but have a lower density than the fingerprint pixels FPXL and PXL in the sensing area SA. can be distributed.

Meanwhile, although FIG. 11A illustrates an embodiment in which the pinholes PIH and the optical sensors PSR have lower densities than the fingerprint pixels FPXL and PXL, the present invention is not limited thereto. According to an exemplary embodiment, the pinholes PIH and the optical sensors PSR may be distributed in the sensing area SA at substantially the same number and spacing so as to correspond one-to-one (1:1). For example, the pinholes PIH and the optical sensors PSR may be disposed to overlap each other in a one-to-one (1:1) pair. According to an embodiment, the pair of pinhole PIH and optical sensor PSR may be disposed to overlap any one of the fingerprint pixels FPXL and PXL disposed in the sensing area SA, but the present invention provides for this. It is not limited. For example, the pinholes PIH and the optical sensors PSR may be alternately disposed not to overlap each other or may be disposed not to overlap the fingerprint pixels FPXL and PXL.

The pinholes PIH and the optical sensors PSR may have the same or different sizes. That is, relative sizes or densities of the pinholes PIH and the light sensors PSR are not particularly limited.

Referring to FIG. 11B , the sensing area SA includes a smaller number of pinholes PIH than the fingerprint pixels FPXL and PXL and a larger number of optical sensors PSR than the fingerprint pixels FPXL and PXL. can do. For example, the pinholes PIH and the optical sensors PSR have a size smaller than the fingerprint pixels FPXL and PXL, but the pinholes PIH are detected with a lower density than the fingerprint pixels FPXL and PXL. Distributed in the area SA, the light sensors PSR may be densely distributed in the sensing area SA with a higher density than the fingerprint pixels FPXL and PXL.

At least some of the optical sensors PSR may overlap at least one pinhole PIH and/or a fingerprint pixel FPXL and PXL, but the present invention is not limited thereto. According to an embodiment, some of the optical sensors PSR are disposed to overlap the pinholes PIH and/or the fingerprint pixels FPXL and PXL, and other portions of the light sensors PSR are disposed in a gap between the fingerprint pixels FPXL and PXL. may be placed in

11C and 11D , the optical sensors PSR may be distributed in the sensing area SA to have a smaller size and higher density than the embodiment illustrated in FIG. 11B . For example, the optical sensors PSR may be distributed in the sensing area SA at a narrow interval of about 1/10 to 1/100 times that of the pinholes PIH. In this case, the optical sensors PSR are so dense in the sensing area SA that one-to-one (1:1) alignment with the fingerprint pixels FPXL, PXL and/or the pinholes PIH is not required. Accordingly, the occurrence of moiré may be prevented or minimized regardless of whether the fingerprint pixels FPXL and PXL and/or the pinholes PIH and the optical sensors PSR are aligned.

According to an embodiment, the pinholes PIH may be distributed in the sensing area SA with the same or different densities. For example, the pinholes PIH are distributed in the sensing area SA at the same density as the fingerprint pixels FPXL and PXL as shown in FIG. 11C , or the fingerprint pixels FPXL and PXL as shown in FIG. 11D . PXL) may be distributed in the sensing area SA with a lower density.

11A to 11D illustrate an embodiment in which the pinholes PIH and the light sensors PSR are arranged in a regular array in the sensing area SA, but the present invention is not limited thereto. According to an embodiment, the pinholes PIH and/or the optical sensors PSR are irregularly distributed in the sensing area SA, or distributed in a different density or arrangement according to each area or section of the sensing area SA. can be

The arrangement structure of the fingerprint pixels FPXL and PXL, the pinholes PIH, and the light sensors PSR in the sensing area SA is not limited to the above-described embodiments. For example, the shape, arrangement, relative size, number, density and/or the fingerprint pixels FPXL and PXL, the pinholes PIH, and/or the optical sensors PSR disposed in the sensing area SA. Alternatively, the mutual arrangement relationship may be variously changed.

12 is an enlarged cross-sectional view of the EA region of FIG. 8 .

12 illustrates the EA region illustrated in FIG. 8 , for example, the pixel circuit unit PCL and the display element unit DPL included in one pixel in more detail. In particular, FIG. 12 illustrates at least one transistor TR provided in the pixel circuit unit PCL and light emitting devices LD and OLED provided in the display element unit DPL.

6 to 12 , one pixel (PXL, hereinafter referred to as a 'pixel') is a substrate SUB, a pixel circuit unit PCL provided on the substrate SUB, and a display provided on the pixel circuit unit PCL It may include a device part DPL.

The pixel circuit unit PCL may include a buffer layer BFL, at least one transistor TR, and a protection layer PSV. The display element unit DPL may include light emitting elements LD and OLED that emit light. For convenience of description, the pixel circuit unit PCL will be described first, and then the display element unit DPL will be described.

The buffer layer BFL may be provided on the substrate SUB. The buffer layer BFL may prevent impurities from diffusing into the transistor TR. The buffer layer BFL may be provided as a single layer, but may also be provided as a multilayer of at least double layers. When the buffer layer BFL is provided in multiple layers, each layer may be formed of the same material or may be formed of different materials. The buffer layer BFL may be omitted depending on the material and/or process conditions of the substrate SUB. The buffer layer BFL may be an inorganic insulating layer including an inorganic material or an organic insulating layer including an organic material.

The transistor TR may include a driving transistor electrically connected to the light emitting devices LD and OLED of the display element unit DPL to drive the light emitting devices LD and OLED. Although not shown directly in the drawing, the transistor TR may include a switching transistor for switching the driving transistor.

The transistor TR may include a semiconductor layer SCL, a gate electrode GE, a first terminal SE, and a second terminal DE. The first terminal SE may be one of a source electrode and a drain electrode, and the second terminal DE may be the other electrode. For example, when the first terminal SE is a source electrode, the second terminal DE may be a drain electrode.

The semiconductor layer SCL may be disposed on the buffer layer BFL. The semiconductor layer SCL may include a first region in contact with the first terminal SE and a second region in contact with the second terminal DE. A region between the first region and the second region may be a channel region. The semiconductor layer SCL may be a semiconductor pattern made of polysilicon, amorphous silicon, oxide semiconductor, or the like. The channel region is a semiconductor pattern that is not doped with impurities, and may be an intrinsic semiconductor. The first region and the second region may be semiconductor patterns doped with impurities.

The gate electrode GE may be provided on the semiconductor layer SCL with the gate insulating layer GI interposed therebetween. The gate insulating layer GI may be an inorganic insulating layer including an inorganic material.

Each of the first terminal SE and the second terminal DE contacts the first region and the second region of the semiconductor layer SCL through a through hole penetrating the interlayer insulating layer ILD and the gate insulating layer GI. can be The interlayer insulating layer ILD may be an inorganic insulating layer including an inorganic material, but the present invention is not limited thereto, and may be an organic insulating layer including an organic material.

In the above embodiment, the first and second terminals SE and DE of the transistor TR have been described as separate electrodes electrically connected to the semiconductor layer SCL, but the present invention is not limited thereto. In some embodiments, the first terminal SE of the transistor TR may be one of the first and second regions adjacent to the channel region of the semiconductor layer SCL, and the second terminal of the transistor TR (DE) may be the remaining region of the first and second regions adjacent to the channel region of the semiconductor layer SCL. In this case, the second terminal DE of the transistor TR may be electrically connected to the light emitting devices LD and OLED of the display device unit DPL through a connection means including a bridge electrode or a contact electrode.

In one embodiment of the present invention, the transistor TR included in the pixel circuit unit PCL may be formed of an LTPS transistor, but the present invention is not limited thereto. it might be Also, in some embodiments, the pixel circuit unit PCL may include at least one transistor TR formed of an LTPS thin film transistor and at least one transistor TR formed of an oxide semiconductor thin film transistor. Additionally, in the exemplary embodiment of the present invention, a case in which the transistor TR is a thin film transistor having a top gate structure has been described as an example, but the present invention is not limited thereto. In some embodiments, the transistor TR may be a thin film transistor having a bottom gate structure.

A passivation layer PSV may be provided on the transistor TR. The passivation layer PSV may be provided and/or formed on the transistor TR to cover the transistor TR. The passivation layer PSV may be provided in a form including an organic insulating layer, an inorganic insulating layer, or an organic insulating layer disposed on the inorganic insulating layer. Here, the inorganic insulating layer may include at least one of silicon oxide (SiOx) and silicon nitride (SiNx). The organic insulating layer may include an insulating material capable of transmitting light. For example, the organic insulating film is made of acrylic resin, epoxy resin, phenolic resin, polyamides resin, polyimides rein, unsaturated polyester. At least one of unsaturated polyesters resin, poly-phenylen ethers resin, poly-phenylene sulfides resin, and benzocyclobutene resin can do.

Next, the display element part DPL will be described.

The display element part DPL may include light emitting elements LD and OLED provided on the passivation layer PSV. The light emitting devices LD and OLED may include a first electrode AE, an emission layer EML, and a second electrode CE.

One of the first electrode AE and the second electrode CE may be an anode electrode, and the other electrode may be a cathode electrode. For example, the first electrode AE may be an anode electrode and the second electrode CE may be a cathode electrode. When the light emitting devices LD and OLED are top emission type organic light emitting devices, the first electrode AE may be a reflective electrode, and the second electrode CE may be a transmissive electrode. In an embodiment of the present invention, a case in which the light emitting devices LD and OLED are top emission type organic light emitting devices and the first electrode AE is an anode will be described as an example.

The first electrode AE may be electrically connected to the transistor TR of the pixel circuit unit PCL through the contact hole CH passing through the passivation layer PSV. For example, the first electrode AE may be electrically connected to the second terminal DE of the transistor TR of the pixel circuit unit PCL through the contact hole CH of the passivation layer PSV. The first electrode AE may include a reflective layer (not shown) capable of reflecting light and a transparent conductive layer (not shown) disposed on or below the reflective layer. At least one of the transparent conductive layer and the reflective layer may be electrically connected to the transistor TR.

A pixel defining layer PDL having an opening OPN exposing a portion of the first electrode AE, for example, a top surface of the first electrode AE, may be further included on the passivation layer PSV. The pixel defining layer PDL may include an organic insulating layer. For example, the pixel defining layer PDL may include polystyrene, polymethylmethacrylate (PMMA, polymethylmethacrylate), polyacrylonitrile (PAN, polyacrylonitrile), polyamide (PA, polyamide), polyimide (PI, polyimide), polyarylether (PAE, polyarylether), heterocyclic polymer, parylene, epoxy, benzocyclobutene (BCB, benzocyclobutene), siloxane based resin and It may include at least one of a silane based resin.

The pixel PXL may be disposed in the pixel area PXA of the display area DA of the display panel PN. In an exemplary embodiment, the pixel area PXA may include a light emitting area EMA from which light is emitted from the light emitting devices LD and OLED and a peripheral area adjacent to the light emitting area. The peripheral area may be an area in which light is not emitted, and the light emitting area EMA may be an area in which the first electrode AE, the light emitting layer EML, and the second electrode CE overlap. The emission area EMA may be defined to correspond to a partial area of the first electrode AE exposed by the opening OPN of the pixel defining layer PDL.

The emission layer EML may be disposed on the upper surface of the first electrode AE exposed by the opening OPN of the pixel defining layer PDL. The emission layer EML may have a multilayer thin film structure including at least a light generation layer. The light emitting layer (EML) is a hole injection layer for injecting holes, a hole transport layer ( hole transport layer), a light generation layer that generates (or emits) light by recombination of injected electrons and holes, and a hole suppression layer ( It may include a hole blocking layer, an electron transport layer for smoothly transporting electrons to the light generating layer, and an electron injection layer for injecting electrons.

The color of light generated in the light generating layer may be one of red, green, blue, and white, but the present embodiment is not limited thereto. For example, the color of light generated in the light generating layer of the emission layer EML may be one of magenta, cyan, and yellow. The hole injection layer, the hole transport layer, the hole suppression layer, the electron transport layer, and the electron injection layer may be a common layer connected in the light emitting area EMA adjacent to each other.

A thin film encapsulation layer TFE covering the second electrode CE may be provided on the second electrode CE.

The thin film encapsulation layer TFE seals the display element part DPL. The thin film encapsulation layer (TFE) may be formed of a single layer, but is not limited thereto, and may be formed of multiple layers according to embodiments. The thin film encapsulation layer TFE may include a plurality of insulating layers covering the display device portion DPL including the light emitting devices LD and OLED. The thin film encapsulation layer TFE may include at least one inorganic layer and at least one organic layer. For example, the thin film encapsulation layer TFE may have a structure in which an inorganic layer and an organic layer are alternately stacked. Also, according to an embodiment, the thin film encapsulation layer TFE may be an encapsulation substrate disposed on the display element unit DPL and bonded to the substrate SUB through a sealant.

A selective light transmitting part LBL may be provided between the substrate SUB and the pixel circuit part PCL. The selective light transmitting part LBL may include a light blocking pattern LBP and at least one pinhole PIH. The pinhole PIH of the selective light transmitting portion LBL may be positioned in the selective light transmitting portion LBL to overlap the light emitting area EMA of the pixel PXL, but the present invention is not limited thereto.

13 is a schematic cross-sectional view of a learner terminal according to an embodiment of the present invention.

Referring to FIG. 13 , when a user's fingerprint comes into contact with the first surface SF1 of the sensing area SA of the learner terminal 104, each of the light emitting elements (FPXL, PXL) of the fingerprint pixels corresponding to the contact area ( LD and OLED may be driven to emit light EL to the first surface SF1 . Hereinafter, for convenience of description, an area to which a user's fingerprint is in direct contact among the first surface SF1 of the sensing area SA is referred to as a fingerprint contact surface SF1 .

The light emitting elements LD and OLED of the fingerprint pixels FPXL corresponding to the fingerprint contact surface SF1 may emit light simultaneously or sequentially. The light emitted by the user's fingerprint may be reflected by the user's fingerprint and may pass through (or transmit) the display module DM and the selective light transmitting unit LBL to be incident on the light sensing array unit LSL. The optical sensors PSR of the light sensing array unit LSL receive the reflected light RL to obtain a user's fingerprint image, and provide the obtained fingerprint image to the fingerprint detection unit FPDP at a predetermined frame rate.

At this time, the light sensing array unit LSL receives the reflected light RL reflected from the outer portion of the user's fingerprint to obtain an image of the outer portion of the user's fingerprint, and provides the obtained image of the outer portion of the fingerprint to the fingerprint detection unit FPDP. can In one embodiment of the present invention, the outer part of the user's fingerprint means that the user's fingerprint is in the vicinity of the portion (A, fingerprint authentication unit) in which the user's fingerprint is in contact with the fingerprint contact surface SF1 of the sensing area SA. It may refer to an area B spaced apart from (or close to) the first surface SF1 without contacting the first surface SF1 . Hereinafter, for convenience of explanation, a portion (A, or fingerprint authentication unit) in which the user's fingerprint contacts the fingerprint contact surface SF1 is referred to as a 'first area (A)', and the outer portion of the first area (A) is referred to as a 'first area (A)'. It is referred to as a 'second region (B)'.

The first area (A) of the fingerprint and the second area (B) of the fingerprint are located between continuous ridges and ridges and include continuous valleys, and a photo-sensing array for each area A difference in brightness (or a difference in gray level) may exist due to a difference in reflected light RL incident to the negative LSL. Using this difference, it may be possible to distinguish the first area A of the fingerprint and the second area B of the fingerprint in the fingerprint image obtained from the light sensing array unit LSL.

13 , when the selective light transmitting part LBL in the sensing area SA sensing a user's fingerprint includes the first to fifth pinholes PIH1 to PIH5, the light sensing array part LSL ) may receive light that passes through each of the first to fifth pinholes PIH1 to PIH5 and travels to the light sensing array unit LSL to obtain a fingerprint image for each unit pinhole PIH. For example, for fingerprint authentication, a first area (A) to which a user's finger is in direct contact within the sensing area (SA) and a second area (B) located at the outer portion of the first area (A) are selective light. When corresponding to the first to fourth pinholes PIH1 to PIH4 of the transmissive part LBL, the light sensing array unit LSL passes through each of the first to fourth pinholes PIH1 to PIH4 and the light sensing array unit A fingerprint image may be acquired for each pinhole (PIH) by receiving the light propagating to the LSL.

The light reflected from the first area A passes through the first to fourth pinholes PIH1 to PIH4 to the light sensing array unit LSL corresponding to each of the first to fourth pinholes PIH1 to PIH4. can proceed. At the same time, the light reflected from the second region B also passes through the first to fourth pinholes PIH1 to PIH4 and a photo sensing array unit corresponding to each of the first to fourth pinholes PIH1 to PIH4 ( LSL).

The amount (or intensity) of light reflected from the first area A and the amount (or intensity) of light reflected from the second area B may be different from each other. For example, the amount (or intensity) of light reflected from the first area A may be greater than the amount (or intensity) of light reflected from the second area B. The first area (A) is an area where the user's fingerprint is in direct contact with the fingerprint contact surface (SF1), and the second area (B) is an area where the user's fingerprint does not touch the fingerprint contact surface (SF1). Differences in reflectance at ridges and valleys may occur. Due to the difference in reflectance, the first area A and the second area B of the user's fingerprint may be distinguished from the fingerprint image obtained by the light sensing array unit LSL.

On the other hand, the fingerprint contact surface SF1 of the learner terminal 104 may be in contact with a user's real finger fingerprint or a fake fingerprint. Here, the forged fingerprint may include a forged fingerprint printed on a transparent film or paper, a forged fingerprint made by pouring a specific material into a fingerprint frame, and the like.

An example of a fingerprint image processed by the fingerprint detection unit FPDP when a user's finger fingerprint (hereinafter referred to as 'biometric fingerprint') comes into contact with the fingerprint contact surface SF1 is the biometric fingerprint in the first area A The formed ridges and valleys are observed, and in the second region (B), although relatively faint compared to the first region (A), the ridges ( ridge) and a valley having continuity with the valley of the first region A may be recognized.

An example of a fingerprint image processed by the fingerprint detection unit FPDP when a forged fingerprint, for example, a forged fingerprint printed on paper comes into contact with the fingerprint contact surface SF1, is a ridge formed on the biometric fingerprint in the first area A Although a pattern having a shape similar to a ridge and a valley appears, it can be seen that the arrangement of the pinholes PIH included in the selective light transmitting part LBL is observed in the second region B. For example, pinholes PIH forming a grid arrangement may be recognized in the second region B.

As described above, the ridges and valleys of the fingerprint can be observed in the first area (A) of each of the biometric fingerprint and the forged fingerprint, but in the second area (B) of each of the biometric and counterfeit fingerprints Different images may be observed. That is, in the second area (B) of the biometric fingerprint, although faint, ridges and valleys of the fingerprint can be observed, but in the second area (B) of the fake fingerprint, the grid arrangement of the pinholes (PIH) is can be observed. In one embodiment of the present invention, it is possible to determine whether the fingerprint in contact with the fingerprint contact surface SF1 is a biometric fingerprint or a forged fingerprint through the images observed in the second region B of each of the biometric fingerprint and the forged fingerprint.

The fingerprint detector FPDP extracts an image corresponding to the first region A and an image corresponding to the second region B from the fingerprint image obtained from the light sensing array unit LSL, After performing image preprocessing and synthesizing, the fingerprint contact surface SF1 of the display device DD is determined by determining whether similarity is present or whether it corresponds to a specific pattern, for example, a lattice arrangement of pinholes PIH. It may be determined whether the touched fingerprint is a biometric fingerprint or a forged fingerprint.

The fingerprint detector FPDP receives an image from the fingerprint sensor FPS, extracts a first image corresponding to the first area of the fingerprint and a second image corresponding to the second area of the fingerprint, and compares the first and second images to determine the degree of similarity, and according to the result, fingerprint authentication can be performed.

The first area of the fingerprint may be one area of the fingerprint in contact with the fingerprint contact surface, and the second area of the fingerprint may be another area of the fingerprint located outside the one area of the fingerprint and not in contact with the fingerprint contact surface.

14 is a block diagram schematically illustrating the fingerprint detection unit shown in FIG. 6 or FIG. 7 .

6 to 14 , the fingerprint detection unit FPDP includes an image receiving unit IRP, a storage unit DBP, an image extraction unit IEP, an image processing unit IPP, a similarity determining unit SDP, and fingerprint authentication. It may include an executor (FPAP).

The fingerprint detector FPDP may be integrally formed with the light sensing array unit LSL as a single module, or may be a separate device connected to the light sensing array unit LSL through a specific communication interface. For example, the fingerprint detection unit FPDP may be a computer device (or a part of a configuration therein) connected to the optical sensing array unit LSL provided in the form of a module and a USB (Universal Serial Bus). In this case, the light sensing array unit LSL must also have a USB interface connected to the fingerprint detection unit FPDP.

The image receiving unit (IRP) may receive all the images provided by the light sensing array unit (LSL) at a predetermined frame rate and transmit them to the image extracting unit (IEP). Also, the image receiving unit (IRP) is a learner terminal among the received images. When the fingerprint touches the fingerprint contact surface SF1 of 104 , the fingerprint image acquired by the light sensing array unit LSL is recognized and stored in a buffer, and then transferred to the image extraction unit 40 . The image receiving unit IRP detects a change in a specific element value (eg, a gray level, a brightness level, the number of pixels with a changed value, etc.) among the received images, checks whether the image is a fingerprint image, and determines whether the image is a fingerprint image. It is stored in the buffer and transmitted to the image extraction unit (IEP).

The storage unit DBP stores the registered fingerprint image. In this case, the storage unit DBP may store the characteristic point information analyzed with respect to the registered fingerprint image. Since the key points are different for each person, the storage unit DBP may obtain a registered fingerprint image from the user, analyze the location and number of the key points, and store it as a database. Here, the feature point may include an ending point through which a ridge passes and a bifurcation point where two or more ridges meet in the registered fingerprint image.

The image extraction unit IEP extracts an image of a desired region from the fingerprint image transmitted from the image reception unit IRP. The image extraction unit IEP uses the brightness difference (or gray level difference) for each region due to the difference in reflected light in the fingerprint image transmitted from the image receiving unit IRP to the image and the fingerprint of the first region A for fingerprint authentication. Images of the second region B, which are the outer regions, may be extracted, respectively.

For example, the image extractor IEP may select a first region A corresponding to a relatively bright region in the fingerprint image as a region of interest (ROI) and extract only an image of the corresponding region. Alternatively, the image extractor IEP may select the second region B corresponding to a relatively dark region in the fingerprint image as a region of interest (ROI) and extract only the image of the corresponding region. According to an embodiment, the image extraction unit (IEP) uses an edge tracing, edge detection, boundary tracing algorithm, etc. to obtain the image of the first area A and the second area ( The image of B) can be extracted. The image extraction unit (IEP) uses various known methods from the fingerprint image transmitted from the image receiving unit (IRP) to the image of the first area (A) for fingerprint authentication and the outer portion of the first area (A). An image of the corresponding second region B may be extracted.

The image of the first region A and the image of the second region B extracted by the image extraction unit IEP may be transmitted to the image processing unit IPP.

The image processing unit IPP may perform image preprocessing on the image of the first region A and the image of the second region B transmitted from the image extraction unit IEP. Here, image preprocessing refers to removing noise (or noise) included in the image of the first region (A) and the image of the second region (B), reducing the amount of data, and finding necessary information. It can mean processing the image in a form that is easy to bet on. Image preprocessing may include a smoothing process, a binarization process, and a thinning process.

The image processing unit (IPP) may perform a smoothing process of increasing contrast and removing noise in order to minimize the influence of noise (or noise) on each of the image of the first region (A) and the image of the second region (B). have. For example, in the smoothing process, smoothing using a median filter that additionally removes fine noise through smoothing using a histogram may be performed. In addition, the image processing unit IPP converts the image of the first region A and the second image B of 256 gray levels from which noise has been removed through a smoothing process to 0 (black) or 1 (white). After performing the process, you can proceed with the thinning process. The image processing unit IPP may synthesize the image of the first region A and the image of the second region B on which image preprocessing has been performed, and then transmit the synthesized image to the similarity determining unit SDP.

The similarity determining unit SDP may determine the degree of similarity between the first region A and the second region B through the image transmitted from the image processing unit IPP.

The similarity determining unit SDP is configured to provide a luminance (or brightness) between the image of the first region A and the image of the second region B based on the boundary between the first region A and the second region B. ), if they have a similar pattern or substantially the same pattern, the image of the first region A and the image of the second region B may be determined as similar images. In one embodiment of the present invention, when the image of the first region (A) and the image of the second region (B) have a similar or substantially the same pattern, the ridge of the fingerprint having continuity ) and may mean a case including a valley. In addition, the similarity determining unit SDP determines that the luminance (or brightness) of the image of the first region A and the luminance (or brightness) of the image of the second region B are different but similar to each other or substantially the same pattern. In the absence of , the image of the first region A and the image of the second region B may be determined to be dissimilar images.

When it is determined that the image of the first region A and the image of the second region B are similar (ie, the difference in luminance (or brightness) between the image of the first region A and the image of the second region B) and including ridges and valleys of fingerprints that are continuous with each other), the similarity determining unit SDP recognizes the fingerprint that has contacted the fingerprint contact surface SF1 as a biometric fingerprint, and the result of the fingerprint authentication performing unit (FPAP). The fingerprint authentication performing unit FPAP performs fingerprint authentication by matching the registered fingerprint image transmitted from the storage DBP with the image of the first area A.

In addition, if it is not determined that the image of the first region A and the image of the second region B are similar (ie, the luminance (or between the image of the first region A) and the image of the second region B brightness) difference, but does not include ridges and valleys of fingerprints that are continuous with each other), and the similarity determining unit SDP recognizes the fingerprint in contact with the fingerprint contact surface SF1 as a counterfeit fingerprint, and as a result is transmitted to the fingerprint authentication performing unit FPAP, and the fingerprint authentication performing unit FPAP does not perform fingerprint authentication according to the result of the similarity determining unit SDP.

Meanwhile, according to another exemplary embodiment, the similarity determining unit SDP may determine the degree of similarity between images by comparing the image transmitted from the image processing unit IPP with a registered reference image. The registered reference image may be an image including a grid arrangement of pinholes PIH observed in the second region B of the forged fingerprint. That is, the similarity determining unit SDP according to another embodiment determines whether the image transmitted from the image processing unit IPP corresponds to a reference image including a grid arrangement of pinholes PIH, and determines whether the result is a fingerprint authentication performing unit (FPAP).

If the image transmitted from the image processing unit IPP does not correspond to the reference image including the grid arrangement of the pinholes PIH, the similarity determining unit SDP determines the fingerprint in contact with the fingerprint contact surface SF1 as a biometric fingerprint, and The result is transmitted to the fingerprint authentication performing unit (FPAP). The fingerprint authentication performing unit FPAP performs fingerprint authentication by matching the registered fingerprint image transmitted from the storage DBP with the image of the first area A. In addition, when the image transmitted from the image processing unit IPP corresponds to the registered reference image including the grid arrangement of the pinholes PIH, the similarity determining unit SDP may counterfeit the fingerprint contacting the fingerprint contact surface SF1. , and transmits the result to the fingerprint authentication performing unit FPAP, and the fingerprint authentication performing unit FPAP does not perform fingerprint authentication according to the result of the similarity determining unit SDP.

As described above, according to an embodiment of the present invention, the light sensing array unit LSL including the light sensors PSR and the selective light transmission unit LBL including the pinholes PIH are used as the fingerprint sensor FPS. By configuring the fingerprint sensor (FPS) can be implemented in a large area. When such a large-area fingerprint sensor (FPS) is used, it is possible to acquire an entire image of the user's finger fingerprint, so that not only the image of the first area (A) of the fingerprint in contact with the fingerprint contact surface (SF1) but also the outside of the first area (A) is possible. Even the image of the denial second region B can be acquired. By analyzing only the image of the first region A and the image of the second region B to determine the degree of similarity, it can be determined whether the fingerprint in contact with the fingerprint contact surface SF1 is a biometric fingerprint or a forged fingerprint.

Accordingly, the learner terminal 104 according to an embodiment of the present invention can easily determine whether a fingerprint is forged without a separate biosensor for determining that it is a biometric fingerprint. In addition, the learner terminal 104 according to an embodiment of the present invention uses a color characteristic of light reflected from a fingerprint in contact with the fingerprint contact surface SF1 to determine whether a fingerprint is forged (eg, a color sensor). Whether or not the fingerprint is forged can be easily determined without provision. In addition, the learner terminal 104 according to an embodiment of the present invention can reduce manufacturing costs due to the provision of a separate biosensor and/or color sensor.

15 is an operation flowchart illustrating a learning method using a finger according to an embodiment of the present invention.

Referring to FIG. 15 , the finger learning system 100 according to an embodiment of the present invention provides a learning service through the learner terminal 104 ( S502 ).

For example, the learner terminal 104 executes the milk T elementary learning service as shown in FIG. 16 in the learning execution unit 508 .

Here, the milk T elementary learning service includes milk T school study, mathematics by level, English by level, creative thinking ability, and milk T middle school general class menu. 16 is a view showing an example of a screen for the Milk T elementary learning service provided through the learner terminal according to an embodiment of the present invention.

Milk T school study menu includes study by subject, special lecture for exam preparation, correctional tutoring, and special lecture menu for middle school. The mathematics menu by level includes storytelling concept learning, mathematics by area, FunFun calculation game, type breaking, and skill charging menu. .

The English menu for each level includes Speaking, Words, Phonics, Reading, Tantan Grammer, English Songs, and English Library menus.

Milk T middle school general class menu includes study manager, multi-learning player, self-directed learning, math English correction service, math question and answer community, and 10-minute math English menu.

Accordingly, the learner may, for example, execute the FunFun calculation game menu among the storytelling concept learning, domain-specific math, FunFun calculation game, type breaking, and skill charging menus in the math menu for each level.

Accordingly, the learner terminal 104 provides a learning page screen for the FunFun arithmetic game of the math menu for each level as shown in FIG. 17 .

At this time, on the learning page screen for the FunFun calculation game, an icon or button for fingerprint registration is displayed in the upper right corner of the screen. 17 is a diagram illustrating a fingerprint registration icon for matching and registering a learner's fingerprint with a learning menu provided through a learner terminal according to an embodiment of the present invention.

The learner requests fingerprint registration for the currently executed learning menu through the fingerprint registration menu (icon or button) displayed on the learning page screen.

Next, the learner terminal 104 matches and registers the current learning menu with the learner's fingerprint according to the input of the fingerprint registration menu (S504).

That is, the learner terminal 104 scans and registers the learner's fingerprint in the process shown in FIGS. 6 to 14, matches the learning page screen for the currently executed FunFun calculation game, and stores the registration in the storage unit. .

After that, the learner can complete the learner's fingerprint registration for the FunFun calculation game menu in the math learning by level of the Milk T elementary learning service, and continue the learning service to end the FunFun calculation game learning of the math by level.

On the other hand, after a certain period of time has elapsed, the learner may want to proceed again to learn about the FunFun calculation game menu in the math learning by level of the Milk T elementary learning service.

Accordingly, the learner executes the application for the milk T elementary learning service through the learner terminal 104 .

Next, the learner terminal 104 provides a fingerprint input screen when the milk T elementary learning service is executed (S506).

Next, the learner terminal 104 receives the learner's fingerprint and compares it with the registered fingerprint (S508).

That is, as shown in FIG. 18 , the learner terminal 104 displays the message Please input your fingerprint here through the pop-up screen, and receives the learner's fingerprint for matching with the currently executed learning menu. 18 is a diagram illustrating an example of receiving a learner's fingerprint for matching with a currently executed learning menu in a learning terminal according to an embodiment of the present invention.

An example of a fingerprint image processed by the fingerprint detection unit FPDP of the learner terminal 104 when the user's finger fingerprint (hereinafter, referred to as 'biometric fingerprint') comes into contact with the fingerprint contact surface SF1 is shown in FIG. It can be like a bar. 19 is a diagram illustrating a state in which a user's finger fingerprint is in contact with the fingerprint contact surface of the learner terminal according to an embodiment of the present invention, and is an enlarged view of a first area of a fingerprint and a second area of the fingerprint, respectively.

As shown in the fingerprint image, ridges and valleys formed in the biometric fingerprint are observed in the first region (A), and relatively faint compared to the first region (A) in the second region (B). It can be seen that a ridge having continuity with the ridge of the first region A and a valley having continuity with the valley of the first region A are observed.

Here, since the process of receiving and registering the learner's fingerprint has been described with reference to FIGS. 6 to 14, detailed operations thereof will be omitted.

Next, when the input fingerprint and the registered fingerprint match, the learner terminal 104 executes a learning menu corresponding to the registered fingerprint ( S510 ).

That is, the learner terminal 104 directly executes the FunFun arithmetic game menu in the math learning for each level of the Milk T elementary learning service corresponding to the registered fingerprint.

Here, the process of comparing the input learner's fingerprint and the registered fingerprint to determine whether they match has been described above with reference to FIGS. 6 to 14 , and thus detailed operations thereof will be omitted.

Accordingly, the learner will be able to execute the desired learning service immediately without going through the cumbersome procedure of running the learning service application and finding the FunFun calculation game menu.

In addition, the learning method using a finger according to an embodiment of the present invention may be recorded as a program in a readable recording medium such as USB, CD, or DVD.

Meanwhile, the learner terminal 104 includes: a display panel including an upper substrate and a lower substrate facing each other; a cover window disposed on the upper substrate and covering a front surface of the display panel; a touch sensor array formed on the cover window; and a fingerprint sensor array formed on the cover window and formed on the same layer as the touch sensor array.

The touch sensor array may include a first driving electrode and a first sensing electrode crossing each other, a first driving line connected to the first driving electrode, and a first sensing line connected to the first sensing electrode.

The fingerprint sensor array includes a second driving electrode formed on the same layer as the first driving electrode, a second sensing electrode formed on the same layer as the first sensing electrode, a second driving line connected to the second driving electrode, and the second driving electrode. It may include a second sensing line connected to the second sensing electrode.

The first driving lines connected to each of the first driving electrodes of the touch sensor array may be arranged on one edge of the touch sensor array in parallel.

At the other edge of the touch sensor array, the fingerprint sensor array that does not overlap the touch sensor array, a second drive line connected to the second drive electrode of the fingerprint sensor array, and the second sensing electrode of the fingerprint sensor array The connected second sensing lines may be arranged side by side.

The fingerprint sensor array may include a fingerprint sensor group including a single second driving electrode disposed in a first row, and the plurality of second sensing electrodes disposed in a second row.

The plurality of second sensing electrodes may be connected to each other at least two by two through the second sensing line, and the fingerprint sensor array may be further disposed in a partial area or an entire area selected from the lower end of the display panel on which a user's touch manipulation unit is disposed. can

The fingerprint sensor array may include the plurality of second driving electrodes arranged in a first direction and the plurality of second sensing electrodes arranged in a second direction perpendicular to the first direction.

As described above, according to the present invention, when a learner's finger or a parent's finger is in contact with a learning application executed in the learner terminal, learning suitable for the learner's level or content learning matched to the finger can be automatically executed. It is possible to provide a learning method using a finger and a recording medium recording the same.

In the above, the embodiments of the present invention have been mainly described, but various changes or modifications can be made at the level of those skilled in the art to which the present invention pertains. Such changes and modifications can be said to belong to the present invention without departing from the scope of the technical spirit provided by the present invention. Accordingly, the scope of the present invention should be judged by the claims described below.

100: learning system using fingers 102: web server
104: learner terminal 110: first server
112: touch input/output unit 114: learning execution unit
120: second server 210: conversion unit
220: embedding processing unit 230: learning induction unit
502: touch input/output unit 504: storage unit
506: matching unit 508: learning execution unit
510: control unit PN: display panel
DCP: drive circuit FPDP: fingerprint detection unit
SUB: Substrate PXL: Pixel
FPXL: Fingerprint Pixels FPS: Fingerprint Sensor
LSL: light sensing array unit LBL: selective light transmission unit
PSR: Fingerprint sensor PIH: Pinhole
SF1: first surface (fingerprint contact surface) IRP: image receiving unit
IEP: image extraction unit DBP: storage unit
IPP: Image processing unit SDP: Similarity determining unit
FPAP: Fingerprint authentication unit

Claims (5)

(a) providing a learning service through a learner terminal;
(b) matching and registering the current learning menu and the learner's fingerprint in the learner terminal;
(c) providing a fingerprint input screen when the learning service is executed in the learner terminal;
(d) receiving the learner's fingerprint from the learner terminal and comparing it with a registered fingerprint; and
(e) when the fingerprint input from the learner terminal matches the registered fingerprint, executing a learning menu corresponding to the registered fingerprint;
In the step of executing the learning menu, the learning execution unit recommends additional related keywords while solving the student's curiosity in real time through the chatbot,
The learning execution unit includes a learning induction unit that checks the learning progress and informs when a conversation is created whether there is necessary learning,
When there is climate change, the learning induction unit asks about the weather in relation to the weather, induces emotional communication with the learner, recommends associative learning through a simple game in the chatbot, and the same vector value as the answer to the question asked by the student A learning method using a finger that induces learning by recommending keywords or associative learning with The method of claim 1,
In the step (b), the learner terminal matches the fingerprint image acquired through the touch input/output unit with the learning content according to the learning menu, stores it in the storage unit, and registers it. 3. The method of claim 2,
The touch input/output unit includes a fingerprint sensor and a fingerprint detection unit,
The fingerprint sensor includes light-emitting elements emitting light, and at least one light sensor that generates an image corresponding to the fingerprint by sensing light emitted from the light-emitting elements and reflected by the fingerprint in contact with the fingerprint contact surface. It includes a sensing array unit and a selective light transmission unit including a plurality of pinholes forming a path of light incident to the optical sensor,
The fingerprint detector receives the image from the fingerprint sensor, extracts a first image corresponding to the first region of the fingerprint and a second image corresponding to the second region of the fingerprint, and compares the first and second images to determine the degree of similarity, and perform fingerprint authentication according to the result;
The first area of the fingerprint is one area of the fingerprint in contact with the fingerprint contact surface, and the second area of the fingerprint is located outside the one area of the fingerprint and is another area of the fingerprint that is not in contact with the fingerprint contact surface. ,
How to learn using your fingers. 4. The method of claim 3,
The fingerprint detection unit receives the image from the fingerprint sensor, extracts the first image and the second image through an image extraction unit, and performs image preprocessing on the first and second images through an image processing unit and comparing the first image and the second image pre-processed by the image processing unit through the similarity determining unit to determine the similarity, and performing fingerprint authentication according to the result of the similarity determining unit through the authentication performing unit,
The similarity determining unit compares the luminance of the first image with the luminance of the second image, checks whether the first and second images have a similar pattern to each other, and determines whether the fingerprint in contact with the fingerprint contact surface is a biometric fingerprint. to judge,
How to learn using your fingers. (a) providing a learning service through a learner terminal;
(b) matching and registering the current learning menu and the learner's fingerprint in the learner terminal;
(c) providing a fingerprint input screen when the learning service is executed in the learner terminal;
(d) receiving the learner's fingerprint from the learner terminal and comparing it with a registered fingerprint; and
(e) when the fingerprint input from the learner terminal matches the registered fingerprint, executing a learning menu corresponding to the registered fingerprint;
In the step of executing the learning menu, the learning execution unit resolves the student's curiosity in real time through the chatbot, and additionally recommends related keywords, the learning execution unit checks the learning progress, and talks whether there is necessary learning includes a learning inducing unit that notifies when is created, wherein the learning inducing unit asks for greetings related to the weather when there is a climate change, induces emotional communication with learners, and recommends associative learning through a simple game in the chatbot, A computer-readable recording medium that records a learning method using a finger to guide learning by recommending keywords or related learning having the same vector value as the answer to a question asked by a student as a program.

Images