TWI696110B - User terminal apparatus and iris recognition method thereof - Google Patents

Abstract

A user terminal apparatus and an iris recognition method thereof are provided. The user terminal apparatus includes an imaging unit configured to generate an image of a user, a display configured to display the generated image, and a controller configured to detect a user eye from the generated image, control the display to display a guide image in a position of the generated image in which the detected user eye is located, and perform iris recognition in response to a difference between a size of an iris included in the detected user eye and a size of the guide image being within a preset range.

Description

User terminal device and iris identification method [Cross-reference to related applications]

This application is based on the 35 USC § 119(a) claim on October 15, 2014, which was filed with the Korean Intellectual Property Office and was granted the Korean Patent Application Serial No. 10-2014-0139306, and on June 5, 2015 The Korean patent application filed with the Korean Intellectual Property Office and granted the rights and interests of the Korean Patent Application Serial No. 10-2015-0080137, the disclosure content of each patent application is incorporated in this case for reference.

The invention relates to a user terminal device and its iris recognition method. More specifically, the present invention relates to a user terminal device displaying a guide image for iris recognition of a user and an iris recognition method thereof.

As personal information and/or content information is stored in individual user terminal devices, the demand for security is increasing. In the prior art, a pass code input method is used to ensure the security of a user terminal device, but in recent years, various authentication methods have been introduced, such as fingerprint recognition, voice recognition, or iris recognition.

In the user terminal device of the prior art, the user attempts iris recognition by looking at a mirror infrared (IR) pass filter called a cold mirror. In the user terminal device, the position and size of the iris recognizable in the user terminal device are presented for iris recognition. However, since the user cannot know the position and size of the iris that can be recognized in the user terminal, it is difficult to perform iris recognition. For example, in a user terminal device equipped with an imaging unit that does not have an auto focus function, it is difficult to determine the distance suitable for iris recognition.

For iris recognition in the user terminal device, a process for registering the user's iris code is necessary. In order to obtain high-quality iris images during user iris code registration, multiple iris images are acquired and analyzed, and one or more iris images determined as high-quality images among the multiple iris images are stored in the iris code data In the library. Even if it corresponds to the user's iris code storing the iris image selected from the acquired multiple iris images, it is still difficult to accurately identify the iris because part of the iris is covered by the eyelid or eyebrows.

The above information is provided as background information only to help understand the present invention. As to whether any of the above can be suitably used as the prior art of the present invention, no judgment or assertion is made.

The aspects of the present invention are intended to solve at least the above problems and/or disadvantages and to provide at least the advantages described below. Therefore, the aspect of the present invention aims to provide a user terminal device and its rainbow capable of performing iris recognition more conveniently and accurately by a user Membrane identification method.

According to an aspect of the present invention, a user terminal device is provided. The user terminal device includes: an imaging unit to generate a user's image; a display to display the generated image; and a controller to: detect the user's eyes from the generated image and control The display displays a guide image at the position of the generated image in which the detected user's eyes are located, and the size of the iris included in the detected user's eye and the guide image The difference in the size is within the preset range to perform iris recognition.

According to another aspect of the present invention, a method of recognizing a user's iris by a user terminal device is provided. The method includes: generating an image of the user; displaying the generated image; detecting a user's eye from the generated image; where the detected user's eye is located in the generated image A guide image is displayed at the position of; and iris recognition is performed in response to the difference between the size of the iris included in the detected user's eye and the size of the guide image being within a preset range.

According to another aspect of the present invention, a user terminal device is provided. The user terminal device includes: a storage unit for storing a plurality of reference images acquired according to different imaging conditions; an imaging unit for generating user images; a display for displaying the generated images; and control For detecting the user's eyes from the generated image, and by comparing the reference images with the imaging conditions corresponding to the detected user's eyes and the detected user's eyes in the plurality of reference images A comparison is performed to perform iris recognition.

According to various embodiments of the present invention, the user can perform iris recognition more conveniently and accurately.

Other aspects, advantages, and salient features of the present invention will become apparent to those having ordinary knowledge in the art through the following detailed description. The detailed description and the accompanying drawings disclose various embodiments of the present invention.

100: user terminal device

110: imaging unit

120: display

130: controller

200: user terminal device

210: communication unit

220: Imaging unit

230: Image processor

240: display

250: storage unit

260: input unit

270: Controller

271: Random Access Memory (RAM)

272: Read Only Memory (ROM)

273: Graphics processor

274: Main central processing unit (CPU)

275-1~275-n: the first interface to the nth interface

276: busbar

310: Eye-guided image

320: Iris guide image

330: Iris line

340: Iris line

480: Boot UI

510: Virtual iris image

540: Guide image

550: Guide image

560: Iris line

700: detected iris area

710, 720, 730: area/area covered by eyelid or eyebrow/occluded area

870: outer corner of eyes

880: the inner corner of the eye

890: Line L

895: S

970-1: Eyelid outline

970-2: contour of eyelid

980: Reference line S

1110: Pre-registered iris code

1120: Iris code

1140: Uncovered area

1400: Home screen

1410: Fixed UI

1420: Guide image

1430: Start screen

1440: entire screen

1500: Home screen

1510: Fixed UI

1520: Guide image

1600: Start screen

1610: Fixed UI

1620: Guide image

1630: screen

1700: Inspection screen

1710: Fixed UI

1720: Guide image

1730: Screen for successful authentication

1910: Eye-like fixed UI

1920:UI

1930: Eye-like fixed UI

1940: Iris recognition success message

2010: Camera

2710: First reference image

2720-1: Second reference image

2720-2: Third reference image

2720-3: Fourth reference image

2810-1: The first image

2810-2: Second image

2810-3: Third image

2810-4: Fourth image

2820-1: fifth image

2820-2: The sixth image

P: center point of iris

S: straight line

S410, S420, S430, S440, S450, S460, S470: Operation

S810, S820, S830, S840, S850, S860: Operation

S910, S920, S930, S940, S950, S960: Operation

S1010, S1020, S1030, S1040: Operation

S1210, S1220, S1230, S1240, S1250: Operation

S1310, S1320, S1330, S1340, S1350, S1360, S1370: Operation

S1810, S1820, S1830, S1840, S1850: Operation

S2110, S2120, S2130, S2140: Operation

S2210, S2220, S2230, S2240, S2250: Operation

S2310, S2320, S2330, S2340: Operation

S2410, S2420, S2430, S2440, S2450, S2460, S2470, S2480, S2490: Operation

S2510, S2520, S2530, S2540, S2550, S2560, S2570, S2580, S2590: Operation

S2610, S2620, S2630, S2640, S2650, S2660, S2670, S2680, S2690: Operation

S2910, S2920, S2930, S2940, S2950, S2960, S2970, S2980, S2990: Operation

By reading the following description in conjunction with the accompanying drawings, the above and other aspects, features, and advantages of some embodiments of the present invention will become more apparent. In the drawings: FIG. 1 illustrates a user terminal according to an embodiment of the present invention A schematic block diagram of the configuration of the device.

2 is a detailed block diagram illustrating the configuration of a user terminal device according to an embodiment of the present invention.

3A and 3B are diagrams illustrating guide images for iris recognition according to various embodiments of the present invention.

4A, 4B, and 4C are diagrams illustrating a method of extracting iris features in consideration of eyelid and eyebrows according to various embodiments of the present invention.

5A, 5B, and 5C are diagrams illustrating a process of performing image processing on a formed image for security purposes according to various embodiments of the present invention.

6A and 6B are diagrams illustrating examples in which the iris is covered by eyebrows or eyelids according to various embodiments of the present invention.

7 is a method for generating an iris code according to an embodiment of the present invention Scheme.

8A, 8B, 9A, 9B, and 10 are diagrams illustrating a method of normalizing an iris area according to various embodiments of the present invention.

FIG. 11 is a diagram illustrating a method of updating an iris code according to an embodiment of the present invention.

12 is a flowchart illustrating an iris recognition method of a user terminal device according to an embodiment of the present invention.

13 is a flowchart illustrating a method of storing and updating an iris code in a user terminal device according to an embodiment of the present invention.

14A, 14B, 14C, 14D, 15A, 15B, 16A, 16B, 17A, 17B, and 18 illustrate iris recognition via a user terminal device according to various embodiments of the present invention For example screen switching.

19A, 19B, 19C, 19D, 20A, 20B, 21, and 22 illustrate iris recognition via an eye-like fixed user interface (UI) according to various embodiments of the invention Example schema.

23, 24, 25, 26, 27A, 27B, 28A, 28B, and 29 illustrate various references according to imaging conditions used in a user terminal device according to various embodiments of the present invention. The image is an example pattern of performing iris recognition.

Throughout the drawings, the same reference numbers will be understood to refer to the same components, assemblies, and structures.

The following description provided with reference to the accompanying drawings is to assist in a comprehensive understanding of various embodiments of the present invention defined by the scope of patent applications and their equivalents. The description includes various specific details to assist in the understanding, but the specific details should be considered exemplary only. Therefore, those of ordinary skill in the art will recognize that various changes and modifications can be made to the various embodiments described herein without departing from the scope and spirit of the invention. In addition, descriptions of well-known functions or constructions may be omitted for clarity and conciseness.

The terms and words used in the following description and patent application scope are not limited to the bibliographic meaning, but are only used by the inventor to achieve a clear and consistent understanding of the present invention. Therefore, those of ordinary knowledge in the art should understand that the following descriptions of various embodiments of the present invention are for illustrative purposes only, and are not intended to limit the present invention defined by the scope of the accompanying patent applications and their equivalents.

It should be understood that unless the context clearly dictates otherwise, the singular forms "a" and "the" include plural indicators. Thus, for example, reference to "a component surface" includes reference to one or more such surfaces.

It should be understood that although the terms first, second, etc. may be used herein when referring to elements of the present invention, such elements should not be interpreted as being limited by these terms. The terminology is only used to distinguish each element.

It should be further understood that when the terms "comprises, including" and/or "includes, including" are used in this text, it means that The presence of features, integers, operations, elements, and/or components does not exclude the presence or addition of one or more other features, integers, operations, elements, components, and/or groups thereof.

In various embodiments of the present invention, a "module" or "unit" can perform at least one function or operation, and can be implemented by hardware, software, or a combination of hardware and software. In addition to the "modules" or "units" that need to be implemented in specific hardware, the "multiple modules" or "multiple units" can be integrated with at least one module to use at least one processor (Fig. (Not shown).

In various embodiments of the present invention, it is assumed that mobile terminal devices and fixed mobile terminal devices (eg, user equipment (UE), mobile station (MS) components, or advanced mobile stations (advanded MS, AMS) ) Element) is collectively referred to as a user terminal device.

Hereinafter, various embodiments of the present invention will be explained in more detail with reference to the drawings. In the following description, unless otherwise stated, when the same elements are depicted in different drawings, the same reference numbers are used to denote the same elements, and will not be repeated.

Specifically, in this patent document, FIGS. 1 to 29 discussed below, and various embodiments for explaining the principles of the present invention are given by way of illustration only and should not be in any way that would limit the scope of the invention be explained. Those of ordinary skill in the art will understand that the principles of the present invention can be implemented in any suitably arranged communication system. The terminology used to illustrate various embodiments is exemplary. It should be understood that these terms are provided only to assist in understanding the description made, and the use and definitions of the terms do not limit the scope of the present invention in any way. Unless expressly stated otherwise, Otherwise the terms first, second, etc. are used to distinguish objects with the same terminology and are by no means intended to represent chronological order. A set is defined as a non-empty set including at least one element.

FIG. 1 is a schematic block diagram illustrating the configuration of a user terminal device according to an embodiment of the present invention.

Referring to FIG. 1, a user terminal device 100 is shown. The user terminal device 100 may include an imaging unit 110, a display 120, and a controller 130.

The imaging unit 110 can image a user. For example, the imaging unit 110 may image the user's face area to recognize the user's iris.

The display 120 can display images according to the control of the controller 130. For example, the display 120 can display the image formed by the imaging unit 110. The display 120 can display a guide image for iris recognition in the resulting image.

The controller 130 can control the overall operation of the user terminal device 100. For example, the controller 130 can detect the user's eyes from the image formed by the imaging unit 110, and control the display 120 to display the guide image at the position of the detected image where the user's eyes are located, and respond to the The difference between the size of the iris detected by the user's eyes and the size of the guide image is within a preset value to perform iris recognition.

For example, the controller 130 may detect the iris boundary from the detected user's eye, estimate the iris diameter based on the detected iris boundary, and based on the detected user's eye and the iris diameter The control display 120 displays a guide image at a position corresponding to the user's eyes and the iris. The guide image may include an eye guide image corresponding to the user's eyes and a user's iris Iris guide image. The eye guide image may be displayed at a position corresponding to the user's eyes, and the iris guide image may be displayed at a position corresponding to the user's iris.

In this example, the guide image may be an image corresponding to the size of the user's eyes and iris at a position matching the focal length of the imaging unit 110. That is, the eye-guided image and the iris-guided image may be larger than the actual user's eyes and iris, because the user is located at a position farther than the focal distance of the imaging unit 110. Since the user is located at a position closer to the focal length of the imaging unit 110, the eye-guided image and iris-guided image may be smaller than the actual user's eye and iris. Therefore, the user can perform iris recognition through the guide image by controlling the distance from the imaging unit 110.

In another example, the controller 130 can detect the user's iris from the formed image. Since the difference between the size of the detected user's iris and the size of the guide image corresponding to the iris is less than or equal to the preset value (ie, because the user is located within the identifiable distance of the iris), the controller 130 can borrow The iris recognition is performed on the user's iris by determining whether the detected user's iris is consistent with the pre-registered iris. In this example, the controller 130 may detect the iris code from the detected user's iris, and perform user authentication by comparison with the pre-registered iris code.

In another example, the controller 130 can detect the user's iris from the resulting image. In response to the difference between the size of the detected user's iris and the size of the guide image corresponding to the iris is less than or equal to the preset value (ie, due to the user being within the identifiable distance of the iris), the controller 130 may output Used to guide the execution of iris recognition Feed message. The feedback message may include at least one of a visual feedback message (for example, an image), an auditory feedback message (for example, an audio signal), and a tactile feedback message (for example, vibration).

In another example, to enhance the security during iris recognition, the controller 130 may process the resulting image so as not to confirm the user's iris pattern during the execution of the iris recognition, and control the display 120 to display the processed image.

In another example, the controller 130 may generate an iris code by normalizing the detected iris area into a rectangular shape. The controller 130 may perform user's iris recognition by masking the area covered by the eyebrows or eyelid of the iris code that is normalized into a rectangular shape, and excluding the masked area of the iris code The remaining area is compared with the pre-registered iris code.

In this example, in response to the detected iris area being recognized as a pre-registered user, the controller may use the remaining area of the detected iris area other than the shaded area to update the iris of the pre-registered user code.

Through the user terminal device 100 described above, the user can perform user authentication by performing iris recognition more accurately and conveniently.

Hereinafter, various embodiments of the present invention will be explained with reference to FIGS. 2 to 7.

2 is a detailed block diagram illustrating the configuration of a user terminal device according to an embodiment of the present invention.

2, a user terminal device 200 is shown. The user terminal device 200 may include a communication unit 210, an imaging unit 220, an image processor 230, a display 240, a storage unit 250, an input unit 260, and a controller 270.

The communication unit 210 may be used to perform communication with various types of external devices according to various types of communication methods. The communication unit 210 may include various communication chips, for example, Wi-Fi chips, Bluetooth chips, near field communication (NFC) chips, wireless communication chips, and infrared (IR) chips. For example, the Wi-Fi chip, Bluetooth chip, NFC chip, and IR chip can perform communication in the Wi-Fi method, Bluetooth method, NFC method, and IR method, respectively. In the communication chip, the NFC chip may be 13.56 MHz in various radio frequency identification (RF-ID) frequency bands (for example, 135 kHz, 13.56 MHz, 433 MHz, 860 MHz to 960 MHz, and 2.45 GHz). The chip whose frequency band operates in NFC mode.

In response to the use of a Wi-Fi chip or a Bluetooth chip, the communication unit 210 can first send/receive various connection information such as a service set identifier (SSID) and session key, and use the connection information to perform a communication connection, And send/receive various information. The wireless communication chip may be a chip for performing communication according to the following various communication standards: for example, Institute of Electrical and Electronics Engineer (IEEE), ZigBee, 3rd generation (3G), The 3G partnership project (3GPP), or long term evolution (LTE).

The imaging unit 220 may perform an imaging operation. The imaging unit 220 may be located in front of the user terminal device 200 and image the user. The imaging unit 220 may include a mirror-type IR pass filter called a cold light mirror.

The image processor 230 can process the image formed by the imaging unit 220. The image processor 230 can perform image processing on the image formed by the imaging unit 220 to enhance security.

The display 240 can display images and user interface (UI) input from various sources. The display 240 may display an image formed by the imaging unit 220, and may display a guide image for iris recognition on the position of the formed image corresponding to the eyes of the user.

The storage unit 250 can store various modules for driving the user terminal device 200. For example, the storage unit 250 may store software including a basic module, a sensing module, a communication module, a presentation module, a web browser module, and a service module.

The basic module may refer to a basic module for processing signals transmitted from the hardware included in the user terminal device 200 and transmitting the processed signals to the upper-layer module.

The sensing module may be a module that collects information from various sensors, and analyzes and manages the collected information. The sensing module may include a face recognition module, a voice recognition module, a motion recognition module, an NFC recognition module, and so on.

The demonstration module may be a module for forming a display screen. The presentation module may include a multimedia module that reproduces multimedia content and outputs the reproduced content, and a UI rendering module that performs UI and graphic processing.

The communication module may be a module for performing communication with an external device. A web browser module can refer to a module that performs web browsing and accesses a web server. The service module may be a module including various application programs for providing various services.

As described above, the storage unit 250 may include various program modules, but the program modules may be partially omitted or deformed according to the type and characteristics of the user terminal device 200, or may be based on the type and characteristics of the user terminal device 200 Feature to add another program module. For example, since the user terminal device 200 is implemented as a tablet personal computer (PC), the basic module may further include position determination for determining the position based on the global positioning system (GPS) Module, and the sensing module may further include a sensing module for detecting a user's movement.

The storage unit 250 may store the iris code corresponding to at least one user. The iris code can be data in which the user's iris area can be normalized into a rectangular shape, and can be used for iris recognition. In response to the successful iris recognition, the user's iris code can be updated in the storage unit 250 via the controller 270.

The input unit 260 can receive various user commands for controlling the user terminal device 200. The input unit 260 can be implemented by touching the screen. The input unit 260 may be implemented with various types of input elements (eg, remote controller, keyboard, mouse, pointing element, etc.).

The controller 270 can use various programs stored in the storage unit 250 to control the overall operation of the user terminal device 200.

As illustrated in FIG. 2, the controller 270 may include a random access memory (random access memory, RAM) 271, a read only memory (ROM) 272, a graphics processor 273, and a main central processing unit (central processing unit, CPU)274, the first interface 275-1 to the nth interface 275-n, And the bus bar 276. The RAM 271, the ROM 272, the image processor 273, the main CPU 274, the first interface 275-1 to the nth interface 275-n, etc. may be electrically coupled via the bus 276.

The command set and the like for system startup are stored in the ROM 272. In response to input of an open command to supply power, the main CPU 274 can copy the operating system (O/S) stored in the storage unit 250 to the RAM 271 according to the command stored in the ROM 272, and execute the operating system to start system. In response to the completion of startup, the main CPU 274 may copy various application programs stored in the storage unit 250 to the RAM 271, and execute the application programs copied to the RAM 271 to perform various operations.

The graphics processor 273 can be used to generate a screen including various objects such as icons, images, text, etc. using an arithmetic unit (not shown in the figure) and a rendering unit (not shown in the figure). The arithmetic unit (not shown) may calculate attribute values, such as coordinate values (where objects are displayed according to the layout of the screen), shape, size, and color, based on control commands input from the input unit 260. The rendering unit (not shown in the figure) may generate screens having various layouts including the objects based on the attribute values calculated in the arithmetic unit. The screen generated in the rendering unit is displayed in the display area of the display 240.

The main CPU 274 accesses the storage unit 250 to perform startup using the operating system stored in the storage unit 250. The main CPU 274 performs various operations using various programs, contents, data, etc. stored in the storage unit 250.

The first interface 275-1 to the nth interface 275-n are coupled to the above components. One of the interfaces may be a network interface coupled to an external device via a network.

The controller 270 can detect the user's eyes from the image formed by the imaging unit 220 and control the display 240 to display the guide image on the position of the generated image where the user's eyes are located.

For example, the controller 270 can detect the user's eyes from the image formed by the imaging unit 220, and detect the iris boundary based on the detected eye position.

In this example, the controller 270 can calculate the center coordinates of the eye (x, y) from the detected eye area. For example, the controller 270 may calculate the center coordinates of the area estimated as the eye area as the center coordinates of the eye. In another example, the controller 270 may calculate the reflected illumination light (ie, flash) formed in the pupil area of the eye area as the central coordinate of the eye.

The controller 270 can detect the iris boundary by analyzing the degree of change of the pixel value based on the center point of the eye. The degree of change of the pixel value can be directly analyzed using the difference in brightness between adjacent pixels or can be analyzed using the first-order differential value or the second-order differential value.

As an example of detecting the iris boundary, the controller 270 may perform a search for a predetermined specific direction based on the center coordinates of the eye to detect the iris boundary. In the search process, the controller 270 may use a method of estimating a circle close to the feature point obtained by analyzing the degree of change of the pixel value. In the method of determining a feature point by analyzing the degree of change of the pixel value, a portion where the degree of change of the pixel value suddenly increases or decreases can be determined as the position of the feature point. As another example of detecting the iris boundary, the controller 270 can detect the iris center and the iris boundary by searching for the following circle: in the circle, while changing the center point and the size of the circle based on the center coordinates of the eye Said The degree of change of the pixel value inside the circle and the degree of change of the pixel value outside the circle are maximized. Even in response to using an ellipse instead of a circle, the controller 270 can still detect the center of the iris and the iris boundary by searching for the following ellipse: in the ellipse, the center, long axis, and center of the ellipse are changed based on the coordinates of the eye center At the same time of the short axis, the degree of change of the pixel value inside the ellipse and the degree of change of the pixel value outside the ellipse are maximized.

The controller 270 may estimate the iris diameter based on the detected iris boundary.

The controller 270 may control the display 240 to display a guide image on a position corresponding to the user's eyes and the iris based on the recognized user's eyes and iris diameter. The guide image may be an image corresponding to the size of the user's eyes and iris at a position matching the focal length of the imaging unit 220. The guide image may include an eye guide image corresponding to the user's eyes, an iris guide image corresponding to the user's iris, and the like. Since the user is located at a position that matches the focal length of the imaging unit 220, the eye-guided image and the iris-guided image may have the statistical values of the user's eyes and iris.

3A and 3B are diagrams illustrating guide images for iris recognition according to various embodiments of the present invention.

Referring to FIG. 3A, an eye guide image 310, an iris guide image 320, and an iris line 330 are shown, in which the controller 270 illustrated in FIG. 2 can control the display 240 illustrated in FIG. 2 to display around the eyes of the detected user The eye guides the image 310 and controls the display 240 to display the iris guide image 320 around the user's iris. The controller 270 can control the display 240 to display the actually detected user's iris line 330 and the guide images 310 and 320, thereby confirming the user's eye contact Location and distance.

In this example, the user can confirm that the iris line 330 is smaller than the iris guide image 320. Therefore, the user can confirm that the user is located away from the user terminal device 200 based on the guide images 310 and 320 and the iris line 330 and move the user terminal device 200 toward the user terminal device 200.

Referring to FIG. 3B, an eye guide image 310, an iris guide image 320, and an iris line 340 are shown, in which the controller 270 can control the display 240 to display the eye guide image 310 around the eyes of the detected user, and control the display 240 to the user An iris guide image 320 is displayed around the iris. The controller 270 can control the display 240 to display the actually detected iris line 340 of the user and the guide images 310 and 320 to confirm the position and distance of the user's eye contact.

In this example, the user can confirm that the iris line 340 is larger than the iris guide image 320. Therefore, the user can confirm that the user is located close to the user terminal device 200 as illustrated in FIG. 2 based on the guide images 310 and 320 and the iris line 340, and point the user terminal device 200 away from the user terminal device 200 mobile.

However, even in response to the detection of the iris area, the iris recognition may not be performed accurately due to the eyelid or eyebrows.

4A, 4B, and 4C are diagrams illustrating a method of extracting iris features in consideration of eyelid and eyebrows according to various embodiments of the present invention.

A flowchart is illustrated with reference to FIG. 4A, where the flowchart illustrates a method for extracting iris features in consideration of eyebrows and eyelids according to an embodiment of the present invention.

Referring to FIG. 4A, at operation S410, the controller 270 as illustrated in FIG. 2 may detect the iris area from the detected image. At operation S420, the controller 270 may detect the eyelid from the detected image. At operation S430, the controller 270 may determine whether the eyelid rate occupied by the eyelid in the iris area is greater than or equal to the first value (eg, 5%). Since the eyelid rate is less than the first value at operation S430, the controller 270 may detect eyelashes from the detected image at operation S440. At operation S450, the controller 270 may determine whether the eyelash rate occupied by eyelashes in the iris area is greater than or equal to the second value. Since the eyelash rate is less than the second value at operation S450, the controller 270 may detect the iris feature for iris recognition at operation S470. That is, the controller 270 may perform iris recognition only in response to the eyelid rate and eyelash rate being less than or equal to the preset value.

Due to the eyelid rate being greater than or equal to the first value at operation S430 or the eyelash rate being greater than or equal to the second value at operation S450, the display 240 illustrated in FIG. 2 may be displayed at operation S460 for smooth iris recognition Guide user interface (UI).

Referring to FIG. 4B, a guidance UI 480 is shown, in which the display 240 illustrated in FIG. 2 may display the guidance UI 480 including a guidance message "open your eyes wide". That is, since iris recognition is not successfully performed due to the eyelids or eyebrows of the user who has not opened their eyes wide, the controller 270 as illustrated in FIG. 2 may display the guide UI 480 for iris recognition.

The following explanation is provided with reference to FIG. 4C: In response to the imaged user’s iris area being partially imaged, the controller 270 illustrated in FIG. 2 may not display the guide image for performing iris recognition.

The security of the iris image as the user's personal information is crucial. Since the iris image is completely displayed, the iris information as the user's personal information can be leaked to the outside world by capturing the displayed image or shooting the displayed image. Therefore, the controller 270 can control the image processor 230 as illustrated in FIG. 2 to process the resulting image in such a way that the resulting iris image is not completely displayed and the user’s iris information may not be confirmed through various methods. The device 270 can control the display 240 to display the image processed by the image processor 230.

5A, 5B, and 5C are diagrams illustrating a process of performing image processing on a formed image for security purposes according to various embodiments of the present invention.

5A, a virtual iris image 510 is shown, in which the controller 270 illustrated in FIG. 2 can control the image processor 230 illustrated in FIG. 2 and the display 240 illustrated in FIG. 2 to completely display the divided images. The remaining part outside the iris area is displayed in the iris area and a virtual iris image 510 is displayed.

An explanation is provided with reference to FIG. 5B, in which the controller 270 illustrated in FIG. 2 can detect the eye area and the iris area, and controls the image processor 230 illustrated in FIG. 2 and the display 240 illustrated in FIG. 2 to display the detected The eye area and iris area serve as edge information and display the remaining area in black and white images.

Referring to FIG. 5C, the guide images 540 and 550 and the iris line 560 are shown, wherein the controller 270 as illustrated in FIG. 2 can control the image processor 230 and the display 240 to not completely display the formed image, and only display and use The iris line 560 and the guide images 540 and 550 corresponding to the actual iris area of the person. In this example, as illustrated in FIG. 5C, the guide images 540 and 550 can be displayed in solid lines, while the iris line 560 can be Shown in dashed lines.

Since the iris image representing the current iris size is represented, the frame per second (FPS) can be reduced according to the performance of the user terminal device 200 as illustrated in FIG. 2. A sudden change (for example, a sharp increase or a sharp decrease) of the iris image representing the current iris size of the user may occur according to the movement of the user terminal device 200. To alleviate this phenomenon, the iris size of the current frame can be determined by reflecting the information of the iris size of the previous frame based on Equation 1 below.

Here, Iris_Diameter _F is the diameter of the iris image, Iris_Diameter _C is the diameter of the iris estimated in the current frame, and Iris_Diameter _P is the diameter of the iris estimated in the previous frame. a is the weight assigned to the current frame and has a value between 0 (zero) and 1.

The embodiment of the present invention has shown that the guide image is a circle or an ellipse. The guide image may be expressed in various figures (for example, a quadrilateral or a triangle).

Since the user is located in the focal position of the imaging unit 220 suitable for performing iris recognition (that is, the difference between the size of the detected user's iris and the size of the iris-guided image is less than or equal to the preset value), the controller 270 may The feedback image used to guide iris recognition is output. For example, because the difference between the size of the detected user's iris and the size of the iris-guided image is within 2% of the user's iris size, the controller 270 may output a feedback message for guiding the execution of iris recognition to execute the Describe iris identification.

For example, the controller 270 may output visual feedback messages. In this example In this case, the controller 270 can control the display 240 to change the color of the iris line 330 due to the difference between the size of the iris guide image 320 as illustrated in FIG. 3A and the size of the iris line 330 as illustrated in FIG. 3A. Or control the display 240 to make the iris line 330 blink. The controller 270 may control the display 240 to blink the entire screen or control the display 240 to display a separate UI for guiding the execution of iris recognition.

In another example, the controller 270 may output auditory feedback messages. In this example, according to the difference between the size of the iris guide image 320 and the size of the iris line 330 is less than or equal to a preset value, the controller 270 may output an alarm sound or an auditory UI (auditory UI for guiding iris recognition, AUI).

In another example, the controller 270 may output tactile feedback information, such as vibration.

According to the fact that the user is located in the focal position of the imaging unit 220 illustrated in FIG. 2 suitable for performing iris recognition (ie, the difference between the size of the detected user’s iris and the size of the iris-guided image is less than or equal to the preset value ), the controller 270 can perform iris recognition by determining whether the detected user's iris is consistent with the iris registered in the storage unit 250 in advance.

6A and 6B are diagrams illustrating examples in which the iris is covered by eyebrows or eyelids according to various embodiments of the present invention.

When performing iris recognition via the controller 270 as illustrated in FIG. 2, the iris may be fully imaged (as illustrated in FIG. 6A) or a part of the iris may be covered by the eyelid or eyebrows (as illustrated in FIG. 6B).

6A provides the following instructions: In response to the iris being completely exposed, the rainbow Membrane recognition can be performed normally. However, referring to FIG. 6B, the following explanation is provided: Since the iris is covered by the eyelid or eyebrow, the success rate of iris recognition may be reduced by the eyelid or eyebrow.

To prevent the reduction, the controller 270 may generate an iris code by normalizing the detected iris area into a rectangular shape as illustrated in FIG. 7.

7 is a diagram illustrating a method of generating an iris code according to an embodiment of the present invention.

Referring to FIG. 7, areas 710, 720, and 730 and a detected iris area 700 are shown, where the controller 270 as illustrated in FIG. 2 may be in the area covered by the eyelid or eyebrows of the iris code that is normalized into a rectangular shape Masking is performed on 710, 720, and 730.

Hereinafter, a method of generating an iris code by normalizing the iris area will be described with reference to FIGS. 8A to 10.

8A to 10 are diagrams illustrating a method of normalizing an iris area according to various embodiments of the present invention.

8A is a flowchart illustrating a method of normalizing the iris area by extracting the inner angle of the eye and the outer angle of the eye according to an embodiment of the present invention.

Referring to FIG. 8A, a flowchart is shown, where at operation S810, the controller 270 as illustrated in FIG. 2 may detect the iris boundary from the eye area of the imaged user.

At operation S820, the controller 270 may calculate the coordinates of the iris center point P.

8A and 8B, at operation S830, the controller 270 may detect the inner angle 880 of the eye and the outer angle 870 of the eye.

At operation S840, the controller 270 may calculate the connection as illustrated in FIG. 8B The straight line L 890 of the inner angle 880 of the eye and the outer angle 870 of the eye.

At operation S850, the controller 270 may calculate a straight line S 895 perpendicular to the iris center point P and the straight line L.

At operation S860, the controller 270 may perform normalization by performing polar coordinate conversion on the iris area based on the reference line S.

9A is a flowchart illustrating a method of normalizing an iris region by extracting eyelid contours according to an embodiment of the present invention.

Referring to FIG. 9A, a flowchart is shown, where at operation S910, the controller 270 as illustrated in FIG. 2 may detect the iris boundary from the eye area of the imaged user. At operation S920, the controller 270 may calculate the coordinates of the iris center point P.

9A and 9B, at operation S930, the controller 270 may detect the contours of the eyelid 970-1 and 970-2.

At operation S940, the controller 270 may measure the distance from the center point P of the iris to the eyelid.

At operation S950, the controller 270 may select a straight line having the shortest distance from the iris center point P to the eyelid as the reference line S980.

At operation S960, the controller 270 may perform normalization by performing polar coordinate conversion on the iris area based on the reference line S980.

10 is a flowchart illustrating a method of normalizing an iris region using both the reference line detected in FIG. 8A and the reference line detected in FIG. 9A according to an embodiment of the present invention.

Referring to FIG. 10, a flowchart is shown, where at operation S1010, as shown in FIG. The illustrated controller 270 may calculate the reference line S1 using the inner angle of the eye and the outer angle of the eye as illustrated in FIG. 8A.

At operation S1020, the controller 270 may calculate the reference line S2 using the eyelid as illustrated in FIG. 9A.

At operation S1030, the controller 270 may use the reference line S1 and the reference line S2 to calculate the final reference line S. For example, the controller 270 may use the average of the reference lines S1 and S2 to calculate the final reference line S, or the controller 270 may calculate the final reference line S by assigning a weight to a specific one of the reference lines S1 and S2 .

At operation S1040, the controller 270 may perform normalization by performing polar coordinate conversion on the iris area based on the final reference line S.

Referring back to FIG. 7, the controller 270 as illustrated in FIG. 2 may record the matching score by comparing the remaining areas of the normalized iris code except the masked areas 710, 720, and 730 with the pre-registered iris code, And the user's iris recognition is performed by judging whether a pre-registered iris code having a matching score greater than or equal to a preset value (for example, 97%) is presented. In response to presenting a pre-registered iris code having a matching score greater than or equal to a preset value, the controller 270 may authenticate the user imaged via the imaging unit 220 as illustrated in FIG. 2 as having a value greater than or equal to the preset The value matches the score of the user corresponding to the pre-registered iris code. For example, as a result of comparing the remaining area other than the masked area with the iris code of the pre-registered first user, the matching score is 98%, as the remaining area other than the masked area and the pre-registration The comparison result of the second user’s iris code is 33%, and it is used as the remaining area except the blocked area and the pre-note The matching score of the third user's iris code is 17%, and the controller 270 can authenticate the imaged user as the first user as the iris recognition result.

FIG. 11 is a diagram illustrating a method of updating an iris code according to an embodiment of the present invention.

In response to the successful identification of the iris, the controller 270 as illustrated in FIG. 2 may analyze the masked area of the iris code registered in advance and the masked area of the iris code used in authentication. In the analysis process, since the masked portion of the pre-registered iris code is not masked in the iris code used in authentication, the controller 270 may use the remaining iris code used in authentication except for the masked area Information to update the obscured part of the pre-registered iris code.

Referring to FIG. 11, an unmasked area (shaded area) 1140 of the iris code 1120 is shown, where the iris code 1120 succeeds in the authentication presented in the shaded area of the pre-registered iris code 1110, and the controller 270 can update and store The unobscured area 1140 of the iris code that succeeded in the authentication.

As described above, the shaded area of the iris code registered in the storage unit 250 can be gradually reduced by repeatedly performing iris recognition, and the portion of the iris code that can be matched in authentication can be increased. Therefore, the iris code registered in the storage unit 250 can be updated by repeated iris recognition, and the reliability of the user can be increased due to the improvement of iris recognition performance.

Hereinafter, the iris recognition method and the iris code update method of the user terminal device will be explained with reference to FIGS. 12 and 13.

12 is a flowchart illustrating an iris recognition method of a user terminal device according to an embodiment of the present invention.

Referring to FIG. 12, a flowchart is shown, in which the user terminal device 200 as illustrated in FIG. 2 can image the user at operation S1210 and display the resulting image at operation S1220.

At operation S1230, the user terminal device 200 can detect the user's eyes from the formed image. The user terminal device 200 can detect the user's iris from the eyes of the detected user.

At operation S1240, the user terminal device 200 may display the guide image at the position corresponding to the detected user's eyes. The user terminal device 200 may display an eye guide image at a position corresponding to the user's eyes, and display an iris guide image at a position corresponding to the user's iris.

At operation S1250, the user terminal device 200 may perform iris recognition. Because the user is located at a distance suitable for iris recognition via the iris-guided image, that is, because the difference between the size of the iris contained in the detected user's eye and the size of the guide image is within a preset range, use The user terminal device 200 can perform iris recognition.

As described above, by displaying the guide image in the detected eye position, the user can perform iris recognition more conveniently.

13 is a flowchart illustrating a method of storing and updating an iris code of a user terminal device according to an embodiment of the present invention.

Referring to FIG. 13, a flowchart is shown, where at operation S1310, as shown in FIG. 2 The user terminal device 200 described above can image the user.

At operation S1320, the user terminal device 200 may determine whether the current mode is the iris code registration mode or the iris authentication mode.

In the iris code registration mode, the user terminal device 200 may generate an iris code from the generated image at operation S1330. For example, the user terminal device 200 can detect the iris area from the formed image, normalize the detected iris area into a rectangular shape, and cover the area of the normalized iris area (eg, eyelid or eyebrow) by masking The iris code is generated.

At operation S1340, the user terminal device 200 may store the generated iris code in the database DB.

In the iris authentication mode, the user terminal device 200 may generate an iris code at operation S1350, and perform masking work on a part (eg, eyelid or eyebrows) that is not necessary for authentication.

At operation S1360, the user terminal device 200 may determine whether the authentication of the iris code on which the masking work has been performed is successful. For example, the user terminal device 200 can judge whether the authentication is successful by comparing the iris code pre-stored in the database DB with the iris code on which the masking work is performed.

In response to the successful authentication at operation S1360, the user terminal device 200 may update the pre-registered iris code using the iris code used in authentication at operation S1370. For example, since the masked portion of the pre-registered iris code is not masked in the iris code used in authentication, the user terminal device 200 can use the user iris code used in authentication except for the masked area Information from the remaining area The masked part of the iris code newly registered in the database DB in advance.

In response to the unsuccessful authentication at operation S1360, the method ends.

As mentioned above, the pre-registered iris code can be updated when the iris recognition is successful, so the reliability of the user can be increased due to the improvement of iris recognition performance.

Hereinafter, an example of switching screens through iris recognition will be explained with reference to FIGS. 14A to 18.

14A to 18 are diagrams illustrating examples of screen switching through iris recognition of a user terminal device according to various embodiments of the present invention.

The controller 270 illustrated in FIG. 2 can control the display 240 illustrated in FIG. 2 to display a fixed UI and guide images on various screens requiring user authentication. In response to performing iris recognition through the fixed UI and the guide image, the controller 270 can control the display 240 to switch the screen requiring user authentication to another screen. The controller 270 can control the display 240 to switch the screen requiring user authentication to another screen by enlarging the fixed UI.

14A shows a main screen 1400, wherein the controller 270 as illustrated in FIG. 2 can control the display 240 illustrated in FIG. 2 to display the main screen 1400.

The home screen 1400 may include a fixed UI 1410 and a guide image 1420. The fixed UI 1410 may be a UI for guiding the point at which the user gazes, and the guide image 1420 may be an image that the user looks at for guiding the position and size of the eyes by the relative position and size to the fixed UI 1410.

Referring to FIG. 15A, a main screen 1500, a fixed UI 1510, and a guide image 1520 are shown, wherein the guide image 1520 is larger than the fixed UI 1510 and is located at a fixed The right side of UI 1510. In this example, the guide image can guide the user's face close to the user terminal device 200 and located on the right side of the user terminal device 200. As the user's face moves away from the user terminal device 200 and moves to the left side of the user terminal device 200, the controller 270 can match the size of the fixed UI 1510 and the guide image 1520 of the main screen 1500, as illustrated in FIG. 15B.

Referring to FIG. 14B, the following main screen 1400, fixed UI 1410, and guide image 1420 are shown: the position and size corresponding to the fixed UI 1410 correspond to the position and size of the guide image 1420, and the controller 270 illustrated in FIG. 2 may Guide iris recognition begins to be performed by changing the type of guide image 1420 (eg, color, flicker, transparency, etc.).

In response to the user authentication performed through iris recognition, referring to FIG. 14C, the following main screen 1400 is shown: The controller 270 can control the display 240 to display the start screen 1430 by gradually enlarging the fixed UI 1410 as illustrated in FIG. 14B Part.

Referring to FIG. 14D, the following entire screen 1440 is shown: The controller 270 illustrated in FIG. 2 can control the display 240 illustrated in FIG. 2 by further enlarging the fixed UI 1410 illustrated in FIG. 14B to display the start screen as the entire Screen 1440.

Referring to FIG. 16A, the following start screen 1600, fixed UI 1610, and guide image 1620 are shown: The controller 270 illustrated in FIG. 2 can control the display 240 illustrated in FIG. 2 at the start screen of the specific application that requires login In 1600, a fixed UI 1610 and a guide image 1620 are displayed.

In response to successful user authentication by performing iris recognition through the fixed UI 1610 and the guide image 1620, referring to FIG. 16B, the controller 270 can control the display 240 to enlarge the fixed UI 1610 after the specific application is logged in. Start screen 1600 switches to screen 1630.

Referring to FIG. 17A, the following verification screen 1700, fixed UI 1710, and guide image 1720 are shown: The controller 270 illustrated in FIG. 2 can control the display 240 illustrated in FIG. 2 to display the fixed UI 1710 and the fixed UI 1710 illustrated in the verification screen 1700. Guide the image 1720. In response to successful user authentication by performing iris recognition through the fixed UI 1710 and the guide image 1720, referring to FIG. 17B, the controller 270 can control the display 240 to switch the verification screen 1700 to the authentication successful screen by enlarging the fixed UI 1710 1730.

18 is a flowchart illustrating an iris recognition method of a user terminal device according to an embodiment of the present invention.

Referring to FIG. 18, a flowchart is shown, where at operation S1810, the user terminal device 200 as illustrated in FIG. 2 may display a fixed UI and a guide image on the first screen. The first screen may be a screen that requires user authentication, and may include a main screen, a login screen, a verification screen, and the like. The fixed UI may be a UI for guiding a user's attention. The guide image may be an image used by the user to guide the position and size of the eyes by the relative position and size with the fixed UI.

At operation S1820, the user terminal device 200 may perform iris recognition using the fixed UI and the guide image.

At operation S1830, the user terminal device 200 may determine to use the iris Whether the identified user authentication is successful.

In response to the successful user authentication at operation S1830, the user terminal device 200 can switch the first screen to the second screen by enlarging the fixed UI at operation S1840. The second screen may be a screen after user authentication, and may include a start screen, a screen after login, a screen for successful authentication, and the like.

In response to the user authentication failure at operation S1830, the user terminal device 200 may output an authentication failure message at operation S1850.

As described above, iris recognition can be more conveniently performed through the fixed UI and the guide image, and a new user experience of switching to another screen through the screen fixation can be provided.

Hereinafter, an example of performing iris recognition via the human eye-like fixed UI is explained with reference to FIGS. 19A to 22.

19A to 22 are diagrams illustrating examples of performing iris recognition via the eye-like fixed UI according to various embodiments of the present invention.

The user must keep his eyes wide open for a long time to increase the success rate of iris recognition. However, this action may be somewhat unusual for the user. Therefore, in the embodiment of the present invention, the user terminal device 200 illustrated in FIG. 2 can display an eye-like fixed UI, and can enable the user to open for a long time by making the user and the virtual character's gaze matching Big eyes.

Referring to FIG. 19A, an eye-shaped fixed UI 1910 is shown, in which the controller 270 as illustrated in FIG. 2 can control the display 240 illustrated in FIG. 2 to display the human eye-shaped fixed UI 1910 on the display screen. The controller 270 can be configured by size repeatedly The form opens the eyes and fixes the eyes of the person included in the UI so that the user can obtain gaze matching.

19B, in response to performing iris recognition, the controller 270 can change the color of the eye-shaped fixed UI 1910, and referring to FIG. 19C, the controller 270 can control the display 240 to be displayed in the eye-shaped fixed UI 1910 as illustrated in FIG. 19C UI 1920 for guiding the degree of iris recognition process.

Referring to FIG. 19D, in response to performing iris recognition, the controller 270 may control the eye-shaped fixed UI 1930 to close the eyes, and control the display 240 to display the iris recognition success message 1940. Iris recognition success message 1940 can be displayed in visual form. The iris recognition success message 1940 can be expressed in an auditory pattern or a vibration pattern.

In response to the user terminal device 200 being used by multiple people, the controller 270 may preferentially determine the person who wishes to control the user terminal device 200 and then perform iris recognition.

Referring to FIG. 20A, a description is provided of the camera 2010 of the user terminal device 200 as illustrated in FIG. 2, wherein the camera 2010 is used to recognize irises and gestures simultaneously.

Referring to FIG. 20B, in response to the user terminal device 200 illustrated in FIG. 2 being used by multiple users (eg, in response to multiple users watching movies or webpages simultaneously), the controller 270 illustrated in FIG. 2 may Prioritize the detection of the user's facial movement and detect the user who is looking at the display screen. The controller 270 may give the detected user interaction priority of the user terminal device 200 by performing iris recognition in the various methods described above.

FIG. 21 illustrates an iris of a user terminal device according to an embodiment of the present invention Flow chart of identification method.

Referring to FIG. 21, a flowchart is shown, in which at step S2110, the user terminal device 200 as illustrated in FIG. 2 may display a human eye-shaped fixed UI on the display screen. The controller 270 as illustrated in FIG. 2 may enable the user to obtain gaze matching by repeatedly opening the human eyes in a size configuration form to fix the human eyes included in the UI 1910 as illustrated in FIG. 19A.

At operation S2120, the user terminal device 200 may determine whether to start iris recognition.

In response to the start of iris recognition at operation S2120, the user terminal device 200 may change the color of the eye-like fixed UI or display the degree of iris recognition progress at operation S2130 to guide the execution of iris recognition.

At operation S2140, the user terminal device 200 may perform iris recognition.

Since the iris recognition is not started at operation S2120, the user terminal device 200 may end the iris recognition method.

22 is a flowchart illustrating an iris recognition method of the user terminal device 200 according to an embodiment of the present invention.

Referring to FIG. 22, a flowchart is shown, wherein at operation S2210, the user terminal device 200 as illustrated in FIG. 2 can image multiple users.

At operation S2220, the user terminal device 200 may detect facial movements of the plurality of users. For example, the user terminal device 200 can determine whether the direction of facial movement of the plurality of users is directed to the user terminal device 200. In another example, the user terminal device 200 may detect the user terminal device 200 The user in front.

In operation S2230, the user terminal device 200 may determine the interaction priority ranker of the plurality of users based on the detected user's facial movements. For example, the user terminal device 200 may determine the user who is gazing at the user terminal device 200.

At operation S2240, the user terminal device 200 may perform iris recognition on the interaction priority sorter.

In response to the successful authentication of the user by performing iris recognition, the user terminal device 200 may give the interaction priority sequencer the interaction priority at operation S2250. For example, the user terminal device 200 may ignore voices, motions, etc. input by other users based on voices, motions, etc. input through the interaction priority sorter rather than the functions of the user terminal device 200.

According to an embodiment of the present invention, the user terminal device 200 may acquire multiple reference images according to imaging conditions, and use the multiple acquired reference images to perform iris recognition. For example, the controller 270 as illustrated in FIG. 2 may acquire multiple reference images according to imaging conditions. The imaging conditions may include at least one of distance, resolution, illumination, focus state, IR light reflection, etc. between the user terminal device 200 and the user. In response to imaging the user to perform iris recognition, the controller 270 can detect the user's eyes from the resulting image, and by comparing the reference image with the imaging conditions corresponding to the detected eyes to the user's eyes Compare and perform iris recognition.

For example, in response to the multiple reference images being registered, the controller 270 The plurality of reference images may be converted into iris codes, and the converted iris codes may be stored in the storage unit 250. However, the controller 270 may store the original data of the multiple reference images in the storage unit 250 as illustrated in FIG. 2.

In another example, in response to the plurality of reference images being registered, the controller 270 may register and store user's imaging condition information (eg, distance information) corresponding to the plurality of reference images and the plurality of reference images Reference image. In response to the execution of iris recognition, the controller 270 can obtain the imaging condition information of the user detected by the detector (not shown in the figure), and based on the acquired imaging condition and stored in the plurality of reference images The imaging condition information selects a reference image for performing iris recognition from the plurality of reference images. The controller 270 may perform iris recognition by comparing the selected reference image with the detected user's eyes.

In response to the generation of the multiple reference images, the controller 270 can acquire the multiple reference images according to the imaging conditions. The controller 270 can obtain the plurality of reference images through image resizing or noise addition. For example, the controller 270 can obtain the first reference image through the imaging unit 220, generate a plurality of reference images according to imaging conditions by performing image size adjustment or noise addition on the first reference image, and The reference images are stored in the storage unit 250.

23 to 29 are diagrams illustrating examples of performing iris recognition using a plurality of reference images according to imaging conditions in a user terminal device according to various embodiments of the present invention.

FIG. 23 is a diagram illustrating a method according to imaging conditions according to an embodiment of the present invention. A flowchart of a method for performing iris recognition by using multiple acquired reference images.

Referring to FIG. 23, a flowchart is shown, where at operation S2310, the user terminal device 200 as illustrated in FIG. 2 may register a plurality of reference images according to imaging conditions. The user terminal device 200 can acquire the plurality of reference images according to imaging conditions. For example, the user terminal device 200 may acquire the multiple reference images according to the distance between the user terminal device 200 and the user. In another example, the user terminal device 200 may obtain the multiple reference images according to the resolution. In another example, the user terminal device 200 may obtain the multiple reference images according to ambient lighting. In another example, the user terminal device 200 may acquire the multiple reference images according to the focus state. In another example, the user terminal device 200 may acquire the multiple reference images according to the point in which the IR light is reflected. In the example, the user terminal device 200 may register and store imaging condition information and the plurality of reference images corresponding to the imaging condition information. The user terminal device 200 can acquire the multiple reference images by imaging the multiple reference images according to the imaging conditions. The user terminal device 200 can generate a plurality of reference images by performing image size adjustment or noise addition on a reference image and according to imaging conditions.

At operation S2320, the user terminal device 200 may image the user's eyes for user authentication.

At operation S2330, the user terminal device 200 can detect the user's eyes and the imaging conditions from the formed image. For example, the user terminal device 200 can detect the user's eyes (specifically, the iris) by detecting the inner and outer boundaries of the iris from the formed image. The user terminal device 200 can detect imaging conditions (for example, use The distance between the user terminal device 200 and the user, etc.). The user terminal device 200 can detect the distance between the user terminal device 200 and the user by a distance detector.

At operation S2340, the user terminal device 200 may perform user authentication by comparing the detected eyes with the reference image. For example, the user terminal device 200 may select a reference image having imaging conditions corresponding to the detected imaging conditions from the plurality of reference images. The user terminal device 200 may perform user authentication by comparing a reference image having an imaging condition corresponding to the detected imaging condition with the detected eye.

As described above, the plurality of reference images acquired according to imaging conditions can be used to perform iris recognition, so even if the eyes of the user are imaged by various imaging conditions, accurate iris recognition can be performed more quickly.

24 is a diagram illustrating a method of registering multiple images and using the multiple reference images to perform user authentication according to an embodiment of the present invention.

Referring to FIG. 24, a flowchart is shown, in which at operation S2410, the user terminal device 200 as illustrated in FIG. 2 can acquire imaging condition information and iris images. For example, the user terminal device 200 may acquire an iris image by imaging an image including the user, and acquire imaging conditions through various sensors (eg, distance detection sensor, illumination sensor, etc.) News.

At operation S2420, the user terminal device 200 can detect the inner and outer iris boundaries. That is, the user terminal device 200 can detect the user's iris by detecting the inner and outer iris boundaries from the iris image.

At operation S2430, the user terminal device 200 may determine whether the current stage is the registration stage or the authentication stage.

Since the current stage is the registration stage, the user terminal device 200 can generate a reference image at operation S2440. For example, the user terminal device 200 can generate a reference image by storing the acquired user iris and the detected imaging condition information.

At operation S2450, the user terminal device 200 may use the reference image to generate the iris code. The method of generating the iris code has been explained, so it will not be explained in detail.

At operation S2460, the user terminal device 200 may store the iris code in the database. That is, the user terminal device 200 may not store the user's iris image in the database, but may store the iris code in the database.

Since the current stage is the authentication stage, the user terminal device 200 may generate an iris code at operation S2470. That is, the user terminal device 200 can generate an iris code through the detected user iris.

At operation S2480, the user terminal device 200 may compare the iris code stored in the database with the generated iris code. For example, the user terminal device 200 may search for an iris code having imaging condition information corresponding to the imaging condition information acquired in operation S2410 among the iris codes stored in the database.

In operation S2490, the user terminal device 200 may perform user authentication based on the result of the comparison in operation S2480. For example, because the comparison result between the iris code searched in the database and the generated iris code is similar If the performance is greater than or equal to the preset value, the user terminal device 200 can perform user authentication. Since the similarity is less than the preset value, the user terminal device 200 may output a user authentication failure message.

As illustrated in FIG. 24, the iris code is stored in the reference image registration instead of the reference image itself, so the operation of converting the reference image into the iris code may not be performed whenever authentication is performed. The iris code rather than the iris image itself can be stored, so even if the database is hacked, the user's personal information can be protected.

FIG. 25 is a diagram illustrating a method of registering multiple images and using the multiple reference images to perform user authentication according to an embodiment of the present invention.

Referring to FIG. 25, a flowchart is shown, in which at operation S2510, the user terminal device 200 as illustrated in FIG. 2 can acquire imaging condition information and iris images. For example, the user terminal device 200 may acquire an iris image by imaging an image including the user, and acquire imaging conditions through various sensors (eg, distance detection sensor, illumination sensor, etc.) News.

At operation S2520, the user terminal device 200 may detect the inner and outer iris boundaries. That is, the user terminal device 200 can use the inner and outer iris boundaries from the iris image to detect the user's iris.

At operation S2530, the user terminal device 200 may determine whether the current stage is the registration stage or the authentication stage.

Since the current stage is the registration stage, the user terminal device 200 can generate a reference image at operation S2540. For example, the user terminal device 200 can generate a reference by storing the acquired user iris and detected imaging condition information image.

At operation S2550, the user terminal device 200 may store the generated reference image in the database. That is, the user terminal device 200 may not store the iris code in the database, but may store the reference image itself in the database.

Since the current stage is the authentication stage, the user terminal device 200 may generate the first iris code at operation S2560. That is, the user terminal device 200 can generate the first iris code through the detected user iris.

At operation S2570, the user terminal device 200 may generate the second iris code of the reference image having the same imaging condition as the detected imaging condition. For example, the user terminal device 200 may search for a reference image having the same imaging condition as the detected imaging condition. The user terminal device 200 can use the searched reference image to generate the second iris code.

At operation S2580, the user terminal device 200 may compare the first iris code with the second iris code.

At operation S2590, the user terminal device 200 may perform user authentication based on the result of the comparison of operation S2580. For example, in response to determining that the similarity between the first iris code and the second iris code is greater than or equal to a preset value, the user terminal device 200 may perform user authentication. In response to determining that the similarity is less than the preset value, the user terminal device 200 may output a user authentication failure message.

As illustrated in FIG. 25, the reference image itself is registered in the reference image registration, so the amount of stored data can be reduced.

A method for generating a plurality of reference images by adjusting the image size of at least one reference image according to an embodiment of the present invention will be described with reference to FIGS. 26 to 28B.

Referring to FIG. 26, a flowchart is shown, in which at operation S2610, the user terminal device 200 as illustrated in FIG. 2 can acquire imaging condition information and iris images.

At operation S2620, the user terminal device 200 may detect the inner and outer iris boundaries.

At operation S2630, the user terminal device 200 may register the detected user iris as a reference image.

At operation S2640, the user terminal device 200 may determine whether there are one or more reference images.

Since there is more than one reference image at operation S2640, the user terminal device 200 may store multiple reference images at operation S2650.

Since the reference image is one at operation S2640, the user terminal device 200 may store a reference image at operation S2660.

At operation S2670, the user terminal device 200 may use the plurality of stored reference images or one stored reference image and perform image size adjustment according to imaging conditions.

For example, referring to FIGS. 27A and 27B, since the first reference image 2710 is acquired in a state in which the distance between the user terminal device 200 and the user is 20 cm, the user terminal device 200 can Perform image size adjustment on the reference image 2710 and obtain the second reference image 2720-1 (if installed on the user terminal The image created at a distance of 50 cm between the user 200 and the user), the third reference image 2720-2 (as formed at a distance of 40 cm between the user terminal device 200 and the user) Image), and a fourth reference image 2720-3 (such as an image formed at a distance of 30 cm between the user terminal device 200 and the user). The embodiments of the present invention have explained that the image is resized in units of 10 cm. Some distance units other than 10 cm (eg, 5 cm, etc.) can be used to resize the image.

Referring to FIGS. 28A and 28B, the first image 2810-1 and the second image 2810-1 and the second image according to the distance between the user terminal device 200 and the user (eg, distances of 50 cm, 40 cm, 30 cm, and 20 cm) are acquired. The second image 2810-2, the third image 2810-3, and the fourth image 2810-4, the user terminal device 200 can perform a size adjustment on the first reference image 2810-1, and generate a fifth image 2820-1 (such as (The image formed at a distance of 45 cm between the user terminal device 200 and the user). Similarly, the user terminal device 200 can perform a size adjustment on the second reference image 2810-2 and generate a sixth image 2820-2 (as shown in the 35 cm distance between the user terminal device 200 and the user) Into the image).

The embodiment of the present invention has explained that the imaging condition is the distance between the user terminal device 200 and the user. The user terminal device 200 can generate a plurality of reference images by performing size adjustment on the images according to the resolution.

Referring back to FIG. 26, at operation S2680, the user terminal device 200 may use the multiple generated reference images to generate multiple iris codes. In addition, at operation S2690, the user terminal device 200 may store the generated iris code in Database.

A method for generating a plurality of reference images by adding noise to at least one reference image according to an embodiment of the present invention will be described with reference to FIG. 29.

Referring to FIG. 29, a flowchart is shown, in which at operation S2910, the user terminal device 200 as illustrated in FIG. 2 can acquire imaging condition information and iris images.

At operation S2920, the user terminal device 200 may detect the inner and outer iris boundaries.

At operation S2930, the user terminal device may register the detected user iris as a reference image.

At operation S2940, the user terminal device 200 may determine whether there are one or more reference images.

Since there is more than one reference image at operation S2940, the user terminal device 200 may store multiple reference images at operation S2950.

Since the reference image is one at operation S2940, the user terminal device 200 may store a reference image at operation S2960.

At operation S2970, the user terminal device 200 may use the plurality of stored reference images or the stored one reference image and perform noise addition according to imaging conditions. For example, in response to acquiring the first reference image under the first illumination value, the user terminal device 200 may perform noise addition (eg, brightness adjustment) on the first reference image and acquire the second reference image ( For example, an image formed under the second illumination value), a third reference image (such as the image formed under the third illumination value), and a fourth reference image (such as the image formed under the fourth illumination value). In response to obtaining all The multiple reference images can also be obtained by adding noise to obtain more reference images.

The embodiment of the present invention has explained that the imaging condition is illumination. The plurality of reference images can be generated by adding noise added by IR light reflection.

At operation S2980, the user terminal device 200 may use the multiple generated reference images to generate multiple iris codes.

At operation S2990, the user terminal device 200 may store the generated iris code in the database.

As illustrated in FIGS. 26 to 29, the multiple reference images according to the imaging conditions can be acquired by image resizing or noise addition, so it is possible to perform more accurate iris recognition even if a smaller number of iris recognitions are performed Iris recognition.

The device according to various embodiments of the present invention may include a processor, a memory for storing and executing program data, a permanent storage (for example, a disk drive), a communication port for communicating with an external device, and a UI device (for example Touch panel, keys, buttons, etc.). Methods implemented with software modules or algorithms can be stored in non-transitory computer-readable recording media as computer-readable codes or program commands that can be executed on the processor. Non-transitory computer-readable recording media may include magnetic storage media (eg, ROM, RAM, floppy disks, hard disks, etc.) and optically readable media (compact disc (CD), digital versatile (digital versatile) disc, DVD, etc.). Non-transitory computer-readable media are distributed in computer systems coupled via a network, and computer-readable codes are stored and executed in a distributed manner. The media can be read by a computer, stored in memory, and executed by a processor.

Various embodiments of the invention can be configured with functional blocks and various processing operations Implement. The functional blocks can be implemented in various hardware and/or software configurations. For example, various embodiments of the invention may employ direct circuit configurations that can perform various functions under the control of one or more microprocessors or other control devices, such as memory, processor, logic, and lookup tables. Similarly, configuration components can be implemented by software programming or software components. Various embodiments of the present invention can be programming languages or scripting languages (such as C, C++, Java, or include various data structures, procedures, routines , And other combinational programming algorithms of stylized configuration). Functional aspects can be implemented in algorithms executed in one or more processors. For electronic environment setting, signal processing, and/or data processing, etc., various embodiments of the present invention may adopt the prior art. Terms such as "mechanism", "component", "unit" and "configuration" can be widely used, not limited to mechanical configuration and physical configuration. The term may refer to a series of routines of software combined with a processor and the like.

The specific implementations set forth in various embodiments of the present invention are illustrative, and the technical scope can be limited in any method. For clarity, the description of the electronic configuration, control system, and software, and other functional aspects of the system in the prior art may be omitted. The coupling of lines or coupling members between components illustrated in the drawings exemplarily illustrate functional coupling and/or physical or circuit coupling, and various functional couplings and physical couplings that can be replaced or added in actual devices , Or circuit coupling.

In the description made (specifically, in the scope of patent application), the term "said" and its similar indicating terms may include both singular and plural forms. The ranges set forth herein may include the individual values included in the range (unless The instructions below clearly indicate otherwise), and are intended to set forth the individual values that make up the range in the description made. Unless the order of operations constituting the method is clearly set forth or otherwise defined, the operations may be performed in an appropriate order. This is not limited to the order of description of the operation.

Various aspects of the invention can also be implemented as a non-transitory computer readable code on a recording medium. A non-transitory computer-readable recording medium is any data storage element that can store data that can be subsequently read by a computer system. Examples of non-transitory computer-readable recording media include read-only memory (ROM), random-access memory (RAM), CD-ROMs, magnetic tapes, floppy disks, and optical data storage devices. Non-transitory computer-readable recording media can also be distributed in network-coupled computer systems so that computer-readable codes are stored and executed in a distributed manner. In addition, the functional programs, codes, and code sections used to achieve the invention can be easily understood by programmers familiar with the technology to which the present invention pertains.

It should be noted in this regard that the various embodiments of the present invention as described above generally involve processing of input data and generation of output data to some extent. This input data processing and output data generation can be implemented by hardware or a combination of software and hardware. For example, specific electronic components may be employed in mobile elements or similar or related circuits for implementing the functions associated with the various embodiments of the present invention as described above. As another option, one or more processors operating according to stored instructions may implement the functions associated with various embodiments of the present invention as described above. If this is the case, it is within the scope of the invention that these instructions can be stored on one or more non-transitory processor-readable media. Examples of processor-readable media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage components. The processor-readable medium can also be distributed in network-coupled computer systems so that instructions are stored and executed in a distributed manner. In addition, the functional computer programs, instructions, and instruction sections used to achieve the invention can be easily understood by programmers familiar with the technology to which the present invention pertains.

Although the invention has been shown and described with reference to various embodiments of the invention, those of ordinary skill in the art will understand that the spirit and scope of the invention as defined by the scope of the accompanying patent application and its equivalents will not be deviated from Various forms and details can be changed.

200‧‧‧User terminal device

210‧‧‧Communication unit

220‧‧‧Imaging unit

230‧‧‧Image processor

240‧‧‧Monitor

250‧‧‧storage unit

260‧‧‧Input unit

270‧‧‧Controller

271‧‧‧ Random Access Memory (RAM)

272‧‧‧Read Only Memory (ROM)

273‧‧‧ graphics processor

274‧‧‧Main central processing unit (CPU)

275-1‧‧‧First interface

275-n‧‧‧nth interface

276‧‧‧Bus

Claims (20)

A user terminal device includes: a camera; a display; at least one processor; and at least one memory configured to store one or more computer programs executed by the at least one processor, the one or The multi-computer program includes instructions to the at least one processor for: capturing the user's image by shooting the user through the camera, and controlling the display to display the acquired image of the user , Detecting the user's eyes from the generated image, controlling the display to display a guide image at the position of the acquired image in which the detected user's eyes are located, and responding to the detected use The difference between the size of the iris included in the eye and the size of the guide image is within a preset range to perform iris recognition. The user terminal device as described in item 1 of the patent scope, wherein the controller is further used to: detect an iris boundary from the detected user's eyes; and estimate the iris diameter based on the detected iris boundary And controlling the display to display the guide image at a position corresponding to the user's eye and the iris based on the diameter of the detected user's eye and the iris. The user terminal device according to item 1 of the patent application, wherein the guide image is an image corresponding to the size of the user's eyes and the iris at a position matching the focal length of the imaging unit . The user terminal device of claim 3, wherein the controller is further used to: detect the iris of the user from the acquired image; and respond to the user's location The difference between the size of the detected iris and the size of the guide image corresponding to the iris is less than or equal to a preset value, by determining whether the detected iris of the user is consistent with a pre-registered iris And the iris recognition of the user is performed. The user terminal device of claim 3, wherein the controller is further used to: detect the iris of the user from the acquired image; and respond to the user's location The difference between the size of the detected iris and the size of the guide image corresponding to the iris is less than or equal to a preset value and a feedback message for guiding execution of the iris recognition is output. The user terminal device according to item 1 of the patent application scope further includes: an image processor, wherein the controller is further used to: control the image processor to process the acquired image so as not to execute the iris Confirm the pattern of the iris of the user during recognition, and control the display to display the processed acquired image. The user terminal device according to item 1 of the patent application scope, wherein the controller is further used to: perform the iris recognition of the user by: by normalizing the detected iris area into a rectangle Shape to obtain an iris code, mask the area of the rectangular shape covered by one of the eyebrows and eyelid in the normalized iris code, and divide the remaining area of the iris code except the masked area Compare with pre-registered iris code. The user terminal device as described in item 7 of the patent application scope, wherein the controller is further used to divide the iris code in response to the detected iris area being recognized as a pre-registered user Update the iris code of the pre-registered user in the remaining area other than the shaded area. The user terminal device according to item 1 of the patent application scope, wherein the guide image includes an eye guide image and an iris guide image, the eye guide image includes a shape corresponding to the user's eye, and the iris guide The image includes a shape corresponding to the iris of the user. The user terminal device according to item 1 of the scope of the patent application, wherein the eye-guided image and the iris-guided image have static sizes of the user's eyes and the user's iris, respectively, corresponding to The focal length of the camera matches the position of the user. The user terminal device as described in item 10 of the patent application range, wherein the guidance image further includes the iris line actually detected by the user. A method for recognizing a user's iris by a user terminal device, the method includes: acquiring an image of the user; displaying the acquired image; detecting a user's eyes from the acquired image; The obtained image in which the guide image is displayed at the position of the detected user's eye; and based on the difference between the size of the iris included in the detected user's eye and the size of the guide image at a preset Iris recognition is performed within the range. The method according to item 12 of the patent application scope, wherein the detecting the user's eyes includes: detecting an iris boundary from the detected user's eye, and estimating the iris based on the detected iris boundary Diameter, and wherein the displaying the guide image includes displaying the guide image at a position corresponding to the user's eye and the iris based on the diameter of the detected user's eye and the iris. The method according to item 12 of the patent application scope, wherein the guide image corresponds to the size of the user's eyes and the iris at a position matching the focal length of the imaging unit of the user terminal device Image. The method according to item 14 of the patent application scope, wherein the performing the iris recognition includes: detecting the iris of the user from the acquired image; and In response to the difference between the size of the detected iris of the user and the size of the guide image corresponding to the iris is less than or equal to a preset value, by judging the detected of the user Whether the iris is consistent with the pre-registered iris is to perform the iris recognition of the user. The method according to item 14 of the patent application scope, wherein the performing the iris recognition includes: detecting the iris of the user from the acquired image; and the location corresponding to the user The difference between the size of the detected iris and the size of the guide image corresponding to the iris is less than or equal to a preset value and a feedback message for guiding the user to perform the iris recognition is output. The method of claim 12 of the patent application, wherein the guide image includes an eye guide image and an iris guide image, the eye guide image includes a shape corresponding to the user's eye, and the iris guide image includes a Corresponding to the shape of the iris of the user. The method according to item 12 of the patent application scope, wherein the eye-guided image and the iris-guided image have static sizes of the user's eyes and the user's iris, respectively, corresponding to the camera Whose focal length matches the location of the user. The method according to item 12 of the patent application scope, wherein the guide image further includes the iris line actually detected by the user. A user terminal device, including: a camera; Display; at least one processor; and at least one memory configured to store one or more computer programs executed by the at least one processor, the one or more computer programs including the at least one processing Instructions for controlling the display to display a user interface (UI) to provide a payment screen, and controlling the display to display guide images on the user interface, including the shape corresponding to the user's iris , In response to a user instruction received through the user interface to make a payment, obtain the user's image through the camera, and control the display to display the user's image during the use The user interface has an area where the guide image is displayed, and performs iris recognition in response to the difference between the size of the iris included in the image of the user and the size of the guide image within a set range .

Images