KR20240052594A - A wearable device for detecting iris information and a method for controlling the same - Google Patents

Abstract

A wearable device that detects iris information and a control method thereof are provided. A wearable device according to an embodiment of the present document includes a display module, at least one camera, and at least one processor, wherein the at least one processor is configured to detect the iris of a user wearing the wearable device. 1 Control the display module so that an execution screen is displayed at a first position, determine whether the iris is detected by the at least one camera to enable performance of iris authentication, and determine whether the iris is detected by the at least one camera to enable performance of iris authentication. Based on the fact that the iris is not detected, the display module is controlled to move the first execution screen for detection of the iris to a second position and be displayed, and based on the iris being detected so that the iris authentication can be performed. Thus, it can be set to perform the iris authentication using the detected iris.

Description

Wearable device for detecting iris information and a method for controlling the same}

This document relates to a wearable device that detects iris information and a control method thereof.

The variety of services and additional functions provided through electronic devices, for example, portable electronic devices such as AR glasses, is gradually increasing. In order to increase the utility value of these electronic devices and satisfy the needs of various users, communication service providers or electronic device manufacturers are competitively developing electronic devices to provide various functions and differentiate themselves from other companies. Accordingly, various functions provided through electronic devices are becoming increasingly sophisticated.

Iris authentication functions or operations are provided through wearable devices as a means of verifying the user's identity (e.g., payment or account authentication) or providing various augmented reality/virtual reality services. However, a wearable device according to the prior art may not completely obtain iris information (e.g., iris image) of a user wearing the wearable device (e.g., to the extent that iris authentication can be performed). The current situation is that the function or operation to obtain it is not provided. For example, if the user is not looking in a specified direction (e.g., the direction in which the iris recognition sensor is located) while wearing the wearable device, the user's iris information may not be completely obtained. In this case, the wearable device according to the prior art only provides the user with information that the iris information has not been completely obtained, and in order to completely obtain the iris information, the location of the object viewed through the wearable device It does not provide functions or operations to change .

According to an embodiment of the present document, a screen for detecting the iris of a user wearing a wearable device (e.g., the first execution screen 510) is completely (e.g., : A wearable device may be provided that allows iris information (enough to perform iris authentication) to be acquired.

According to an embodiment of the present document, when iris information of a user wearing a wearable device is not completely acquired or when it is predicted that iris information will not be completely obtained, a screen for iris detection (e.g., first execution A control method of a wearable device may be provided that allows complete (e.g., sufficient to enable iris authentication) iris information to be obtained by adaptively/actively determining the position at which the screen 510 is shown to the user. there is.

A wearable device according to an embodiment of the present document includes a display module, at least one camera, and at least one processor, wherein the at least one processor is configured to detect the iris of a user wearing the wearable device. 1 Control the display module so that an execution screen is displayed at a first location (e.g., a first location in the real world or a first location in virtual reality), and the iris is connected to the at least one camera to enable iris authentication. Based on whether the iris is detected to enable the iris authentication to be performed, the first execution screen for detection of the iris is displayed at a second location (e.g., a second location in the real world). or control the display module to be moved to a second location in virtual reality and set to perform the iris authentication using the detected iris based on the iris being detected to enable the iris authentication. It can be.

A method of controlling a wearable device according to an embodiment of the present document includes controlling a display module of the wearable device so that a first execution screen for detecting the iris of a user wearing the wearable device is displayed at a first position. and, determining whether the iris is detected by at least one camera of the wearable device to enable iris authentication, based on whether the iris is not detected to enable iris authentication, An operation of controlling the display module so that the first execution screen for iris detection is moved to a second position and displayed, and based on the iris being detected to enable the iris authentication, using the detected iris Thus, the operation of performing the iris authentication may be included.

According to an embodiment of the present document, a screen for detecting the iris of a user wearing a wearable device (e.g., the first execution screen 510) is completely (e.g., : A wearable device that allows iris information (enough to perform iris authentication) to be obtained may be provided.

Effects according to various embodiments of this document are not limited to the effects described above, and it is clear to those skilled in the art that various effects are inherent in this document.

1 is a block diagram of an electronic device in a network environment, according to various embodiments of this document.
FIG. 2A is a perspective view of an electronic device (eg, a wearable device) according to an embodiment of the present document.
FIG. 2B is a perspective view for explaining the internal configuration of an electronic device (eg, a wearable device) according to an embodiment of the present document.
FIG. 2C is an exploded perspective view of an electronic device (eg, a wearable device) according to an embodiment of the present document.
3A and 3B are diagrams showing the front and back of a wearable device according to an embodiment of the present document.
FIG. 4 illustrates a function or operation in which a wearable device according to an embodiment of this document maintains or changes the position where an execution screen for iris detection (e.g., iris authentication) is displayed based on whether or not the iris is completely detected. This is an example drawing for the following.
FIG. 5A is an example diagram illustrating a state in which a virtual object for iris detection (e.g., first execution screen) is shown in a location where iris detection is not completely performed, according to an embodiment of the present document.
FIGS. 5B to 5D are example diagrams for explaining the iris detection rate according to the gaze of a user wearing a wearable device according to an embodiment of the present document.
FIG. 5E shows a function for changing the position where a virtual object (e.g., first execution screen) for iris detection is shown in order to completely obtain iris information of a user wearing a wearable device according to an embodiment of this document. Or, it is an example drawing to explain the operation.
FIG. 5F is an example diagram illustrating a case where a virtual object for iris detection (eg, first execution screen 510) is an authentication screen for performing payment, according to an embodiment of this document.
FIG. 5G is an example diagram illustrating a case where a virtual object for iris detection (e.g., first execution screen 510) is an execution screen (e.g., login screen) of a specific application, according to an embodiment of this document. am.
Figure 6 shows that when additional iris information is needed for iris detection according to an embodiment of this document, the position where the virtual object for iris detection (e.g., the first execution screen) is displayed is changed to obtain additional iris information. This is an example drawing to explain a function or operation.
FIGS. 7A and 7B are example diagrams for explaining the function or operation described in FIG. 6 from a user interface perspective.
FIG. 7C illustrates a function or operation in which a wearable device according to an embodiment of the present document provides information about the location where the iris is detected to enable iris authentication at a previous time point relative to the current time point. This is an example drawing for:
FIG. 8 shows that a wearable device according to an embodiment of the present document is used for iris detection at a designated location for iris detection, which is different from the display location of the execution screen (e.g., second execution screen) of the application currently being shown. This is an example diagram for explaining a function or operation of determining a display position of a virtual object (eg, first execution screen) so that the virtual object (eg, first execution screen) is displayed.
FIGS. 9A and 9B are example diagrams for explaining the function or operation described in FIG. 8 from a user interface perspective.
9C and 9D show a location designated by the wearable device according to an embodiment of the present document by changing the execution screen (e.g., the second execution screen) of the application currently being displayed to another screen (e.g., the first execution screen). These are example drawings to explain the function or operation of controlling the display module to show a different screen.
FIG. 10 is an example diagram illustrating a function or operation of a wearable device according to an embodiment of this document to register (e.g., store) iris information and user's gaze information by matching them with each other.
11 shows a screen for iris registration (e.g., a third execution screen) in order for the wearable device according to an embodiment of this document to register (e.g., store) iris information and user's gaze information by matching each other. This is an example diagram to explain a function or operation of controlling a display module to be viewed from a plurality of locations in the world from a user interface perspective.
Figure 12 shows that a wearable device according to an embodiment of this document determines a position where a virtual object (e.g., first execution screen) for iris detection is displayed based on the user's gaze information, and performs iris detection at the determined position. A virtual object (e.g. the first execution screen) may be displayed? This is an example diagram to explain the function or operation of controlling the display module.
FIG. 13 is an example diagram illustrating a function or operation of a wearable device according to an embodiment of the present document to provide a specified visual effect for the first execution screen depending on whether iris information is completely detected.
FIGS. 14A and 14B are example diagrams for explaining the function or operation described in FIG. 13 from a user interface perspective.
FIG. 15 is an example diagram illustrating a function or operation of a wearable device according to an embodiment of the present document to control a display module to display a designated notification around the first execution screen depending on whether iris information is completely detected.
FIG. 16 is an example diagram for explaining the function or operation described in FIG. 15 from a user interface perspective.
FIG. 17 shows a comparison result of the wearable device according to an embodiment of this document between the brightness of a virtual object (e.g., first execution screen) for iris detection and the brightness of the execution screen that was being displayed at the time of registering iris information. Based on this, this is an example diagram for explaining a function or operation of adjusting the brightness of a virtual object (eg, first execution screen) for iris detection.
FIG. 18 is an example diagram for explaining the function or operation described in FIG. 17 from a user interface perspective.
FIG. 19 illustrates a function or operation in which a wearable device according to an embodiment of this document adjusts the brightness of a light-emitting module based on a comparison result between the current ambient illuminance and the ambient illuminance at the time of registration of iris information. This is an example drawing for:

1 is a block diagram of an electronic device 101 in a network environment 100, according to various embodiments.

Referring to FIG. 1, in the network environment 100, the electronic device 101 communicates with the electronic device 102 through a first network 198 (e.g., a short-range wireless communication network) or a second network 199. It is possible to communicate with at least one of the electronic device 104 or the server 108 through (e.g., a long-distance wireless communication network). According to one embodiment, the electronic device 101 may communicate with the electronic device 104 through the server 108. According to one embodiment, the electronic device 101 includes a processor 120, a memory 130, an input module 150, an audio output module 155, a display module 160, an audio module 170, and a sensor module ( 176), interface 177, connection terminal 178, haptic module 179, camera module 180, power management module 188, battery 189, communication module 190, subscriber identification module 196 , or may include an antenna module 197. In some embodiments, at least one of these components (eg, the connection terminal 178) may be omitted or one or more other components may be added to the electronic device 101. In some embodiments, some of these components (e.g., sensor module 176, camera module 180, or antenna module 197) are integrated into one component (e.g., display module 160). It can be.

The processor 120, for example, executes software (e.g., program 140) to operate at least one other component (e.g., hardware or software component) of the electronic device 101 connected to the processor 120. It can be controlled and various data processing or operations can be performed. According to one embodiment, as at least part of data processing or computation, the processor 120 stores commands or data received from another component (e.g., sensor module 176 or communication module 190) in volatile memory 132. The commands or data stored in the volatile memory 132 can be processed, and the resulting data can be stored in the non-volatile memory 134. According to one embodiment, the processor 120 includes the main processor 121 (e.g., a central processing unit or an application processor) or an auxiliary processor 123 that can operate independently or together (e.g., a graphics processing unit, a neural network processing unit ( It may include a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, if the electronic device 101 includes a main processor 121 and a secondary processor 123, the secondary processor 123 may be set to use lower power than the main processor 121 or be specialized for a designated function. You can. The auxiliary processor 123 may be implemented separately from the main processor 121 or as part of it.

The auxiliary processor 123 may, for example, act on behalf of the main processor 121 while the main processor 121 is in an inactive (e.g., sleep) state, or while the main processor 121 is in an active (e.g., application execution) state. ), together with the main processor 121, at least one of the components of the electronic device 101 (e.g., the display module 160, the sensor module 176, or the communication module 190) At least some of the functions or states related to can be controlled. According to one embodiment, co-processor 123 (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module 180 or communication module 190). there is. According to one embodiment, the auxiliary processor 123 (eg, neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. Artificial intelligence models can be created through machine learning. For example, such learning may be performed in the electronic device 101 itself on which the artificial intelligence model is performed, or may be performed through a separate server (e.g., server 108). Learning algorithms may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but It is not limited. An artificial intelligence model may include multiple artificial neural network layers. Artificial neural networks include deep neural network (DNN), convolutional neural network (CNN), recurrent neural network (RNN), restricted boltzmann machine (RBM), belief deep network (DBN), bidirectional recurrent deep neural network (BRDNN), It may be one of deep Q-networks or a combination of two or more of the above, but is not limited to the examples described above. In addition to hardware structures, artificial intelligence models may additionally or alternatively include software structures.

The memory 130 may store various data used by at least one component (eg, the processor 120 or the sensor module 176) of the electronic device 101. Data may include, for example, input data or output data for software (e.g., program 140) and instructions related thereto. Memory 130 may include volatile memory 132 or non-volatile memory 134.

The program 140 may be stored as software in the memory 130 and may include, for example, an operating system 142, middleware 144, or application 146.

The input module 150 may receive commands or data to be used in a component of the electronic device 101 (e.g., the processor 120) from outside the electronic device 101 (e.g., a user). The input module 150 may include, for example, a microphone, mouse, keyboard, keys (eg, buttons), or digital pen (eg, stylus pen).

The sound output module 155 may output sound signals to the outside of the electronic device 101. The sound output module 155 may include, for example, a speaker or a receiver. Speakers can be used for general purposes such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part of it.

The display module 160 can visually provide information to the outside of the electronic device 101 (eg, a user). The display module 160 may include, for example, a display, a hologram device, or a projector, and a control circuit for controlling the device. According to one embodiment, the display module 160 may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of force generated by the touch.

The audio module 170 can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module 170 acquires sound through the input module 150, the sound output module 155, or an external electronic device (e.g., directly or wirelessly connected to the electronic device 101). Sound may be output through the electronic device 102 (e.g., speaker or headphone).

The sensor module 176 detects the operating state (e.g., power or temperature) of the electronic device 101 or the external environmental state (e.g., user state) and generates an electrical signal or data value corresponding to the detected state. can do. According to one embodiment, the sensor module 176 includes, for example, a gesture sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, It may include a temperature sensor, humidity sensor, or light sensor.

The interface 177 may support one or more designated protocols that can be used to connect the electronic device 101 directly or wirelessly with an external electronic device (eg, the electronic device 102). According to one embodiment, the interface 177 may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

The connection terminal 178 may include a connector through which the electronic device 101 can be physically connected to an external electronic device (eg, the electronic device 102). According to one embodiment, the connection terminal 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (eg, a headphone connector).

The haptic module 179 can convert electrical signals into mechanical stimulation (e.g., vibration or movement) or electrical stimulation that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

The camera module 180 can capture still images and moving images. According to one embodiment, the camera module 180 may include one or more lenses, image sensors, image signal processors, or flashes.

The power management module 188 can manage power supplied to the electronic device 101. According to one embodiment, the power management module 188 may be implemented as at least a part of, for example, a power management integrated circuit (PMIC).

Battery 189 may supply power to at least one component of electronic device 101. According to one embodiment, the battery 189 may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

Communication module 190 is configured to provide a direct (e.g., wired) communication channel or wireless communication channel between electronic device 101 and an external electronic device (e.g., electronic device 102, electronic device 104, or server 108). It can support establishment and communication through established communication channels. The communication module 190 operates independently of the processor 120 (e.g., an application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module 190 is a wireless communication module 192 (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (e.g. : LAN (local area network) communication module, or power line communication module) may be included. Among these communication modules, the corresponding communication module is a first network 198 (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network 199 (e.g., legacy It may communicate with an external electronic device 104 through a telecommunication network such as a cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., LAN or WAN). These various types of communication modules may be integrated into one component (e.g., a single chip) or may be implemented as a plurality of separate components (e.g., multiple chips). The wireless communication module 192 uses subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module 196 within a communication network such as the first network 198 or the second network 199. The electronic device 101 can be confirmed or authenticated.

The wireless communication module 192 may support 5G networks after 4G networks and next-generation communication technologies, for example, NR access technology (new radio access technology). NR access technology provides high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and access to multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low latency). -latency communications)) can be supported. The wireless communication module 192 may support a high frequency band (eg, mmWave band), for example, to achieve a high data rate. The wireless communication module 192 uses various technologies to secure performance in high frequency bands, for example, beamforming, massive array multiple-input and multiple-output (MIMO), and full-dimensional multiplexing. It can support technologies such as input/output (FD-MIMO: full dimensional MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module 192 may support various requirements specified in the electronic device 101, an external electronic device (e.g., electronic device 104), or a network system (e.g., second network 199). According to one embodiment, the wireless communication module 192 supports peak data rate (e.g., 20 Gbps or more) for realizing eMBB, loss coverage (e.g., 164 dB or less) for realizing mmTC, or U-plane latency (e.g., 164 dB or less) for realizing URLLC. Example: Downlink (DL) and uplink (UL) each of 0.5 ms or less, or round trip 1 ms or less) can be supported.

The antenna module 197 may transmit or receive signals or power to or from the outside (eg, an external electronic device). According to one embodiment, the antenna module 197 may include an antenna including a radiator made of a conductor or a conductive pattern formed on a substrate (eg, PCB). According to one embodiment, the antenna module 197 may include a plurality of antennas (eg, an array antenna). In this case, at least one antenna suitable for the communication method used in the communication network, such as the first network 198 or the second network 199, is connected to the plurality of antennas by, for example, the communication module 190. can be selected. Signals or power may be transmitted or received between the communication module 190 and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (eg, radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module 197.

According to various embodiments, the antenna module 197 may form a mmWave antenna module. According to one embodiment, a mmWave antenna module includes a printed circuit board, an RFIC disposed on or adjacent to a first side (e.g., bottom side) of the printed circuit board and capable of supporting a designated high frequency band (e.g., mmWave band), And a plurality of antennas (e.g., array antennas) disposed on or adjacent to the second side (e.g., top or side) of the printed circuit board and capable of transmitting or receiving signals in the designated high frequency band. can do.

At least some of the components are connected to each other through a communication method between peripheral devices (e.g., bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI)) and signal ( (e.g. commands or data) can be exchanged with each other.

According to one embodiment, commands or data may be transmitted or received between the electronic device 101 and the external electronic device 104 through the server 108 connected to the second network 199. Each of the external electronic devices 102 or 104 may be of the same or different type as the electronic device 101. According to one embodiment, all or part of the operations performed in the electronic device 101 may be executed in one or more of the external electronic devices 102, 104, or 108. For example, when the electronic device 101 needs to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device 101 may perform the function or service instead of executing the function or service on its own. Alternatively, or additionally, one or more external electronic devices may be requested to perform at least part of the function or service. One or more external electronic devices that have received the request may execute at least part of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device 101. The electronic device 101 may process the result as is or additionally and provide it as at least part of a response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology can be used. The electronic device 101 may provide an ultra-low latency service using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device 104 may include an Internet of Things (IoT) device. Server 108 may be an intelligent server using machine learning and/or neural networks. According to one embodiment, the external electronic device 104 or server 108 may be included in the second network 199. The electronic device 101 may be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

FIG. 2A is a perspective view of a wearable device 200 according to an embodiment of the present disclosure.

Referring to FIG. 2A, the wearable device 200 is an electronic device in the form of glasses, and a user can visually perceive surrounding objects or environments while wearing the wearable device 200. For example, the wearable device 200 may be a head mounting device (HMD) or smart glasses that can provide images directly in front of the user's eyes. The configuration of the wearable device 200 of FIG. 2A may be completely or partially the same as the configuration of the electronic device 101 of FIG. 1 .

According to an embodiment of this document, the wearable device 200 may include a housing 210 that forms the exterior of the wearable device 200. The housing 210 may provide a space where components of the wearable device 200 can be placed. For example, the housing 210 may include a lens frame 202 and at least one wearing member 203.

According to an embodiment of this document, the wearable device 200 may include a display member 201 that can provide visual information to the user. For example, the display member 201 may include a module equipped with a lens or a second window member, a display, a waveguide, and/or a touch circuit. According to one embodiment, the display member 201 may be formed to be transparent or translucent. According to one embodiment, the display member 201 may include a translucent glass material or a window member whose light transmittance can be adjusted by adjusting the coloring density. According to one embodiment, the display members 201 are provided as a pair and can be respectively disposed on the user's left and right eyes while the wearable device 200 is worn on the user's body.

According to an embodiment of the present document, the lens frame 202 may accommodate at least a portion of the display member 201. For example, the lens frame 202 may surround at least a portion of the edge of the display member 201. According to one embodiment, the lens frame 202 may position at least one of the display members 201 to correspond to the user's eye. According to one embodiment, the lens frame 202 may be a rim of a general eyeglass structure. According to one embodiment, the lens frame 202 may include at least one closed curve surrounding the display member 201.

According to one embodiment of the present document, the wearing member 203 may extend from the lens frame 202. For example, the wearing member 203 extends from an end of the lens frame 202 and, together with the lens frame 202, can be supported or positioned on the user's body (eg, ears). According to one embodiment, the wearing member 203 may be rotatably coupled to the lens frame 202 through the hinge structure 229. According to one embodiment, the wearing member 203 may include an inner side 231c configured to face the user's body and an outer side 231d opposite the inner side.

According to an embodiment of the present document, the wearable device 200 may include a hinge structure 229 configured to fold the wearing member 203 with respect to the lens frame 202. The hinge structure 229 may be disposed between the lens frame 202 and the wearing member 203. When not wearing the wearable device 200, the user can fold the wearing member 203 so that a portion overlaps the lens frame 202 and carry or store it.

FIG. 2B is a perspective view for explaining the internal configuration of the wearable device 200 according to an embodiment of the present disclosure. FIG. 2C is an exploded perspective view of the wearable device 200 according to an embodiment of the present disclosure.

Referring to FIGS. 2B and 2C , the wearable device 200 includes components (e.g., at least one circuit board 241 (e.g., printed circuit board (PCB)), printed board assembly (PBA)) accommodated in the housing 210. , flexible PCB (FPCB) or rigid-flexible PCB (RFPCB)), at least one battery 243, at least one speaker module 245, at least one power delivery structure 246, and/or camera module 250 ))) may be included. The configuration of the housing 210 in FIG. 2B may be the same in whole or in part as the configuration of the display member 201, lens frame 202, wearing member 203, and hinge structure 229 in FIG. 2A.

According to an embodiment of the present document, the wearable device 200 uses a camera module 250 (e.g., the camera module 180 in FIG. 1) to capture the direction in which the user is looking or the wearable device 200 is pointing ( Example: acquire and/or recognize a visual image of an object or environment in the -Y direction, and acquire and/or recognize an external electronic device through a network (e.g., the first network 198 or the second network 199 in FIG. 1) Information about an object or environment may be provided from (e.g., the electronic devices 102 and 104 or the server 108 of FIG. 1). In another embodiment, the wearable device 200 may provide information about the provided object or environment to the user in audio or visual form. The wearable device 200 may provide information about the provided object or environment to the user through the display member 201 in a visual form using a display module (e.g., the display module 160 of FIG. 1). For example, the wearable device 200 can implement augmented reality by implementing information about objects or the environment in a visual form and combining it with actual images of the user's surrounding environment.

According to an embodiment of the present document, the display member 201 has a first surface F1 facing the direction in which external light is incident (e.g. -Y direction) and a direction opposite to the first surface F1 (e.g. the -Y direction). : +Y direction) may include a second surface (F2) facing. When the user is wearing the wearable device 200, at least a portion of the light or image incident through the first surface F1 is directed to the second surface of the display member 201 disposed to face the left and/or right eye of the user. It may pass through the face F2 and enter the user's left and/or right eye.

According to an embodiment of this document, the lens frame 202 may include at least two or more frames. For example, the lens frame 202 may include a first frame 202a and a second frame 202b. According to one embodiment, when a user wears the wearable device 200, the first frame 202a is a frame that faces the user's face, and the second frame 202b is attached to the first frame 202a. It may be a part of the lens frame 202 spaced apart in the user's gaze direction (eg, -Y direction).

According to an embodiment of this document, the light output module 211 may provide images and/or videos to the user. For example, the light output module 211 includes a display panel (not shown) capable of outputting an image, and a lens (not shown) that corresponds to the user's eyes and guides the image to the display member 201. can do. For example, a user may obtain an image output from the display panel of the light output module 211 through the lens of the light output module 211. According to an embodiment of this document, the light output module 211 may include a device configured to display various information. For example, the light output module 211 may be a liquid crystal display (LCD), a digital mirror device (DMD), a liquid crystal on silicon (LCoS), or an organic light emitting diode. It may include at least one of an organic light emitting diode (OLED) or a micro LED (micro light emitting diode, micro LED). According to one embodiment, when the light output module 211 and/or the display member 201 includes one of LCD, DMD, or LCoS, the wearable device 200 includes the light output module 211 and/or It may include a light source that radiates light to the display area of the display member 201. According to another embodiment, when the light output module 211 and/or the display member 201 includes one of OLED or micro LED, the wearable device 200 does not include a separate light source and provides a virtual image to the user. can be provided.

According to an embodiment of the present document, at least a portion of the light output module 211 may be disposed within the housing 210. For example, the light output module 211 may be disposed on the wearing member 203 or the lens frame 202 to correspond to the user's right eye and left eye, respectively. According to one embodiment, the light output module 211 is connected to the display member 201 and can provide an image to the user through the display member 201.

According to an embodiment of this document, the circuit board 241 may include components for driving the wearable device 200. For example, the circuit board 241 may include at least one integrated circuit chip, such as the processor 120, memory 130, power management module 188, or communication module of FIG. 1. At least one of (190) may be provided in the integrated circuit chip. According to one embodiment, the circuit board 241 may be disposed within the wearing member 203 of the housing 210. According to one embodiment, the circuit board 241 may be electrically connected to the battery 243 through the power transmission structure 246. According to one embodiment, the circuit board 241 is connected to the flexible printed circuit board 205, and electronic components of the electronic device (e.g., the optical output module 211, According to one embodiment, the circuit board 241 may be a circuit board including an interposer.

According to various embodiments, flexible printed circuit board 205 may extend from circuit board 241 across hinge structure 229 and into the interior of lens frame 202. It may be disposed at least partially around the display member 201.

According to an embodiment of the present document, the battery 243 (e.g., battery 189 in FIG. 1) is a component of the wearable device 200 (e.g., an optical output module 211, a circuit board 241, a speaker module). 245, the microphone module 247, and the camera module 250), and may supply power to components of the wearable device 200.

According to an embodiment of the present document, at least a portion of the battery 243 may be disposed on the wearing member 203. According to one embodiment, the battery 243 may be disposed at the ends 203a and 203b of the wearing member 203. For example, the battery 243 may include a first battery 243a disposed at the first end 203a of the wearing member 203 and a second battery 243b disposed at the second end 203b. there is.

According to various embodiments, the speaker module 245 (eg, the audio module 170 or the sound output module 155 of FIG. 1) may convert an electrical signal into sound. At least a portion of the speaker module 245 may be disposed within the wearing member 203 of the housing 210. According to one embodiment, the speaker module 245 may be located within the wearing member 203 to correspond to the user's ears. For example, the speaker module 245 may be disposed between the circuit board 241 and the battery 243.

According to an embodiment of this document, the power transmission structure 246 may transmit power from the battery 243 to an electronic component (eg, the optical output module 211) of the wearable device 200. For example, the power transmission structure 246 is electrically connected to the battery 243 and/or the circuit board 241, and the circuit board 241 outputs power received through the power transmission structure 246 as light. It can be transmitted to module 211. According to one embodiment, the power transmission structure 246 may be connected to the circuit board 241 through the speaker module 245. For example, when the wearable device 200 is viewed from the side (eg, Z-axis direction), the power transmission structure 246 may at least partially overlap the speaker module 245.

According to an embodiment of this document, the power transmission structure 246 may be a configuration capable of transmitting power. For example, power delivery structure 246 may include a flexible printed circuit board or wire. For example, a wire may include a plurality of cables (not shown). In various embodiments, the shape of the power transmission structure 246 may be varied in consideration of the number and/or type of cables.

According to an embodiment of the present document, the microphone module 247 (e.g., the input module 150 and/or the audio module 170 in FIG. 1) may convert sound into an electrical signal. According to one embodiment, the microphone module 247 may be disposed on at least a portion of the lens frame 202. For example, at least one microphone module 247 may be disposed at the bottom (eg, towards the -X axis) and/or top (eg, towards the X axis) of the wearable device 200. According to an embodiment of this document, the wearable device 200 can recognize the user's voice more clearly using voice information (e.g., sound) acquired from at least one microphone module 247. For example, the wearable device 200 may distinguish between voice information and surrounding noise based on the acquired voice information and/or additional information (eg, low-frequency vibration of the user's skin and bones). For example, the wearable device 200 can clearly recognize the user's voice and perform a function to reduce surrounding noise (eg, noise canceling).

According to an embodiment of this document, the camera module 250 can capture still images and/or moving images. The camera module 250 may include at least one of a lens, at least one image sensor, an image signal processor, or a flash. According to one embodiment, the camera module 250 may be disposed within the lens frame 202 and around the display member 201.

According to an embodiment of this document, the camera module 250 may include at least one first camera module 251. According to one embodiment, the first camera module 251 may photograph the user's eye (eg, pupil) or gaze trajectory. For example, the first camera module 251 may photograph a reflection pattern of light emitted by the light emitting unit to the user's eyes. For example, the light emitting unit 330 may emit light in the infrared band for tracking the gaze trajectory using the first camera module 251. For example, the light emitting unit 330 may include an IR LED. According to one embodiment, the processor (e.g., processor 120 in FIG. 1) may adjust the position of the virtual image projected on the display member 201 so that it corresponds to the direction in which the user's pupils are gazing. According to one embodiment, the first camera module 251 may include a global shutter (GS) type camera, and a plurality of first camera modules 251 of the same standard and performance are used to capture the user's eyes or The trajectory of gaze can be tracked.

According to an embodiment of the present document, the first camera module 251 periodically sends information (e.g., trajectory information) related to the trajectory of the user's eyes or gaze to a processor (e.g., the processor 120 in FIG. 1). Alternatively, it can be transmitted aperiodically. According to another embodiment, when the first camera module 251 detects that the user's gaze has changed based on the trajectory information (e.g., the eyes move more than a reference value while the head is not moving), the first camera module 251 processes the trajectory information into a processor. It can be sent to .

According to an embodiment of this document, the first camera module 251 may detect (eg, recognize) the iris of the user 500 wearing the wearable device 200. The wearable device 200 according to an embodiment of this document may separate the iris image from the user's eye image acquired using the first camera module 251 (eg, an infrared sensor). The wearable device 200 according to an embodiment of this document can flatten the separated iris image and convert it into a coordinate system. The wearable device 200 according to an embodiment of this document may detect the iris of the user 500 (e.g., recognize the iris characteristics of the user 500) by coding the iris image converted into a coordinate system. The wearable device 200 according to an embodiment of this document may include an infrared (IR) light emitting module to acquire an iris image of the user 500. The infrared light emitting module according to an embodiment of this document may be placed around the first camera module 251, but is not limited thereto. The wearable device 200 (e.g., the first camera module 251) according to an embodiment of this document receives the infrared light output from the infrared light-emitting module and reflects from the user's eyeball, thereby creating an image of the user's eyeball. And/or an iris image may be acquired.

According to an embodiment of this document, the camera module 250 may include a second camera module 253. According to one embodiment, the second camera module 253 can capture external images. According to one embodiment, the second camera module 253 may be a global shutter type camera or a rolling shutter (RS) type camera. According to one embodiment, the second camera module 253 may capture an external image through the second optical hole 223 formed in the second frame 202b. For example, the second camera module 253 may include a high-resolution color camera and may be a high resolution (HR) or photo video (PV) camera. Additionally, the second camera module 253 may provide an auto focus function (AF) and an optical image stabilizer (OIS) function. The second camera module 253 according to an embodiment of this document may include one camera or a plurality of cameras.

According to an embodiment of this document, the wearable device 200 may include a flash (not shown) located adjacent to the second camera module 253. For example, a flash (not shown) may provide light to increase brightness (e.g., illuminance) around the wearable device 200 when acquiring an external image of the second camera module 253, in a dark environment, Difficulty in obtaining images due to mixing of various light sources and/or reflection of light can be reduced.

According to an embodiment of this document, the camera module 250 may include at least one third camera module 255. According to one embodiment, the third camera module 255 may capture the user's movements through the first optical hole 221 formed in the lens frame 202. For example, the third camera module 255 may capture a user's gestures (eg, hand movements). The third camera module 255 and/or the first optical hole 221 are located at both ends of the lens frame 202 (e.g., the second frame 202b), for example, in the X direction. (For example, it may be disposed at both ends of the second frame 202b). According to one embodiment, the third camera module 255 may be a global shutter (GS) type camera. For example, the third camera module 255 is a camera that supports 3DoF (degrees of freedom) or 6DoF, which can provide 360-degree spatial (e.g., omnidirectional), position recognition, and/or movement recognition. You can. According to one embodiment, the third camera module 255 is a stereo camera that uses a plurality of global shutter cameras of the same standard and performance to perform a movement path tracking function (simultaneous localization and mapping, SLAM) and user movement recognition. It can perform its function. According to one embodiment, the third camera module 255 may include an infrared (IR) camera (eg, a time of flight (TOF) camera, or a structured light camera). For example, the IR camera may be operated as at least a part of a sensor module (eg, sensor module 176 in FIG. 1) to detect the distance to the subject.

According to one embodiment, at least one of the first camera module 251 or the third camera module 255 may be replaced with a sensor module (e.g., the sensor module 176 of FIG. 1) (e.g., a Lidar sensor). . For example, the sensor module may include at least one of a vertical cavity surface emitting laser (VCSEL), an infrared sensor, and/or a photodiode. For example, the photo diode may include a positive intrinsic negative (PIN) photo diode, or an avalanche photo diode (APD). The photo diode may be referred to as a photo detector or photo sensor.

According to one embodiment, at least one of the first camera module 251, the second camera module 253, or the third camera module 255 may include a plurality of camera modules (not shown). For example, the second camera module 253 consists of a plurality of lenses (e.g., wide-angle and telephoto lenses) and image sensors and is disposed on one side (e.g., the side facing the -Y axis) of the wearable device 200. It can be. For example, the wearable device 200 may include a plurality of camera modules, each with different properties (e.g., angle of view) or function, and change the angle of view of the camera modules based on the user's selection and/or trajectory information. You can control it to do so. For example, at least one of the plurality of camera modules may be a wide-angle camera, and at least another one may be a telephoto camera.

According to various embodiments, the processor (e.g., processor 120 of FIG. 1) acquires information using at least one of a gesture sensor, a gyro sensor, or an acceleration sensor of a sensor module (e.g., sensor module 176 of FIG. 1). Movement of the wearable device 200 using information on the wearable device 200 and the user's motion (e.g., approach of the user's body to the wearable device 200) obtained using the first camera module 251. And/or the user's movement may be determined. According to one embodiment, in addition to the described sensor, the wearable device 200 includes a magnetic (geomagnetic) sensor capable of measuring orientation using a magnetic field and magnetoelectric force, and/or motion information (e.g., movement) using the strength of the magnetic field. It may include a Hall sensor capable of acquiring direction or movement distance. For example, the processor may determine the movement of the wearable device 200 and/or the user's movement based on information obtained from a magnetic (geomagnetic) sensor and/or a hall sensor.

According to various embodiments (not shown), the wearable device 200 may perform an input function (eg, touch and/or pressure sensing function) that allows interaction with the user. For example, components configured to perform touch and/or pressure sensing functions (e.g., touch sensors and/or pressure sensors) may be disposed on at least a portion of the wearing member 203 . The wearable device 200 can control a virtual image output through the display member 201 based on information acquired through the components. For example, sensors related to touch and/or pressure sensing functions may be resistive type, capacitive type, electro-magnetic type (EM), or optical type. ) can be configured in various ways, such as: According to one embodiment, components configured to perform the touch and/or pressure sensing function may be completely or partially identical to the configuration of the input module 150 of FIG. 1 .

According to an embodiment of the present document, the wearable device 200 may include a reinforcing member 260 disposed in the inner space of the lens frame 202 and formed to have a higher rigidity than the rigidity of the lens frame 202. there is.

According to an embodiment of this document, the wearable device 200 may include a lens structure 270. The lens structure 270 may refract at least a portion of light. For example, the lens structure 270 may be a prescription lens with a predetermined refractive power. According to one embodiment, the lens structure 270 may be disposed behind the second window member of the display member 201 (eg, in the +Y direction). For example, the lens structure 270 may be positioned between the display member 201 and the user's eyes. For example, the lens structure 270 may face one side of the display member.

According to one embodiment of the present document, the housing 210 may include a hinge cover 227 that can conceal a portion of the hinge structure 229. Another part of the hinge structure 229 may be accommodated or hidden between the inner case 231 and the outer case 233, which will be described later.

According to an embodiment of the present document, the wearing member 203 may include an inner case 231 and an outer case 233. The inner case 231 is, for example, a case configured to face or directly contact the user's body, and may be made of a material with low thermal conductivity, for example, synthetic resin. According to one embodiment, the inner case 231 may include an inner side (eg, inner side 231c in FIG. 2A) that faces the user's body. The outer case 233 includes, for example, a material capable of at least partially transferring heat (eg, a metal material) and may be coupled to face the inner case 231 . According to one embodiment, the outer case 233 may include an outer side opposite to the inner side 231c (eg, the outer side 231d in FIG. 2A). In one embodiment, at least one of the circuit board 241 or the speaker module 245 may be accommodated in a space separate from the battery 243 within the wearing member 203. In the illustrated embodiment, the inner case 231 may include a first case 231a containing a circuit board 241 or a speaker module 245, and a second case 231b containing the battery 243. The outer case 233 may include a third case 233a coupled to face the first case 231a and a fourth case 233b coupled to face the second case 231b. . For example, the first case 231a and the third case 233a are combined (hereinafter referred to as 'first case portions 231a, 233a') to accommodate the circuit board 241 and/or the speaker module 245. The battery 243 can be accommodated by combining the second case 231b and the fourth case 233b (hereinafter referred to as 'second case parts 231b, 233b').

According to various embodiments, the first case portions 231a and 233a are rotatably coupled to the lens frame 202 through a hinge structure 229, and the second case portions 231b and 233b are connected to the connection member 235. It can be connected to or mounted on the ends of the first case portions 231a and 233a. In some embodiments, the portion of the connection member 235 that is in contact with the user's body may be made of a material with low thermal conductivity, for example, an elastomer material such as silicone, polyurethane, or rubber. , parts that are not in contact with the user's body may be made of a material with high thermal conductivity (e.g., a metal material). For example, when heat is generated from the circuit board 241 or the battery 243, the connection member 235 blocks heat from being transferred to the part that is in contact with the user's body and dissipates heat through the part that is not in contact with the user's body. It can be dispersed or released. According to one embodiment, the part of the connecting member 235 that is in contact with the user's body can be interpreted as a part of the inner case 231, and the part of the connecting member 235 that does not contact the user's body can be interpreted as a part of the outer case ( 233). According to one embodiment (not shown), the first case 231a and the second case 231b are formed as one piece without a connecting member 235, and the third case 233a and the fourth case 233b are connected. It can be configured as an integrated piece without the member 235. According to an embodiment of this document, in addition to the components shown, other components (e.g., the antenna module 197 in FIG. 1) may be further included, and the communication module 190 may be used to connect the network (e.g., FIG. Information about an object or environment is provided from an external electronic device (e.g., the electronic devices 102, 104 or the server 108 of FIG. 1) through the first network 198 or the second network 199 in FIG. 1. You can receive it.

2A to 2C, only the wearable device 200 is shown and described, but it is not limited thereto, and some components of the wearable device 200 shown in FIGS. 2A to 2C can also be used in electronic devices such as smartphones and tablet PCs. may be included.

Figures 3a and 3b are diagrams showing the front and back sides of the wearable device 200 according to an embodiment of this document.

Referring to FIGS. 3A and 3B , in one embodiment, camera modules 311, 312, 313, 314 are installed on the first side 310 of the housing to obtain information related to the surrounding environment of the wearable device 300. 315, 316) and/or depth sensor 317 may be disposed. In one embodiment, the camera modules 311 and 312 may acquire images related to the environment surrounding the wearable device. In one embodiment, the camera modules 313, 314, 315, and 316 may acquire images while the wearable device 300 is worn by a user. Camera modules 313, 314, 315, and 316 according to an embodiment of this document may be used for hand detection, tracking, or recognition of user gestures (eg, hand movements). The camera modules 313, 314, 315, and 316 according to an embodiment of this document may be used for 3DoF, 6DoF head tracking, location (space, environment) recognition, and/or movement recognition. In one embodiment, camera modules 311 and 312 may be used for hand detection and tracking, or user gestures. In one embodiment, the depth sensor 317 may be configured to transmit a signal and receive a signal reflected from an object, and may be used to determine the distance to an object, such as time of flight (TOF). You can. According to one embodiment, camera modules 325 and 326 and/or a display 321 (and/or lens) for face recognition may be disposed on the second surface 320 of the housing. In one embodiment, the face recognition camera modules 325 and 326 adjacent to the display may be used to recognize the user's face, or may recognize and/or track both eyes of the user. According to an embodiment of this document, the face recognition camera modules 325 and 326 may detect (eg, recognize) the iris of the user 500 wearing the wearable device 300. The wearable device 300 according to an embodiment of this document may separate the iris image from the user's eye image acquired using the face recognition camera modules 325 and 326 (eg, an infrared sensor). The wearable device 300 according to an embodiment of this document can flatten the separated iris image and convert it into a coordinate system. The wearable device 300 according to an embodiment of this document may detect the iris of the user 500 (e.g., recognize the iris characteristics of the user 500) by coding the iris image converted into a coordinate system. The wearable device 300 according to an embodiment of this document may include an infrared (IR) light emitting module to acquire an iris image of the user 500. The infrared light emitting module according to an embodiment of this document may be disposed around the face recognition camera modules 325 and 326, but is not limited thereto. The wearable device 300 (e.g., camera modules 325 and 326 for face recognition) according to an embodiment of this document receives the infrared light output from the infrared light-emitting module and reflects from the user's eyeball. Images and/or iris images may be acquired. In one embodiment, the display 321 (and/or lens) may be disposed on the second side 320 of the wearable device 300. In one embodiment, the wearable device 300 may not include camera modules 315 and 316 among the plurality of camera modules 313, 314, 315, and 316. Although not shown in FIGS. 3A and 3B , the wearable device 300 may further include at least one of the configurations shown in FIG. 2 . As described above, the wearable device 300 according to one embodiment may have a form factor to be worn on the user's head. The wearable device 300 may further include a strap and/or a wearing member to be fixed on a part of the user's body. The wearable device 101 may provide a user experience based on augmented reality, virtual reality, and/or mixed reality while worn on the user's head.

FIG. 4 shows an execution screen (e.g., first execution screen 510) for iris detection (e.g., iris authentication) based on whether or not the iris is completely detected by the wearable device 200 according to an embodiment of the present document. This is an example drawing to explain the function or operation of maintaining or changing the displayed position. FIG. 5A is an example diagram illustrating a state in which a virtual object for iris detection (e.g., first execution screen 510) is displayed at a designated location (e.g., first location) according to an embodiment of the present document. am. FIGS. 5B to 5D are example diagrams for explaining the iris detection rate according to the gaze of a user 500 wearing the wearable device 200 according to an embodiment of the present document. FIG. 5E shows a virtual object (e.g., first execution screen 510) for iris detection in order to completely obtain iris information of the user 500 wearing the wearable device 200 according to an embodiment of the present document. ) is an example diagram to explain the function or operation of changing the visible position. FIG. 5F is an example diagram illustrating a case where a virtual object for iris detection (eg, first execution screen 510) is an authentication screen for performing payment, according to an embodiment of this document.

Referring to FIG. 4, in operation 410, the wearable device 200 according to an embodiment of the present document displays the first execution screen 510 for iris detection at a first location (e.g., a designated location in the real world, It is a position where iris detection of the user 500 can be completely performed, or a position where iris detection of the user 500 cannot be completely performed, and is visible from the upper portion of the field of view (FoV) area of the wearable device 200. The display module (e.g., the light output module 211) may be controlled to be displayed (e.g., shown as a virtual object). The first location according to an embodiment of this document may be a location designated in advance or a location designated by the user so that the first execution screen 510 for iris detection is displayed. The first execution screen 510 according to an embodiment of this document may be displayed to the user 500 at a first location based on the user 500's request for iris authentication and/or iris registration. When the camera module (e.g., the first camera module 251) according to an embodiment of the present document is disposed at the bottom of the wearable device 200 as shown in FIG. 5A, the first execution shown at the first position The gaze of the user 500 may be directed upward to look at the screen 510. In this case, as shown in FIG. 5C, the iris image of the user 500 may be obtained incompletely (eg, to the extent that iris detection and/or iris recognition cannot be performed). In FIG. 5C, as the gaze of the user 500 is directed to the first position (e.g., the actual center coordinate of the first execution screen 510), only about 60% of the iris image of the eye 530 is obtained. A case in which this occurs is shown by way of example. Iris detection and/or iris authentication according to an embodiment of this document can be performed only when 90% or more of the iris image of the user 500 is substantially acquired (e.g., the iris image is completely acquired), but in this description There is no limit to the value of the ratio (e.g., 90%) at which it is determined that the iris image has been completely acquired. The first location according to an embodiment of the present document may refer to any coordinate within the viewing angle area of the wearable device 200, which indicates the actual center of the first execution screen 510. Alternatively, the first location according to an embodiment of the present document may refer to a plurality of coordinate values within the viewing angle area of the wearable device 200. In FIG. 5A , for convenience of explanation, an embodiment in which the first execution screen 510 is displayed at the top of the viewing angle area of the wearable device 200 is shown. However, according to various embodiments of this document, the first execution screen ( 510) may be displayed at the bottom of the viewing angle area. In this case, the gaze of the user 500 may be directed downward, and as shown in FIG. 5D, the iris image of the user 500 may be incompletely obtained. The first execution screen 510 according to an embodiment of this document may include an authentication screen for performing payment, as shown in FIG. 5F.

The wearable device 200 according to an embodiment of this document may determine whether the iris has been detected in operation 420. The wearable device 200 according to an embodiment of this document may determine whether the iris image of the user 500 has been completely acquired (eg, the iris has been recognized). In order to determine whether the iris image has been completely acquired, the wearable device 200 according to an embodiment of the present document determines whether the iris shape of the user 500 is completely (e.g., substantially Information about the included iris image (a ratio greater than 90%, which is sufficient to code iris characteristics) can be obtained. According to an embodiment of the present document, information about an iris image (e.g., a reference iris image) containing the iris shape of the user 500 completely (e.g., substantially 90%) is stored in the wearable device 200. It may be stored in or may be stored in an external electronic device (eg, server) that is operable connected to the wearable device 200. In operation 420, the wearable device 200 according to an embodiment of the present document determines the rate at which the iris image of the user 500 acquired by the first camera module 251 is obtained (e.g., based on the reference iris image). A function or operation to determine (60% shown in FIG. 5C) can be performed. The wearable device 200 according to an embodiment of the present document is configured to use the wearable device 200 when the iris image is obtained at a specified rate (e.g., substantially more than 90%, a rate sufficient to code iris characteristics). It may be determined that the iris of the user 500 wearing the device has been detected. Alternatively, the wearable device 200 according to an embodiment of the present document codes the iris characteristics without determining the percentage at which the iris image of the user 500 is obtained (e.g., 60% shown in FIG. 5C). It may also be determined only whether sufficient iris images have been acquired. In this case, the wearable device 200 according to an embodiment of this document does not perform the function or operation of determining the rate at which the iris image was acquired based on the reference iris image, as shown in FIGS. 5B to 5D. Maybe not. Alternatively, according to an embodiment of this document, the function or operation of determining the iris ratio of the user 500 may be performed based on the iris image of any user other than the legitimate user 500 of the wearable device 200. It may be possible.

If it is determined in operation 430 that the iris has not been detected (e.g., operation 420 - No), the wearable device 200 according to an embodiment of the present document displays the first execution screen 510 for iris detection in the second A display module (e.g., light output module 211) can be controlled to be viewed as a virtual object at a location (e.g., a designated location in the real world). The second location according to an embodiment of this document is a location surrounding the portion where the first camera module 251 is placed, and may be a pre-designated location. The second location according to an embodiment of the present document may refer to any coordinate within the viewing angle area of the wearable device 200, which represents the actual center of the first execution screen 510. Alternatively, the second location according to an embodiment of the present document may refer to a plurality of coordinate values within the viewing angle area of the wearable device 200. As shown in FIG. 5E, the wearable device 200 according to an embodiment of the present document includes a display module (e.g., a light output module) so that the first execution screen 510 for iris detection is displayed at a second position. 211)) is controlled (e.g., by changing the angle of light (e.g., image data) incident on the display member 201 so that the first execution screen 510 is displayed at a second position that is relatively lower than the first position. display module) can be controlled. In this case, the gaze of the user 500 is directed toward the area where the first camera module 251 according to an embodiment of the present document is placed, and a complete iris image is detected (e.g., substantially as shown in FIG. 5B). 100%) can be achieved.

The wearable device 200 according to an embodiment of this document may detect the iris based on the first execution screen 510 displayed as a virtual object at a second location in operation 440. According to an embodiment of the present document, when the first execution screen 510 is shown at the second location, the iris image can be completely detected, based on the pre-stored iris image of the user 500 (e.g. Iris authentication may be performed by comparing the pre-stored iris image of the user 500 with the iris image acquired by the first camera module 251. The wearable device 200 according to an embodiment of this document may determine, in operation 460, whether the iris is detected within a specified number of times (eg, 5 times). If it is determined that an iris has been detected (e.g., operation 460 - Yes), the wearable device 200 according to an embodiment of this document may perform iris authentication based on the detected iris in operation 470. If it is determined that the iris has not been detected (e.g., operation 460-No), the wearable device 200 according to an embodiment of this document may stop iris detection (e.g., output an iris detection failure message as a virtual object). You can.

The wearable device 200 according to an embodiment of the present document, in operation 450, when it is determined that the iris has been detected (e.g., operation 420 - e.g., when the first location is a location where iris detection can be completely performed) ), iris authentication can be performed based on the detected iris. The wearable device 200 according to an embodiment of this document is based on a pre-stored iris image of the user 500 (e.g., the stored iris image of the user 500 is acquired by the first camera module 251). Comparison with iris image) Iris authentication can be performed.

FIG. 5G is an example diagram illustrating a case where a virtual object for iris detection (e.g., first execution screen 510) is an execution screen (e.g., login screen) of a specific application, according to an embodiment of this document. am. Referring to FIG. 5G, the wearable device 200 according to an embodiment of this document displays an execution screen (e.g., login screen) of a specific application while the execution screen (e.g., login screen) of a specific application is being displayed. User input (e.g., virtual touch input) can be obtained. The wearable device 200 according to an embodiment of this document may move the execution screen (eg, login screen) of a specific application to a designated location based on acquisition of user input. The wearable device 200 according to an embodiment of this document may perform iris authentication while being moved to a designated location. The wearable device 200 according to an embodiment of this document may perform iris authentication in the background while the execution screen (eg, login screen) of a specific application is being displayed.

FIG. 6 shows a virtual object for iris detection (e.g., first execution screen 510) to obtain additional iris information when additional iris information is needed for iris detection according to an embodiment of this document. This is an example diagram to explain the function or operation of changing the location. FIGS. 7A and 7B are example diagrams for explaining the function or operation described in FIG. 6 from a user interface perspective. FIG. 7C shows a function or operation in which the wearable device 200 according to an embodiment of this document provides information about the location where the iris is detected to enable iris authentication at a previous point in time relative to the current point in time. This is an example drawing to explain. The function or operation shown in FIG. 6 may be understood by those skilled in the art as an embodiment of performing additional iris detection after initial iris detection fails.

Referring to FIG. 6, in operation 610, the wearable device 200 according to an embodiment of the present document displays the first execution screen 510 for iris detection at a first location (e.g., a first location in the real world). The display module (e.g., the light output module 211) can be controlled to be viewed as a virtual object. The first location according to an embodiment of this document may be a location designated in advance or a location designated by the user so that the first execution screen 510 for iris detection is displayed. The first execution screen 510 according to an embodiment of this document may be displayed to the user 500 at a first location based on the user's 500 request for iris authentication and/or iris registration. When the camera module (e.g., the first camera module 251) according to an embodiment of the present document is disposed at the bottom of the wearable device 200 as shown in FIG. 7A, the first execution shown at the first position The gaze of the user 500 may be directed downward to look at the screen 510. In this case, as shown in FIG. 5D, the iris image of the user 500 may be obtained incompletely (eg, to the extent that iris detection and/or iris recognition cannot be performed). In FIG. 5D, as the gaze of the user 500 is directed to the first position (e.g., the actual center coordinate of the first execution screen 510), only about 70% of the iris image of the eye 530 is obtained. A case in which this occurs is shown by way of example.

The wearable device 200 according to an embodiment of this document may determine whether an additional iris image is needed for iris detection and/or iris authentication in operation 620. The wearable device 200 according to an embodiment of this document may determine whether the iris image obtained in operation 610 is an incomplete iris image. The wearable device 200 according to an embodiment of the present document transmits information about the incompletely acquired iris image to the wearable device 200 or an external electronic device operably connected to the wearable device 200 (e.g., server). In FIG. 7A, as the gaze of the user 500 is directed downward, the wearable device 200 according to an embodiment of the present document detects the eye 530 of the user 500 for iris detection and/or iris authentication. ) It may be determined that an additional lower iris image is necessary.

If the wearable device 200 according to an embodiment of the present document determines that an additional iris image is needed in operation 630 (e.g., operation 620-Yes), the first execution screen 510 for iris detection is displayed. The display module (e.g., the light output module 211) may be controlled to be viewed as a virtual object in a second location (e.g., a second location in the real world). The second location according to an embodiment of the present document is a location closer to the upper end of the viewing angle area of the wearable device 200 than the first location, and may be a pre-designated location. Alternatively, the second location according to an embodiment of this document may be a location determined based on the ratio of the detected iris image of the user 500. In this case, the wearable device 200 according to an embodiment of the present document includes a look-up table defining the relationship between the detection rate of the iris image and the second position and/or the moving distance of the first execution screen 510. The second position can be determined using . The wearable device 200 according to an embodiment of the present document moves the position where the first execution screen 510 is shown to a position closer to the upper part of the viewing angle area of the wearable device 200, as shown in FIG. 7B. By moving and directing the gaze of the user 500 to the second position, a lower iris image can be additionally obtained.

The wearable device 200 according to an embodiment of this document may detect the iris based on the first execution screen 510 displayed as a virtual object at a second location in operation 640. According to an embodiment of this document, when the first execution screen 510 is displayed in the second position, it is possible to obtain a complete iris image by additionally acquiring the lower iris image, so that the pre-stored image of the user 500 Iris authentication may be performed based on the iris image. The wearable device 200 according to an embodiment of the present document includes an iris image acquired at a first location and an iris image acquired at a second location for iris detection and/or iris authentication (e.g., to generate a complete iris image). Iris images can be synthesized. According to an embodiment of this document, the function or operation of synthesizing an iris image may be performed by an external electronic device (eg, server) operably connected to the wearable device 200. The wearable device 200 according to an embodiment of this document may determine whether the iris is detected within a specified number of times (eg, 5 times) in operation 660. When it is determined that an iris has been detected (e.g., operation 660 - Yes), the wearable device 200 according to an embodiment of this document may perform iris authentication based on the detected iris in operation 670. If it is determined that the iris has not been detected (e.g., operation 660-No), the wearable device 200 according to an embodiment of this document may stop iris detection (e.g., output an iris detection failure message as a virtual object). You can.

The wearable device 200 according to an embodiment of the present document may perform iris authentication based on the detected iris if, in operation 650, it is determined that an additional iris image is not needed (e.g., operation 620-No). there is. The wearable device 200 according to an embodiment of this document is based on the stored iris image of the user 500 (e.g., the pre-stored iris image of the user 500 is acquired by the first camera module 251). Comparison with iris image) Iris authentication can be performed.

The wearable device 200 according to an embodiment of this document provides information on the location where iris detection and/or iris authentication was or was not performed at a previous point in time based on the current point in time (e.g., the point in time when operation 610 is performed). Information can be saved. The wearable device 200 according to an embodiment of this document provides information on the location where iris detection and/or iris authentication was or was not performed at a previous point in time based on the current point in time (e.g., the point in time when operation 610 is performed). Information may be provided to the user 500 visually, audibly, and/or tactilely. In FIG. 7C , an embodiment in which information about a location where iris detection and/or iris authentication was performed in the past is displayed around the first execution screen 510 is exemplarily shown. The wearable device 200 according to an embodiment of the present document colors the area containing the location where iris detection and/or iris authentication was performed in the past (e.g., the first location in FIG. 7A) in a designated color (e.g., black). Alternatively, the display module (e.g., the light output module 211) can be controlled to appear darker than other parts. The wearable device 200 according to an embodiment of the present document colors the area containing the location where iris detection and/or iris authentication failed in the past (e.g., the second location in FIG. 7B) with a designated color (e.g., white). Alternatively, the display module (e.g., the light output module 211) can be controlled to appear brighter than other parts. The wearable device 200 according to an embodiment of the present document displays the first execution screen 510 at a location where iris detection and/or iris authentication was performed in the past (e.g., the first execution screen 510 The display module (e.g., light output module 211) can be controlled by moving the visible position. The wearable device 200 according to an embodiment of the present document tracks the gaze of the user 500, and when it is determined that the user's gaze is directed to the location where iris detection and/or iris authentication was performed in the past, the speaker module Using , a guidance message indicating that the location currently being viewed is a location where iris detection and/or iris authentication was performed in the past can be output. The wearable device 200 according to an embodiment of the present document tracks the gaze of the user 500, and when it is determined that the user's gaze is directed to a location where iris detection and/or iris authentication was performed in the past, the motor module You can output vibration using . The wearable device 200 according to an embodiment of the present document provides a notification indicating that the first execution screen 510 will be displayed at a designated location (e.g., a location where iris detection and/or iris authentication was performed in the past) and/ Alternatively, a notification indicating that iris recognition is scheduled to be performed may be provided through the first execution screen 510 at a designated location. Through this function or operation, the effect of intuitively informing the user that the location the user is currently looking at is a location where iris detection can be completely or incompletely performed can be achieved.

FIG. 8 shows that the wearable device 200 according to an embodiment of the present document is displayed at a designated location for iris detection, which is different from the display position of the execution screen (e.g., the second execution screen 910) of the application currently being displayed. An example diagram for explaining a function or operation of determining the display position of a virtual object (e.g., the first execution screen 510) so that the virtual object for iris detection (e.g., the first execution screen 510) is displayed at the location. am. FIGS. 9A and 9B are example diagrams for explaining the function or operation described in FIG. 8 from a user interface perspective.

Referring to FIG. 8, in operation 810, the wearable device 200 according to an embodiment of the present document displays the second execution screen 910 of the first application (e.g., an Internet application) at a first location (e.g., actual location). As a designated location in the world, the display module (eg, light output module 211) can be controlled to be viewed as a virtual object in the upper right corner of the viewing angle area of the wearable device 200. The wearable device 200 according to an embodiment of the present document displays the second execution screen 910 of the first application (e.g., an Internet application) at a designated location (e.g., as shown in FIG. 9A) according to a user's request. : The display module (e.g., the light output module 211) can be controlled to be displayed at the first position.

The wearable device 200 according to an embodiment of this document may obtain an iris authentication request in operation 820. The wearable device 200 according to an embodiment of this document performs a virtual touch input and/or a voice command from the user 500 on the second execution screen 910 of the first application (e.g., an Internet application). You can obtain an iris authentication request through.

In the wearable device 200 according to an embodiment of the present document, in operation 830, the first location where the second execution screen 910 of the first application (e.g., Internet application) is displayed is the designated location for iris detection. You can determine whether it is recognized or not. The wearable device 200 according to an embodiment of the present document is located at a first location (e.g., the second execution screen 910) where the second execution screen 910 of the first application (e.g., an Internet application) is displayed. A second execution screen 910 of the first application (e.g., an Internet application) is displayed based on whether the actual center coordinates) substantially coincide with a predetermined position for iris detection (e.g., the second position in FIG. 5E). It may be determined whether the first position being held is a designated position for iris detection. When an iris authentication request is obtained in operation 820, the wearable device 200 according to an embodiment of this document is configured to use the first application (e.g., Internet application) based on whether the user's iris image is completely detected. It may be determined whether the first location where the second execution screen 910 is displayed is a designated location for iris detection.

In operation 840, the wearable device 200 according to an embodiment of the present document configures the display module so that the first execution screen 510 of the first application for performing iris authentication is displayed as a virtual object at a designated location for iris detection. (For example, the optical output module 211) can be controlled. The designated location according to an embodiment of the present document is a pre-designated location close to the area where the first camera module 251 is placed (e.g., the second location in FIG. 5E), and when only part of the iris image is detected, the remaining part Locations for acquiring iris images, locations where iris detection and/or iris authentication has been performed in the past, locations with the highest iris recognition rates (e.g., locations where the acquisition rate of iris images is substantially 100%), and/or users. It may include at least one location among the locations specified by 500. In FIG. 9B, as an example, a case where the second execution screen 510 is displayed at a pre-designated location (eg, the second location in FIG. 5E) close to the area where the first camera module 251 is disposed is shown. In one embodiment, referring to FIG. 9C, the second execution screen 910 may not be displayed, and the first execution screen 510 may be displayed at a second location instead of the first execution screen 910.

The wearable device 200 according to an embodiment of this document may detect the iris based on the first execution screen 510 displayed at a designated location in operation 850. The wearable device 200 according to an embodiment of this document is based on the stored iris image of the user 500 (e.g., the pre-stored iris image of the user 500 is acquired by the first camera module 251). Comparison with iris image) Iris authentication can be performed. The wearable device 200 according to an embodiment of this document may determine whether the iris is detected within a specified number of times (eg, 5 times) in operation 870. When it is determined that an iris has been detected (e.g., operation 870 - Yes), the wearable device 200 according to an embodiment of this document may perform iris authentication based on the detected iris in operation 880. If it is determined that the iris has not been detected (e.g., operation 870-No), the wearable device 200 according to an embodiment of this document may stop iris detection (e.g., output an iris detection failure message as a virtual object). You can.

When the wearable device 200 according to an embodiment of the present document determines that the location of the second execution screen 910 is a designated location for iris detection in operation 830, iris authentication is performed at the first location in operation 860. It can be done. In other words, the wearable device 200 according to an embodiment of this document may not display the first execution screen 510. The wearable device 200 according to an embodiment of this document is based on the stored iris image of the user 500 (e.g., the pre-stored iris image of the user 500 is acquired by the first camera module 251). Comparison with iris image) Iris authentication can be performed.

In the wearable device 200 according to an embodiment of this document, when an iris authentication request is obtained, the execution screen (e.g., second execution screen 910) of the first application (e.g., Internet application) performs iris detection and/or Alternatively, the display module (e.g., the light output module 211) can be controlled to change and display the execution screen for iris authentication (e.g., the first execution screen 510). Referring to FIG. 9C, the wearable device 200 according to an embodiment of the present document may control the display module so that the second execution screen 910, which has been changed to the first execution screen 510, is displayed at a designated location. . Referring to FIG. 9D, when iris authentication is performed, the wearable device 200 according to an embodiment of this document changes the first execution screen 510 to the second execution screen 910 and displays the second execution screen 910. ) can be controlled to display the second execution screen 910 again at the position where it was being displayed.

FIG. 10 is an example diagram illustrating a function or operation of the wearable device 200 according to an embodiment of the present document to register (e.g., store) iris information and gaze information of the user 500 by matching them with each other. . FIG. 11 shows a screen for iris registration (e.g., a third execution screen) in order for the wearable device 200 according to an embodiment of this document to match and register (e.g., store) iris information and user's gaze information. This is an example diagram for explaining the function or operation of controlling the display module to display (1110)) at multiple locations (e.g., multiple locations in the real world) from a user interface perspective.

Referring to FIG. 10, in operation 1010, the wearable device 200 according to an embodiment of the present document displays a first execution screen (e.g., a first execution screen for registering iris information for iris authentication). 3 The display module (eg, the light output module 211) can be controlled so that the execution screen 1110 is sequentially displayed as a virtual object at a plurality of locations. As shown in FIG. 11, the wearable device 200 according to an embodiment of the present document is located at a plurality of locations designated in advance or by the user (e.g., left, top, The display module (e.g., light output module 211) can be controlled so that the first execution screen (e.g., third execution screen 1110) for registering iris information is sequentially displayed on the right and/or bottom side. .

In operation 1020, the wearable device 200 according to an embodiment of the present document collects iris information and the user at a plurality of locations (e.g., left, top, right, and/or bottom in the viewing angle area of the wearable device 200). Gaze information of (500) can be obtained. The wearable device 200 according to an embodiment of this document may acquire iris information and gaze information of the user 500 using a camera module (eg, the first camera module 251).

The wearable device 200 according to an embodiment of this document may store the iris information obtained in operation 1020 and the gaze information of the user 500 by matching them in operation 1030. The wearable device 200 according to an embodiment of this document includes the direction in which the user 500's gaze is directed, iris information corresponding to the gaze direction, and one position corresponding to the gaze direction among a plurality of positions. Information may be grouped and stored in the wearable device 200 or an external electronic device (eg, server) that is operable connected to the wearable device 200. The wearable device 200 according to an embodiment of this document may group and store information about the iris recognition rate (eg, iris image acquisition rate) at each of a plurality of locations. Through this function or operation, the gaze information and iris information of the user 500 can be matched and registered.

FIG. 12 shows a location where a virtual object (e.g., first execution screen 510) for iris detection is displayed based on gaze information of the user 500 by the wearable device 200 according to an embodiment of this document. An example diagram for explaining a function or operation of determining and controlling a display module (e.g., light output module 211) so that a virtual object for iris detection (e.g., first execution screen 510) is displayed at the determined location. am.

Referring to FIG. 12, the wearable device 200 according to an embodiment of this document may obtain an iris authentication request in operation 1210. The wearable device 200 according to an embodiment of the present document may, for example, use a virtual touch input on the second execution screen 910 of the first application (e.g., an Internet application) and/or the user's 500 Iris authentication requests can be obtained through voice commands, etc.

The wearable device 200 according to an embodiment of this document may acquire the user's gaze information in operation 1220 based on the acquisition of the iris authentication request in operation 1210. The wearable device 200 according to an embodiment of this document can acquire gaze information of the user 500 based on the user's eye image acquired by a camera module (e.g., the first camera module 251). there is.

In operation 1230, the wearable device 200 according to an embodiment of the present document displays a first execution screen 510 for performing iris authentication based on the gaze information of the user 500 obtained in operation 1220. You can decide where it will be displayed (e.g. where it will be displayed as a virtual object in the real world). The wearable device 200 according to an embodiment of the present document includes a first device for performing iris authentication based on information on the gaze direction of the user 500 registered in operation 1030 and information on the position corresponding to the gaze direction. The location where the execution screen 510 will be displayed can be determined. The wearable device 200 according to an embodiment of this document may determine a position where the first execution screen 510 for performing iris authentication will be displayed based on information about the iris recognition rate.

In operation 1240, the wearable device 200 according to an embodiment of the present document controls the display module (e.g., the light output module 211) to display the first execution screen 510 at the position determined according to operation 1230. can do. The wearable device 200 according to an embodiment of this document may acquire an iris image based on the first execution screen 510. The wearable device 200 according to an embodiment of this document may perform iris authentication using iris information corresponding to the gaze direction of the registered user 500 in operation 1030.

FIG. 13 is an example diagram illustrating a function or operation of the wearable device 200 according to an embodiment of the present document to provide a specified visual effect for the first execution screen 510 depending on whether iris information is completely detected. am. FIGS. 14A and 14B are example diagrams for explaining the function or operation described in FIG. 13 from a user interface perspective.

Referring to FIG. 13, in operation 1310, the wearable device 200 according to an embodiment of the present document displays the first execution screen 510 for iris detection at a first location (e.g., a designated location for iris detection or The display module (e.g., the light output module 211) can be controlled to be viewed as a virtual object at an arbitrary location.

The wearable device 200 according to an embodiment of this document may determine, in operation 1320, whether iris detection has been performed normally (e.g., detection of an iris image has been completely performed at a designated location).

The wearable device 200 according to an embodiment of the present document, in operation 1330, when it is determined that iris detection was not normally performed according to operation 1320 (operation 1320-No), the wearable device 200 displays the screen designated for the first execution screen 510. Visual effects can be applied. The wearable device 200 according to an embodiment of this document is used when a designated location is not specified or when iris detection is not completely performed even though the first execution screen 510 is displayed at the designated location (e.g. : If the iris recognition rate is below the specified value), this can be judged as a case where iris detection was not performed properly. The wearable device 200 according to an embodiment of this document may enlarge the size of the first execution screen 510 as shown in FIG. 14A as a designated visual effect. The wearable device 200 according to an embodiment of the present document includes, as a designated visual effect, a display module (e.g., a light output module) that reduces the resolution of the first execution screen 510 to make it appear blurry, as shown in FIG. 14B. (211)) can be controlled. The wearable device 200 according to an embodiment of this document may control the display module (e.g., the light output module 211) so that the first execution screen 510 moves in the direction of the user 500 and is displayed. . By applying such a visual effect, the effect of increasing the recognition rate of iris detection can be achieved by allowing the user 500 to open his eyes wider. The wearable device 200 according to an embodiment of this document may acquire an iris image based on the first execution screen 510 to which visual effects are applied. The wearable device 200 according to an embodiment of this document may perform iris authentication based on the acquired iris image.

In operation 1340, the wearable device 200 according to an embodiment of the present document detects the iris based on the first execution screen 510 when it is determined that iris detection has been performed normally according to operation 1320 (operation 1320 - Yes). Detection can be performed. The wearable device 200 according to an embodiment of this document may perform iris authentication based on an iris image obtained as a result of iris detection.

FIG. 15 shows a display module (e.g., light output) so that a designated notification 1610 is displayed around the first execution screen 510 depending on whether the wearable device 200 according to an embodiment of the present document completely detects iris information. This is an example diagram to explain the function or operation that controls the module 211). FIG. 16 is an example diagram for explaining the function or operation described in FIG. 15 from a user interface perspective.

Referring to FIG. 15, in operation 1510, the wearable device 200 according to an embodiment of the present document displays the first execution screen 510 for iris detection at a first location (e.g., a designated location for iris detection or The display module (e.g., the light output module 211) can be controlled to be viewed as a virtual object at an arbitrary location.

The wearable device 200 according to an embodiment of this document may determine, in operation 1520, whether iris detection has been performed normally (e.g., detection of an iris image has been completely performed at a designated location).

The wearable device 200 according to an embodiment of the present document, in operation 1530, when it is determined that iris detection was not normally performed according to operation 1320 (operation 1320-No), displays a screen designated around the first execution screen 510. The display module (e.g., the light output module 211) can be controlled so that the notification 1610 is displayed. As shown in FIG. 16, the wearable device 200 according to an embodiment of the present document displays a designated notification 1610 (e.g., “Galaxy S22 for free!”) at the top of the first execution screen 510. The display module (e.g., the optical output module 211) can be controlled to display the display module. The wearable device 200 according to an embodiment of the present document determines the content of the specified notification 1610 based on log information of the specified application (e.g., Internet application) and/or interest information specified by the user 500. You can. For example, if the user 500's history of frequent visits to a smart phone sales site is stored as log information, the wearable device 200 according to an embodiment of this document may be stored in the smart phone sales site. The display module (eg, the light output module 211) can be controlled so that at least some of the content is displayed as a designated notification 1610. A designated notification according to an embodiment of this document may include a still image or video. According to the provision of such a designated notification 1610, the user 500 can unconsciously open his eyes wide, thereby increasing the recognition rate of iris detection. The wearable device 200 according to an embodiment of this document may provide a designated notification 1610 (eg, content of the notification) to the user audibly through at least one speaker.

In operation 1540, the wearable device 200 according to an embodiment of the present document detects the iris based on the first execution screen 510 when it is determined that iris detection has been performed normally according to operation 1520 (operation 1520 - Yes). Detection can be performed. The wearable device 200 according to an embodiment of this document may perform iris authentication based on an iris image obtained as a result of iris detection.

FIG. 17 shows an execution screen that was being displayed when the wearable device 200 according to an embodiment of the present document registered the brightness of a virtual object (e.g., the first execution screen 510) and iris information for iris detection. This is an example diagram for explaining a function or operation of adjusting the brightness of a virtual object (eg, the first execution screen 510) for iris detection based on the result of comparison with the brightness. FIG. 18 is an example diagram for explaining the function or operation described in FIG. 17 from a user interface perspective.

Referring to FIG. 17, in operation 1710, the wearable device 200 according to an embodiment of the present document displays the first execution screen 510 for iris detection at a first location (e.g., a designated location for iris detection). The display module (e.g., the light output module 211) can be controlled to be viewed as a virtual object. The designated location according to an embodiment of the present document is a designated location close to the area where the first camera module 251 is placed (e.g., the second location in FIG. 5E), and when only part of the iris image is detected, the remaining part is Location for acquiring iris images, location where iris detection and/or iris authentication has been performed in the past, location with the highest iris recognition rate (e.g., location with an iris image acquisition rate of substantially 100%), and/or user ( 500) may include at least one of the positions specified by .

In operation 1720, the wearable device 200 according to an embodiment of the present document displays the brightness of the first execution screen 510 and the second execution screen (e.g., the third execution screen) that was being displayed at the time of registering the iris information. (1110)) can be compared in brightness. The wearable device 200 according to an embodiment of this document adjusts the brightness of the second execution screen (e.g., the third execution screen 1110) that was being displayed at the time of registering the iris information stored in the wearable device 200. Based on the information, the brightness of the currently displayed first execution screen 510 is compared with the brightness of the second execution screen (e.g., the third execution screen 1110) that was being displayed at the time of registering the iris information. You can.

In operation 1730, the wearable device 200 according to an embodiment of the present document displays the brightness of the first execution screen 510 and the second execution screen (e.g., the third execution screen) that was being displayed at the time of registering the iris information. It is possible to determine whether the brightness of (1110)) is different from each other.

In operation 1740, the wearable device 200 according to an embodiment of the present document displays the brightness of the first execution screen 510 and the second execution screen (e.g., the third execution screen) that was being displayed at the time of registering the iris information. When the brightness of the first execution screen 510 is different (operation 1730 - example), the brightness of the first execution screen 510 is different from that of the second execution screen (e.g., the third execution screen 1110) that was being displayed at the time of registering the iris information. ) can be controlled to be substantially the same as the brightness of the display module (e.g., the light output module 211). The wearable device 200 according to an embodiment of this document determines that the brightness of the first execution screen 510 that is currently being displayed is determined by the second execution screen (e.g., the second execution screen ( 910)), as shown in FIG. 18, the display module (e.g., light output module 211) can be controlled to increase the brightness of the currently displayed first execution screen 510. You can.

The wearable device 200 according to an embodiment of this document may detect the iris based on the first execution screen 510 whose brightness has been adjusted in operation 1750. The wearable device 200 according to an embodiment of this document may perform iris authentication based on an iris image obtained as a result of iris detection. The wearable device 200 according to an embodiment of this document may determine whether the iris is detected within a specified number of times (eg, 5 times) in operation 1770. If it is determined that an iris has been detected (e.g., operation 1770 - Yes), the wearable device 200 according to an embodiment of this document may perform iris authentication based on the detected iris in operation 1780. If it is determined that the iris has not been detected (e.g., operation 1770-No), the wearable device 200 according to an embodiment of the present document may stop iris detection (e.g., output an iris detection failure message as a virtual object). You can.

In operation 1760, the wearable device 200 according to an embodiment of the present document displays the brightness of the first execution screen 510 and the second execution screen (e.g., the third execution screen) that was being displayed at the time of registering the iris information. If the brightnesses (1110)) are substantially the same (e.g., within a specified error range) (operation 1730-No), the iris may be detected based on the first execution screen 510. The wearable device 200 according to an embodiment of this document may perform iris authentication based on an iris image obtained as a result of iris detection. According to this function or operation, the iris is detected in a situation similar to the situation at the time of registering the iris information, which can have the effect of increasing the iris recognition rate. The wearable device 200 according to an embodiment of this document may include a variable light transmittance lens (eg, an electrochromic (EC) lens). When iris authentication is performed, the wearable device 200 according to an embodiment of this document may increase the iris recognition rate by controlling the light transmittance of the lens (eg, reducing the light transmittance of the lens). The wearable device 200 according to an embodiment of this document may control the light transmittance of the lens so that the light transmittance of the lens at the time of registering iris information and the light transmittance at the time of iris authentication are substantially the same. To this end, the wearable device 200 according to an embodiment of this document may store information about the light transmittance of the lens at the time of iris registration. Figure 19 shows that the wearable device 200 according to an embodiment of this document adjusts the brightness of the light emitting module (not shown) based on a comparison result between the current ambient illuminance and the ambient illuminance at the time of registration of iris information. This is an example drawing to explain a function or operation.

Referring to FIG. 19, in operation 1910, the wearable device 200 according to an embodiment of the present document displays the first execution screen 510 for iris detection at a first location (e.g., a designated location for iris detection). The display module (e.g., the light output module 211) can be controlled to be viewed as a virtual object. The designated location according to an embodiment of the present document is a designated location close to the area where the first camera module 251 is placed (e.g., the second location in FIG. 5E), and when only part of the iris image is detected, the remaining part is Location for acquiring iris images, location where iris detection and/or iris authentication has been performed in the past, location with the highest iris recognition rate (e.g., location with an iris image acquisition rate of substantially 100%), and/or user ( 500) may include at least one of the positions specified by .

In operation 1920, the wearable device 200 according to an embodiment of this document may compare the current ambient illuminance of the wearable device 200 with the ambient illuminance at the time of registering iris information. The wearable device 200 according to an embodiment of this document may include at least one sensor module for identifying ambient illumination. The wearable device 200 according to an embodiment of this document is based on information about the surrounding illuminance at the time of registering the iris information stored in the wearable device 200, and determines the current surrounding illuminance of the wearable device 200 and the iris. You can compare the surrounding illumination at the time of registering information.

The wearable device 200 according to an embodiment of this document may determine, in operation 1930, whether the current ambient illuminance of the wearable device 200 is different from the ambient illuminance at the time of registering iris information.

The wearable device 200 according to an embodiment of the present document, in operation 1940, if the current ambient illuminance of the wearable device 200 and the ambient illuminance at the time of registering the iris information are different from each other (operation 1930 - example), the current ambient illuminance is different from that of the wearable device 200. The light emitting module can be controlled so that the surrounding illumination of the wearable device 200 is substantially the same as the surrounding illumination at the time of registering the iris information. The wearable device 200 according to an embodiment of this document may include a light emitting module that outputs visible light. The light emitting module according to an embodiment of this document may be arranged to output visible light in the direction of the user's eyeball 530, but the location where the light emitting module is placed is not limited thereby.

In operation 1950, the wearable device 200 according to an embodiment of this document may detect the iris using the first execution screen 510 while the brightness of the light emitting module is adjusted. The wearable device 200 according to an embodiment of this document may perform iris authentication based on an iris image obtained as a result of iris detection. The wearable device 200 according to an embodiment of this document may determine whether the iris is detected within a specified number of times (eg, 5 times) in operation 1970. When it is determined that an iris has been detected (e.g., operation 1970-Yes), the wearable device 200 according to an embodiment of this document may perform iris authentication based on the detected iris in operation 1980. The wearable device 200 according to an embodiment of this document may stop iris detection (e.g., output an iris detection failure message as a virtual object) when it is determined that the iris has not been detected (e.g., operation 1970-No). You can.

The wearable device 200 according to an embodiment of the present document, in operation 1960, when the current peripheral illuminance of the wearable device 200 and the ambient illuminance at the time of registering the iris information are substantially the same (operation 1930-No). , when the brightness of the light emitting module is not adjusted (e.g., without changing the output illuminance value of the light emitting module currently being output, or keeping the light emitting module in the off state if the light emitting module is OFF). 1 Iris can be detected based on the execution screen 510. The wearable device 200 according to an embodiment of this document may perform iris authentication based on an iris image obtained as a result of iris detection. According to this function or operation, the iris is detected in a situation similar to the situation at the time of registering the iris information, which can have the effect of increasing the iris recognition rate.

In this document, as an example of the wearable device 200, various embodiments of the present document are based on a wearable device 200 configured to provide augmented reality (e.g., the wearable device 200 shown in FIGS. 2A to 2C). have been explained. However, according to another embodiment of this document, the wearable device 200 for providing virtual reality (e.g., the wearable device 200 shown in FIGS. 3A and 3B) also includes various devices according to an embodiment of this document. Functions or operations may be equally applied.

A wearable device according to an embodiment of the present document includes a display module (e.g., light output module 211), at least one camera (e.g., first camera module 251), and at least one processor (e.g., processor (120)), wherein the at least one processor controls the display module to display a first execution screen for detecting the iris of a user wearing the wearable device as a virtual object at a first location, Determining whether an iris is detected by the at least one camera to enable iris authentication, and based on whether the iris is not detected to enable iris authentication, the first device for detecting the iris Control the display module so that the execution screen is moved to a second position and displayed as the virtual object, and based on the iris being detected to enable the iris authentication, the iris authentication is performed using the detected iris. It can be set to do so.

Electronic devices according to various embodiments disclosed in this document may be of various types. Electronic devices may include, for example, portable communication devices (e.g., smart phones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to embodiments of this document are not limited to the above-described devices.

The various embodiments of this document and the terms used herein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various changes, equivalents, or replacements of the embodiments. In connection with the description of the drawings, similar reference numbers may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the above items, unless the relevant context clearly indicates otherwise. As used herein, “A or B”, “at least one of A and B”, “at least one of A or B”, “A, B or C”, “at least one of A, B and C”, and “A Each of phrases such as “at least one of , B, or C” may include any one of the items listed together in the corresponding phrase, or any possible combination thereof. Terms such as "first", "second", or "first" or "second" may be used simply to distinguish one component from another, and to refer to those components in other respects (e.g., importance or order) is not limited. One (e.g., first) component is said to be “coupled” or “connected” to another (e.g., second) component, with or without the terms “functionally” or “communicatively.” When mentioned, it means that any of the components can be connected to the other components directly (e.g. wired), wirelessly, or through a third component.

The term “module” used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and is interchangeable with terms such as logic, logic block, component, or circuit, for example. It can be used as A module may be an integrated part or a minimum unit of the parts or a part thereof that performs one or more functions. For example, according to one embodiment, the module may be implemented in the form of an application-specific integrated circuit (ASIC).

Various embodiments of the present document are one or more instructions stored in a storage medium (e.g., built-in memory 2536 or external memory 2538) that can be read by a machine (e.g., electronic device 2501). It may be implemented as software (e.g., program 2540) including these. For example, a processor of a device (e.g., electronic device 2501) may call at least one command among one or more commands stored from a storage medium and execute it. This allows the device to be operated to perform at least one function according to the at least one instruction called. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. A storage medium that can be read by a device may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' only means that the storage medium is a tangible device and does not contain signals (e.g. electromagnetic waves), and this term refers to cases where data is semi-permanently stored in the storage medium. There is no distinction between temporary storage cases.

According to one embodiment, methods according to various embodiments disclosed in this document may be provided and included in a computer program product. Computer program products are commodities and can be traded between sellers and buyers. The computer program product may be distributed in the form of a machine-readable storage medium (e.g. compact disc read only memory (CD-ROM)) or through an application store (e.g. Play StoreTM) or on two user devices (e.g. It can be distributed (e.g. downloaded or uploaded) directly between smart phones) or online. In the case of online distribution, at least a portion of the computer program product may be at least temporarily stored or temporarily created in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

According to an embodiment of this document, each component (e.g., module or program) of the above-described components may include a single or multiple entities, and some of the multiple entities are separately arranged in other components. It could be. According to an embodiment of the present document, one or more of the components or operations described above may be omitted, or one or more other components or operations may be added. Alternatively or additionally, multiple components (eg, modules or programs) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each component of the plurality of components identically or similarly to those performed by the corresponding component of the plurality of components prior to the integration. . According to an embodiment of the present document, operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order. , may be omitted, or one or more other operations may be added.

Claims (20)

In wearable devices,
display module,
at least one camera, and
Comprising at least one processor, wherein the at least one processor includes:
Controlling the display module so that a first execution screen for detecting the iris of a user wearing the wearable device is displayed at a first position,
Determine whether the iris is detected by the at least one camera to enable iris authentication,
Based on the fact that the iris is not detected to enable the iris authentication, control the display module to move the first execution screen for detection of the iris to a second position and display it,
A wearable device, characterized in that it is set to perform the iris authentication using the detected iris, based on the iris being detected so that the iris authentication can be performed. According to paragraph 1,
wherein the at least one processor is further set to determine whether the iris has been detected to enable the iris authentication based on whether more than a specified ratio of iris images registered through the wearable device has been detected, Wearable devices. According to claim 1 or 2,
The second location is a wearable device that is closer to the location where the at least one camera is located than the first location. According to any one of claims 1 to 3,
The second location is a wearable device characterized in that it includes a location where the iris is detected to enable the iris authentication to be performed at a previous time point relative to the current time point. According to any one of claims 1 to 4,
The at least one processor is further configured to determine whether an additional iris image is required for iris authentication before controlling the display module to move the first execution screen to the second location and display it. A wearable device that does. According to any one of claims 1 to 5,
The second location is a wearable device, characterized in that it includes a location for acquiring the additional iris image. According to any one of claims 1 to 6,
The at least one processor,
Controlling the display module so that the execution screen of the first application is displayed at the third location,
Based on obtaining the request for iris authentication, determine whether the third location is a designated location for the iris authentication, and
The wearable device is further configured to control the display module to display the first execution screen for iris authentication at the second location, based on a result of determining whether it is the designated location. According to any one of claims 1 to 7,
The at least one processor,
To register iris information for the iris authentication, obtain the iris information and the user's gaze information at a plurality of locations,
Registering the obtained iris information and the gaze information by matching them with each other,
Based on the request for iris authentication being obtained, the user's gaze information is obtained to determine a position where the first execution screen will be displayed, and
The wearable device is further configured to control the display module to display the first execution screen at the determined location. According to any one of claims 1 to 8,
The at least one processor, based on the iris not being detected to enable performance of the iris authentication, does not move the first execution screen to the second location and displays a visual effect specified for the first execution screen. The wearable device is further set to control the display module to apply or display a designated notification around the first execution screen. According to any one of claims 1 to 9,
The wearable device further includes at least one light emitting module,
The at least one processor configures the display module or the at least one light emitting device to make the brightness of the first execution screen or the peripheral illuminance of the wearable device substantially the same as the brightness or peripheral illuminance at the time of registering the iris information. A wearable device, characterized in that it is further configured to control a module. In a method of controlling a wearable device,
Controlling a display module of the wearable device to display a first execution screen for detecting the iris of a user wearing the wearable device at a first position;
determining whether the iris is detected by at least one camera of the wearable device to enable iris authentication;
Based on the fact that the iris is not detected to enable the iris authentication, controlling the display module so that the first execution screen for detecting the iris is moved to a second position and displayed; and
A method of controlling a wearable device, comprising an operation of performing the iris authentication using the detected iris, based on the iris being detected to enable the iris authentication. According to clause 11,
The method for controlling the wearable device further includes the operation of determining whether the iris has been detected to enable the iris authentication based on whether more than a specified ratio of iris images registered through the wearable device has been detected. A method of controlling a wearable device, characterized in that. According to claim 11 or 12,
The second location is a location closer to the location where the at least one camera is located than the first location. According to any one of claims 11 to 13,
The second location is a method for controlling a wearable device, characterized in that it includes a location where the iris is detected to enable the iris authentication at a previous time point relative to the current time point. According to any one of claims 11 to 14,
The method of controlling the wearable device further includes determining whether an additional iris image is required for iris authentication before controlling the display module so that the first execution screen is moved to the second position and displayed. A method of controlling a wearable device, comprising: According to any one of claims 11 to 15,
The second location includes a location for acquiring the additional iris image. According to any one of claims 11 to 16,
The method for controlling the wearable device is:
Controlling the display module so that an execution screen of a first application is displayed at a third location;
Based on obtaining the request for iris authentication, determining whether the third location is a designated location for iris authentication; and
Based on the result of determining whether it is the designated location, the wearable device further comprises controlling the display module to display the first execution screen for iris authentication at the second location. How to control it. According to any one of claims 11 to 17,
The method for controlling the wearable device is:
Obtaining the iris information and the user's gaze information at a plurality of locations to register iris information for the iris authentication;
An operation of matching and registering the obtained iris information and the gaze information;
Based on the request for iris authentication being obtained, obtaining gaze information of the user and determining a position at which the first execution screen will be displayed; and
A method of controlling a wearable device, further comprising controlling the display module to display the first execution screen at the determined location. According to any one of claims 11 to 18,
The method of controlling the wearable device includes not moving the first execution screen to the second location based on the iris not being detected to enable the iris authentication to be performed, and moving the first execution screen to the second location designated for the first execution screen. A method of controlling a wearable device, further comprising controlling the display module to apply a visual effect or display a designated notification around the first execution screen. According to any one of claims 11 to 19,
The wearable device further includes at least one light emitting module,
The method of controlling the wearable device includes: the display module or the at least one such that the brightness of the first execution screen or the peripheral illumination of the wearable device is substantially the same as the brightness or peripheral illumination at the time of registering the iris information. A method of controlling a wearable device, further comprising controlling the light emitting module.

Images