WO2022196984A1

WO2022196984A1 - Electronic device comprising camera and operating method thereof

Info

Publication number: WO2022196984A1
Application number: PCT/KR2022/003005
Authority: WO
Inventors: 김보성; 김성오; 프루신스키발레리; 여형석; 이기혁; 최재성; 홍준의
Original assignee: 삼성전자 주식회사
Priority date: 2021-03-15
Filing date: 2022-03-03
Publication date: 2022-09-22
Also published as: KR20220128735A

Abstract

A wearable electronic device according to various embodiments comprises: a first sensor; a display; a camera arranged under the rear surface of the display that is not exposed to the outside; and a processor, wherein the processor may be set to: identify, through the first sensor, a first gesture regarding the wearable electronic device; identify the face and gaze of a user on the basis of an image captured through the camera, the image comprising the face of the user; determine whether to turn on the display on the basis of the first gesture, the face, and the gaze; and when it is determined to turn on the display, adjust a screen displayed on the display on the basis of the distance between the face and the wearable electronic device, the direction of the face, and the angle of the gaze.

Description

Electronic device including camera and operating method thereof

Various embodiments of the present disclosure relate to an electronic device including a camera and an operating method thereof.

Recently, electronic devices have been developed in various forms for the convenience of users, and various services or functions are provided.

In recent years, in order to effectively display content and information according to the execution of various services or functions and to increase a user's immersion feeling, electronic devices have been implemented to extend the display screen by reducing the display bezel to make the most of the exposed areas of the display. To this end, the electronic device may be implemented by including an under display camera (UDC) in which the front camera is mounted under the display panel instead of in the form of a notch or hole display.

Various services and additional functions provided through portable electronic devices such as wearable electronic devices are gradually expanding. In order to increase the utility value of the wearable electronic device and satisfy the needs of various users, communication service providers or electronic device manufacturers are competitively developing electronic devices to provide various functions and differentiate them from other companies.

In particular, a smart watch is a type of wearable electronic device in which a watch and an electronic device are combined. A smart watch performs not only a function as a watch, but also performs various functions by mounting devices (eg, a camera, a sensor, etc.) capable of performing various functions.

The smart watch may provide a list-up function. For example, the list-up function may mean a 'watch watch by raising wrist' function. For example, the smart watch may turn on the screen with only a gesture of raising the wrist without a separate input while the user wears the smart watch.

When a smart watch is equipped with a camera, usability may deteriorate due to the arrangement of the camera. For example, even if the camera is not located on the same surface as the watch face, or even if it is located on the same surface, the size of the smart watch may increase and usability may decrease. In addition, since it is inconvenient for the user to look at the camera instead of looking at the watch face in order to use the camera, usability may be reduced.

In addition, since the list-up function provided by the smart watch detects a gesture through a sensor, there is a possibility that misrecognition may occur. For example, when the user drives or exercises, an unintentional list-up function may be performed. In order to reduce such misrecognition, a large number of sensors may be mounted on the smart watch, but this may increase the size of the smart watch or increase the manufacturing cost.

According to various embodiments of the present disclosure, a gesture for performing a list-up function is checked through a sensor, a user's face and gaze are checked through a UDC mounted on the wearable electronic device, and the gesture, the face, and a gaze are performed. It is possible to provide an electronic device capable of determining whether to turn on a display included in the wearable electronic device based on , and adjusting a screen displayed on the display, and an operating method thereof.

A wearable electronic device according to various embodiments of the present disclosure includes a first sensor, a display, a camera disposed under a rear surface that is not exposed to the outside of the display, and a processor, wherein the processor, through the first sensor, A first gesture for the wearable electronic device is checked, and the face and gaze of the user are checked based on an image including the user's face captured through the camera, and the first gesture, the face, and the gaze determines whether to turn on the display based on It can be set to adjust the screen.

The method of operating the wearable electronic device according to various embodiments of the present disclosure may include an operation of identifying a first gesture for the wearable electronic device through a first sensor included in the wearable electronic device, and exposing the wearable electronic device to the outside of a display Based on an image including the user's face photographed through a camera disposed below the rear surface, the operation of confirming the user's face and gaze, the first gesture, the face, and the gaze Determining whether to turn on the display of the electronic device, and when the display is turned on, the display is displayed on the display based on the distance between the face and the wearable electronic device, the direction of the face, and the angle of the gaze It may include an operation for adjusting the screen to be displayed.

In the non-transitory storage medium according to various embodiments of the present disclosure, when the program is executed by a processor, the processor checks a first gesture for the wearable electronic device through a first sensor included in the wearable electronic device operation, the operation of confirming the user's face and gaze based on an image including the user's face photographed through a camera disposed under the rear surface of the display of the wearable electronic device that is not exposed to the outside; 1 The operation of determining whether to turn on the display of the electronic device based on the gesture, the face, and the gaze, and determining whether to turn on the display, the distance between the face and the wearable electronic device, the direction of the face, and an instruction executable to adjust the screen displayed on the display based on the angle of the gaze.

A wearable electronic device according to various embodiments of the present disclosure may reduce misrecognition of a list-up function and increase usability by using UDC.

1 is a diagram illustrating a network environment according to various embodiments of the present disclosure;

2 is a diagram illustrating a configuration example of a wearable electronic device according to various embodiments of the present disclosure.

3 is a diagram illustrating a configuration example of an electronic device according to various embodiments of the present disclosure.

4 is a block diagram illustrating a schematic configuration of a wearable electronic device according to various embodiments of the present disclosure.

5 is a flowchart illustrating a method of operating a wearable electronic device according to various embodiments of the present disclosure.

6 is a diagram for describing an operation of a wearable electronic device checking a user's face and gaze, according to various embodiments of the present disclosure;

7 is a flowchart illustrating an operation of adjusting a screen of a wearable electronic device based on a distance from a user's face, according to various embodiments of the present disclosure.

8 is a diagram for describing an operation of adjusting a screen of a wearable electronic device based on a distance from a user's face according to various embodiments of the present disclosure;

9A is a flowchart illustrating an operation of adjusting a screen of a wearable electronic device when a user other than a user is identified, according to various embodiments of the present disclosure;

FIG. 9B is a diagram for describing an operation of adjusting a screen of a wearable electronic device when a user other than a user is identified, according to various embodiments of the present disclosure;

10 is a diagram for explaining an operation of adjusting a screen based on a user's gaze by a wearable electronic device according to various embodiments of the present disclosure;

11 is a flowchart illustrating an operation in which a wearable electronic device adjusts a screen based on user's biometric information according to various embodiments of the present disclosure.

12 is a diagram for explaining an operation of adjusting a screen of a wearable electronic device based on user's biometric information according to various embodiments of the present disclosure;

13 is a flowchart illustrating an operation in which a wearable electronic device adjusts a screen based on a user's gesture according to various embodiments of the present disclosure;

14 is a diagram for explaining an operation of adjusting a screen of a wearable electronic device based on a user's gesture according to various embodiments of the present disclosure;

15 is a flowchart illustrating an operation in which a wearable electronic device transmits an unlock signal to another electronic device according to various embodiments of the present disclosure;

Hereinafter, an electronic device according to various embodiments will be described with reference to the accompanying drawings. The term user used in various embodiments may refer to a person who uses an electronic device or a device (eg, an artificial intelligence electronic device) using the electronic device.

1 is a block diagram of an electronic device 101 in a network environment 100, according to various embodiments. Referring to FIG. 1 , in a network environment 100 , the electronic device 101 communicates with the electronic device 102 through a first network 198 (eg, a short-range wireless communication network) or a second network 199 . It may communicate with the electronic device 104 or the server 108 through (eg, a long-distance wireless communication network). According to an embodiment, the electronic device 101 may communicate with the electronic device 104 through the server 108 . According to an embodiment, the electronic device 101 includes a processor 120 , a memory 130 , an input module 150 , a sound output module 155 , a display module 160 , an audio module 170 , and a sensor module ( 176), interface 177, connection terminal 178, haptic module 179, camera module 180, power management module 188, battery 189, communication module 190, subscriber identification module 196 , or an antenna module 197 . In some embodiments, at least one of these components (eg, the connection terminal 178 ) may be omitted or one or more other components may be added to the electronic device 101 . In some embodiments, some of these components (eg, sensor module 176 , camera module 180 , or antenna module 197 ) are integrated into one component (eg, display module 160 ). can be

The processor 120, for example, executes software (eg, a program 140) to execute at least one other component (eg, a hardware or software component) of the electronic device 101 connected to the processor 120. It can control and perform various data processing or operations. According to one embodiment, as at least part of data processing or operation, the processor 120 converts commands or data received from other components (eg, the sensor module 176 or the communication module 190 ) to the volatile memory 132 . may be stored in , process commands or data stored in the volatile memory 132 , and store the result data in the non-volatile memory 134 . According to an embodiment, the processor 120 is the main processor 121 (eg, a central processing unit or an application processor) or a secondary processor 123 (eg, a graphic processing unit, a neural network processing unit (eg, a graphic processing unit, a neural network processing unit) a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, when the electronic device 101 includes the main processor 121 and the sub-processor 123 , the sub-processor 123 uses less power than the main processor 121 or is set to be specialized for a specified function. can The auxiliary processor 123 may be implemented separately from or as a part of the main processor 121 .

The secondary processor 123 may, for example, act on behalf of the main processor 121 while the main processor 121 is in an inactive (eg, sleep) state, or when the main processor 121 is active (eg, executing an application). ), together with the main processor 121, at least one of the components of the electronic device 101 (eg, the display module 160, the sensor module 176, or the communication module 190) It is possible to control at least some of the related functions or states. According to an embodiment, the coprocessor 123 (eg, an image signal processor or a communication processor) may be implemented as part of another functionally related component (eg, the camera module 180 or the communication module 190 ). have. According to an embodiment, the auxiliary processor 123 (eg, a neural network processing device) may include a hardware structure specialized for processing an artificial intelligence model. Artificial intelligence models can be created through machine learning. Such learning may be performed, for example, in the electronic device 101 itself on which artificial intelligence is performed, or may be performed through a separate server (eg, the server 108). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but in the above example not limited The artificial intelligence model may include a plurality of artificial neural network layers. Artificial neural networks include deep neural networks (DNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), restricted boltzmann machines (RBMs), deep belief networks (DBNs), bidirectional recurrent deep neural networks (BRDNNs), It may be one of deep Q-networks or a combination of two or more of the above, but is not limited to the above example. The artificial intelligence model may include, in addition to, or alternatively, a software structure in addition to the hardware structure.

The memory 130 may store various data used by at least one component (eg, the processor 120 or the sensor module 176 ) of the electronic device 101 . The data may include, for example, input data or output data for software (eg, the program 140 ) and instructions related thereto. The memory 130 may include a volatile memory 132 or a non-volatile memory 134 .

The program 140 may be stored as software in the memory 130 , and may include, for example, an operating system 142 , middleware 144 , or an application 146 .

The input module 150 may receive a command or data to be used by a component (eg, the processor 120 ) of the electronic device 101 from the outside (eg, a user) of the electronic device 101 . The input module 150 may include, for example, a microphone, a mouse, a keyboard, a key (eg, a button), or a digital pen (eg, a stylus pen).

The sound output module 155 may output a sound signal to the outside of the electronic device 101 . The sound output module 155 may include, for example, a speaker or a receiver. The speaker can be used for general purposes such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from or as part of the speaker.

The display module 160 may visually provide information to the outside (eg, a user) of the electronic device 101 . The display module 160 may include, for example, a control circuit for controlling a display, a hologram device, or a projector and a corresponding device. According to an embodiment, the display module 160 may include a touch sensor configured to sense a touch or a pressure sensor configured to measure the intensity of a force generated by the touch.

The audio module 170 may convert a sound into an electric signal or, conversely, convert an electric signal into a sound. According to an embodiment, the audio module 170 acquires a sound through the input module 150 , or an external electronic device (eg, a sound output module 155 ) connected directly or wirelessly with the electronic device 101 . The electronic device 102) (eg, a speaker or headphones) may output a sound.

The sensor module 176 detects an operating state (eg, power or temperature) of the electronic device 101 or an external environmental state (eg, a user state), and generates an electrical signal or data value corresponding to the sensed state. can do. According to an embodiment, the sensor module 176 may include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, It may include a temperature sensor, a humidity sensor, or an illuminance sensor.

The interface 177 may support one or more specified protocols that may be used by the electronic device 101 to directly or wirelessly connect with an external electronic device (eg, the electronic device 102 ). According to an embodiment, the interface 177 may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

The connection terminal 178 may include a connector through which the electronic device 101 can be physically connected to an external electronic device (eg, the electronic device 102 ). According to an embodiment, the connection terminal 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (eg, a headphone connector).

The haptic module 179 may convert an electrical signal into a mechanical stimulus (eg, vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic sense. According to an embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

The camera module 180 may capture still images and moving images. According to an embodiment, the camera module 180 may include one or more lenses, image sensors, image signal processors, or flashes.

The power management module 188 may manage power supplied to the electronic device 101 . According to an embodiment, the power management module 188 may be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

The battery 189 may supply power to at least one component of the electronic device 101 . According to one embodiment, battery 189 may include, for example, a non-rechargeable primary cell, a rechargeable secondary cell, or a fuel cell.

The communication module 190 is a direct (eg, wired) communication channel or a wireless communication channel between the electronic device 101 and an external electronic device (eg, the electronic device 102, the electronic device 104, or the server 108). It can support establishment and communication performance through the established communication channel. The communication module 190 may include one or more communication processors that operate independently of the processor 120 (eg, an application processor) and support direct (eg, wired) communication or wireless communication. According to one embodiment, the communication module 190 is a wireless communication module 192 (eg, a cellular communication module, a short-range communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (eg, : It may include a local area network (LAN) communication module, or a power line communication module). A corresponding communication module among these communication modules is a first network 198 (eg, a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network 199 (eg, legacy It may communicate with the external electronic device 104 through a cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (eg, a telecommunication network such as a LAN or a WAN). These various types of communication modules may be integrated into one component (eg, a single chip) or may be implemented as a plurality of components (eg, multiple chips) separate from each other. The wireless communication module 192 uses subscriber information (eg, International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module 196 within a communication network such as the first network 198 or the second network 199 . The electronic device 101 may be identified or authenticated.

The wireless communication module 192 may support a 5G network after a 4G network and a next-generation communication technology, for example, a new radio access technology (NR). NR access technology includes high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and access to multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency) -latency communications)). The wireless communication module 192 may support a high frequency band (eg, mmWave band) to achieve a high data rate, for example. The wireless communication module 192 uses various techniques for securing performance in a high-frequency band, for example, beamforming, massive multiple-input and multiple-output (MIMO), all-dimensional multiplexing. It may support technologies such as full dimensional MIMO (FD-MIMO), an array antenna, analog beam-forming, or a large scale antenna.

The wireless communication module 192 may support various requirements defined in the electronic device 101 , an external electronic device (eg, the electronic device 104 ), or a network system (eg, the second network 199 ). According to an embodiment, the wireless communication module 192 may provide a peak data rate (eg, 20 Gbps or more) for realizing 1eMBB, loss coverage (eg, 164 dB or less) for realizing mMTC, or U-plane latency for realizing URLLC ( Example: Downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) can be supported.

The antenna module 197 may transmit or receive a signal or power to the outside (eg, an external electronic device). According to an embodiment, the antenna module 197 may include an antenna including a conductor formed on a substrate (eg, a PCB) or a radiator formed of a conductive pattern. According to an embodiment, the antenna module 197 may include a plurality of antennas (eg, an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network such as the first network 198 or the second network 199 is connected from the plurality of antennas by, for example, the communication module 190 . can be selected. A signal or power may be transmitted or received between the communication module 190 and an external electronic device through the selected at least one antenna. According to some embodiments, other components (eg, a radio frequency integrated circuit (RFIC)) other than the radiator may be additionally formed as a part of the antenna module 197 .

According to various embodiments, the antenna module 197 may form a mmWave antenna module. According to one embodiment, the mmWave antenna module comprises a printed circuit board, an RFIC disposed on or adjacent to a first side (eg, bottom side) of the printed circuit board and capable of supporting a designated high frequency band (eg, mmWave band); and a plurality of antennas (eg, an array antenna) disposed on or adjacent to a second side (eg, top or side) of the printed circuit board and capable of transmitting or receiving signals of the designated high frequency band. can do.

At least some of the components are connected to each other through a communication method between peripheral devices (eg, a bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI)) and a signal ( e.g. commands or data) can be exchanged with each other.

According to an embodiment, the command or data may be transmitted or received between the electronic device 101 and the external electronic device 104 through the server 108 connected to the second network 199 . Each of the external

electronic devices

102 or 104 may be the same as or different from the electronic device 101 . According to an embodiment, all or a part of operations executed in the electronic device 101 may be executed in one or more external

electronic devices

102 , 104 , or 108 . For example, when the electronic device 101 needs to perform a function or service automatically or in response to a request from a user or other device, the electronic device 101 may perform the function or service itself instead of executing the function or service itself. Alternatively or additionally, one or more external electronic devices may be requested to perform at least a part of the function or the service. One or more external electronic devices that have received the request may execute at least a part of the requested function or service, or an additional function or service related to the request, and transmit a result of the execution to the electronic device 101 . The electronic device 101 may process the result as it is or additionally and provide it as at least a part of a response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. 111 The electronic device 101 may provide an ultra-low latency service using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device 104 may include an Internet of things (IoT) device. The server 108 may be an intelligent server using machine learning and/or neural networks. According to an embodiment, the external electronic device 104 or the server 108 may be included in the second network 199 . The electronic device 101 may be applied to an intelligent service (eg, smart home, smart city, smart car, or health care) based on 5G communication technology and IoT-related technology.

The electronic device according to various embodiments disclosed in this document may have various types of devices. The electronic device may include, for example, a portable communication device (eg, a smart phone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a home appliance device. The electronic device according to the embodiment of the present document is not limited to the above-described devices.

The various embodiments of this document and terms used therein are not intended to limit the technical features described in this document to specific embodiments, but it should be understood to include various modifications, equivalents, or substitutions of the embodiments. In connection with the description of the drawings, like reference numerals may be used for similar or related components. The singular form of the noun corresponding to the item may include one or more of the item, unless the relevant context clearly dictates otherwise. As used herein, "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C", and "A , B, or C," each of which may include any one of the items listed together in the corresponding one of the phrases, or all possible combinations thereof. Terms such as "first", "second", or "first" or "second" may simply be used to distinguish an element from other elements in question, and may refer elements to other aspects (e.g., importance or order) is not limited. It is said that one (eg, first) component is “coupled” or “connected” to another (eg, second) component, with or without the terms “functionally” or “communicatively”. When referenced, it means that one component can be connected to the other component directly (eg by wire), wirelessly, or through a third component.

The term “module” used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and is interchangeable with terms such as, for example, logic, logic block, component, or circuit. can be used as A module may be an integrally formed part or a minimum unit or a part of the part that performs one or more functions. For example, according to an embodiment, the module may be implemented in the form of an application-specific integrated circuit (ASIC).

Various embodiments of the present document include one or more instructions stored in a storage medium (eg, internal memory 136 or external memory 138) readable by a machine (eg, electronic device 101). may be implemented as software (eg, the program 140) including For example, the processor (eg, the processor 120 ) of the device (eg, the electronic device 101 ) may call at least one of the one or more instructions stored from the storage medium and execute it. This makes it possible for the device to be operated to perform at least one function according to the called at least one command. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' only means that the storage medium is a tangible device and does not contain a signal (eg, electromagnetic wave), and this term is used in cases where data is semi-permanently stored in the storage medium and It does not distinguish between temporary storage cases.

According to one embodiment, the method according to various embodiments disclosed in this document may be provided in a computer program product (computer program product). Computer program products may be traded between sellers and buyers as commodities. The computer program product is distributed in the form of a machine-readable storage medium (eg compact disc read only memory (CD-ROM)), or via an application store (eg Play Store ^TM ) or on two user devices ( It can be distributed (eg downloaded or uploaded) directly or online between smartphones (eg: smartphones). In the case of online distribution, at least a part of the computer program product may be temporarily stored or temporarily created in a machine-readable storage medium such as a memory of a server of a manufacturer, a server of an application store, or a relay server.

According to various embodiments, each component (eg, a module or a program) of the above-described components may include a singular or a plurality of entities, and some of the plurality of entities may be separately disposed in other components. have. According to various embodiments, one or more components or operations among the above-described corresponding components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (eg, a module or a program) may be integrated into one component. In this case, the integrated component may perform one or more functions of each component of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. . According to various embodiments, operations performed by a module, program, or other component are executed sequentially, in parallel, repeatedly, or heuristically, or one or more of the operations are executed in a different order, or omitted. , or one or more other operations may be added.

2 is a diagram illustrating a configuration example of an electronic device according to various embodiments, and FIG. 3 is a diagram illustrating a configuration example of an electronic device according to various embodiments.

Referring to FIG. 2 , an electronic device 101 (eg, the electronic device 101 of FIG. 1 ) according to an embodiment includes a display module 160 (eg, FIG. 1 ) disposed on a front surface 201 of a housing 200 . 1 ) and a camera 210 (eg, the camera module 180 of FIG. 1 ). The electronic device 101 according to an embodiment may include a memory 130 , a display module 160 , a camera 210 , and at least one processor 120 electrically connected to the memory 130 . In addition to this, the electronic device 101 according to an embodiment may further include other components described with reference to FIG. 1 . According to an embodiment, the electronic device 101 may be implemented as a wearable electronic device. For example, the electronic device 101 may be implemented as a watch-type wearable electronic device (eg, a smart watch).

Referring to FIG. 3 , according to an embodiment, the display module 160 is disposed to be exposed on the front surface 201 of the housing 200 , and the window 310 is disposed such that the first surface is exposed to the outside, and the window The display panel 320 may be disposed under the second surface (rear surface) of the 310 . The display panel 320 may include a substrate (eg, a flexible circuit board (FPCB)) 340 and a display element layer 311 disposed on the substrate 340 . The display panel 320 may be configured in the form of a touch sensitive panel (TSP). The display element layer 311 is a circuit layer including a thin film transistor (TFT) (not shown), an organic light emitting diode (OLED) (not shown) as a display element, and an insulating layer (IL) therebetween (not shown). ) may be included. The display panel 320 may include a display driver integrated circuit (not shown). According to an embodiment, the transparent glass layer 310 and the display panel 320 may be configured to have at least a part curved shape. The display panel 320 may be formed of a flexible polymer film, and may include, for example, polyimide, polyethylene terephthalate, or other polymer material. The display panel 320 may include a first polymer layer 313 (eg, polyimide) and a second polymer layer 315 (eg, polyethylene terephthalate) disposed below the display element layer 311 . have.

According to an exemplary embodiment, the camera 210 may be disposed under a rear surface (eg, a second surface) that is not exposed to the outside of the display module 160 . As shown in FIG. 3 , the camera 210 may be an under display camera (eg, an under display camera (UDC) camera) 210 in which at least a portion of the camera module 180 is disposed under the display panel 320 . . The camera 210 may be disposed between the display panel 320 and the substrate 340 . For example, the camera 210 may be at least a part of the camera module 180 of FIG. 1 , and may not be exposed because it is included in the housing 200 . The camera 210 is a camera sensor (eg, an image sensor) (not shown) that detects light incident through the window 310 through a lens and converts the light into a digital signal to obtain an image, an image processing image It may be configured to include at least one of a processing module (not shown) or a memory (not shown) for storing images.

Referring to FIG. 4 , according to various embodiments of the present disclosure, the wearable electronic device 401 includes a camera 410 , a processor 420 , a memory 430 , a first sensor 440 , a second sensor 450 , and a display. 460 , and a communication module 480 . For example, the wearable electronic device 401 may be implemented in the same or similar manner to the electronic device 101 of FIG. 1 . For example, the wearable electronic device 401 may be implemented as a watch-type wearable electronic device (eg, a smart watch).

According to various embodiments, the camera 410 may be implemented as an under display camera (UDC). For example, the camera 410 may be implemented in the same or similar manner to the camera 210 of FIG. 2 . For example, the camera 410 may be located in a central region of the display 460 (eg, the display 160 of FIG. 2 ).

According to various embodiments, the processor 420 may control the overall operation of the wearable electronic device 420 . For example, the processor 420 may be implemented in the same or similar manner to the processor 120 of FIG. 1 .

According to various embodiments, the processor 420 may identify a first gesture with respect to the wearable electronic device 401 through the first sensor 440 . For example, the first sensor 440 may sense a gesture with respect to the wearable electronic device 401 . For example, the first sensor 440 may include at least one of an acceleration sensor, a speed sensor, and a motion sensor. For example, the first gesture may include a gesture of raising the wrist while the user wears the wearable electronic device 401 (eg, a smart watch).

According to various embodiments, the processor 420 may capture and acquire an image including the user's face through the camera 410 . The processor 420 may check the user's face and gaze based on the image including the user's face captured through the camera 410 . For example, the processor 420 may check the size of the user's face, the direction of the face, and the direction of the gaze, based on the image. For example, the processor 420 may determine the direction of the face based on the nose and/or both eyes included in the face. For example, the processor 420 may determine the direction of the gaze by checking the two eyes included in the face. The processor 420 may check whether the direction of the face and the direction of the gaze coincide with the camera 410 .

According to various embodiments, the processor 420 may determine whether to turn on the display 460 based on the first gesture, the user's face, and the user's gaze. For example, the processor 420 may raise the wrist while the first gesture is a gesture (eg, the user wearing the wearable electronic device 401 ) pre-stored in the memory 430 (eg, the memory 130 of FIG. 1 ). You can check whether it matches the raising gesture). Also, the processor 420 may check whether the direction of the user's face and/or the direction of the gaze coincides with the camera 410 . The processor 420 may turn on the display when the first gesture matches the pre-stored gesture and the direction of the user's face and the direction of the gaze coincide with the camera 410 . Alternatively, the processor 420 may turn on the display when the first gesture matches a pre-stored gesture and the user's gaze direction matches the camera 410 . For example, the processor 420 may turn on the display 460 if the first gesture matches the pre-stored gesture and the direction of the gaze matches the camera 410 even if the direction of the face does not match the camera 410 . can

According to various embodiments, when the processor 420 determines to turn on the display 460 , the processor 420 may display a screen on the display 460 . For example, the screen may include at least one of a watch face, a message, a user interface, an image, an application execution screen, and a home screen. The processor 420 is configured to be displayed on the display based on the distance between the user's face and the wearable electronic device 401 (or the camera 410), the direction of the user's face, and the direction (or angle) of the user's gaze. You can adjust the screen. For example, the adjustment of the screen may include changing at least one object included in the screen (eg, changing a color, changing a location, and/or changing a size) or changing from an existing screen to another screen.

According to various embodiments, when the user's gaze remains the same for a specified time, the processor 420 determines the size of the screen displayed on the display 460 based on the distance between the user's face and the wearable electronic device 401 . can be enlarged or reduced. For example, the processor 420 may enlarge the size of the screen as the distance between the user's face and the wearable electronic device 401 increases. Alternatively, the processor 420 may reduce the size of the screen as the distance between the user's face and the wearable electronic device 401 increases. For example, the processor 420 checks the size of the user's face and the distance between the user's eyes based on the image including the user's face, and based on the size of the user's face and the distance between the user's eyes The distance between the user's face and the wearable electronic device may be checked.

According to various embodiments, when a user other than a user previously registered in an image captured by the camera 410 is identified, the processor 420 may display only some information on a screen displayed on the display 460 . For example, the processor 420 may check the privacy level of content displayed on the screen and determine information displayed on the screen according to the privacy level. In addition, the processor 420 may change the display position of the information displayed on the display 460 when a user other than the user previously registered in the image captured by the camera 410 is identified. For example, the processor 420 may display information in an area (eg, an area away from other users) that other users other than the pre-registered user cannot see in detail.

According to various embodiments, the processor 420 may adjust the position of the notification window displayed on the display 460 based on the direction of the face and the direction (or angle) of the gaze. For example, the processor 420 may display a notification window at a position toward which the user's face and gaze are directed. According to another embodiment, the processor 420 may adjust the position of the watch face displayed on the display 460 based on the direction of the face and the direction (or angle) of the gaze. For example, the processor 420 may display the watch face at a position toward which the user's face and gaze are directed.

According to various embodiments, the processor 420 checks a second gesture (eg, a gesture of spreading the hand) for the movement of the user's hand through the camera 410, and displays it on the display 460 based on the second gesture. The displayed screen may be adjusted (eg, enlarged or reduced in screen size). For example, the processor 420 checks a second gesture for the user's movement based on images continuously captured by the camera 410 while the screen is displayed on the display 460 , and the second gesture You can check whether or not matches the specified gesture. When the second gesture matches the designated gesture, the processor 420 may adjust the screen of the display 460 according to a function corresponding to the second gesture.

According to various embodiments, the processor 420 may acquire the user's biometric information through the second sensor 450 . For example, the second sensor 450 may include a PPG sensor, an EKG sensor, and/or an ECG sensor capable of measuring a user's biosignal. The processor 420 may update the screen displayed on the display 460 based on the biometric information obtained through the second sensor 450 . For example, the processor 420 may check the user's mood state and/or health state through the biometric information, and update the screen according to the confirmed mood state and/or health state.

According to various embodiments, the processor 420 may transmit/receive data (or signals) to and from the external electronic device 402 through the communication module 480 . For example, the external electronic device 402 may include a terminal, a smartphone, and an AI speaker. The processor 420 may determine whether the user is a pre-registered user based on the user's face and gaze included in the image captured by the camera 410 . When the user is identified as a pre-registered user, the processor 420 may transmit a signal for requesting unlocking to the external electronic device 402 .

According to various embodiments, when the user is identified as a pre-registered user, the processor 420 may display data received from the external electronic device 402 on the display 460 . In addition, after the user is identified as a pre-registered user, the processor 420 receives the privacy-related data received from the external electronic device 402 when the user's face and gaze are directed toward the wearable electronic device 401 . It may be displayed on the display 460 .

Meanwhile, at least some of the following operations of the electronic device may be performed by the processor 420 of FIG. 4 . However, hereinafter, it is assumed that the processor 420 performs an operation for convenience of description.

Referring to FIG. 5 , according to various embodiments, in operation 501 , the wearable electronic device (eg, the wearable electronic device 401 of FIG. 4 ) includes a first sensor (eg, the first sensor 440 of FIG. 4 ). Through , the first gesture for the wearable electronic device 401 may be confirmed. For example, the processor 420 may check whether the first gesture matches a pre-registered gesture.

According to various embodiments of the present disclosure, in operation 503 , the wearable electronic device 401 receives the user's face based on the image including the user's face captured through the camera UDC (eg, the camera 410 of FIG. 4 ). Face and gaze can be checked. For example, the processor 420 may determine whether the user is using or looking at the wearable electronic device 401 based on the user's face direction and the angle of the gaze.

According to various embodiments, in operation 505 , the wearable electronic device 401 determines whether to turn on a display (eg, the display 460 of FIG. 4 ) based on the first gesture, face, and gaze. can For example, the processor 420 turns on the display 460 when it is confirmed that the first gesture matches the pre-registered gesture and the user is looking at the wearable electronic device 401 based on the face direction and the gaze angle. on) can be turned on.

According to various embodiments of the present disclosure, in operation 507 , when the wearable electronic device 401 determines to turn on the display, the display 460 is based on the distance between the user's face and the wearable electronic device, the direction of the face, and the angle of the gaze. ) can be adjusted. For example, the processor 420 may change the size of the screen or change the display position based on the distance between the user's face and the wearable electronic device, the direction of the face, and the angle of the gaze.

Referring to FIG. 6 , according to various embodiments, a wearable electronic device (eg, the wearable electronic device 401 of FIG. 4 ) uses a camera UDC (eg, the camera 410 of FIG. 4 ) by a user It may be determined whether the wearable electronic device 401 is being used or not. For example, the wearable electronic device 401 checks the direction of the user's face and the angle of the gaze (or the direction of the gaze) in the image projected through the camera (UDC) 410 , and determines the direction of the face and the angle of the gaze. Based on the determination, whether the user is using the wearable electronic device 401 may be determined.

According to various embodiments of the present disclosure, the wearable electronic device 401 allows the user to select the wearable electronic device 401 based on the direction of the gaze while the front of the user's face is confirmed in the image captured by the camera 410 . You can determine whether it is being used or not.

According to various embodiments, assuming that the camera 410 is positioned in front, the wearable electronic device 401 may acquire an

image

610 , 620 , 630 , or 640 including the user's face. For example, as shown in FIG. 6A , when the wearable electronic device 401 confirms that the direction of the face and the direction of the gaze coincide with the camera 410 in the image 610 including the user's face, the user may determine that the wearable electronic device 401 is being used or intends to be used. Also, as shown in FIG. 6B , in the wearable electronic device 401 , in the image 620 including the user's face, the direction of the face does not coincide with the camera 410 , but the direction of the gaze is the camera 410 . If matching with , it may be determined that the user is using or intends to use the wearable electronic device 401 . On the other hand, as shown in FIG. 6C , in the wearable electronic device 401 , in the image 630 including the user's face, the direction of the face coincides with the camera 410 , but the direction of the gaze is the camera 410 . If it is confirmed that the user is not using the wearable electronic device 401 , it may be determined that the user is not using the wearable electronic device 401 or does not intend to use the wearable electronic device 401 . Also, as shown in (d) of FIG. 6 , when the wearable electronic device 401 confirms that the direction of the face and the direction of the gaze do not coincide with the camera 410 in the image 640 including the user's face, It may be determined that the user is not using the wearable electronic device 401 or does not intend to use it.

According to various embodiments of the present disclosure, when the user's face is not identified or the front of the face is not identified in the image captured by the camera 410 , the wearable electronic device 401 indicates that the user is not using the wearable electronic device 401 . Or it may be determined that there is no intention to use it.

According to various embodiments, the wearable electronic device 401 turns on a display (eg, the display 460 of FIG. 4 ) based on the first gesture, the direction of the face, and the angle of the gaze (or the direction of the gaze). on) can be decided.

According to various embodiments of the present disclosure, after turning on the display 460 , the wearable electronic device 401 displays a screen displayed on the display 460 based on the user's face and gaze in the image captured by the camera 410 . Can be adjusted.

Referring to FIG. 7 , according to various embodiments, in operation 701 , the wearable electronic device 401 (continuously) captures a plurality of images through the camera 410 to maintain the same gaze of the user for a specified time. can check whether For example, the wearable electronic device 401 may check whether the user's gaze looks at the display 460 of the camera 410 for a specified time.

According to various embodiments, in operation 703 , the wearable electronic device 401 may check the size of the user's face and the distance between the eyes included in the face.

According to various embodiments, in operation 705 , the wearable electronic device 401 may identify a distance between the user's face and the wearable electronic device 401 based on the size of the user's face and the distance between the eyes.

According to various embodiments, in operation 707 , the wearable electronic device 401 may enlarge or reduce the size of the screen displayed on the display 460 based on the distance between the user's face and the wearable electronic device 401 . have. For example, the wearable electronic device 401 may enlarge the size of the screen as the distance between the user's face and the wearable electronic device 401 increases. Also, the wearable electronic device 401 may reduce the size of the screen as the distance between the user's face and the wearable electronic device 401 increases.

Referring to FIG. 8 , the wearable electronic device (eg, the wearable electronic device 401 of FIG. 4 ) checks the user's face included in the image captured by the camera 410 , and determines the size and eyes of the user's face. The distance between the user's face and the wearable electronic device 401 may be checked based on the distance between them.

According to various embodiments of the present disclosure, the wearable electronic device 401 may identify the user face 810 of the first size. The wearable electronic device 401 may determine that the distance between the user's face and the wearable electronic device 401 is the first distance d1 based on the user's face 810 of the first size. The wearable electronic device 401 may display the first screen 820 on the display (eg, the display 460 of FIG. 4 ) based on the first distance d1 .

According to various embodiments, the wearable electronic device 401 may identify the user face 830 of the second size. For example, the second size may be smaller than the first size. The wearable electronic device 401 may determine that the distance between the user's face and the wearable electronic device 401 is the second distance d2 based on the user's face 830 of the second size. The wearable electronic device 401 may display the second screen 840 on the display 460 based on the second distance d2 . For example, the size of the second screen 840 may be larger than the size of the first screen 820 . The second distance d2 may be longer than the first distance.

Although FIG. 8 describes only a method of determining the size of a screen based on the size of a face, the technical spirit of the present invention may not be limited thereto. For example, the wearable electronic device 401 may check the distance between the user's face and the wearable electronic device 401 based on a change in the distance between the user's eyes in the same or similar manner as described above.

Referring to FIG. 9A , according to various embodiments, in operation 901 , the wearable electronic device (the wearable electronic device 401 of FIG. 4 ) takes a picture through a camera UDC (eg, the camera 410 of FIG. 4 ). It can be checked whether a user other than the user is included in the image including the user's face. For example, the wearable electronic device 401 may check whether a user other than a pre-registered user (eg, a user of the wearable electronic device 401) is included in the image.

According to various embodiments of the present disclosure, if it is determined that another user is included (eg in operation 903 ), in operation 905 , the wearable electronic device 401 displays a screen displayed on a display (eg, the display 460 in FIG. 4 ). Only some information in the message can be displayed. For example, the displayed information among the information of the message may be determined by the privacy level of the corresponding message. For example, for a message with the highest privacy level (eg, a secure authentication message), only the message subject may be displayed. Alternatively, in a message having an intermediate privacy level (eg, a private message), a message subject and a part of message content may be displayed.

According to various embodiments, if it is not confirmed that other users are included (No in operation 903 ), in operation 907 , the wearable electronic device 401 displays all information of the message on the screen displayed on the display 460 . can However, the wearable electronic device 401 may set to display only some information of the message according to the privacy level according to the user's setting.

Referring to (a) of FIG. 9B , according to various embodiments, a wearable electronic device (eg, the wearable electronic device 401 of FIG. 4 ) displays a message 950 on a display (eg, the display 460 of FIG. 4 ). can be displayed.

According to various embodiments of the present disclosure, the wearable electronic device 401 checks whether a user other than the user is included in the image including the user's face captured through the camera UDC (eg, the camera 410 of FIG. 4 ). can When it is confirmed that another user is included, the wearable electronic device 401 may display only partial information of a message displayed on the display 460 . For example, referring to (b) of FIG. 9B , the wearable electronic device 401 may display only the title 960 of the message on the display 460 . The wearable electronic device 401 may display the display position of the title 960 of the message on the lower area of the display 460 . Also, the size at which the title 960 of the message is displayed may be reduced compared to the size of the existing message 950 .

Referring to FIG. 10 , according to various embodiments of the present disclosure, the wearable electronic device (eg, the wearable electronic device 401 of FIG. 4 ) has an image captured through a camera UDC (eg, the camera 410 of FIG. 4 ). You can check the user's face included in the . The wearable electronic device 401 may adjust the position of the

notification window

1010 , 1020 , 1030 , 1040 , or 1050 displayed on the display 460 based on the face direction and the gaze direction (or angle).

According to various embodiments of the present disclosure, the wearable electronic device 401 may display a notification window at a position toward which the user's face and gaze are directed. For example, when the position to which the user's face and gaze are directed is confirmed as the first position (eg, the central region of the display 460 ), the wearable electronic device 401 may display an area corresponding to the first position (eg, the display 460 ). ) may display a notification window 1010 in the middle area). For example, when the position to which the user's face and gaze are directed is confirmed as the second position (eg, the left region of the display 460 ), the wearable electronic device 401 may display an area corresponding to the second position (eg, the display 460 ). ), a notification window 1020 may be displayed. For example, when the position to which the user's face and gaze are directed is confirmed as the third position (eg, the upper region of the display 460 ), the wearable electronic device 401 may display an area corresponding to the third position (eg, the display 460 ). ) may display a notification window 1030 in the upper area). For example, when the position to which the user's face and gaze are directed is confirmed as the fourth position (eg, the region to the right of the display 460 ), the wearable electronic device 401 may display an area corresponding to the fourth position (eg, the display 460 ). ), a notification window 1040 may be displayed. For example, in the wearable electronic device 401 , when the position to which the user's face and gaze are directed is identified as the fifth position (eg, the lower region of the display 460 ), the region corresponding to the fifth position (eg, the display 460 ) ), a notification window 1050 may be displayed. In this case, the wearable electronic device 401 may adjust the shape of the notification window displayed on the display 460 based on the position where the notification window is displayed.

Referring to FIG. 11 , according to various embodiments, in operation 1101 , the wearable electronic device (eg, the wearable electronic device 401 of FIG. 4 ) may acquire the user's biometric information through the second sensor.

According to various embodiments, in operation 1103 , the wearable electronic device 401 may update a screen displayed on a display (eg, the display 460 of FIG. 4 ) in consideration of the user's biometric information. For example, the wearable electronic device 401 may check the user's mood or health status based on the user's biometric information. The wearable electronic device 401 may update the watch face in consideration of the user's mood or health condition.

According to various embodiments, the wearable electronic device (eg, the wearable electronic device 401 of FIG. 4 ) may display the first watch face 1210 on a display (eg, the display 460 of FIG. 4 ). For example, the first watch face 1210 may be a basic watch face.

According to various embodiments, the wearable electronic device 401 may acquire the user's biometric information through a second sensor (eg, the second sensor 450 of FIG. 4 ). For example, the wearable electronic device 401 may check the user's mood state (or health state) based on the user's biometric information.

According to various embodiments, the wearable electronic device 401 may update the watch face 1210 in consideration of the user's mood state (or health state). For example, when the user is "tired" as a result of checking the biometric information, the wearable electronic device 401 may update the first watch face 1210 to the second watch face 1220 . For example, the second watch face 1220 may include a color indicating the degree of “tiredness”. For example, when the user is in a “depressed” state as a result of checking the biometric information, the wearable electronic device 401 may update the first watch face 1210 to the third watch face 1230 . For example, the third watch face 1230 may include a color indicating the degree of "depressed". For example, the wearable electronic device 401 may update the first watch face 1210 to the fourth watch face 1240 when the user is in a "nervous" state as a result of checking the biometric information. For example, the fourth watch face 1240 may include a color indicating the degree of “tension”.

According to various embodiments, the wearable electronic device 401 may provide a notification or feedback indicating the user's mood state (or health state) using sound or vibration while updating the watch face.

Referring to FIG. 13 , according to various embodiments, in operation 1301 , the wearable electronic device (eg, the wearable electronic device 401 of FIG. 4 ) uses a camera UDC (eg, the camera 410 of FIG. 4 ). Through this, the user's second gesture can be checked. For example, the second gesture may include a movement of a specific user's hand (or finger) and/or a shape of the hand. For example, the wearable electronic device 401 may set a face detection area and a hand detection area through the camera (UDC) 410 , and may check the user's face and hand through this.

According to various embodiments of the present disclosure, in operation 1303 , the wearable electronic device 401 displays the camera (UDC) 410 by further considering the second gesture to the face and gaze of the user identified in the photographed image (eg, FIG. The screen displayed on the display 460 of 4 may be adjusted. According to another embodiment, the wearable electronic device 401 may adjust the screen displayed on the display 460 in consideration of only the second gesture identified by the camera (UDC) 410 .

Referring to FIG. 14 , according to various embodiments, the wearable electronic device (eg, the wearable electronic device 401 of FIG. 4 ) uses a camera UDC (eg, the camera 410 of FIG. 4 ) of a user's hand The movement (or the shape of the hand) 1405 can be confirmed. The wearable electronic device 401 may determine whether the movement of the user's hand matches the specified gesture.

According to various embodiments, when the movement of the user's hand matches the specified gesture, the wearable electronic device 401 may perform a function corresponding to the specified gesture. For example, the wearable electronic device 401 may change the first screen 1410 having the first size displayed on the display (eg, the display 460 of FIG. 4 ) to the second screen 1420 having the second size. have. That is, the wearable electronic device 401 may enlarge the size of the screen previously displayed on the display 1460 based on the movement of the user's hand. According to another embodiment, the wearable electronic device 401 may change the second screen 1420 having the second size displayed on the display 460 to the first screen 1410 having the first size. That is, the wearable electronic device 401 may reduce the size of the screen previously displayed on the display 1460 based on the movement of the user's hand.

Although FIG. 14 describes only the operation of enlarging or reducing the size of the screen, the technical spirit of the present invention may not be limited thereto. For example, the wearable electronic device 401 may adjust the screen in various forms based on a gesture including the movement of the user's hand (or the shape of the hand) confirmed through the camera (UDC) 410 .

Referring to FIG. 15 , according to various embodiments, in operation 1501 , the wearable electronic device (eg, the wearable electronic device 401 of FIG. 4 ) uses a camera UDC (eg, the camera 410 of FIG. 4 ). Through this, the user's face and gaze can be checked.

According to various embodiments, in operation 1503 , the wearable electronic device 401 may determine whether the user matches the registered user based on the user's face and gaze. For example, the wearable electronic device 401 may include user information (eg, face and gaze information) stored in a memory (eg, memory 430 of FIG. 4 ) and the user's face and You can check whether the eyes match. When the user's information (eg, information about face and gaze) stored in the memory 430) and the user's face and gaze confirmed through the camera 410 match, the wearable electronic device 401 includes the camera 410 It can be determined that the user identified through the is a registered user.

According to various embodiments, if it is determined that the user does not match the registered user (No in operation 1503 ), the wearable electronic device 401 may not perform a separate additional operation. In this case, the wearable electronic device 401 may monitor whether a new user is identified through the camera 410 .

According to various embodiments of the present disclosure, if it is determined that the user matches the registered user (eg in operation 1503 ), in operation 1505 , the wearable electronic device 401 is configured to display an external electronic device (eg, the external electronic device 402 of FIG. 4 ). )) to send a signal requesting unlocking. For example, the external electronic device 402 may unlock or maintain the unlocked state of the external electronic device 402 based on reception of a signal for requesting unlocking.

The processor may be configured to enlarge or reduce the size of the screen based on the distance between the face and the wearable electronic device when the gaze remains the same for a specified time.

The processor may be configured to identify the distance between the face and the wearable electronic device based on the size of the face and the distance between the eyes of the face.

The processor may be configured to display only some information on the screen when a user other than the user is identified through the camera.

The processor may be configured to change a display position of the partial information included in the screen when the other user other than the user is identified through the camera.

The processor may be set to adjust the position of the notification window displayed on the display based on the direction of the face and the angle of the gaze.

The processor may be configured to check a second gesture for the movement of the user's hand through the camera, and to adjust the configuration of the screen based on the second gesture.

The electronic device may further include a second sensor for acquiring biometric information, and the processor may be configured to acquire the user's biometric information through the second sensor and update the screen based on the biometric information. can

The processor may be configured to check whether the user is a pre-registered user based on the face and the gaze, and to transmit a signal for requesting unlocking to an external electronic device when the user is identified as the pre-registered user. can

The wearable electronic device may be implemented as a smart watch.

The operation of adjusting the screen may include an operation of enlarging or reducing the size of the screen based on the distance between the face and the wearable electronic device when the gaze remains the same for a specified time.

The adjusting the screen may include checking the distance between the face and the wearable electronic device based on the size of the face and the distance between the eyes of the face.

The operation of adjusting the screen may include an operation of displaying only partial information on the screen when a user other than the user is identified through the camera.

The operation of adjusting the screen may include an operation of changing a display position of the partial information included in the screen when the other user other than the user is identified through the camera.

The operation of adjusting the screen may include adjusting the position of the notification window displayed on the display based on the direction of the face and the angle of the gaze.

The method of operating the electronic device may further include checking a second gesture for the movement of the user's hand through the camera and adjusting the configuration of the screen based on the second gesture.

The method of operating the electronic device may further include acquiring the user's biometric information through a second sensor included in the wearable electronic device and updating the screen based on the biometric information.

The method of operating the electronic device includes an operation of confirming whether the user is a pre-registered user based on the face and the gaze, and a signal for requesting unlocking of an external electronic device when the user is identified as the pre-registered user It may further include the operation of transmitting.

Claims

A wearable electronic device comprising:

a first sensor;

display;

a camera disposed under a rear surface of the display that is not exposed to the outside; and

A processor comprising:

Checking a first gesture for the wearable electronic device through the first sensor,

Based on the image including the user's face taken through the camera, to check the face and gaze of the user,

determining whether to turn on the display based on the first gesture, the face, and the gaze;

A wearable electronic device configured to adjust a screen displayed on the display based on a distance between the face and the wearable electronic device, a direction of the face, and an angle of the gaze when the display is determined to be turned on.
The method of claim 1, wherein the processor comprises:

The wearable electronic device is set to enlarge or reduce the size of the screen based on the distance between the face and the wearable electronic device when the gaze remains the same for a specified time.
The method of claim 2, wherein the processor comprises:

The electronic device is configured to check the distance between the face and the wearable electronic device based on the size of the face and the distance between the eyes of the face.
The method of claim 1, wherein the processor comprises:

A wearable electronic device configured to display only partial information on the screen when a user other than the user is identified through the camera.
The method of claim 4, wherein the processor comprises:

An electronic device configured to change a display position of the partial information included in the screen when the other user is identified through the camera.
The method of claim 1, wherein the processor comprises:

A wearable electronic device configured to adjust a position of a notification window displayed on the display based on the direction of the face and the angle of the gaze.
The method of claim 1, wherein the processor comprises:

Confirming a second gesture for the movement of the user's hand through the camera,

The wearable electronic device is set to adjust the configuration of the screen based on the second gesture.
According to claim 1,

Further comprising a second sensor for obtaining biometric information,

The processor obtains the user's biometric information through the second sensor,

A wearable electronic device configured to update the screen based on the biometric information.
The method of claim 1, wherein the processor comprises:

Check whether the user is a pre-registered user based on the face and the gaze,

An electronic device configured to transmit a signal for requesting unlocking to an external electronic device when the user is identified as the pre-registered user.
According to claim 1,

The wearable electronic device is a wearable electronic device implemented as a smart watch.
A method of operating a wearable electronic device, comprising:

checking a first gesture with respect to the wearable electronic device through a first sensor included in the wearable electronic device;

checking the user's face and gaze based on an image including the user's face photographed through a camera disposed below the rear surface of the display of the wearable electronic device that is not exposed to the outside;

determining whether to turn on the display of the electronic device based on the first gesture, the face, and the gaze; and

and adjusting a screen displayed on the display based on a distance between the face and the wearable electronic device, a direction of the face, and an angle of the gaze when it is determined to turn on the display; Way.
The method of claim 11, wherein adjusting the screen comprises:

and enlarging or reducing the size of the screen based on the distance between the face and the wearable electronic device when the gaze remains the same for a specified time.
The method of claim 12, wherein adjusting the screen comprises:

and determining the distance between the face and the wearable electronic device based on the size of the face and the distance between the eyes of the face.
The method of claim 11, wherein adjusting the screen comprises:

and displaying only partial information on the screen when a user other than the user is identified through the camera.
In the non-transitory storage medium, the program, when executed by a processor, the processor

checking a first gesture with respect to the wearable electronic device through a first sensor included in the wearable electronic device;

checking the user's face and gaze based on an image including the user's face photographed through a camera disposed below the rear surface of the display of the wearable electronic device that is not exposed to the outside;

determining whether to turn on the display of the electronic device based on the first gesture, the face, and the gaze; and

When it is determined to turn on the display, an executable instruction is stored to adjust the screen displayed on the display based on the distance between the face and the wearable electronic device, the direction of the face, and the angle of the gaze. storage medium.