KR20230108186A - A wearable device that provides a plurality of application execution screens grouped, and a method for controlling the same - Google Patents

Abstract

A wearable device for grouping and providing a plurality of application execution screens and a control method thereof are disclosed. The wearable device according to an embodiment of the present document includes a display module and at least one processor. The least one processor is configured to control the display module to display the execution screen of a first application as a virtual object, wherein the execution screen of the first application is displayed according to a first type; detect a first user input for executing a second application while the execution screen of the first application is displayed as the virtual object; after a second user input is detected, determine whether the second user input requesting that the execution screen of the second application be viewed while being included in the same group as the execution screen of the first application is detected; and display the execution screen of the second application independently from the execution screen of the first application or while forming a group with the execution screen of the first application based on the result of the determination. Accordingly, the wearable device can provide convenience to a user in controlling a plurality of provided application execution screens.

Description

A wearable device that provides a plurality of application execution screens grouped and a method for controlling the same

This document relates to a wearable device that groups and provides a plurality of application execution screens and a control method thereof.

Various services and additional functions provided through portable electronic devices such as wearable devices, for example, augmented reality glasses (AR glasses), are gradually increasing. In order to increase the utility value of these electronic devices and satisfy the needs of various users, communication service providers or electronic device manufacturers are competitively developing electronic devices to provide various functions and to differentiate themselves from other companies. Accordingly, various functions provided through wearable devices are gradually becoming more advanced.

In an AR environment, various application execution screens may be provided as virtual objects. However, according to a conventional wearable device (eg, AR glasses) that provides various application execution screens in an AR environment, a plurality of application execution screens are provided independently (eg, independently), so that the wearable device (eg, AR glasses) ) may cause an inconvenient situation in controlling a plurality of application execution screens. For example, when a user wants to move a location where some of the execution screens of a plurality of application execution screens are displayed to a specific location at once, according to a conventional wearable device, the user individually designates some of the execution screens. This may cause inconvenience in having to move to a specific location. In addition, according to the conventional wearable device (eg, AR glasses), a plurality of application execution screens are provided independently (eg, independently), so that the user feels that the plurality of virtual objects are visually disorganized. can do.

According to various embodiments of the present document, a plurality of application execution screens provided as virtual objects are grouped and provided, and by providing various embodiments capable of integrally controlling the grouped application execution screens, a user may use a plurality of application execution screens. A wearable device capable of providing convenience in controlling execution screens may be provided.

According to various embodiments of the present document, a plurality of application execution screens provided as virtual objects are grouped and provided, and by providing various embodiments capable of integrally controlling the grouped application execution screens, the user is provided with a virtual object. A wearable device may be provided that allows the user to receive a visually tidy feeling.

According to various embodiments of the present document, a plurality of application execution screens provided as virtual objects are grouped and provided, and by providing various embodiments capable of integrally controlling the grouped application execution screens, a user may use a plurality of application execution screens. A control method of a wearable device capable of providing convenience in controlling execution screens may be provided.

According to various embodiments of the present document, a plurality of application execution screens provided as virtual objects are grouped and provided, and by providing various embodiments capable of integrally controlling the grouped application execution screens, the user is provided with a virtual object. A control method of a wearable device may be provided so that a user may receive a visually tidy feeling.

A wearable device according to an embodiment of the present document includes a display module and at least one processor, wherein the at least one processor controls the display module to display an execution screen of a first application as a virtual object; In a state where the execution screen of the first application is displayed according to a first type and the execution screen of the first application is displayed as the virtual object, a first user input for executing a second application is detected; After the second user input is detected, it is determined whether or not a second user input requesting to be displayed while the execution screen of the second application is included in the same group as the execution screen of the first application is detected, and as a result of the determination Based on the above, the display module may be set to be controlled so that the execution screen of the second application is displayed independently of the execution screen of the first application or while forming a group with the execution screen of the first application.

A method for controlling a wearable device according to an embodiment of the present document includes an operation of controlling a display module of the wearable device so that an execution screen of a first application is displayed as a virtual object, and the execution screen of the first application includes a first An operation of detecting a first user input for executing a second application in a state in which the screen is displayed according to the type and the execution screen of the first application is displayed as the virtual object, and after the second user input is detected , determining whether or not a second user input requesting to be displayed while being included in the same group as the execution screen of the first application is detected, and based on the result of the determination, the and controlling the display module to display the execution screen of the second application independently of the execution screen of the first application or while forming a group with the execution screen of the first application.

According to various embodiments of the present document, a plurality of application execution screens provided as virtual objects are grouped and provided, and by providing various embodiments capable of integrally controlling the grouped application execution screens, a user may use a plurality of application execution screens. A wearable device capable of providing convenience in controlling execution screens may be provided.

According to various embodiments of the present document, a plurality of application execution screens provided as virtual objects are grouped and provided, and by providing various embodiments capable of integrally controlling the grouped application execution screens, the user is provided with a virtual object. A wearable device may be provided that allows the user to receive a visually tidy feeling.

Effects according to various embodiments are not limited to the effects described above, and it is obvious to those skilled in the art that various effects are inherent in the present disclosure.

1 is an exemplary diagram for explaining a wearable device according to an embodiment of the present document.
2A is an exemplary diagram for explaining a structure of a wearable device according to an embodiment of the present document.
2b and 2c are exemplary diagrams for explaining a connection relationship of wearable devices according to an embodiment of the present document.
3 is an exemplary diagram for explaining a structure of an eye tracking camera of a wearable device according to an embodiment of the present document.
4 is an exemplary view for explaining a function or operation in which a plurality of application execution screens are displayed independently or included in the same group according to a user's input (eg, a user's gaze direction) according to an embodiment of the present document. am.
5A to 5G are exemplary diagrams for explaining the function or operation described in FIG. 4 from a graphical user interface point of view.
6A to 6F are exemplary diagrams for describing virtual objects shown as a first type or a second type according to an embodiment of the present document.
7 is a diagram showing information about at least one virtual object (eg, an application execution screen) displayed through the wearable device in an external electronic device operatively connected to the wearable device according to an embodiment of the present document. It is an exemplary diagram for explaining a function or operation provided through
8A to 8E are exemplary diagrams for explaining the function or operation described in FIG. 7 from the perspective of a graphic user interface.
9 is an exemplary diagram for explaining a function or operation provided by a wearable device according to an embodiment of the present document by grouping a plurality of application execution screens based on a user's gaze direction.
10A and 10B are exemplary diagrams for explaining the function or operation described in FIG. 9 from the viewpoint of a graphic user interface.
11 is a control to include at least one execution screen among a plurality of grouped application execution screens in the same group as a plurality of other grouped application execution screens, or independently It is an exemplary diagram for explaining a function or operation for controlling to be shown.
12A to 12D are exemplary diagrams for explaining the function or operation described in FIG. 11 from the viewpoint of a graphic user interface.
11 is an exemplary diagram for explaining a function or operation of enlarging or reducing at least one application execution screen according to a user's gesture by a wearable device according to an embodiment of the present document.
14A to 14C are exemplary diagrams for explaining the function or operation described in FIG. 11 from the viewpoint of a graphic user interface.
15 illustrates a function of storing a layout of at least one application execution screen in which wearable devices are grouped according to an embodiment of the present document and providing at least one application execution screen according to the stored layout based on a user's input; or It is an exemplary diagram for explaining the operation.
16A to 16D are exemplary diagrams for explaining the function or operation described in FIG. 15 from a graphical user interface point of view.
17 is an exemplary diagram for explaining a function or operation of grouping a plurality of application execution screens into the same group based on movement of the application execution screen by the wearable device according to an embodiment of the present document.
18A to 18E are exemplary diagrams for explaining the function or operation described in FIG. 17 from the viewpoint of a graphic user interface.
19 is a diagram of a wearable device according to an embodiment of the present document moving a location where at least one application execution screen is displayed according to a user's input (eg, user's gaze), and the moved at least one application execution screen; It is an exemplary diagram for explaining a function or operation of grouping at least one other application execution screen into one group.
20A to 20D are exemplary diagrams for explaining the function or operation described in FIG. 19 from the viewpoint of a graphic user interface.
21 is an exemplary diagram for explaining a function or operation of grouping at least one application execution screen grouped and at least one other application execution screen grouped into the same group by a wearable device according to an embodiment of the present document.
22A to 22C are exemplary diagrams for explaining the function or operation described in FIG. 21 from the viewpoint of a graphic user interface.
23 is a function for controlling the wearable device to display an execution screen of an application running on an external electronic device operably connected to the wearable device as a virtual object according to a user's gesture, according to an embodiment of the present document. Or, it is an exemplary drawing for explaining an operation.
24a to 24c are exemplary diagrams for explaining the function or operation described in FIG. 19 from the viewpoint of a graphic user interface.
25 is a block diagram of an electronic device in a network environment, according to various embodiments.

1 is an exemplary diagram for explaining a wearable device 100 according to an embodiment of the present document. In this document, the wearable device 100 is described as an example, but is not limited thereto. For example, the various embodiments described in this document may be an accessory type (eg, a watch, ring, bracelet, anklet, necklace, glasses, contact lens, or a head mounted device (HMD) or a head mounted display device ( head mounted display (HMD)), fabric or clothing integral type (eg, electronic clothing), body attachment type (eg, skin pad or tattoo), or bio-implantable circuitry. The wearable device 100 according to an embodiment of the present document includes a first camera 111, a second camera 112, a third camera 113, a processor 120, a PMIC 130, and a battery 135. , a memory 140, a display module 150, an audio module 161, an audio input device 162, an audio output device 163, a communication circuit 170, and a sensor 180. there is.

According to an embodiment of the present document, an image acquired through the first camera 111 may be used for detecting a hand gesture by a user, tracking a user's head, and/or spatial recognition. According to an embodiment of the present document, the first camera 111 may include a global shutter (GS) camera. According to an embodiment of the present document, the first camera 111 may perform a simultaneous localization and mapping (SLAM) operation through depth imaging. According to an embodiment of the present document, the first camera 111 may perform spatial recognition for 6DoF. When the first camera 111 according to an embodiment of the present document is worn by the first user, it may be set to capture the real world corresponding to the gaze of the first user.

According to an embodiment of the present document, an image obtained through the second camera 112 may be used for detection and tracking of the user's pupil. According to one embodiment of this document, the second camera 112 may include a GS camera. According to one embodiment of the present document, the second cameras 112 may correspond to the left eye and the right eye, respectively, and the performance of the second cameras 112 may be the same or similar to each other. The second camera 112 according to an embodiment of the present document may be configured to acquire an image of a user's facial expression. At least one processor 120 according to an embodiment of the present document may be configured to detect a change in the user's facial expression based on the image of the user's facial expression acquired by the second camera 112 .

According to one embodiment of the present document, the third camera 113 may include a higher resolution camera than the first camera 111 and the second camera 112 . According to an embodiment of the present document, the third camera 113 may perform an auto-focusing (AF) function and a shake correction function. According to an embodiment of the present document, the third camera 113 may include a GS camera or a rolling shutter (RS) camera. When the first camera 111 according to an embodiment of the present document is worn by the first user, it may be set to capture the real world corresponding to the gaze of the first user.

The processor 120 according to an embodiment of the present document includes components of the wearable device 100, for example, a first camera 111, a second camera 112, a third camera 113, and a PMIC 130, the memory 140, the display module 150, the audio module 161, the communication circuit 170, and the sensor 180 can be controlled, and various data processing or calculations can be performed. The processor 120 according to an embodiment of the present document, for example, executes software (eg, a program) to perform at least one other component (eg, hardware or software component), and can perform various data processing or calculations. According to one embodiment of the present document, as at least part of data processing or operation, processor 120 transfers instructions or data received from other components (eg, sensor module 176 or communication module 190) to volatile memory. , process commands or data stored in volatile memory, and store resultant data in non-volatile memory. According to an embodiment of the present document, the processor 120 may include a main processor (eg, a central processing unit or an application processor) or a secondary processor (eg, a graphic processing unit, a neural network processing unit (NPU) that may operate independently of or together with the main processor) processing unit), image signal processor, sensor hub processor, or communication processor). For example, when the wearable device 100 includes a main processor and an auxiliary processor, the auxiliary processor may use less power than the main processor or may be set to be specialized for a designated function. A secondary processor may be implemented separately from, or as part of, the main processor.

The PMIC 130 according to an embodiment of the present document converts the power stored in the battery 135 to have a current or voltage required by other components of the wearable device 100 so that the other components of the wearable device 100 can supply to

The memory 140 according to an embodiment of the present document may store various data used by at least one component (eg, the processor 120 or the sensor module 180) of the wearable device 100 . The memory 140 according to an embodiment of the present document may include a volatile memory or a non-volatile memory.

The display module 150 according to an embodiment of the present document may display a screen to be provided to the user. The display module 150 according to an embodiment of the present document may output data (eg, RGB data) for expressing a virtual object (eg, an augmented reality guide). The display module 150 according to an embodiment of the present document includes a first display 251, a second display 252, one or more input optical members 253-1 and 253-2, and one or more transparent members 290-250. 1 and 290-2), and one or more screen display parts 254-1 and 254-2.

The audio module 161 according to an embodiment of the present document is connected to the audio input device 162 and the audio output device 163, converts data input through the audio input device 162, and converts data input to the audio output device ( 163) can convert the data to be output. The audio input device 162 according to an embodiment of the present document may include at least one microphone, and the audio output device 163 may include at least one speaker and amplifier. The communication circuit 170 according to an embodiment of the present document may support establishment of a wireless communication channel with an external electronic device connected to the wearable device 100 and communication through the established communication channel. The sensor 180 according to an embodiment of the present document may include a 6-axis sensor 181 , a magnetic sensor 182 , a proximity sensor 183 , and/or an optical sensor 184 .

2A is an exemplary diagram for explaining the structure of the wearable device 100 according to an embodiment of the present document.

Referring to FIG. 2A , the wearable device 100 may include one or more light emitting elements 214-1 and 214-2. The light emitting devices 214-1 and 214-2 according to an embodiment of the present document may be different from light sources to be described later, which radiate light to a screen output area of a display. According to an embodiment of the present document, the light emitting devices 214-1 and 214-2 detect and track the user's pupils through one or more second cameras 212-1 and 212-2. Light may be irradiated to facilitate detection. According to one embodiment of this document, each of the light emitting elements 214-1 and 214-2 may include an LED. According to one embodiment of this document, the light emitting devices 214-1 and 214-2 may emit light in the infrared region. According to one embodiment of this document, the light emitting elements 214-1 and 214-2 may be attached around the frame of the wearable device 100. According to one embodiment of this document, the light emitting elements 214-1 and 214-2 are located around one or more first cameras 211-1 and 211-2, and when the wearable device 100 is used in a dark environment Gesture detection, head tracking, and spatial recognition by one or more first cameras 211-1 and 211-2 may be assisted. According to one embodiment of this document, the light emitting devices 214-1 and 214-2 are located around one or more third cameras 213, and when the wearable device 100 is used in a dark environment, one or more third cameras ( 213) to assist image acquisition.

According to one embodiment of this document, the wearable device 100 may include batteries 235-1 and 235-2. The batteries 235 - 1 and 235 - 2 may store power for operating the remaining components of the wearable device 100 .

According to one embodiment of this document, the wearable device 100 includes a first display 251, a second display 252, one or more input optical members 253-1 and 253-2, and one or more transparent members 290. -1 and 290-2), and one or more screen display parts 254-1 and 254-2. According to an embodiment of the present document, the first display 251 and the second display 252 may include, for example, a liquid crystal display (LCD), a digital mirror device (DMD), It may include a liquid crystal on silicon (LCoS), an organic light emitting diode (OLED), or a micro light emitting diode (micro LED). According to one embodiment of this document, when the first display 251 and the second display 252 are made of one of a liquid crystal display, a digital mirror display, and a silicon liquid crystal display, the wearable device 100 displays the screen of the display A light source for radiating light to the output area may be included. According to one embodiment of this document, when the first display 251 and the second display 252 can generate light by themselves (eg, made of either an organic light emitting diode or a micro LED), a wearable device ( 100) can provide a relatively good quality virtual image to the user even without a separate light source.

According to one embodiment of this document, one or more transparent members 290-1 and 290-2 may be disposed to face the user's eyes when the user wears the wearable device 100. According to one embodiment of the present document, the one or more transparent members 290-1 and 290-2 may include at least one of a glass plate, a plastic plate, or a polymer. According to one embodiment of this document, a user may see the outside world through one or more transparent members 290-1 and 290-2 when wearing the wearable device 100. According to one embodiment of the present document, one or more input optical members 253-1 and 253-2 may guide light generated by the first display 251 and the second display 252 to the user's eyes. . According to one embodiment of this document, a first display 251 and a second display ( An image based on the light generated in step 252 is formed, and the user can see the image formed on one or more screen display parts 254-1 and 254-2.

According to one embodiment of this document, the wearable device 100 may include one or more optical waveguides (not shown). The optical waveguide according to an embodiment of the present document may transmit light generated by the first display 251 and the second display 252 to the user's eyes. The wearable device 100 may include one optical waveguide corresponding to each of the left and right eyes. According to one embodiment of the present document, the optical waveguide may include at least one of glass, plastic, or polymer. According to an embodiment of the present document, the optical waveguide may include a nanopattern formed on an inner or outer surface, for example, a polygonal or curved grating structure. According to an embodiment of the present document, the optical waveguide may include a free-form prism, and in this case, the optical waveguide may provide incident light to a user through a reflective mirror. According to an embodiment of the present document, the optical waveguide includes at least one of at least one diffractive element (eg, a diffractive optical element (DOE), a holographic optical element (HOE)) or a reflective element (eg, a reflective mirror), Display light emitted from a light source may be guided to the eyes of a user by using at least one diffractive element or reflective element included in the optical waveguide. According to one embodiment of this document, the diffractive element may include input/output optical elements. According to one embodiment of the present document, the reflective element may include a member that causes total internal reflection.

According to one embodiment of this document, the wearable device 100 includes one or more audio input devices 262-1, 262-2, and 262-3 and one or more audio output devices 263-1 and 263-2. can do.

According to one embodiment of this document, the wearable device 100 may include a first PCB 270-1 and a second PCB 270-2. The first PCB 270-1 and the second PCB 270-2 include a first camera 111, a second camera 112, a third camera 113, a display module 150, and an audio module 161. , and may be set to transmit electrical signals to components included in the wearable device 100 , such as the sensor 180 . According to one embodiment of this document, the first PCB 270-1 and the second PCB 270-2 may include a flexible printed circuit board (FPCB). According to one embodiment of the present document, the first PCB 270-1 and the second PCB 270-2 each include a first substrate, a second substrate, and an interface disposed between the first substrate and the second substrate. Posers may be included.

2B and 2C are exemplary diagrams for explaining a connection relationship of the wearable device 100 according to an embodiment of the present document.

Referring to FIG. 2B , the first user's wearable device 100 according to an embodiment of the present document includes an external electronic device (eg, smart glasses 310, an external electronic device 800, and a second external electronic device 2420). )) and can be directly connected through the network. A network according to an embodiment of the present document is a first network (eg, a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (eg, a legacy cellular network, 5G network, a next-generation communications network, the Internet, or a telecommunications network such as a computer network (eg, LAN or WAN). Alternatively, as shown in FIG. 2C , the wearable device 100 of the first user according to an embodiment of the present document is an external electronic device (eg, smart glasses 310, an external electronic device 800, a second external The electronic device 2420) may be connected through a server. A server according to an embodiment of the present document may include an intelligent server using machine learning and/or neural networks. In FIGS. 2B and 2C , the smart glasses 300 and the smart phone 310 are shown as examples of external electronic devices, but the types of external electronic devices are not limited thereto.

3 is an exemplary diagram for explaining an eye tracking camera structure of the wearable device 100 according to an embodiment of the present document. Referring to FIG. 3 , the wearable device 100 includes an eye tracking (ET) camera 310 (eg, second cameras 212-1 and 212-2), a display 321, an input optical member 322, At least one of a first waveguide 323 , an output optical member 324 , a first splitter 341 , a second waveguide 342 , and a second splitter 343 may be included.

ET An image may be captured by the camera 310 . The ET camera 310 according to an embodiment of the present document may track the user's gaze by detecting the pupil 330 in the captured image and confirming the motion of the detected pupil 330 .

According to an embodiment of the present document, an image output through the display 321 may be reflected through the input optical member 322 and the first waveguide 323 and displayed through the output optical member 324. . The wearable device 300 according to an embodiment of the present document outputs an image through the display 321 and at the same time checks the movement of the user's pupil 330 to determine the user's gaze (eg, the direction of the user's gaze) can be tracked (e.g. identified).

4 is an exemplary view for explaining a function or operation in which a plurality of application execution screens are displayed independently or included in the same group according to a user's input (eg, a user's gaze direction) according to an embodiment of the present document. am. 5A to 5G are exemplary diagrams for explaining the function or operation described in FIG. 4 from a graphical user interface point of view.

Referring to FIG. 4 , the wearable device 100 (eg, the processor 120) according to an embodiment of the present document, in operation 410, displays the execution screen of the first application as a virtual object through the display module 150. can control. Referring to FIG. 5A , the wearable device 100 according to an embodiment of the present document includes a plurality of visual elements (eg, a first icon 561, a second icon 562, a third icon 563, The display module 150 may be controlled to display the fourth icon 564 , the fifth icon 567 , the sixth icon 568 , or the seventh icon 569 on the application tray 560 . The wearable device 100 according to an embodiment of the present document includes a plurality of visual elements (eg, a first icon 561, a second icon 562, a third icon 563, and a fourth icon 564). , the fifth icon 567, the sixth icon 568, or the seventh icon 569) and the application tray 560 are moved and displayed according to the movement of the user's body (eg, when the head is turned to the right). The display module 150 may be controlled so as to be displayed. The first icon 561 according to an embodiment of the present document may include various applications installed in the wearable device 100 (eg, stored in memory) according to a user's virtual touch input to the first icon 561, and /or may be a visual element for providing icons corresponding to various applications installed in an external electronic device operatively connected to the wearable device 100 . The second icon 562 according to an embodiment of the present document is a visual element for providing information on an application currently being executed through the wearable device 100 (eg, an application execution screen being displayed as a virtual object) can be The third icon 563 and the fourth icon 564 according to an embodiment of the present document are fixedly displayed on the application tray 560 (eg, provided together when the application tray 560 is displayed). An icon corresponding to at least one application set by a user or pre-designated may be included. In FIG. 5A , an embodiment in which a music application and a text message application are provided as fixed applications is illustrated as an example. The fifth icon 567 according to an embodiment of the present document may include an icon corresponding to an application (eg, a market application) for downloading at least one application from an external electronic device (eg, a server). The sixth icon 568 according to an embodiment of this document may include an icon corresponding to an application for searching various information stored in the wearable device 100 . The seventh icon 569 according to an embodiment of the present document may include an icon corresponding to an application for setting various states of the wearable device 100 . In this document, for convenience of description, a method of being displayed in the form of an icon on the application tray 560 may be referred to as the term “second type”. According to one embodiment of this document, the application tray 560 may not be provided, and at least one visual element (eg, a first icon 561, a second icon 562, a third icon 563, Only the fourth icon 564, the fifth icon 567, the sixth icon 568, or the seventh icon 569 may be provided as virtual objects. Referring to FIG. 5B , the wearable device 100 according to an embodiment of the present document includes at least one visual element (eg, a first icon 561, a second icon 562, a third icon 563, Selection input for any one visual element (eg, the fourth icon 564) among the fourth icon 564, the fifth icon 567, the sixth icon 568, or the seventh icon 569 ( Example: virtual touch input) may be received. In the wearable device 100 according to an embodiment of the present document, at least one image of a user's body part (eg, right hand) obtained using at least one camera (eg, the first camera 111) is displayed. Visual elements (eg, first icon 561, second icon 562, third icon 563, fourth icon 564, fifth icon 567, sixth icon 568, or seventh icon 568). When it is determined that at least a part of the icon 569 is overlapped with any one visual element (eg, the fourth icon 564), or any one visual element (eg, the fourth icon 564) in an overlapped state ))), it may be determined that a virtual touch input for any one visual element (eg, the fourth icon 564) is received. According to another embodiment of the present document, it is possible to determine whether a virtual touch is input through a separate device for determining a virtual touch (eg, a sensor provided on a user's hand). According to another embodiment of the present document, coordinates in a space where a user's fingertip is located are recognized using a plurality of cameras, and the coordinates in the recognized space are a visual element (eg, a fourth icon 564). ) is matched with the coordinates on the displayed space, it can be determined whether a virtual touch is input. In addition, various algorithms (eg, artificial intelligence model algorithms) for determining that a virtual touch has been performed may be applied. Referring to FIG. 5C , the wearable device 100 according to an embodiment of the present document displays an application execution screen (eg, SMS application execution) corresponding to a visual element (eg, a fourth icon 564) selected by a user. The display module 150 may be controlled to display the screen 510 as a virtual object. The wearable device 100 according to an embodiment of the present document identifies a user's gaze direction, and displays an application execution screen (eg, SMS application execution screen 510) at a location corresponding to the identified user's gaze direction. The display module 150 may be controlled to be displayed as a virtual object. The wearable device 100 according to an embodiment of this document controls an application execution screen (eg, SMS application execution screen 510) together with an application execution screen (eg, SMS application execution screen 510). handlers (eg, the first handler 501a, the second handler 501b, and the third handler 501c) and a visual element indicating that at least one application execution screen is included in the same group (eg, a border 502) can provide. However, according to another embodiment of this document, a handler (eg, a first handler 501a, a second handler 501b) and/or a handler for controlling an application execution screen (eg, the SMS application execution screen 510) 3 Handler 501c) and at least one visual element indicating that at least one application execution screen is included in the same group (eg, border 502) may be controlled by the wearable device 100 not to be displayed. According to another embodiment of the present document, when a visual element indicating that at least one application execution screen is included in the same group (eg, the boundary line 502) is not shown, a handler (eg, the first handler 501a, Whether they belong to the same group can be identified by the second handler 501b and/or the third handler 501c. For example, a plurality of application execution screens belonging to the same group may share a handler (eg, a first handler 501a, a second handler 501b, and/or a third handler 501c). Execution of at least one grouped application according to a user's gesture for handlers (eg, the first handler 501a, the second handler 501b, and/or the third handler 501c) according to an embodiment of the present document. Screens may be controlled together (eg, at least one grouped application execution screen is moved together, or all of the grouped at least one application execution screen is terminated). The wearable device 100 according to an embodiment of the present document may perform a user's gesture (eg, the first handler 501a, the second handler 501b, and/or the third handler 501c) with respect to the handler (eg, the first handler 501a, the second handler 501b, and/or the third handler 501c). When a virtual touch input to the handler) is detected, various menus (eg, icons for ending all grouped at least one application execution screen, etc.) for controlling the at least one application execution screen grouped together are displayed. The module 150 can be controlled. In this document, the term "included in the same group" refers to a specific user's input (eg, handler (eg, the first handler 501a, the second handler 501b, and/or the third handler 501c)). It may mean that at least one application execution screen is substantially equally controlled (eg, simultaneously moved or simultaneously removed) according to a grip gesture and a drop gesture. Alternatively, in this document, the term "included in the same group" may mean a state in which all or part of at least one application execution screen is included within the same boundary 502 . As mentioned in this document, the meaning of the term "application included in the same group" and the meaning of the term "application execution screen included in the same group" may be referred to as substantially the same meaning. In the wearable device 100 according to an embodiment of the present document, a location where a specific visual object (eg, the SMS application execution screen 510) is being displayed (eg, displayed by the wearable device 100) and / or a specified gesture (eg, a gesture of gripping a visual object or a visual When a gesture of dropping an object) is detected, it may be identified that a gesture for controlling a specific visual object (eg, the SMS application execution screen 510) is input from the user. For convenience of explanation in this document, visual elements including handlers (eg, the first handler 501a, the second handler 501b, and the third handler 501c), the boundary line 502, and the application execution screen 510 are may be referred to by the term "slate". Accordingly, the term "included in the same slate" may mean "included in the same group at least one application execution screen". When a specific application execution screen (eg, the SMS application execution screen 510) is provided as a slate, the wearable device 100 according to an embodiment of the present document displays a visual element corresponding to the slate (eg, an eighth icon ( 565) may control the display module 150 to be displayed on the application tray 560 . A visual element (eg, the eighth icon 565) corresponding to the slate according to an embodiment of the present document may be displayed to be visually distinguished from other icons. For example, a visual element corresponding to a slate according to an embodiment of the present document (eg, the eighth icon 565) has a thicker border than other icons, or a color different from that of other icons. can be shown As shown in FIG. 5D , the wearable device 100 according to an embodiment of the present document, around a visual element (eg, an eighth icon 565) corresponding to a currently focused slate (eg, an application execution screen) A predetermined visual effect 565a (eg, a blur effect) may be provided. The wearable device 100 according to an embodiment of the present document detects that the user's gaze is directed to a specific slate (eg, SMS application execution screen 510), or detects a specific slate (eg, SMS application execution screen 510) by the user. When the screen 510) is selected, it may be determined that a specific slate is focused. In FIG. 5D , an embodiment in which a designated visual effect 565a is provided around an eighth icon 565 based on the fact that the user is looking at the first slate 500 is illustrated as an example.

In operation 420, the wearable device 100 (eg, the processor 120) according to an embodiment of the present document may detect a first user input for executing a second application (eg, a music application). As shown in FIG. 5E , the wearable device 100 according to an embodiment of the present document may detect a virtual touch input for a specific application icon (eg, a third icon 563 indicating a music application). . Alternatively, according to another embodiment of the present document, the wearable device 100 detects a virtual touch input for any one icon among application icons included in an application list provided separately from the application tray 560, thereby detecting a second virtual touch input. A first user input for executing an application may be detected.

In operation 430, the wearable device 100 (eg, the processor 120) according to an embodiment of the present document displays an execution screen of a second application (eg, a music application) of a first application (eg, an SMS application). It may be determined whether a second user input requesting to be displayed while being included in the same group as the execution screen is detected. As shown in FIG. 5F , the wearable device 100 according to an embodiment of the present document may detect that the user's gaze is directed toward a specific slate (eg, the first slate 500). In the wearable device 100 according to an embodiment of the present document, the virtual touch input for a specific application icon (eg, the third icon 563 indicating a music application) is detected, and the virtual touch input is detected. Points of time before a predetermined time elapses (eg, 2 seconds after the virtual touch input is detected), points of time before the virtual touch input is detected (eg, 2 seconds after the virtual touch input is detected) Before), and/or when a virtual touch input is predicted to be input (eg, when predicted through an artificial intelligence model), the user's gaze may be detected. In this case, in the wearable device 100 according to an embodiment of the present document, while the execution screen of the second application (eg, music application) is included in the same group as the execution screen of the first application (eg, SMS application) It may be determined that the second user input requesting to be viewed has been obtained. In other words, the second user input may include a direction of the user's gaze toward a specific slate. According to another embodiment of the present document, the second user input is a user input (eg, virtual touch input) for a specific slate (eg, the SMS application execution screen 510), or at least one provided separately from the application execution screen. It may include a user input (eg, a virtual touch input) for selecting one slate from one slate list.

The wearable device 100 (eg, the processor 120) according to an embodiment of the present document, in operation 440, based on the determination result of operation 430, displays an execution screen of a second application (eg, a music application). The display module 150 may be controlled to be displayed independently of the execution screen of one application or while forming the same group. As shown in FIG. 5F , the wearable device 100 according to an embodiment of the present document, when it is detected that the user's gaze is directed toward a specific slate (eg, the first slate 500), displays a second application The display module 150 may be controlled so that an execution screen of (eg, a music application) is displayed while being included in the same group as an execution screen of a first application (eg, an SMS application). In this case, the wearable device 100 according to an embodiment of the present document updates and provides an icon (eg, an eighth icon 565) corresponding to the first slate 500 as shown in FIG. 5F. can For example, as shown in FIG. 5F , the wearable device 100 according to an embodiment of the present document displays an icon representing a first application (eg, an SMS application) and a second application (eg, a music application). A single icon (eg, the eighth icon 565) may be generated and provided. As shown in FIG. 5G , the wearable device 100 according to an embodiment of the present document detects that the user's gaze is not directed toward a specific slate (eg, the first slate 500) (eg, any When it is identified that the user is looking at the real world where the application execution screen is not shown), the slate (eg, the second slate 505) including the execution screen of the second application (eg, music application) is displayed on the first slate ( 500), the display module 150 can be controlled to be displayed independently (eg, so that the SMS application execution screen 510 and the music application execution screen 520 are not included in the same group). In this case, the wearable device 100 according to an embodiment of the present document, as shown in FIG. 5G, displays an icon (eg, a ninth icon 566) corresponding to the second slate 505 as the first slate. It may be provided on the application tray 560 separately from the icon corresponding to 500 (eg, the eighth icon 565). In the wearable device 100 according to an embodiment of the present document, an icon corresponding to the second slate 505 (eg, the ninth icon 566) is surrounded by a target icon and an icon corresponding to the second slate 505 The display module 150 may be controlled to display information indicating that the current state is displayed (eg, “slate 2”). In FIG. 5G , an embodiment in which a designated visual effect 565a is provided around a ninth icon 566 based on the fact that the user is looking at the second slate 505 is shown as an example.

6A to 6F are exemplary diagrams for describing virtual objects shown as a first type or a second type according to an embodiment of the present document.

Referring to FIGS. 6A and 6B , the wearable device 100 according to an embodiment of the present document includes at least one slate (eg, a first slate 500 and/or a second slate 500) including at least one application execution screen. 2 The display module 150 may be controlled so that the slate 505 is displayed as a virtual object. In this document, for convenience of description, a case in which a specific visual object (eg, an application execution screen) is displayed as a slate may be referred to as “first type” or “displayed based on the first type”. In FIG. 6A , a case in which a music application, an e-mail application, a file application providing information on the currently used and remaining memory capacity, and a gallery application are designated as one group is shown as an example. In this case, the execution screen 520 of the music application, the execution screen 530 of the email application, the execution screen 540 of the file application, and the execution screen 550 of the gallery application are the same slate (eg, the second slate 505). ) and can be controlled by the wearable device 100 to be displayed. In addition, in FIG. 6A, a case in which the SMS application is designated to be included in a different group from the music application, the email application, the file application, and the gallery application is illustrated as an example. In this case, the execution screen 510 of the SMS application may be displayed through a slate (eg, the first slate 500 ) provided visually and/or spatially separated from the second slate 505 . 6A and 6B, an embodiment in which an eighth icon 565 corresponding to the first slate 500 and a ninth icon 566 corresponding to the second slate 505 are displayed on the application tray 560 is exemplified. depicted negatively. When a user input for the eighth icon 565 is obtained, the wearable device 100 according to an embodiment of the present document may control the display module 150 to display the first slate 500 . When a user input for the ninth icon 566 is obtained, the wearable device 100 according to an embodiment of the present document may control the display module 150 to display the second slate 505 . The wearable device 100 according to an embodiment of the present document corresponds to a specific slate, as shown in FIG. Display module (eg, display module (eg, "+1") to display information indicating that the icon (eg, the ninth icon 566) further includes application execution screens exceeding a predetermined number (eg, 3). 150) can be controlled.

Referring to FIG. 6C , when a user input for a first icon 561 is detected, the wearable device 100 according to an embodiment of the present document performs an operation with the wearable device 100 and/or the wearable device 100 The display module 150 may be controlled to display a list of icons of at least one application installed in an operably connected external electronic device.

Referring to FIGS. 6D and 6E , the wearable device 100 according to an embodiment of the present document may detect a user input for a second icon 562 . When a user input for the second icon 562 is detected, the wearable device 100 according to an embodiment of the present document, as shown in FIG. 6E , at least one slate currently shown as the first type ( Example: The first slate 500 and the second slate 505) and/or an external electronic device operatively connected to the wearable device 100 (eg, a smart phone ("S21+" ^® ) and a tablet ("Tab S7") ^® )) at least one of the running applications (e.g., applications (e.g., including the currently running application) that were recently (e.g., from the current point in time to a specific point in the past) running in the background and foreground. Display module (eg, the tenth icon 621, the eleventh icon 622, the twelfth icon 623, the thirteenth icon 631, and the fourteenth icon 632) representing the application of 150) can be controlled. The wearable device 100 according to an embodiment of the present document shows visual elements (eg, icons) representing external electronic devices (eg, a smart phone ("S21+" ^® ) and a tablet ("Tab S7" ^® )). The display module 150 may be controlled so as to be displayed. Additionally, when a user input for the second icon 562 is detected, the wearable device 100 according to an embodiment of the present document stores information stored in the wearable device 100 and/or stored in an external electronic device through a network. The display module 150 may be controlled to display a visual element (eg, a search bar 610) for searching for and providing information. When a user input for the second icon 562 is detected, the wearable device 100 according to an embodiment of the present document provides a visual element for providing an execution screen of a setting application (eg, the fifteenth icon 630) It is possible to control the display module to show. When a user input for the second icon 562 is detected, the wearable device 100 according to an embodiment of the present document displays at least one slate and/or at least one external display through the wearable device 100. The display module may be controlled to display a visual element (eg, the sixteenth icon 640) for terminating all applications running in the electronic device (eg, removing at least one slate). However, the wearable device 100 according to an embodiment of the present document terminates all applications running on at least one slate and/or at least one external electronic device being displayed through the wearable device 100 (eg: Even if a user input for a visual element (eg, the sixteenth icon 640) for removing at least one slate is detected, the execution screen of a fixed application (eg, SMS application) may not be removed. In FIG. 6E , an embodiment in which a visual element (eg, a pin 660 ) indicating that the corresponding application is a fixed application is displayed around an application execution screen is illustrated as an example. The wearable device 100 according to an embodiment of the present document shows a visual element (eg, the seventeenth icon 670) for removing only a specific slate (eg, the second slate 650) around each slate. The display module 150 may be controlled so as to be displayed. However, in FIG. 6E, since the first slate 500 is a fixed slate including a fixed application, a visual element (eg, a 17th icon 670) for removing only a specific slate (eg, the second slate 650). )) is shown as an example. The wearable device 100 according to an embodiment of the present document provides a list of icons of at least one application installed in the wearable device 100 and/or an external electronic device operatively connected to the wearable device 100. The display module 150 may be controlled to display a visual element (for example, the fifteenth icon 650) for display.

Referring to FIG. 6F , the wearable device 100 according to an embodiment of the present document may detect a user input for a visual element (eg, a tenth icon 621) representing an application running on an external electronic device. there is. When a user input for a visual element (eg, the tenth icon 621) representing an application being executed in the external electronic device is detected, the wearable device 100 according to an embodiment of the present document is detected by the external electronic device (eg, the external electronic device). Control the display module 150 so that the execution screen of the application running on the "S21+" ^® as a smart phone (eg, the execution screen 680 of the calculator application and the execution screen 690 of the download application) is displayed as a virtual object. can To this end, the wearable device 100 (eg, communication module) according to an embodiment of the present document receives a user input for a visual element (eg, the tenth icon 621) representing an application running in the external electronic device. Upon detection, a request is made to provide information on an application running on the external electronic device (eg, information on an execution screen currently being displayed on the external electronic device) corresponding to the selected visual element (eg, the tenth icon 621). can In response to a request for providing information, the wearable device 100 according to an embodiment of the present document obtains information on an execution screen currently being displayed on the external electronic device from the external electronic device, and displays the display module as a virtual object. (150) can be controlled. Various visual elements (eg, 10th icon 621, 11th icon 622, 12th icon 623, 13th icon 631, 14th icon 632) shown according to an embodiment of the present document )), only icons corresponding to applications commonly installed in the wearable device 100 and the external electronic device may be displayed. For example, when the music application is not installed in the wearable device 100, the twelfth icon 623 corresponding to the music application may not be displayed. However, according to another embodiment of this document, when a specific application (eg, music application) is not installed in the wearable device 100, the wearable device 100, as an external electronic device (eg, a smart phone, "S21+ After receiving data about a specific application from " ^® ) and installing the specific application, the display module 150 may be controlled so that the twelfth icon 623 is displayed. In this document, for convenience of description, embodiments in which various visual elements are displayed on a 2D plane are illustrated, but according to other embodiments of the present document, various visual elements may be shown as 3D objects.

7 shows at least one external electronic device operably connected to the wearable device 100 according to an embodiment of the present document (eg, "S21+" ^® as a smart phone) being displayed through the wearable device 100. It is an exemplary diagram for explaining a function or operation provided by information on a virtual object (eg, application execution screen) through an external electronic device (eg, Galaxy ^® Fold Z™ as a smart phone). 8A to 8E are exemplary diagrams for explaining the function or operation described in FIG. 7 from the perspective of a graphic user interface.

Referring to FIG. 7 , the wearable device 710 according to an embodiment of the present document may provide, in operation 710, a virtual object for at least one grouped application execution screen. The scope of the term "grouping" mentioned in this document may also include cases in which one application is grouped. As shown in FIG. 8A , the wearable device 710 according to an embodiment of the present document displays various virtual objects (eg, the first slate 500 and the second slate 505) through a display module ( 150) can be controlled.

In operation 720, the wearable device 710 according to an embodiment of the present document may identify that an external electronic device (eg, a Galaxy ^® Fold Z™ as a smart phone) is connected. The wearable device 710 according to an embodiment of the present document, in operation 730, displays information on the virtual object being provided through the wearable device 100 on the external electronic device 800. may be transmitted to an external electronic device. An external electronic device according to an embodiment of the present document, upon detection of a user input (eg, a swipe gesture for displaying a side bar), based on information acquired from the wearable device 100, provides a wearable device. Information about at least one virtual object (eg, the first slate 500 and the second slate 505) being displayed through the device 100 (eg, the first slate icon 803 and the second slate icon ( 804)) can be displayed. The external electronic device according to an embodiment of the present document includes at least one list of applications executable through the external electronic device 800 (eg, a first application icon 801 and a second application icon 802). Information (eg, the first slate icon 803 and the second slate icon 804) about the virtual objects (eg, the first slate 500 and the second slate 505) may be displayed. Referring to FIGS. 8B and 8C , the external electronic device 800 according to an embodiment of the present document may detect a selection input for at least one application icon (eg, the second slate icon 804). According to one embodiment of this document, for an application icon for which an execution screen of a specific application may be provided as a virtual object through the wearable device 100, another application icon (eg, an execution screen only through the external electronic device 800) It may be displayed on the external electronic device 800 to be visually distinguished from icons corresponding to applications that may be provided. For example, an application icon for which an execution screen of a specific application may be provided as a virtual object through the wearable device 100 is displayed so that the background color of the icon is distinguished from other application icons so that it can be visually distinguished from other icons. Alternatively, the color of the border of the icon may be displayed in a different color, or a predetermined visual effect (eg, a blur effect) may be provided around the icon. According to one embodiment of this document, an application for which an execution screen of a specific application may be provided as a virtual object through the wearable device 100 is a user's input from the wearable device 100 and/or the external electronic device 800. It may be set by or pre-specified. In addition, the types of various visual effects provided for an application icon for which an execution screen of a specific application may be provided as a virtual object are pre-specified or provided to the user through the wearable device 100 and/or the external electronic device 800. can be set by In the external electronic device 800 according to an embodiment of the present document, a plurality of application execution screens are displayed on the wearable device 100 in excess of the number of applications (eg, 3) that can be displayed as multi-windows on the external electronic device. If there is (eg, when four application execution screens are formed as one group in the wearable device 100), in order to be selected whether to display the execution screen of any application as a multi-window on the external electronic device 800, Among the icons included in the second slate icon 804, a list of icons of applications (eg, setting applications and file applications) other than applications (eg, music applications and email applications) displayed as multi-windows may be displayed. In FIG. 8C , while the execution screen 810 of the music application displayed on the external electronic device 800 and the execution screen 820 of the email application displayed on the external electronic device 800 are displayed as multi-windows, the setting application An embodiment in which the icon 830 and the file application icon 840 are displayed is shown as an example. Referring to FIGS. 8D and 8E , the external electronic device 800 according to an embodiment of the present document provides a selection input (eg, a file application icon 840) for an application (eg, a file application) to be displayed as a multi-window. When a touch input to ) is detected, the execution screen of the selected application (eg, the execution screen 810 of the file application displayed on the external electronic device 800) may be displayed as a multi-window. In FIGS. 7 to 8E , a smart phone is illustratively illustrated and referred to as an external electronic device, but is not limited thereto to various embodiments of this document. For example, according to other embodiments of the present document, the external electronic device may include a device for providing virtual reality or augmented reality, such as a head mounted device (HMD), and the implementations shown in FIGS. 7 to 8E Examples may be provided to the user in the manner of virtual reality or augmented reality.

9 is an exemplary diagram for explaining a function or operation provided by a wearable device according to an embodiment of the present document by grouping a plurality of application execution screens based on a user's gaze direction. 10A and 10B are exemplary diagrams for explaining the function or operation described in FIG. 9 from the viewpoint of a graphic user interface.

Referring to FIG. 9 , the wearable device 100 according to an embodiment of the present document may receive an application execution request in operation 910 . The wearable device 100 according to an embodiment of the present document may receive an application execution request by detecting a user input (eg, a virtual touch input to the third icon 563) as shown in FIG. 5E. .

In operation 920, the wearable device 100 according to an embodiment of the present document may obtain the user's gaze direction information. In operation 930, the wearable device 100 according to an embodiment of the present document may group and provide application execution screens to correspond to the user's gaze direction. In FIGS. 10A and 10B , an embodiment in which the user's line of sight is directed toward the first slate 500 is illustratively illustrated. The wearable device 100 according to an embodiment of the present document groups the execution screen 520 of a selected application (eg, a music application) with execution screens of other applications as shown in FIGS. 10A and 10B and shows the The display module 150 may be controlled so as to be displayed.

11 is a control for the wearable device 100 according to an embodiment of the present document to include at least one execution screen among a plurality of grouped application execution screens in the same group as a plurality of other grouped application execution screens, or , It is an exemplary diagram for explaining a function or operation that is independently controlled to be displayed. 12A to 12D are exemplary diagrams for explaining the function or operation described in FIG. 11 from a graphical user interface point of view.

Referring to FIG. 11 , the wearable device 100 according to an embodiment of the present document may detect, in operation 1110, a gesture for selecting at least one execution screen from among a plurality of grouped application execution screens. As exemplarily shown in FIG. 12A , the wearable device 100 according to an embodiment of the present document includes a music application execution screen 520, an email application execution screen 530, a file application execution screen 540, and A user input for selecting the file application execution screen 540 among the gallery application execution screens 550 (eg, a grip gesture for the execution screen) may be detected. The wearable device 100 according to an embodiment of the present document may display a selected application execution screen (eg, a file application execution screen 540) to visually distinguish a selected application execution screen (eg, a file application execution screen 540) from other application execution screens. A visual effect (eg, a shading effect) may be provided to the execution screen 540 . As shown in FIG. 12B , the wearable device 100 according to an embodiment of the present document may detect a user input for moving the selected application execution screen (eg, the file application execution screen 540). The wearable device 100 according to an embodiment of the present document provides a visual effect (eg, a shadow effect) even while a user input for moving a selected application execution screen (eg, the file application execution screen 540) is detected can do.

The wearable device 100 according to an embodiment of the present document, in operation 1120, performs a gesture (eg, a drop gesture) to deselect the selected at least one application execution screen (eg, the file application execution screen 540). can be detected. In operation 1130, the wearable device 100 according to an embodiment of the present document determines whether another slate (eg, the third slate 1200) includes an execution screen (eg, the file application execution screen 540). can judge In the wearable device 100 according to an embodiment of the present document, in operation 1140, an application execution screen included in another slate and at least one selected application execution screen (eg, the file application execution screen 540) are displayed on the same slate. To belong, at least one selected application execution screen (eg, the file application execution screen 540) may be arranged. As shown in FIG. 12C , the wearable device 100 according to an embodiment of the present document displays the selected at least one application execution screen (eg, the file application execution screen 540) to another slate (eg, SMS application execution screen). A drop gesture may be detected at a location where the third slate 1200 including the screen 520 and the Internet application execution screen 1220 is displayed. In this case, the wearable device 100 according to an embodiment of the present document displays the selected at least one application execution screen (eg, the file application execution screen 540) on another slate (eg, the third slate 1200). The display module 150 may be controlled to be displayed while being included in the same group as the included application execution screen. The wearable device 100 according to an embodiment of the present document, when at least one selected application execution screen (eg, the file application execution screen 540) is displayed on another slate, provides a visual effect (eg, a shading effect) ) may be controlled so that the display module 150 is not displayed.

In operation 1150, in operation 1130, the wearable device 100 according to an embodiment of the present document, if another slate does not include an execution screen, selects at least one application execution screen (eg, a file application execution screen). (540)) can be arranged to be included as a new slate. For example, as shown in FIG. 12B , the wearable device 100 according to an embodiment of the present document displays at least one selected application execution screen (eg, a file application execution screen 540) on another slate (eg, a file application execution screen 540). : The display module 150 can be controlled to be displayed independently of the third slate 1200). The wearable device 100 according to an embodiment of the present document, when at least one selected application execution screen (eg, the file application execution screen 540) is independently displayed, provides a visual effect (eg, a shading effect) The display module 150 may be controlled so that is not shown.

13 is an exemplary diagram for explaining a function or operation of enlarging or reducing at least one application execution screen according to a user's gesture by a wearable device according to an embodiment of the present document. 14A to 14C are exemplary diagrams for explaining the function or operation described in FIG. 11 from the viewpoint of a graphic user interface.

Referring to FIG. 13 , the wearable device 100 according to an embodiment of the present document may, in operation 1310, detect an enlargement gesture for enlarging at least one application execution screen. In operation 1320, the wearable device 100 according to an embodiment of the present document may enlarge and provide at least one application execution screen. As shown in FIG. 14A, the wearable device 100 according to an embodiment of the present document detects a gesture of gripping a specific application execution screen (eg, a file application execution screen 540) and moving it to the side where the user is located In this case, it may be determined that an enlargement gesture for enlarging at least one application execution screen (eg, the file application execution screen 540) is detected. When a zooming gesture is detected, the wearable device 100 according to an embodiment of the present document, as shown in FIG. 14A, displays at least one selected application execution screen (eg, the file application execution screen 540) as The display module 150 may be controlled to be shown enlarged at a ratio.

In operation 1330, the wearable device 100 according to an embodiment of the present document may detect a reduction gesture for reducing the enlarged at least one application execution screen (eg, the file application execution screen 540). In operation 1340, the wearable device 100 according to an embodiment of the present document may provide a reduced application execution screen at a position where the execution screen of at least one application is disposed. As shown in FIG. 14B , the wearable device 100 according to an embodiment of the present document may detect a gesture of pushing back an enlarged specific application execution screen. As shown in FIG. 14C , the wearable device 100 according to an embodiment of the present document executes a reduced (eg, size before enlargement) application again at a position where an execution screen of at least one application is arranged. screen can be provided. The wearable device 100 according to an embodiment of the present document specifies a display state of an icon (eg, the eighth icon 565) corresponding to a slate including a specific application execution screen even when the specific application execution screen is enlarged. The application execution screen can be maintained the same as before expansion.

15 shows a layout of at least one application execution screen grouped by the wearable device 100 according to an embodiment of the present document, and providing at least one application execution screen according to the stored layout based on a user's input. It is an exemplary diagram for explaining a function or operation to be performed. 16A to 16D are exemplary diagrams for explaining the function or operation described in FIG. 15 from a graphical user interface point of view.

Referring to FIG. 15 , the wearable device 100 according to an embodiment of the present document may, in operation 1510, group and provide a plurality of application execution screens. In operation 1520, the wearable device 100 according to an embodiment of the present document may detect a gesture for saving a layout of at least one grouped application execution screen. Referring to FIG. 16A , the wearable device 100 according to an embodiment of the present document shows a plurality of application execution screens (eg, a music application execution screen 520 and an email application execution screen 530) being displayed. In the state, a selection input for a visual object set to store a layout of at least one application execution screen being displayed through the wearable device 100 may be detected. When the wearable device 100 according to an embodiment of the present document detects a selection input for a visual object set to store a layout of at least one application execution screen being displayed through the wearable device 100, the layout The display module 150 may be controlled so that a visual object (eg, the eighth icon 565) corresponding to is shown on the application tray 560 .

In operation 1530, the wearable device 100 according to an embodiment of the present document displays at least one application execution screen (eg, a music application execution screen 520 and an e-mail application execution screen 530) according to the stored layout. A gesture to do may be detected. As shown in FIG. 16B , the wearable device 100 according to an embodiment of the present document may detect a user input for selecting a visual object (eg, the eighth icon 565) corresponding to the layout.

The wearable device 100 according to an embodiment of the present document may, in operation 1540, provide at least one application execution screen grouped based on the stored layout. When a gesture according to operation 1530 is detected while another slate is being viewed, the wearable device 100 according to an embodiment of the present document displays an application execution screen (eg, an SMS application execution screen 510) included in the other slate. )) and/or reducing the size according to a specified ratio, displaying at least one grouped application execution screen overlapped with an application execution screen (eg, SMS application execution screen 510) included in another slate. The module 150 can be controlled. As shown in FIG. 16D , the wearable device 100 according to an embodiment of the present document may detect a gesture of moving at least one grouped application execution screen through the second handler 501b. In this case, the wearable device 100 according to an embodiment of the present document moves and/or reduces the application execution screen (eg, the SMS application execution screen 510) that has been moved backward and/or reduced to the previous position. The display module 150 may be controlled to be enlarged to a size before being displayed.

17 is an exemplary diagram for explaining a function or operation of grouping a plurality of application execution screens into the same group based on movement of the application execution screen by the wearable device 100 according to an embodiment of the present document. 18A to 18E are exemplary diagrams for explaining the function or operation described in FIG. 17 from the viewpoint of a graphic user interface.

Referring to FIG. 17 , in the wearable device 100 according to an embodiment of the present document, in operation 1710, among a plurality of application execution screens, a first application execution screen (eg, a music application execution screen 520) Moving closer to the second application execution screen (eg, the email application execution screen 530) may be detected. As shown in FIG. 18A , the wearable device 100 according to an embodiment of the present document includes a plurality of slates (eg, a first slate 500 and a second slate (eg, a first slate 500) each including a plurality of application execution screens. 505) and the fourth slate 1840) may be controlled to display the display module 150. As shown in FIG. 18B , the wearable device 100 according to an embodiment of the present document uses a second handler 501b to transfer a specific slate (eg, the second slate 505) to another specific slate (eg, the second slate 505). : A gesture moving to approach the fourth slate 1840 (eg, within a predetermined distance) may be detected. The wearable device 100 according to an embodiment of the present document approaches (eg, within a predetermined distance) a specific slate (eg, the second slate 505) to another specific slate (eg, the fourth slate 1840). 1850, a pre-specified visual effect indicating that application execution screens included in each slate can be formed as one group, when it is detected that it is moved so as to be included) (eg, a visual element including a specified color) 1850 ) may be controlled to display the display module 150 .

In operation 1720, the wearable device 100 according to an embodiment of the present document displays a first application execution screen (eg, music application execution screen 520) and a second application execution screen (eg, email application execution screen 530). )) can be grouped into one group. For example, as shown in FIG. 18C , the wearable device 100 according to an embodiment of the present document includes a first application execution screen (eg, a music application execution screen 520) and a second application execution screen ( Example: The display module 150 may be controlled so that the email application execution screen 530 is included in one slate (eg, the second slate 505). According to an embodiment of the present document, the wearable device 100 may approach (eg, within a predetermined distance) a specific slate (eg, the second slate 505) to another specific slate (eg, the fourth slate 1840). included), the first application execution screen (eg, music application execution screen 520) and the second application execution screen (eg, regardless of whether a user's selection input is received or not) are automatically (eg, regardless of whether a user's selection input is received or not) : The email application execution screen 530) may be grouped. Alternatively, according to another embodiment of the present document, the wearable device 100 displays a guide message asking whether to set a plurality of application execution screens to be included in a group together with a pre-specified visual effect 1850. The display module 150 may be controlled. In this case, the wearable device 100 according to an embodiment of the present document displays a first application execution screen (eg, a music application execution screen 520) and a second application execution screen ( Example: An email application execution screen 530) may be grouped.

In operation 1730, the wearable device 100 according to an embodiment of the present document displays a grouped first application execution screen (eg, music application execution screen 520) and a second application execution screen (eg, email application execution screen). 530) may be moved closer to the third application execution screen (eg, the SMS application execution screen 510). As shown in FIG. 18D , the wearable device 100 according to an embodiment of the present document includes a grouped first application execution screen (eg, a music application execution screen 520) and a second application execution screen (eg, a music application execution screen 520). It may be detected that the email application execution screen 530 is moved closer to the third application execution screen (eg, the SMS application execution screen 510). In this case, the wearable device 100 according to an embodiment of the present document may control the display module 150 to display a pre-specified visual effect 1850 .

In operation 1740, the wearable device 100 according to an embodiment of the present document displays a first application execution screen (eg, music application execution screen 520) and a second application execution screen (eg, email application execution screen 530). )) and the third application execution screen (eg, the SMS application execution screen 510) may be grouped into one group. As shown in FIG. 18E , the wearable device 100 according to an embodiment of the present document includes a first application execution screen (eg, a music application execution screen 520) and a second application execution screen (eg, an email application). The display module 150 may be controlled so that the execution screen 530) and the third application execution screen (eg, the SMS application execution screen 510) are displayed on the same slate.

19 is a diagram of a wearable device according to an embodiment of the present document moving a location where at least one application execution screen is displayed according to a user's input (eg, user's gaze), and the moved at least one application execution screen; It is an exemplary diagram for explaining a function or operation of grouping at least one other application execution screen into one group. 20A to 20D are exemplary diagrams for explaining the function or operation described in FIG. 19 from the viewpoint of a graphic user interface.

Referring to FIG. 19 , the wearable device 100 according to an embodiment of the present document may, in operation 1910, detect the user's gaze on the first application execution screen (eg, the SMS application execution screen 510). there is. In operation 1920, the wearable device 100 according to an embodiment of the present document may move the location of the first application execution screen. Referring to FIG. 20A , the wearable device 100 according to an embodiment of the present document, when it is identified that the user's gaze direction is facing the first application execution screen (eg, the SMS application execution screen 510), A visual effect 1850 indicating that one application execution screen (eg, the SMS application execution screen 510) is selected may be provided. In this case, in the wearable device 100 according to an embodiment of the present document, as shown in FIG. 20B, the location where the first application execution screen (eg, the SMS application execution screen 510) is displayed is moved and displayed. The display module 150 may be controlled.

In operation 1930, the wearable device 100 according to an embodiment of the present document detects that a first application execution screen (eg, the SMS application execution screen 510) is moved to approach the grouped at least one application execution screen. can do. In operation 1940, the wearable device 100 according to an embodiment of the present document may group the grouped at least one application execution screen and the first application execution screen into one group. As shown in FIG. 20C , in the wearable device 100 according to an embodiment of the present document, a first application execution screen (eg, SMS application execution screen 510) is close to the grouped at least one application execution screen. It can be detected that it is moved to In this case, a visual effect 1850 according to an embodiment of the present document may be displayed. As shown in FIG. 20D , the wearable device 100 according to an embodiment of the present document includes at least one grouped application execution screen and a first application execution screen (eg, SMS application execution screen 510). It is possible to control the display module 150 so that it is grouped into groups of .

21 is an example for explaining a function or operation of grouping at least one application execution screen grouped and at least one other application execution screen grouped into the same group by the wearable device 100 according to an embodiment of the present document. it is a drawing 22A to 22C are exemplary diagrams for explaining the function or operation described in FIG. 21 from the viewpoint of a graphic user interface.

Referring to FIG. 21 , in the wearable device 100 according to an embodiment of the present document, in operation 2110, a first slate including at least one grouped application execution screen displays another grouped at least one application execution screen. It can be detected that it is moved to the second slate including . As shown in FIG. 22A , the wearable device 100 according to an embodiment of the present document may detect a gesture of gripping a specific slate (eg, the first slate 500). As shown in FIG. 22B , the wearable device 100 according to an embodiment of this document may detect a gesture of moving a specific slate (eg, the first slate 500).

In operation 2120, the wearable device 100 according to an embodiment of the present document divides at least one application execution screen included in the first slate and at least one other application execution screen included in the second slate into one group. can be grouped into As shown in FIG. 22C , the wearable device 100 according to an embodiment of this document executes a plurality of applications included in each slate (eg, the first slate 500 and the second slate 505). The display module 150 may be controlled so that screens are grouped into one group and displayed.

23 is a diagram so that the wearable device 100 according to an embodiment of this document displays an execution screen of an application running on an external electronic device operatively connected to the wearable device 100 as a virtual object according to a user's gesture. It is an exemplary diagram for explaining a function or operation of controlling a wearable device. 24a to 24c are exemplary diagrams for explaining the function or operation described in FIG. 19 from the viewpoint of a graphic user interface.

Referring to FIG. 23 , the wearable device 100 according to an embodiment of the present document, in operation 2310, displays at least one grouped application execution screen (eg, a music application execution screen 520 and an e-mail application execution screen 530). ), the proximity of an external electronic device (eg, the second external electronic device 2420) to the file application execution screen 540 and/or the gallery application execution screen 550 may be detected. The wearable device 100 according to an embodiment of the present document identifies the shape of an external electronic device (eg, the second external electronic device 2420) based on an image acquired by a camera of the wearable device 100 and , The type of external electronic device (eg, the second external electronic device 2420) and / or the external electronic device (eg, the second external electronic device 2420) are compared in the real world by comparing the identified shape with a pre-stored shape. location can be identified. In the wearable device 100 according to an embodiment of the present document, when the shape of the identified external electronic device overlaps with the execution screen of the application, at least one grouped application execution screen (eg, music application execution screen 520) , It may be determined that an external electronic device (eg, the second external electronic device 2420) is approaching the email application execution screen 530, the file application execution screen 540, and/or the gallery application execution screen 550. . When the wearable device 100 according to an embodiment of the present document determines that an external electronic device (eg, the second external electronic device 2420) is approaching, a predetermined visual effect (eg, a blur effect and an arrow) ( 2432) may control the display module 150 to be displayed around the external electronic device (eg, the second external electronic device 2420).

In operation 2320, the wearable device 100 according to an embodiment of the present document displays an application execution screen (eg, a calendar application execution screen 2422) being displayed on an external electronic device (eg, the second external electronic device 2420). ) is newly grouped with at least one application execution screen grouped (eg, music application execution screen 520, e-mail application execution screen 530, file application execution screen 540, and/or gallery application execution screen 550). A user input (eg, a swipe gesture) may be detected. As shown in FIG. 24B , the wearable device 100 according to an embodiment of the present document may detect that a swipe gesture for an external electronic device (eg, the second external electronic device 2420) is input. . The wearable device 100 according to an embodiment of the present document performs a swipe gesture for an external electronic device (eg, the second external electronic device 2420) based on an image acquired by a camera of the wearable device 100 It can be detected that has been entered. Alternatively, the wearable device 100 according to an embodiment of the present document may receive a signal indicating that a swipe gesture is input from an external electronic device (eg, the second external electronic device 2420) through a network. there is. The wearable device 100 according to an embodiment of the present document detects that a swipe gesture for an external electronic device (eg, the second external electronic device 2420) is input based on a signal indicating that the swipe gesture is input can do.

In operation 2330, the wearable device 100 according to an embodiment of the present document converts an application execution screen being displayed on an external electronic device (eg, the second external electronic device 2420) into at least one grouped application execution screen. can be provided to belong to the same group as As shown in FIG. 24C , the wearable device 100 according to an embodiment of the present document displays an application execution screen (eg, calendar application execution) being displayed on an external electronic device (eg, the second external electronic device 2420). The display module 150 may be controlled so that the virtual object 2450 corresponding to the screen 2422 is displayed on the same slate. According to an embodiment of this document, when a specific application (eg, a calendar application) is not installed in the wearable device 100, the wearable device 100 displays an external electronic device (eg, the second external electronic device 2420). ))), and after installing the specific application, the application execution screen being displayed on the external electronic device (eg, the second external electronic device 2420) (eg, the calendar application execution screen 2422) The display module 150 may be controlled so that the virtual object 2450 corresponding to is displayed.

25 is a block diagram of an electronic device 2501 within a network environment 2500, according to various embodiments. Referring to FIG. 25 , in a network environment 2500, an electronic device 2501 communicates with an electronic device 2502 through a first network 2598 (eg, a short-range wireless communication network) or through a second network 2599. It may communicate with at least one of the electronic device 2504 or the server 2508 through (eg, a long-distance wireless communication network). According to an embodiment, the electronic device 2501 may communicate with the electronic device 2504 through the server 2508. According to an embodiment, the electronic device 2501 includes a processor 2520, a memory 2530, an input module 2550, an audio output module 2555, a display module 2560, an audio module 2570, a sensor module ( 2576), interface (2577), connection terminal (2578), haptic module (2579), camera module (2580), power management module (2588), battery (2589), communication module (2590), subscriber identification module (2596) , or an antenna module 2597. In some embodiments, in the electronic device 2501, at least one of these components (eg, the connection terminal 2578) may be omitted or one or more other components may be added. In some embodiments, some of these components (eg, sensor module 2576, camera module 2580, or antenna module 2597) are integrated into a single component (eg, display module 2560). It can be.

The processor 2520, for example, executes software (eg, the program 2540) to cause at least one other component (eg, hardware or software component) of the electronic device 2501 connected to the processor 2520. It can control and perform various data processing or calculations. According to one embodiment, as at least part of data processing or operation, processor 2520 transfers instructions or data received from other components (e.g., sensor module 2576 or communication module 2590) to volatile memory 2532. , process commands or data stored in the volatile memory 2532, and store resultant data in the non-volatile memory 2534. According to one embodiment, the processor 2520 may include a main processor 2521 (eg, a central processing unit or an application processor) or an auxiliary processor 2523 (eg, a graphic processing unit, a neural network processing unit ( NPU: neural processing unit (NPU), image signal processor, sensor hub processor, or communication processor). For example, when the electronic device 2501 includes a main processor 2521 and an auxiliary processor 2523, the auxiliary processor 2523 may use less power than the main processor 2521 or be set to be specialized for a designated function. can The auxiliary processor 2523 may be implemented separately from or as part of the main processor 2521 .

The auxiliary processor 2523 may, for example, take the place of the main processor 2521 while the main processor 2521 is in an inactive (eg, sleep) state, or when the main processor 2521 is active (eg, running an application). ) state, together with the main processor 2521, at least one of the components of the electronic device 2501 (eg, the display module 2560, the sensor module 2576, or the communication module 2590) It is possible to control at least some of the related functions or states. According to one embodiment, the co-processor 2523 (eg, image signal processor or communication processor) may be implemented as part of other functionally related components (eg, camera module 2580 or communication module 2590). there is. According to an embodiment, the auxiliary processor 2523 (eg, a neural network processing device) may include a hardware structure specialized for processing an artificial intelligence model. AI models can be created through machine learning. Such learning may be performed, for example, in the electronic device 2501 itself where the artificial intelligence model is performed, or may be performed through a separate server (eg, the server 2508). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning or reinforcement learning, but in the above example Not limited. The artificial intelligence model may include a plurality of artificial neural network layers. Artificial neural networks include deep neural networks (DNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), restricted boltzmann machines (RBMs), deep belief networks (DBNs), bidirectional recurrent deep neural networks (BRDNNs), It may be one of deep Q-networks or a combination of two or more of the foregoing, but is not limited to the foregoing examples. The artificial intelligence model may include, in addition or alternatively, software structures in addition to hardware structures.

The memory 2530 may store various data used by at least one component (eg, the processor 2520 or the sensor module 2576) of the electronic device 2501 . The data may include, for example, input data or output data for software (eg, the program 2540) and commands related thereto. The memory 2530 may include a volatile memory 2532 or a non-volatile memory 2534 .

The program 2540 may be stored as software in the memory 2530, and may include, for example, an operating system 2542, middleware 2544, or an application 2546.

The input module 2550 may receive a command or data to be used by a component (eg, the processor 2520) of the electronic device 2501 from an outside of the electronic device 2501 (eg, a user). The input module 2550 may include, for example, a microphone, a mouse, a keyboard, a key (eg, a button), or a digital pen (eg, a stylus pen).

The sound output module 2555 may output sound signals to the outside of the electronic device 2501 . The sound output module 2555 may include, for example, a speaker or receiver. The speaker can be used for general purposes such as multimedia playback or recording playback. A receiver may be used to receive an incoming call. According to one embodiment, the receiver may be implemented separately from the speaker or as part of it.

The display module 2560 can visually provide information to the outside of the electronic device 2501 (eg, a user). The display module 2560 may include, for example, a display, a hologram device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module 2560 may include a touch sensor configured to detect a touch or a pressure sensor configured to measure the intensity of force generated by the touch.

The audio module 2570 may convert sound into an electrical signal or vice versa. According to an embodiment, the audio module 2570 acquires sound through the input module 2550, the sound output module 2555, or an external electronic device connected directly or wirelessly to the electronic device 2501 (eg: Sound may be output through the electronic device 2502 (eg, a speaker or a headphone).

The sensor module 2576 detects an operating state (eg, power or temperature) of the electronic device 2501 or an external environmental state (eg, a user state), and generates an electrical signal or data value corresponding to the detected state. can do. According to one embodiment, the sensor module 2576 may include, for example, a gesture sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, It may include a temperature sensor, humidity sensor, or light sensor.

The interface 2577 may support one or more specified protocols that may be used to directly or wirelessly connect the electronic device 2501 to an external electronic device (eg, the electronic device 2502). According to one embodiment, the interface 2577 may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

The connection terminal 2578 may include a connector through which the electronic device 2501 may be physically connected to an external electronic device (eg, the electronic device 2502). According to one embodiment, the connection terminal 2578 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (eg, a headphone connector).

The haptic module 2579 may convert electrical signals into mechanical stimuli (eg, vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module 2579 may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

The camera module 2580 may capture still images and moving images. According to one embodiment, the camera module 2580 may include one or more lenses, image sensors, image signal processors, or flashes.

The power management module 2588 may manage power supplied to the electronic device 2501 . According to one embodiment, the power management module 2588 may be implemented as at least part of a power management integrated circuit (PMIC), for example.

The battery 2589 may supply power to at least one component of the electronic device 2501 . According to one embodiment, the battery 2589 may include, for example, a non-rechargeable primary cell, a rechargeable secondary cell, or a fuel cell.

The communication module 2590 is a direct (eg, wired) communication channel or wireless communication channel between the electronic device 2501 and an external electronic device (eg, the electronic device 2502, the electronic device 2504, or the server 2508). Establishment and communication through the established communication channel may be supported. The communication module 2590 may include one or more communication processors that operate independently of the processor 2520 (eg, an application processor) and support direct (eg, wired) communication or wireless communication. According to one embodiment, the communication module 2590 may be a wireless communication module 2592 (eg, a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 2594 (eg, a cellular communication module). : a local area network (LAN) communication module or a power line communication module). Among these communication modules, a corresponding communication module is a first network 2598 (eg, a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network 2599 (eg, legacy It may communicate with the external electronic device 2504 through a cellular network, a 5G network, a next-generation communication network, the Internet, or a telecommunications network such as a computer network (eg, LAN or WAN). These various types of communication modules may be integrated as one component (eg, a single chip) or implemented as a plurality of separate components (eg, multiple chips). The wireless communication module 2592 uses subscriber information (eg, International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module 2596 within a communication network such as the first network 2598 or the second network 2599. The electronic device 2501 may be identified or authenticated.

The wireless communication module 2592 may support a 5G network after a 4G network and a next-generation communication technology, for example, NR access technology (new radio access technology). NR access technologies include high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and access of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low latency (URLLC)). -latency communications)) can be supported. The wireless communication module 2592 may support a high frequency band (eg, mmWave band) to achieve a high data rate, for example. The wireless communication module 2592 uses various technologies for securing performance in a high frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), and full-dimensional multiplexing. Technologies such as input/output (FD-MIMO: full dimensional MIMO), array antenna, analog beam-forming, or large scale antenna may be supported. The wireless communication module 2592 may support various requirements defined for the electronic device 2501, an external electronic device (eg, the electronic device 2504), or a network system (eg, the second network 2599). According to one embodiment, the wireless communication module 2592 may include a peak data rate (eg, 20 Gbps or more) for eMBB realization, a loss coverage (eg, 164 dB or less) for mMTC realization, or a U-plane latency (eg, URLLC realization). Example: downlink (DL) and uplink (UL) each of 0.5 ms or less, or round trip 1 ms or less) may be supported.

The antenna module 2597 may transmit or receive signals or power to the outside (eg, an external electronic device). According to one embodiment, the antenna module 2597 may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (eg, PCB). According to one embodiment, the antenna module 2597 may include a plurality of antennas (eg, an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network such as the first network 2598 or the second network 2599 is selected from the plurality of antennas by, for example, the communication module 2590. can be chosen A signal or power may be transmitted or received between the communication module 2590 and an external electronic device through the selected at least one antenna. According to some embodiments, other components (eg, a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module 2597 in addition to the radiator.

According to various embodiments, the antenna module 2597 may form a mmWave antenna module. According to one embodiment, the mmWave antenna module includes a printed circuit board, an RFIC disposed on or adjacent to a first surface (eg, a lower surface) of the printed circuit board and capable of supporting a designated high frequency band (eg, mmWave band); and a plurality of antennas (eg, array antennas) disposed on or adjacent to a second surface (eg, a top surface or a side surface) of the printed circuit board and capable of transmitting or receiving signals of the designated high frequency band. can do.

At least some of the components are connected to each other through a communication method between peripheral devices (eg, a bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI)) and signal ( e.g. commands or data) can be exchanged with each other.

According to an embodiment, commands or data may be transmitted or received between the electronic device 2501 and the external electronic device 2504 through the server 2508 connected to the second network 2599 . Each of the external electronic devices 2502 or 2504 may be the same as or different from the electronic device 2501 . According to an embodiment, all or part of operations executed in the electronic device 2501 may be executed in one or more external electronic devices among the external electronic devices 2502 , 2504 , or 2508 . For example, when the electronic device 2501 needs to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device 2501 instead of executing the function or service by itself. Alternatively or additionally, one or more external electronic devices may be requested to perform the function or at least part of the service. One or more external electronic devices receiving the request may execute at least a part of the requested function or service or an additional function or service related to the request, and deliver the execution result to the electronic device 2501 . The electronic device 2501 may provide the result as at least part of a response to the request as it is or additionally processed. To this end, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device 2501 may provide an ultra-low latency service using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device 2504 may include an internet of things (IoT) device. Server 2508 may be an intelligent server using machine learning and/or neural networks. According to an embodiment, the external electronic device 2504 or server 2508 may be included in the second network 2599. The electronic device 2501 may be applied to intelligent services (eg, smart home, smart city, smart car, or health care) based on 5G communication technology and IoT-related technology.

Electronic devices according to various embodiments disclosed in this document may be devices of various types. The electronic device may include, for example, a portable communication device (eg, a smart phone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a home appliance. An electronic device according to an embodiment of the present document is not limited to the aforementioned devices.

Various embodiments of this document and terms used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, like reference numbers may be used for like or related elements. The singular form of a noun corresponding to an item may include one item or a plurality of items, unless the relevant context clearly dictates otherwise. In this document, "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C", and "A Each of the phrases such as "at least one of , B, or C" may include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first", "second", or "first" or "secondary" may simply be used to distinguish a given component from other corresponding components, and may be used to refer to a given component in another aspect (eg, importance or order) is not limited. A (e.g., first) component is said to be “coupled” or “connected” to another (e.g., second) component, with or without the terms “functionally” or “communicatively.” When mentioned, it means that the certain component may be connected to the other component directly (eg by wire), wirelessly, or through a third component.

The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and is interchangeable with terms such as, for example, logic, logical blocks, parts, or circuits. can be used as A module may be an integrally constructed component or a minimal unit of components or a portion thereof that performs one or more functions. For example, according to one embodiment, the module may be implemented in the form of an application-specific integrated circuit (ASIC).

Various embodiments of this document provide one or more instructions stored in a storage medium (eg, internal memory 136 or external memory 138) readable by a machine (eg, electronic device 101). It may be implemented as software (eg, the program 140) including them. For example, a processor (eg, the processor 120) of a device (eg, the electronic device 101) may call at least one command among one or more instructions stored from a storage medium and execute it. This enables the device to be operated to perform at least one function according to the at least one command invoked. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, "non-transitory" only means that the storage medium is a tangible device and does not contain a signal (e.g. electromagnetic wave), and this term refers to the case where data is stored semi-permanently in the storage medium. It does not discriminate when it is temporarily stored.

According to one embodiment, the method according to various embodiments disclosed in this document may be included and provided in a computer program product. Computer program products may be traded between sellers and buyers as commodities. A computer program product is distributed in the form of a device-readable storage medium (e.g. compact disc read only memory (CD-ROM)), or through an application store (e.g. Play Store™) or on two user devices (e.g. It can be distributed (eg downloaded or uploaded) online, directly between smart phones. In the case of online distribution, at least part of the computer program product may be temporarily stored or temporarily created in a storage medium readable by a device such as a manufacturer's server, an application store server, or a relay server's memory.

According to one embodiment of this document, each component (eg, module or program) of the components described above may include a single object or a plurality of objects, and some of the multiple objects are separated from other components. It could be. According to one embodiment of the present document, one or more components or operations among the aforementioned corresponding components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (eg modules or programs) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by a corresponding component among the plurality of components prior to the integration. . According to one embodiment of this document, operations performed by modules, programs, or other components are executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations are executed in a different order, or , may be omitted, or one or more other operations may be added.

Claims (20)

In the wearable device,
a display module; and
including at least one processor, wherein the at least one processor comprises:
Control the display module so that an execution screen of a first application is displayed as a virtual object, and the execution screen of the first application is displayed according to a first type;
detecting a first user input for executing a second application while the execution screen of the first application is displayed as the virtual object;
After the second user input is detected, it is determined whether or not a second user input requesting that the execution screen of the second application be included in the same group as the execution screen of the first application is detected;
Based on the result of the determination, it is configured to control the display module to display the execution screen of the second application independently of the execution screen of the first application or while forming a group with the execution screen of the first application. Characterized in that, a wearable device. According to claim 1,
A visual element corresponding to the execution screen of the second application is spatially separated from the execution screen of the first application and displayed according to a second type different from the first type. device. According to claim 1,
The wearable device, characterized in that the visual element for executing the first application is spatially separated from the execution screen of the first application and displayed according to a second type different from the first type. According to claim 1,
The visual element corresponding to the execution screen of the first application and the visual element corresponding to the execution screen of the second application included in the same group form one group and are of a second type different from the first type. A wearable device, characterized in that it is visible. According to claim 1,
The wearable device further includes a communication module,
The at least one processor may cause a plurality of visual elements representing the first application execution screen and the second application execution screen included in the same group in an external electronic device operatively connected to the wearable device through the communication module. Characterized in that, the wearable device is further configured to transmit information on the first application execution screen and the second application execution screen to the external electronic device so as to be displayed. According to claim 1,
The at least one processor may include a second type visual element for providing information about at least one application execution screen displayed through the wearable device and an application running in an external electronic device operably connected to the wearable device. Characterized in that further set to control the display module to be shown according to, wearable device. According to claim 1,
The second user input may include a gaze direction of a user wearing the wearable device. According to claim 7,
The wearable device, characterized in that the at least one processor is further set to provide a predetermined visual effect around a visual element corresponding to the execution screen of a specific application that the user is looking at, based on the gaze direction. According to claim 1,
The at least one processor, while the execution screen of the first application and the execution screen of the second application are displayed independently of each other, the execution screen of the first application and the execution screen of the second application are displayed by the user's input. If it is detected that they overlap with each other according to , the wearable device is further configured to control the display module to show the execution screen of the first application and the execution screen of the second application as the same group. According to claim 1,
When the execution screen of the first application and the execution screen of the second application are shown to be included in the same group, the at least one processor detects a request to insert an execution screen through an external electronic device, the external electronic device. Characterized in that, the wearable device is further set to control the display module so that an execution screen of an application displayed on the electronic device is included in the same group and displayed. A method for controlling a wearable device,
controlling a display module of the wearable device so that an execution screen of a first application is displayed as a virtual object, and the execution screen of the first application is displayed according to a first type;
detecting a first user input for executing a second application while the execution screen of the first application is displayed as the virtual object;
After the second user input is detected, determining whether or not a second user input requesting that the execution screen of the second application be included in the same group as the execution screen of the first application is detected;
Based on the result of the determination, controlling the display module to display the execution screen of the second application independently of the execution screen of the first application or while forming a group with the execution screen of the first application. Characterized in that it comprises, a method for controlling a wearable device. According to claim 11,
A visual element corresponding to the execution screen of the second application is spatially separated from the execution screen of the first application and displayed according to a second type different from the first type. method. According to claim 11,
The method of controlling a wearable device, characterized in that the visual element for executing the first application is spatially separated from the execution screen of the first application and displayed according to a second type different from the first type. According to claim 11,
The visual element corresponding to the execution screen of the first application and the visual element corresponding to the execution screen of the second application included in the same group form one group and are of a second type different from the first type. A method of controlling a wearable device, characterized in that shown. According to claim 11,
The wearable device further includes a communication module,
The method of controlling the wearable device may include a plurality of visual images indicating the first application execution screen and the second application execution screen included in the same group in an external electronic device operatively connected to the wearable device through the communication module. The method of controlling a wearable device of claim 1, further comprising transmitting information on the first application execution screen and the second application execution screen to the external electronic device so that elements are displayed. According to claim 11,
The method of controlling the wearable device includes a visual element for providing information on at least one application execution screen displayed through the wearable device and an application running on an external electronic device operably connected to the wearable device. Characterized in that it further comprises an operation of controlling the display module to be shown according to two types, the method of controlling the wearable device. According to claim 11,
The second user input may include a gaze direction of a user wearing the wearable device. According to claim 17,
The method of controlling the wearable device further comprises providing a predetermined visual effect around a visual element corresponding to an execution screen of a specific application that the user is looking at, based on the gaze direction. , How to control a wearable device. According to claim 11,
The method of controlling the wearable device may include displaying the execution screen of the first application and the execution screen of the second application while the execution screen of the first application and the execution screen of the second application are displayed independently of each other. The wearable device further comprises controlling the display module to show the execution screen of the first application and the execution screen of the second application as the same group when it is detected that they overlap each other according to the input of How to control. According to claim 11,
The method of controlling the wearable device may include, when an execution screen of the first application and an execution screen of the second application are shown to be included in the same group, when a request to insert an execution screen is detected through an external electronic device, The method of controlling the wearable device of claim 1 , further comprising controlling the display module so that an execution screen of an application being displayed on the external electronic device is displayed while being included in the same group.

Images