KR20240043029A - Method for providing image and wearable electronic device for supporting the same - Google Patents

Abstract

A wearable electronic device according to an embodiment includes a display and at least one processor, and the at least one processor may be configured to obtain a distance to an external electronic device. The at least one processor may be configured to obtain an input for displaying the first image displayed on the external electronic device using the display. The at least one processor may be configured to obtain, based on the input, a plurality of second images related to the first image and corresponding to the distance. The at least one processor may be configured to continuously display the number of second images using the display.

Description

Method for providing images and wearable electronic device supporting the same {METHOD FOR PROVIDING IMAGE AND WEARABLE ELECTRONIC DEVICE FOR SUPPORTING THE SAME}

This disclosure relates to a method for providing images and a wearable electronic device that supports the same.

As communication technology develops, wearable electronic devices are becoming smaller and lighter to the point where they can be used without significant discomfort even when worn on the user's body. For example, wearable electronic devices such as head mounted display devices (HMDs), smart watches (or bands), contact lens-type devices, ring-type devices, glove-type devices, shoe-type devices, or clothing-type devices are commercialized. there is. Since wearable electronic devices are worn directly on the body, portability and user accessibility can be improved.

A wearable electronic device (eg, HMD device) is a device that is worn on a user's head or face and can provide augmented reality (AR) to the user. For example, AR glasses that provide augmented reality can be implemented in the form of glasses and provide information about objects in the form of images or text to the user in at least a portion of the user's field of view. VR (virtual reality) glasses can provide virtual reality to users. For example, VR glasses can provide an excellent sense of immersion by outputting independent images to both eyes of the user and outputting content provided from an external input to the user in the form of video or sound. Additionally, wearable electronic devices may provide mixed reality (MR) and/or extended reality (XR).

A wearable electronic device (eg, AR glasses) can display a screen displayed on an external electronic device (eg, a smart phone) to the user when worn by the user. A wearable electronic device may display a screen corresponding to the screen displayed on the external electronic device (e.g., a screen that is substantially the same as the screen displayed on the external electronic device), based on input input while the screen (e.g., content) is displayed on the external electronic device. ) can be displayed through the display.

When the wearable electronic device (e.g., AR glasses) displays a screen corresponding to the screen displayed on the external electronic device through the display, the user can see the image that is viewed through the wearable electronic device (e.g., a transparent member of the wearable electronic device). ) It may feel that the screen of the external electronic device and the screen displayed through the display of the wearable electronic device are not continuous. For example, the position and size of the screen of an external electronic device displayed through a wearable electronic device (e.g., a transparent member of the wearable electronic device) may be different from the position and size of the screen displayed through the display of the wearable electronic device. You can. Due to this difference, the user may feel that there is no continuity between the screen displayed through the wearable electronic device and the screen displayed through the display of the wearable electronic device, and the user's sense of immersion in the screen (e.g. content) may be reduced. can be reduced.

In one embodiment of the present disclosure, when a wearable electronic device displays a screen corresponding to a screen displayed on an external electronic device through a display, the screen displayed through the wearable electronic device and the screen displayed through the display of the wearable electronic device It relates to a method of providing images so that there is continuity between screens and a wearable electronic device that supports the same.

The technical problems to be achieved by the present disclosure are not limited to the technical problems mentioned above, and other technical problems not mentioned can be clearly understood by those skilled in the art related to this document from the description below. There will be.

A wearable electronic device according to an embodiment includes a display and at least one processor, and the at least one processor may be configured to obtain a distance to an external electronic device. The at least one processor may be configured to obtain an input for displaying the first image displayed on the external electronic device using the display. The at least one processor may be configured to obtain, based on the input, a plurality of second images related to the first image and corresponding to the distance. The at least one processor may be configured to continuously display the number of second images using the display.

In one embodiment, a method of providing an image from an electronic device may include obtaining a distance to an external electronic device. The method may include obtaining an input for displaying a first image displayed on the external electronic device using a display of the wearable electronic device. The method may include obtaining, based on the input, a plurality of second images related to the first image and corresponding to the distance. The method may include sequentially displaying the plurality of second images using the display.

In one embodiment, a non-transitory computer-readable medium having computer-executable instructions recorded thereon, wherein the computer-executable instructions, when executed, cause an electronic device including at least one processor to adjust a zoom factor based on a user input. You can change it. When executed, the computer-executable instructions may cause an electronic device including at least one processor to acquire a distance to an external electronic device. When executed, the computer-executable instructions may cause an electronic device including at least one processor to obtain an input for displaying a first image displayed on the external electronic device using the display. The computer-executable instructions, when executed, cause an electronic device including at least one processor to cause the at least one processor to, based on the input, generate a plurality of second images associated with the first image and a number corresponding to the distance. Images can be acquired. When executed, the computer-executable instructions may cause an electronic device including at least one processor to continuously display the number of second images using the display.

1 is a block diagram of an electronic device in a network environment, according to one embodiment.
FIG. 2 is an example diagram for explaining the structure of an electronic device, according to an embodiment.
FIG. 3 is an example diagram for explaining the structure of an eye tracking camera of an electronic device, according to an embodiment.
Figure 4 is a block diagram of an electronic device, according to one embodiment.
Figure 5 is a flowchart for explaining a method of providing an image, according to one embodiment.
FIG. 6 is a diagram illustrating an operation of obtaining a distance to an external electronic device based on the posture of the electronic device and the posture of the external electronic device, according to one embodiment.
FIG. 7 is a diagram illustrating an operation of obtaining a distance to an external electronic device based on an image of the external electronic device, according to an embodiment.
FIG. 8 is a diagram for explaining an image conversion input to an electronic device, according to an embodiment.
FIG. 9 is a diagram for explaining a plurality of second images, according to an embodiment.
FIG. 10 is a diagram for explaining a method of determining sizes of a plurality of second images and areas in which the plurality of second images will be displayed, according to an embodiment.
FIG. 11 is a diagram illustrating a method of determining positions of a plurality of second images based on the posture of an electronic device and the posture of an external electronic device, according to an embodiment.
Figure 12 is a flowchart for explaining a method of providing an image, according to an embodiment.
FIG. 13 is a diagram for explaining a method of providing an image, according to an embodiment.
Figure 14 is a flowchart for explaining a method of providing an image, according to one embodiment.
FIG. 15 is a diagram for explaining a method of providing an image, according to an embodiment.
FIG. 16 is a diagram illustrating a method of providing an image based on binocular disparity, according to an embodiment.
FIG. 17 is a diagram for explaining a method of providing an image, according to an embodiment.
FIG. 18 is a diagram for explaining a method of providing an image, according to an embodiment.
FIG. 19 is a diagram for explaining a method of providing an image according to an embodiment.

Hereinafter, with reference to the drawings, embodiments of the present disclosure will be described in detail so that those skilled in the art can easily practice them. However, the present disclosure may be implemented in many different forms and is not limited to the embodiments described herein. In relation to the description of the drawings, identical or similar reference numerals may be used for identical or similar components. Additionally, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and brevity.

1 is a block diagram of an electronic device 101 in a network environment 100, according to one embodiment.

Referring to FIG. 1, in the network environment 100, the electronic device 101 communicates with the electronic device 102 through a first network 198 (e.g., a short-range wireless communication network) or a second network 199. It is possible to communicate with at least one of the electronic device 104 or the server 108 through (e.g., a long-distance wireless communication network). According to one embodiment, the electronic device 101 may communicate with the electronic device 104 through the server 108. According to one embodiment, the electronic device 101 includes a processor 120, a memory 130, an input module 150, an audio output module 155, a display module 160, an audio module 170, and a sensor module ( 176), interface 177, connection terminal 178, haptic module 179, camera module 180, power management module 188, battery 189, communication module 190, subscriber identification module 196 , or may include an antenna module 197. In some embodiments, at least one of these components (eg, the connection terminal 178) may be omitted or one or more other components may be added to the electronic device 101. In some embodiments, some of these components (e.g., sensor module 176, camera module 180, or antenna module 197) are integrated into one component (e.g., display module 160). It can be.

The processor 120, for example, executes software (e.g., program 140) to operate at least one other component (e.g., hardware or software component) of the electronic device 101 connected to the processor 120. It can be controlled and various data processing or calculations can be performed. According to one embodiment, as at least part of data processing or computation, the processor 120 stores instructions or data received from another component (e.g., sensor module 176 or communication module 190) in volatile memory 132. The commands or data stored in the volatile memory 132 can be processed, and the resulting data can be stored in the non-volatile memory 134. According to one embodiment, the processor 120 includes the main processor 121 (e.g., a central processing unit or an application processor) or an auxiliary processor 123 that can operate independently or together (e.g., a graphics processing unit, a neural network processing unit ( It may include a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, if the electronic device 101 includes a main processor 121 and a secondary processor 123, the secondary processor 123 may be set to use lower power than the main processor 121 or be specialized for a designated function. You can. The auxiliary processor 123 may be implemented separately from the main processor 121 or as part of it.

The auxiliary processor 123 may, for example, act on behalf of the main processor 121 while the main processor 121 is in an inactive (e.g., sleep) state, or while the main processor 121 is in an active (e.g., application execution) state. ), together with the main processor 121, at least one of the components of the electronic device 101 (e.g., the display module 160, the sensor module 176, or the communication module 190) At least some of the functions or states related to can be controlled. According to one embodiment, coprocessor 123 (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module 180 or communication module 190). there is. According to one embodiment, the auxiliary processor 123 (eg, neural network processing device) may include a hardware structure specialized for processing artificial intelligence models. Artificial intelligence models can be created through machine learning. For example, such learning may be performed in the electronic device 101 itself on which the artificial intelligence model is performed, or may be performed through a separate server (e.g., server 108). Learning algorithms may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but It is not limited. An artificial intelligence model may include multiple artificial neural network layers. Artificial neural networks include deep neural network (DNN), convolutional neural network (CNN), recurrent neural network (RNN), restricted boltzmann machine (RBM), belief deep network (DBN), bidirectional recurrent deep neural network (BRDNN), It may be one of deep Q-networks or a combination of two or more of the above, but is not limited to the examples described above. In addition to hardware structures, artificial intelligence models may additionally or alternatively include software structures.

The memory 130 may store various data used by at least one component (eg, the processor 120 or the sensor module 176) of the electronic device 101. Data may include, for example, input data or output data for software (e.g., program 140) and instructions related thereto. Memory 130 may include volatile memory 132 or non-volatile memory 134.

The program 140 may be stored as software in the memory 130 and may include, for example, an operating system 142, middleware 144, or application 146.

The input module 150 may receive commands or data to be used in a component of the electronic device 101 (e.g., the processor 120) from outside the electronic device 101 (e.g., a user). The input module 150 may include, for example, a microphone, mouse, keyboard, keys (eg, buttons), or digital pen (eg, stylus pen).

The sound output module 155 may output sound signals to the outside of the electronic device 101. The sound output module 155 may include, for example, a speaker or a receiver. Speakers can be used for general purposes such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part of it.

The display module 160 can visually provide information to the outside of the electronic device 101 (eg, a user). The display module 160 may include, for example, a display, a hologram device, or a projector, and a control circuit for controlling the device. According to one embodiment, the display module 160 may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of force generated by the touch.

The audio module 170 can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module 170 acquires sound through the input module 150, the sound output module 155, or an external electronic device (e.g., directly or wirelessly connected to the electronic device 101). Sound may be output through the electronic device 102 (e.g., speaker or headphone).

The sensor module 176 detects the operating state (e.g., power or temperature) of the electronic device 101 or the external environmental state (e.g., user state) and generates an electrical signal or data value corresponding to the detected state. can do. According to one embodiment, the sensor module 176 includes, for example, a gesture sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, It may include a temperature sensor, humidity sensor, or light sensor.

The interface 177 may support one or more designated protocols that can be used to connect the electronic device 101 directly or wirelessly with an external electronic device (eg, the electronic device 102). According to one embodiment, the interface 177 may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

The connection terminal 178 may include a connector through which the electronic device 101 can be physically connected to an external electronic device (eg, the electronic device 102). According to one embodiment, the connection terminal 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (eg, a headphone connector).

The haptic module 179 can convert electrical signals into mechanical stimulation (e.g., vibration or movement) or electrical stimulation that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

The camera module 180 can capture still images and moving images. According to one embodiment, the camera module 180 may include one or more lenses, image sensors, image signal processors, or flashes.

The power management module 188 can manage power supplied to the electronic device 101. According to one embodiment, the power management module 188 may be implemented as at least a part of, for example, a power management integrated circuit (PMIC).

The battery 189 may supply power to at least one component of the electronic device 101. According to one embodiment, the battery 189 may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

Communication module 190 is configured to provide a direct (e.g., wired) communication channel or wireless communication channel between electronic device 101 and an external electronic device (e.g., electronic device 102, electronic device 104, or server 108). It can support establishment and communication through established communication channels. Communication module 190 operates independently of processor 120 (e.g., an application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module 190 may be a wireless communication module 192 (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (e.g., : LAN (local area network) communication module, or power line communication module) may be included. Among these communication modules, the corresponding communication module is a first network 198 (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network 199 (e.g., legacy It may communicate with an external electronic device 104 through a telecommunication network such as a cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., LAN or WAN). These various types of communication modules may be integrated into one component (e.g., a single chip) or may be implemented as a plurality of separate components (e.g., multiple chips). The wireless communication module 192 uses subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module 196 within a communication network such as the first network 198 or the second network 199. The electronic device 101 can be confirmed or authenticated.

The wireless communication module 192 may support 5G networks after 4G networks and next-generation communication technologies, for example, NR access technology (new radio access technology). NR access technology provides high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and access to multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low latency). -latency communications)) can be supported. The wireless communication module 192 may support high frequency bands (eg, mmWave bands), for example, to achieve high data rates. The wireless communication module 192 uses various technologies to secure performance in high frequency bands, for example, beamforming, massive array multiple-input and multiple-output (MIMO), and full-dimensional multiplexing. It can support technologies such as input/output (FD-MIMO: full dimensional MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module 192 may support various requirements specified in the electronic device 101, an external electronic device (e.g., electronic device 104), or a network system (e.g., second network 199). According to one embodiment, the wireless communication module 192 supports Peak data rate (e.g., 20 Gbps or more) for realizing eMBB, loss coverage (e.g., 164 dB or less) for realizing mmTC, or U-plane latency (e.g., 164 dB or less) for realizing URLLC. Example: Downlink (DL) and uplink (UL) each of 0.5 ms or less, or round trip 1 ms or less) can be supported.

The antenna module 197 may transmit or receive signals or power to or from the outside (eg, an external electronic device). According to one embodiment, the antenna module 197 may include an antenna including a radiator made of a conductor or a conductive pattern formed on a substrate (eg, PCB). According to one embodiment, the antenna module 197 may include a plurality of antennas (eg, an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network such as the first network 198 or the second network 199 is connected to the plurality of antennas by, for example, the communication module 190. can be selected. Signals or power may be transmitted or received between the communication module 190 and an external electronic device through the at least one selected antenna. According to some embodiments, in addition to the radiator, other components (eg, radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module 197.

According to various embodiments, the antenna module 197 may form a mmWave antenna module. According to one embodiment, a mmWave antenna module includes: a printed circuit board, an RFIC disposed on or adjacent to a first side (e.g., bottom side) of the printed circuit board and capable of supporting a designated high frequency band (e.g., mmWave band); And a plurality of antennas (e.g., array antennas) disposed on or adjacent to the second side (e.g., top or side) of the printed circuit board and capable of transmitting or receiving signals in the designated high frequency band. can do.

At least some of the components are connected to each other through a communication method between peripheral devices (e.g., bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI)) and signal ( (e.g. commands or data) can be exchanged with each other.

According to one embodiment, commands or data may be transmitted or received between the electronic device 101 and the external electronic device 104 through the server 108 connected to the second network 199. Each of the external electronic devices 102 or 104 may be of the same or different type as the electronic device 101. According to one embodiment, all or part of the operations performed in the electronic device 101 may be executed in one or more of the external electronic devices 102, 104, or 108. For example, when the electronic device 101 needs to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device 101 may perform the function or service instead of executing the function or service on its own. Alternatively, or additionally, one or more external electronic devices may be requested to perform at least part of the function or service. One or more external electronic devices that have received the request may execute at least part of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device 101. The electronic device 101 may process the result as is or additionally and provide it as at least part of a response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology can be used. The electronic device 101 may provide an ultra-low latency service using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device 104 may include an Internet of Things (IoT) device. Server 108 may be an intelligent server using machine learning and/or neural networks. According to one embodiment, the external electronic device 104 or server 108 may be included in the second network 199. The electronic device 101 may be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

FIG. 2 is an example diagram for explaining the structure of the electronic device 201 according to an embodiment.

Referring to FIG. 2, in one embodiment, the electronic device 201 (e.g., the electronic device 101 of FIG. 1) includes one or more first cameras 211-1 and 211-2 and one or more second cameras. It may include cameras 212-1 and 212-2, and one or more third cameras 213. In one embodiment, images acquired through one or more first cameras 211-1 and 211-2 may be used for detecting a hand gesture by a user, tracking the user's head, and/or spatial recognition. In one embodiment, one or more first cameras 211-1 and 211-2 may be global shutter (GS) cameras.

In one embodiment, one or more first cameras 211-1 and 211-2 may perform a simultaneous localization and mapping (SLAM) operation through depth photography. In one embodiment, one or more first cameras 211-1 and 211-2 may perform spatial recognition for 6 degrees of freedom (6DoF).

In one embodiment, images acquired through one or more second cameras 212-1 and 212-2 may be used to detect and track the user's pupils. In one embodiment, one or more of the second cameras 212-1 and 212-2 may be GS cameras. In one embodiment, the one or more second cameras 212-1 and 212-2 may correspond to the left and right eyes, respectively, and the performance of the one or more second cameras 212-1 and 212-2 is substantially can be the same.

In one embodiment, one or more third cameras 213 may be high-resolution cameras. In one embodiment, one or more third cameras 213 may perform an auto-focusing (AF) function and a shake correction function. In one embodiment, the one or more third cameras 213 may be GS cameras or rolling shutter (RS) cameras.

In one embodiment, the electronic device 201 may include one or more light emitting elements 214-1 and 214-2. In one embodiment, the light emitting elements 214-1 and 214-2 may be different from the light source described later that irradiates light to the screen output area of the display. In one embodiment, the light emitting elements (214-1, 214-2) perform eye detection when detecting and tracking the user's pupils through one or more second cameras (212-1, 212-2). Light can be irradiated to facilitate this.

In one embodiment, the light emitting elements 214-1 and 214-2 may each include a light emitting diode (LED). In one embodiment, the light emitting elements 214-1 and 214-2 may emit light in the infrared region. In one embodiment, the light emitting elements 214-1 and 214-2 may be attached around the frame of the electronic device 201. In one embodiment, the light-emitting elements 214-1 and 214-2 are located around one or more first cameras 211-1 and 211-2, and when the electronic device 201 is used in a dark environment, one or more Gesture detection, head tracking, and spatial recognition by the first cameras 211-1 and 211-2 can be assisted. In one embodiment, the light-emitting elements 214-1 and 214-2 are located around one or more third cameras 213, and when the electronic device 201 is used in a dark environment, the one or more third cameras 213 ) can assist in image acquisition.

In one embodiment, the electronic device 201 may include batteries 235-1 and 235-2. The batteries 235-1 and 235-2 may store power to operate the remaining components of the electronic device 201.

In one embodiment, the electronic device 201 includes a first display 251, a second display 252, one or more input optical members 253-1 and 253-2, and one or more transparent members 290-1. , 290-2), and one or more screen display portions 254-1 and 254-2.

In one embodiment, the first display 251 and the second display 252 may be, for example, a liquid crystal display (LCD), a digital mirror device (DMD), or a silicon liquid crystal display. The device may include a liquid crystal on silicon (LCoS), an organic light emitting diode (OLED), or a micro light emitting diode (micro LED).

In one embodiment, when the first display 251 and the second display 252 are made of one of a liquid crystal display device, a digital mirror display device, or a silicon liquid crystal display device, the electronic device 201 transmits light to the screen output area of the display. It may include a light source that irradiates. In one embodiment, when the first display 251 and the second display 252 are capable of generating light on their own (for example, when made of one of organic light emitting diodes or micro LEDs), the electronic device 201 is installed separately. Even if it does not include a light source, a virtual image of relatively good quality can be provided to the user.

In one embodiment, one or more transparent members 290-1 and 290-2 may be disposed to face the user's eyes when the user wears the electronic device 201. In one embodiment, the one or more transparent members 290-1 and 290-2 may include at least one of a glass plate, a plastic plate, or a polymer. In one embodiment, when wearing the electronic device 201, the user can view the outside world through one or more transparent members 290-1 and 290-2. In one embodiment, one or more input optical members 253-1 and 253-2 may guide light generated by the first display 251 and the second display 252 to the user's eyes. In one embodiment, a first display 251 and a second display 252 are displayed on one or more screen display portions 254-1 and 254-2 on one or more transparent members 290-1 and 290-2. An image based on the light generated is formed, and the user can see the image formed on one or more screen display portions 254-1 and 254-2.

In one embodiment, the electronic device 201 may include one or more optical waveguides (not shown). The optical waveguide can transmit light generated by the first display 251 and the second display 252 to the user's eyes. The electronic device 201 may include one optical waveguide corresponding to the left eye and the right eye. In one embodiment, the optical waveguide may include at least one of glass, plastic, or polymer. In one embodiment, the optical waveguide may include a nanopattern formed on one of its inner or outer surfaces, for example, a polygonal or curved grating structure. In one embodiment, the optical waveguide may include a free-form prism. In this case, the optical waveguide may provide incident light to the user through a reflection mirror. In one embodiment, the optical waveguide includes at least one of at least one diffractive element (e.g., a diffractive optical element (DOE), a holographic optical element (HOE)) or a reflective element (e.g., a reflective mirror), and is included in the optical waveguide. Display light emitted from a light source may be guided to the user's eyes using at least one diffractive element or reflective element. In one embodiment, the diffractive element may include an input/output optical member. In one embodiment, the reflective element may include a member that causes total reflection.

In one embodiment, the electronic device 201 may include one or more voice input devices 262-1, 262-2, and 262-3 and one or more voice output devices 263-1 and 263-2. there is.

In one embodiment, the electronic device 201 may include a first PCB 270-1 and a second PCB 270-2. The first PCB (270-1) and the second PCB (270-2) include one or more first cameras (211-1, 211-2), one or more second cameras (212-1, 212-2), It may be set to transmit electrical signals to components included in the electronic device 201, such as one or more third cameras 213, displays, audio modules, and sensors. In one embodiment, the first PCB 270-1 and the second PCB 270-2 may include a flexible printed circuit board (FPCB). In one embodiment, the first PCB 270-1 and the second PCB 270-2 may each include a first substrate, a second substrate, and an interposer disposed between the first substrate and the second substrate. You can.

FIG. 3 is an example diagram for explaining the structure of an eye tracking camera of the electronic device 201, according to an embodiment.

Referring to FIG. 3, the electronic device 201 includes an eye tracking (ET) camera 311 (e.g., one or more second cameras 212-1 and 212-2 of FIG. 2), a display 301, and an input. It may include at least one of an optical member 305, a first waveguide 307, an output optical member 309, a first splitter 313, a second waveguide 315, or a second splitter 317. You can.

In one embodiment, the user's pupil 319 is connected to the ET camera 311 through the first splitter 313 (e.g., a splitter for eye tracking), the second waveguide 315, and the second splitter 317. can be photographed. In one embodiment, the ET camera 311 may track the user's gaze by detecting the pupil 319 in the captured image and confirming the movement of the detected pupil 319.

In one embodiment, the image output through the display 301 may be reflected through the input optical member 305 and the first waveguide 307 and displayed through the output optical member 309. In one embodiment, the electronic device 201 outputs an image through the display 301 and simultaneously tracks the user's gaze (e.g., the direction of the user's gaze) by checking the movement of the user's eyes 319 ( Example: identification) can be done.

Figure 4 is a block diagram of an electronic device 401, according to one embodiment.

Referring to FIG. 4 , in one embodiment, the electronic device 401 may be the electronic device 101 of FIG. 1 and/or the electronic device 201 of FIG. 2 .

In one embodiment, the electronic device 401 may include a communication module 410, a display 420, a camera 430, a sensor 440, a memory 450, and/or a processor 460. .

In one embodiment, communication module 410 may be communication module 190 of FIG. 1 .

In one embodiment, the communication module 410 receives an image (e.g., screen image) (or content) being displayed on the external electronic device (e.g., the electronic devices 102 and 104 of FIG. 1). You can receive it. For example, the communication module 410 may receive data (eg, a file) of an image being displayed on the external electronic device from an external electronic device under the control of the processor 460.

In one embodiment, the communication module 410 communicates with the external electronic device the pose of the external electronic device obtained from the external electronic device and/or calculated by the external electronic device between the external electronic device and the electronic device 401. distance can be received.

However, the information that the communication module 410 receives from an external electronic device is not limited to the above-described examples. In addition, although the above-mentioned example illustrates that the communication module 410 receives information from an external electronic device, the communication module 410 receives information corresponding to the information received from the external electronic device under the control of the processor 460. (or similar information) can be transmitted to an external electronic device.

In one embodiment, the display 420 may be the display module 160 of FIG. 1 or the first display 251 and the second display 252 of FIG. 2 .

In one embodiment, the display 420 displays an animation (or animation effects) can be displayed. The animation displayed by the display 420 will be described in detail later.

In one embodiment, the camera 430 includes the camera module 180 of FIG. 1 or the cameras of FIG. 2 (e.g., one or more first cameras 211-1 and 211-2, one or more second cameras) (212-1, 212-2), and one or more third cameras 213).

In one embodiment, the camera 430 may acquire an image of a screen displayed on an external electronic device. The acquired image can be used while performing an image providing operation, which will be described in detail later.

In one embodiment, sensor 440 may be sensor module 176 of FIG. 1 .

In one embodiment, the sensor 440 may include a first sensor 441 and a second sensor 442.

In one embodiment, the first sensor 441 may detect the distance between the electronic device 401 and an external electronic device. For example, the first sensor 441 may be a depth sensor, a time of flight (TOF) sensor (e.g., Laser AF (auto focus)) that can detect the distance between the electronic device 401 and an external electronic device. ) sensor, Lidar, Range sensor), and/or proximity sensor. However, the sensors included in the first sensor 441 are not limited to the above-described example, and may include all sensors capable of detecting the distance between the electronic device 401 and an external electronic device. Additionally, instead of or additionally to the first sensor 441, the distance between the electronic device 401 and an external electronic device may be detected using an image acquired through the camera 430.

In one embodiment, the second sensor 442 determines the posture of the electronic device 401 (e.g., the location of the electronic device 401 and/or the direction the electronic device 401 is facing) and/or the electronic device 401 movement can be detected. For example, the second sensor 442 may include an acceleration sensor and/or a gyro sensor.

In one embodiment, memory 450 may be memory 130 of FIG. 1 .

In one embodiment, the memory 450 may store information for performing an operation to provide an image. Information stored by the memory 450 will be described later.

In one embodiment, processor 460 may be processor 120 of FIG. 1 .

In one embodiment, the processor 460 may control overall operations for providing images. In one embodiment, processor 460 may include one or more processors to provide images. Operations performed by the processor 460 to provide images will be described later with reference to FIGS. 5 to 19.

In Figure 4, the electronic device 401 is illustrated as including a communication module 410, a display 420, a camera 430, a sensor 440, a memory 450, and/or a processor 460. , but is not limited to this. For example, the electronic device 101 further includes at least one component shown in FIG. 1 or 2 (e.g., one or more transparent members 290-1 and 290-2 in FIG. 2). can do.

Although not shown in FIG. 4 , in one embodiment, the electronic device 401 may have a form factor for being worn on the user's head. The electronic device 401 may further include a strap and/or a wearing member for securing onto a user's body part.

The wearable electronic device 401 according to one embodiment includes a display 420 and at least one processor 460, and the at least one processor 460 is configured to obtain a distance to an external electronic device. You can. The at least one processor 460 may be configured to obtain an input for displaying the first image displayed on the external electronic device using the display 420. The at least one processor 460 may be configured to obtain, based on the input, a plurality of second images related to the first image and corresponding to the distance. The at least one processor 460 may be configured to continuously display the number of second images using the display 420.

In one embodiment, the at least one processor 460 determines the number of the plurality of second images to be proportional to the distance based on the distance to the external electronic device, and and determine sizes of the plurality of second images based on the distance.

In one embodiment, the wearable electronic device 401 further includes a depth sensor and/or a proximity sensor, and the at least one processor 460 uses the depth sensor and/or the proximity sensor to detect the external electronic device. It may be configured to obtain the distance to the device.

In one embodiment, the at least one processor 460 detects movement of the external electronic device relative to the movement of the wearable electronic device 401 based on the input, and detects movement of the external electronic device based on the movement of the external electronic device. Thus, it may be configured to set a frame per second (FPS) for displaying the plurality of second images.

In one embodiment, the at least one processor 460 acquires the posture of the wearable electronic device 401 and the posture of the external electronic device, and acquires the posture of the wearable electronic device 401 and the posture of the external electronic device. Based on the posture, the device may be configured to determine positions of the plurality of second images, and set an FPS for displaying the plurality of second images based on the positions of the plurality of second images.

In one embodiment, the at least one processor 460 obtains a first object included in the first image, and obtains a plurality of second objects related to the first object and corresponding to the distance. and may be configured to continuously display the number of second objects using the display 410.

In one embodiment, the at least one processor 460 obtains binocular parallax for the first image, and, based on the obtained binocular parallax for the first image, positions the plurality of second images. It can be configured to determine them.

In one embodiment, the at least one processor 460 may be configured to obtain a user's gesture and determine positions of the plurality of second images based on the direction the user's gesture is facing.

In one embodiment, the at least one processor 460 acquires the posture of the wearable electronic device 401 and the posture of the external electronic device, and acquires the posture of the wearable electronic device 401 and the external electronic device. If the posture satisfies a specified condition, it may be configured to obtain the distance to the external electronic device.

In one embodiment, the at least one processor 460 acquires an image of the external electronic device through the camera 430 of the wearable electronic device 401, and based on the acquired image, Confirm the content displayed on the external electronic device, and based on the input, receive the content from the external electronic device through the communication module 410 of the wearable electronic device 401, and If the received content is the same, it may be configured to obtain the distance to the external electronic device.

FIG. 5 is a flowchart 500 for explaining a method of providing an image, according to an embodiment.

In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel.

According to one embodiment, operations 501 to 507 may be understood as being performed by a processor (e.g., processor 460 of FIG. 4) of an electronic device (e.g., electronic device 401 of FIG. 4).

Referring to FIG. 5 , in operation 501, in one embodiment, the processor 460 may obtain the distance to an external electronic device.

In one embodiment, the processor 460 may obtain the distance to the external electronic device through a first sensor (e.g., the first sensor 441 in FIG. 4). For example, the processor 460 may obtain the distance between the electronic device 401 and an external electronic device through a depth sensor, a TOF sensor, and/or a proximity sensor. However, it is not limited to this. For example, the processor 460 may acquire an image of an external electronic device through the camera 430. The processor 460 determines the size of the portion corresponding to the external electronic device (or display of the external electronic device) in the acquired image and the size of the external electronic device (or By comparing the size of the display of the external electronic device, the distance to the external electronic device can be obtained.

In one embodiment, the processor 460 may perform an operation to obtain the distance to an external electronic device when a specified condition is satisfied. For example, if a specified condition is satisfied, the processor 460 may activate the first sensor 441 and start an operation to obtain the distance to the external electronic device through the activated first sensor 441. . Hereinafter, with reference to FIGS. 6 and 7, an operation for obtaining the distance to an external electronic device performed when a specified condition is satisfied will be described.

FIG. 6 is a diagram illustrating an operation of obtaining a distance to an external electronic device based on the posture of the electronic device 401 and the posture of the external electronic device, according to an embodiment.

Referring to FIG. 6 , in one embodiment, the processor (e.g., processor 460 of FIG. 4 ) connects the external electronic device with the external electronic device when the posture of the electronic device 401 and the posture of the external electronic device satisfy specified conditions. You can start the operation to obtain the distance.

In one embodiment, the processor 460 controls the direction in which the electronic device 401 faces (e.g., the direction in which one or more transparent members 290-1 and 290-2 of FIG. 2 face) and the external electronic device 610. If the direction in which the display (not shown) of the external electronic device 610 faces (e.g., the direction in which the display (not shown) of the external electronic device 610 faces) satisfies a specified condition, an operation to obtain the distance to the external electronic device 610 may begin. For example, at reference numeral 601, the processor 460, through a second sensor (e.g., the second sensor 442 of 4), allows the electronic device 401 to move substantially in the direction indicated by the arrow 621. It can be detected that they are facing in the same direction. The processor 460 receives a signal from the external electronic device 610 through a communication module (e.g., the communication module 410 of FIG. 4), so that the external electronic device 610 receives a signal that is substantially the same as the direction indicated by the arrow 622. You can receive information indicating that you are in a directional state. As indicated by reference numeral 601, the processor 460 operates when the direction in which the electronic device 401 faces and the direction in which the external electronic device 610 faces are not parallel (e.g., the direction in which the electronic device 401 faces and the direction in which the external electronic device 610 faces are not parallel). (If the directions 610 face are not opposite to each other), the operation to obtain the distance to the external electronic device 610 may not be performed. For example, at reference numeral 603, the processor 460 detects, through the second sensor 442, that the electronic device 401 is facing in substantially the same direction as the direction indicated by the arrow 631. can do. The processor 460 receives information from the external electronic device 610 through the communication module 410, indicating that the external electronic device 610 is facing in substantially the same direction as the direction indicated by the arrow 632. can receive. The processor 460 starts an operation to obtain the distance to the external electronic device 610 when the direction in which the electronic device 401 faces and the direction in which the external electronic device 610 faces are opposite to each other, as indicated by reference numeral 602. You can.

In the above-described example, the direction in which the external electronic device 610 faces is described as being received through the communication module 410, but it is not limited thereto. For example, the processor 460 may acquire an image of the external electronic device 610 through a camera (eg, camera 430 in FIG. 4). The processor 460 may obtain the direction in which the external electronic device 610 is facing relative to the direction in which the electronic device 401 is facing, based on the portion corresponding to the external electronic device 610 included in the acquired image. there is. The processor 460 may determine whether to perform an operation to obtain the distance to the external electronic device 610 based on the direction in which the external electronic device 610 faces relative to the direction in which the electronic device 401 faces. .

In the above-mentioned example, whether to perform an operation to obtain the distance to the external electronic device 610 is determined depending on whether the direction toward which the electronic device 401 faces and the direction toward which the external electronic device 610 meets satisfies specified conditions. Although it is described as doing so, it is not limited to this. For example, the processor 460 determines that the direction in which the electronic device 401 is facing and the direction in which the external electronic device 610 is facing satisfy specified conditions, and the external electronic device 610 is wearing the electronic device 401. When located within the viewing range, an operation to obtain the distance to the external electronic device 610 may be started.

FIG. 7 is a diagram 700 illustrating an operation of acquiring a distance to an external electronic device based on an image of the external electronic device, according to an embodiment.

Referring to FIG. 7 , in one embodiment, a processor (eg, processor 460 of FIG. 4 ) may obtain an image of the external electronic device 710 obtained through the camera 430 . The processor 460 may obtain, from the acquired image, a portion 711 corresponding to an image (or screen image, or content) displayed (eg, being displayed) on the external electronic device 710. The processor 460 receives an image (e.g., image data) displayed on the external electronic device 710 from the external electronic device 710 through a communication module (e.g., the communication module 410 of FIG. 4). You can. The processor 460 determines that the portion 711 corresponding to the image displayed on the external electronic device 710 within the image acquired through the camera 430 module and the image received through the communication module 410 are substantially the same. If applicable to content, it can be determined that the specified conditions are satisfied. If a specified condition is satisfied, the processor 460 may start an operation to obtain the distance to the external electronic device 710.

In one embodiment, when a plurality of external electronic devices are located around the electronic device 401, a plurality of external electronic devices may exist within the field of view of the first sensor 441 (eg, TOF sensor). In this case, the external electronic device from which the user wishes to obtain the distance to the electronic device 401 among the plurality of external electronic devices may not be identified (eg, specified). The processor 460 determines that the portion 711 corresponding to the image displayed on the external electronic device 710 within the image acquired through the camera 430 and the image received through the communication module 410 are substantially the same content. By determining whether it corresponds to , the external electronic device 710 for which the distance to the electronic device 401 is to be obtained can be identified among the plurality of external electronic devices. When the external electronic device 710 for distance acquisition is identified, the processor 460 may start an operation to obtain the distance to the identified external electronic device 710. In one embodiment, when the external electronic device 710 is identified, the processor 460 performs an operation to authenticate whether the external electronic device 710 corresponds to the electronic device 401 owned by the user. If authentication is successful, an operation to obtain the distance to the identified external electronic device 710 may be started.

Referring again to FIG. 5, in one embodiment, the processor 460 displays the first image displayed on the external electronic device, which will be described through operation 503, using a display (e.g., the display 420 of FIG. 4). An operation to obtain the distance to an external electronic device may be performed in response to an input for doing so.

In one embodiment, the processor 460 may repeatedly perform the operation of obtaining the distance to the external electronic device in operation 501 while performing operations 503 to 507, which will be described later, or after performing operation 507. there is.

In operation 503, in one embodiment, the processor 460 may obtain an input for displaying the first image displayed on the external electronic device using the display 420.

In one embodiment, the processor 460 processes an image (hereinafter referred to as “first image”) displayed through the display 420 of an external electronic device (e.g., a screen displayed through the display of an external electronic device, an external An input (hereinafter referred to as “image conversion input to the electronic device”) for displaying content (content displayed through the display of the electronic device) using the display 420 of the electronic device 401 can be obtained.

In one embodiment, the image change input to the electronic device may include a user input for displaying the first image displayed on the external electronic device using the display 420. For example, the image conversion input to the electronic device may be a user input on the touch pad of the electronic device 401, a user input using a controller connected to communication with the electronic device 401 (or a user input on an external electronic device) may include. However, it is not limited to this. For example, an image change input to an electronic device may include an input by a designated movement of an external electronic device. Hereinafter, the input by designated movement of an external electronic device as an image conversion input to the electronic device in FIG. 8 will be described.

FIG. 8 is a diagram for explaining an image conversion input to an electronic device, according to an embodiment.

Referring to FIG. 8 , in one embodiment, an image change input to an electronic device may include an input due to movement of an external electronic device.

In one embodiment, as shown at reference numeral 801, the image switching input to the electronic device is determined by the distance between the electronic device 401 and the external electronic device 810 (e.g., the external electronic device 401 relative to the position of the electronic device 401). It may include an input for moving the external electronic device 810 (e.g., an input for moving the external electronic device 810 in the direction indicated by the arrow 821) so that the position of the electronic device 810 becomes closer. . For example, the image switching input to the electronic device causes the distance between the electronic device 401 and the external electronic device 810 to become closer, and the input is caused by the movement of the external electronic device 810 with an acceleration greater than the specified acceleration. may include. The processor (e.g., the processor 460 in FIG. 4) detects the movement of the electronic device 401 by the second sensor (e.g., the second sensor 442 in FIG. 4) and the communication module (e.g., the second sensor 442 in FIG. 4). Based on the movement of the external electronic device 810 received from the external electronic device 810 through the communication module 410, input by the movement of the external electronic device 810 can be obtained as an image conversion input to the electronic device. .

In one embodiment, the image change input to the electronic device 401 may include an input by a user's action of tilting the external electronic device in the first direction by a specified angle or more. The processor 460 provides the external electronic device as an image conversion input to the electronic device based on a user's action of tilting the external electronic device received from the external electronic device through the communication module 410 at a specified angle or more in the first direction. Input can be obtained by a user's motion that tilts more than a specified angle in one direction.

However, image conversion input to the electronic device is not limited to the above-described examples.

In one embodiment, the processor 460 may set an image conversion input to the electronic device based on user input. For example, the processor 460 may set the movement of the electronic device 401 and/or the movement of an external electronic device input by the user as an image conversion input to the electronic device. For example, when the processor 460 closes the distance between the electronic device 401 and an external electronic device and sets the input due to the movement of the external electronic device with an acceleration greater than the specified acceleration as the image conversion input to the electronic device. , the size of the specified acceleration can be set based on user input.

In the above examples, the image conversion input to the electronic device has been described as an input for displaying an image displayed through the display of an external electronic device using the display 420 of the electronic device 401, but the image conversion input to the electronic device is not limited thereto. No. For example, the processor 460 may provide an input for displaying an image displayed using the display 420 of the electronic device 401 through the display of an external electronic device (hereinafter referred to as “image conversion input to an external electronic device”). ") can be obtained.

In one embodiment, the image conversion input to the external electronic device includes a user input on the touch pad of the electronic device 401, a user input using a controller communicated with the electronic device 401 (or a user input on the external electronic device 401). input) may be included.

In one embodiment, as shown at reference numeral 802, the image conversion input to the external electronic device is based on the distance between the electronic device 401 and the external electronic device 810 (e.g., relative to the position of the electronic device 401). An input for moving the external electronic device 810 (e.g., an input for moving the external electronic device 810 in the direction indicated by the arrow 822) so that the location of the external electronic device moves away may be included. For example, an image switching input to an external electronic device may include an input caused by movement of the external electronic device that increases the distance between the electronic device 401 and the external electronic device and has an acceleration greater than a specified acceleration. . The processor 460 transmits data to the external electronic device based on the movement of the electronic device 401 detected by the second sensor 442 and the movement of the external electronic device received from the external electronic device through the communication module 410. As an image conversion input, input through the movement of an external electronic device can be obtained.

In one embodiment, the image change input to the electronic device may include an input by a user's action of tilting the external electronic device in a second direction opposite to the above-described first direction by a specified angle or more. The processor 460 uses the external electronic device as an image conversion input to the external electronic device based on a user's action of tilting the external electronic device received from the external electronic device through the communication module 410 in the second direction by a specified angle or more. An input may be obtained through a user's motion of tilting the device in the second direction by a specified angle or more.

Referring again to FIG. 5 , in one embodiment, as described above, the processor 460 may perform an operation of obtaining the distance to an external electronic device in response to an image change input to the electronic device.

In operation 505, in one embodiment, the processor 460 may acquire a plurality of second images that are related to the first image and correspond to the distance (distance to the external electronic device).

In one embodiment, a plurality of second images (hereinafter referred to as “plurality of second images”) are displayed using the display 420 in response to an image transition input to the electronic device and constitute an animation. These may be images that do. Hereinafter, with reference to FIG. 9, the plurality of second images will be described.

FIG. 9 is a diagram 900 for explaining a plurality of second images, according to an embodiment.

In one embodiment, a plurality of second images may be related to the first image.

In one embodiment, a plurality of second images may be acquired based on data of the first image displayed through an external electronic device when an image conversion is input to the electronic device. Data of the first image may always be transmitted from an external electronic device to the electronic device 401 through the communication module 410 when obtaining an image conversion input to the electronic device.

In one embodiment, the content represented by each of the plurality of second images may be substantially the same as the content represented by the first image. However, it is not limited to this. For example, when the first image displayed through an external electronic device when an image conversion is input to the electronic device is one image frame constituting a video, the plurality of second images are the first image included in the video. The images may be acquired based on data of image frames following the image (e.g., image frames to be sequentially displayed after the first image in the video).

In one embodiment, the plurality of second images create the impression that the first image is coming from an external electronic device (e.g., the location of the external electronic device) to the user's eyes (or a transparent member of the electronic device 401). These may be images that can be provided to the user. For example, as shown in FIG. 9, a plurality of second images are sequentially transmitted from the position of the external electronic device 910 to the left eye of the user wearing the electronic device 401, from image 921 to image ( 923 is approaching, and may be images that can provide the user with the feeling that images 931 to 933 are sequentially approaching the right eye of the user wearing the electronic device 401.

In one embodiment, the plurality of second images may be images representing substantially the same content (eg, a screen including a globe) as the first image, as shown in FIG. 9 . However, it is not limited thereto, and as described above, when the first image displayed through an external electronic device when an image conversion is input to the electronic device is one image frame constituting a video, the plurality of second images are the image frame. These may be images obtained based on data of image frames following the first image included in .

Referring again to FIG. 5, in one embodiment, each of the plurality of second images may be 2D (2D). However, the present invention is not limited thereto, and each of the plurality of second images may be an image.

In one embodiment, when each of the plurality of second images is a 2D image, the processor 460 determines the positions of areas where the plurality of second images will be displayed so that the plurality of second images are visible to the user in 3D form. You can set it. This will be described later with reference to FIG. 16.

In one embodiment, the processor 460 may determine the number of second images based on the distance to the external electronic device. For example, the processor 460 may determine the number of a plurality of second images to correspond to the distance to the external electronic device (e.g., the distance to the external electronic device obtained upon inputting an image conversion to the electronic device). there is. For example, the processor 460 may determine the number of second images to be proportional to the distance to the external electronic device. For example, the processor 460 may determine the number of second images such that it increases as the distance to the external electronic device increases and decreases as the distance to the external electronic device decreases.

In one embodiment, the processor 460 may determine sizes of the plurality of second images and areas in which the plurality of second images will be displayed. Hereinafter, with reference to FIG. 10, a method of determining the sizes of the plurality of second images and the areas in which the plurality of second images will be displayed will be described.

FIG. 10 is a diagram for explaining a method of determining sizes of a plurality of second images and areas in which the plurality of second images will be displayed, according to an embodiment.

Referring to FIG. 10 , in one embodiment, reference numeral 1001 represents an external image visible on a transparent member (e.g., transparent member 290-2 in FIG. 2) through the left eye of a user wearing the electronic device 401. A plurality of second images may be displayed using the electronic device 1002 and a display (eg, the display 420 of FIG. 4).

In one embodiment, a processor (eg, processor 460 of FIG. 4) may determine the sizes of the plurality of second images based on the distance from the external electronic device 1002.

In one embodiment, the processor 460 determines the size (or resolution) of a portion corresponding to the first image in an image acquired through a camera (e.g., the camera 430 of FIG. 4) and a plurality of second images. The sizes of the plurality of second images may be determined based on the size of the second image to be displayed last.

In one embodiment, the processor 460 may acquire an image of the external electronic device 1002 through the camera 430. The processor 460 may check the size of the portion corresponding to the first image displayed through the external electronic device 1002 (hereinafter referred to as “portion corresponding to the first image”) within the acquired image. . The processor 460 sets the size of the portion corresponding to the first image to the size of the second image 1011 (hereinafter referred to as the “2-1 image”) to be displayed first among the plurality of second images. You can decide. However, it is not limited to this, and the size obtained by adjusting the size of the portion corresponding to the first image to a specified ratio (or specified size) may be determined as the size of the 2-1 image. However, the method of determining the size of the 2-1 image is not limited to the above-described example.

In one embodiment, the processor 460 determines the size (or resolution) of the area 1013 for display on the transparent member (e.g., the screen display portion 254-2 of FIG. 2) and the communication module (e.g., FIG. Based on the size of the first image received from the external electronic device 1002 through the communication module 410 of 4), the second image 1012 to be displayed last among the plurality of second images (hereinafter referred to as “the first image 1012”) The size of the image (referred to as “2-2 image”) can be determined. In one embodiment, the processor 460 is set to display the first image received from the external electronic device 1002 through the communication module 410 after the display of the animation including the plurality of second images ends. The size can be determined as the size of the 2-2 image. However, the method of determining the size of the 2-2 image is not limited to the above-described example.

In one embodiment, the processor 460 may determine the sizes of the remaining second images among the plurality of second images based on the size of the 2-1 image and the size of the 2-2 image. For example, as shown in reference numeral 1001, the processor 460 moves from the 2-1 image to the 2-2 image when the size of the 2-1 image is smaller than the size of the 2-2 image. In order, the sizes of the remaining second images may be determined so that the size of the second image increases at a specified rate (or by a specified size). If the size of the 2-1 image is larger than the size of the 2-2 image, the processor 460 displays the second image at a specified ratio (or by a specified size) in order from the 2-1 image to the 2-2 image. The sizes of the remaining second images may be determined so that the size of the image is reduced. If the size of the 2-1 image is the same as the size of the 2-2 image, the processor 460 determines the sizes of the remaining second images as the size of the 2-1 image and the size of the 2-2 image. You can.

In one embodiment, the processor 460 may determine areas (or positions) where the plurality of second images will be displayed based on the area (or position) of the external electronic device 1002 visible on the transparent member. there is.

In one embodiment, the processor 460 creates a display on the transparent member based on the area (or position) of the external electronic device 1002 visible on the transparent member and the area (or position) of the 2-2 image. Areas (or positions) where a plurality of second images will be displayed within the area 1013 may be determined.

In one embodiment, the processor 460 may determine the area of the external electronic device 1002 visible on the transparent member as the area where the 2-1 image will be displayed on the transparent member. However, the method of determining the area of the 2-1 image is not limited to the above-described example.

In one embodiment, the processor 460 may determine a designated area of the area for display 1013 on the transparent member as an area where the 2-2 image will be displayed. In one embodiment, the processor 460 receives the first image from the external electronic device through the communication module 410 after the display of the animation including the plurality of second images ends on the display area 1013. The area set for displaying the image can be determined as the area where the 2-2 image will be displayed. However, the method of determining the area of the 2-2 image is not limited to the above-described example.

In one embodiment, the processor 460 selects the remaining second images among the plurality of second images based on the area (or position) of the 2-1 image and the area (or position) of the 2-2 image. Areas can be determined. For example, as shown in reference numeral 1002, the processor 460, when the position of the 2-1 image 1021 and the position of the 2-2 image 1024 are determined, the 2-1 image ( The positions moved at a specified interval from the position of 1021) to the position of the 2-2 image 1024 can be determined as the positions of the remaining second images (eg, images 1022 and 1023). However, the method of determining the areas of the remaining second images among the plurality of second images is not limited to the above-described example.

10 , for convenience of explanation, a plurality of second images viewed through the left eye (or transparent member 290-2) of the user wearing the electronic device 401 are shown and described, but the electronic device 401 The above-described examples may be applied in the same or similar manner to a plurality of second images viewed through the right eye (or transparent member 290-1) of the user wearing the .

In FIG. 10, the line 1014 connecting the positions of the plurality of second images is illustrated as a straight line, but is not limited thereto. For example, the processor 460 may operate according to the posture of the electronic device 401 (e.g., the position and direction of the electronic device 401) and the posture of the external electronic device (e.g., the position and direction of the external electronic device). The line connecting the positions of the plurality of second images may not be a straight line. Hereinafter, with reference to FIG. 11, a method of determining the positions of a plurality of second images will be described.

FIG. 11 is a diagram illustrating a method of determining the positions of a plurality of second images based on the posture of the electronic device 401 and the posture of an external electronic device, according to an embodiment.

Referring to FIG. 11 , in one embodiment, reference numeral 1101 represents an example of displaying a plurality of second images while the direction of the electronic device 401 and the direction of the external electronic device 1110 are substantially parallel. It can be expressed.

In one embodiment, when the direction of the electronic device 401 and the direction of the external electronic device 1110 are substantially parallel, the processor (e.g., processor 460 in FIG. 4), as shown by reference numeral 1101, , the images experienced by the user (e.g., image 1111 (and image 1112) to image 1113 (and image 1114)) appear to move in the straight direction indicated by the arrow 1115. For example, the positions of the plurality of second images may be determined so that the positions of the plurality of second images (eg, a trace of the positions of the plurality of second images) form a straight line.

In one embodiment, when the direction of the electronic device 401 and the direction of the external electronic device 1110 are not parallel, the processor 460, as shown by reference numeral 1102, displays images experienced by the user (e.g. : so that the images 1121 (and images 1122) to images 1123 (and images 1124) appear to move (and rotate) in the curved direction indicated by the arrow 1125, The positions (and rotation direction) of the plurality of second images may be determined such that the positions of the two images form a curve.

Referring again to FIG. 5 , at operation 507, in one embodiment, the processor 460, using the display 420, displays a plurality of second images of the number (the number of second plurality images determined in operation 505). Images can be displayed sequentially.

In one embodiment, the processor 460 uses the display 420, as shown in FIG. 10, to display a plurality of images including the 2-1st image 1011 to the 2-2nd image 1012. 2 Images can be displayed sequentially. In one embodiment, the processor 460 may use the display 420 to sequentially display a plurality of second images such that each of the plurality of second images disappears after being displayed.

In one embodiment, the processor 460 sets a frame per second (FPS) (hereinafter referred to as “FPS of the plurality of second images”) for displaying the plurality of second images using the display 420. (or can be adjusted).

In one embodiment, the FPS of the plurality of second images may be the number of second images displayed per second. For example, when the FPS of the plurality of second images is 50, 50 second images may be displayed per second from the total number of the plurality of second images. In one embodiment, the operation of setting the FPS of the plurality of second images includes a time interval for displaying each of the plurality of second images (e.g., a plurality of image frames) (e.g., the 2-1 image in FIG. 10). (1021) and the time interval for displaying the image (1022) following the 2-1 image (1021). For example, when the FPS of the plurality of second images is set to the first FPS, the processor 460 sets the time interval for displaying each of the determined number of second images through operation 505 to the first FPS. It can be set at time intervals. When the FPS of the plurality of second images is set to a second FPS that is higher than the first FPS, the processor 460 determines a time interval for displaying each of the determined number of second images through operation 505. The second time interval can be set to be shorter than the 1 hour interval.

In one embodiment, the lower the FPS of the plurality of second images is set, the slower the plurality of second images appear to the user to move on the transparent member, and the higher the FPS of the plurality of second images is set, the more slowly the plurality of second images appear to the user. It may appear to the user to be moving quickly on a transparent member.

In one embodiment, the processor 460, among the plurality of second images, includes a specified number of second images following the 2-1 image, including the 2-1 image, and a 2-2 image. A designated number of second images before the 2-2 image may be displayed at the first FPS, and the remaining second images may be displayed at a second FPS that is higher than the first FPS. Through this, among the plurality of second images, the second images displayed at the beginning of the animation and the second images displayed at the end of the animation appear to move slowly, and the second images displayed at the middle of the animation are seen as moving slowly. 2 Images can be made to appear to be moving quickly.

In one embodiment, when the positions of the plurality of second images (e.g., traces of the positions of the plurality of second images) form a straight line, as indicated by reference numeral 1101, the processor 460: When the positions of the plurality of second images form a curve, as indicated by reference numeral 1102, the FPS may be set higher than the set FPS.

Although not shown in FIG. 5, in one embodiment, the processor 460 changes the first image that was displayed on the external electronic device to the external electronic device while the plurality of second images are displayed or in response to an image switching input to the electronic device. External electronic devices can be controlled so that they are not displayed on the electronic device. However, the present invention is not limited thereto, and for example, the processor 460 may control the external electronic device to continuously display the first image while the plurality of second images are displayed. For example, if the first image is an image, the processor 460 may display the image at the time that the external electronic device continuously reproduces the image while the plurality of second images are displayed or when an image change input to the electronic device is input. An external electronic device can be controlled to display a paused image.

5 to 11 illustrate that a plurality of second images are related to a first image displayed through an external electronic device (eg, a screen image displayed through an external electronic device), but the present invention is not limited thereto. For example, the plurality of second images may be related to an object detected in the first image. When a plurality of second images are related to an object detected in the first image, a method of providing the images will be described below with reference to FIGS. 12 and 13.

FIG. 12 is a flowchart 1200 for explaining a method of providing an image, according to an embodiment.

FIG. 13 is a diagram 1300 for explaining a method of providing an image, according to an embodiment.

In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel.

According to one embodiment, operations 1201 to 1205 may be understood as being performed by a processor (e.g., processor 460 of FIG. 4) of an electronic device (e.g., electronic device 401 of FIG. 4).

Referring to FIGS. 12 and 13 , in one embodiment, operations 1201 and 1203 are operations included in operation 505 of FIG. 5 (or performed in addition to operation 505), and operation 1205 is operation 507 of FIG. 5. It may be an operation that is included (or replaces operation 507).

In operation 1201, in one embodiment, the processor 460 may obtain a first object included in the first image.

In one embodiment, the first object may be a character included in the first image. In one embodiment, the first object may be an object selected by user input from among one or more objects included in the first image.

In one embodiment, an external electronic device may obtain a user input for an area containing the first object within the first image. For example, as shown in FIG. 13, the external electronic device 1310 obtains a user input for an area containing the first object 1321 (e.g., a dog character) within the first image 1320. can do.

In one embodiment, the external electronic device 1310 may check the coordinates at which the user input was obtained within the first image. The external electronic device 1310 may detect the first object (eg, an area including the first object) within the first image based on the confirmed coordinates. The processor 460 receives the first object detected in the external electronic device 1310 from the external electronic device 1310 through a communication module (e.g., the communication module 410 of FIG. 4), thereby detecting the first object. can be obtained. However, the method of acquiring the first object is not limited to the above-described example.

In operation 1203, in one embodiment, the processor 460 may obtain a plurality of second objects that are related to the first object and correspond to the distance (distance to the external electronic device).

In operation 1203, a plurality of second objects (e.g., in response to an image change input to the electronic device) associated with the first object and corresponding to the distance from the external electronic device to the display (e.g., the display 420 of FIG. 4 The operation of acquiring the objects displayed using )) and constituting the animation includes acquiring a plurality of second images related to the first image in operation 505 of FIG. 5 and corresponding to the distance to the external electronic device. Since at least some of the operations are the same or similar, detailed description will be omitted.

At operation 1205, in one embodiment, the processor 460 may use the display 420 to sequentially display the number (the number of the plurality of second objects determined in operation 1203) of a plurality of second objects. there is.

Since at least part of the operation of operation 1205 is the same or similar to operation 507, redundant description will be omitted.

In one embodiment, the processor 460, as shown in FIG. 13, configures the display 420 so that objects 1331 to 1332 appear to move sequentially, as shown in FIG. 10. Using this, a plurality of second objects can be displayed sequentially. In FIG. 13 , for convenience of explanation, objects 1331 to 1332 experienced through the right eye (or transparent member (e.g., transparent member 290-1 of FIG. 2)) of the user wearing the electronic device 401. ) is shown, but the processor 460 senses objects similar to the objects 1331 and 1332 through the user's left eye (or a transparent member (e.g., transparent member 290-2 in FIG. 2)). Preferably, a plurality of second objects can be sequentially displayed using the display 420.

In one embodiment, the processor 460, based on the movement of the first object, sets an FPS for displaying a plurality of second objects using the display 420 (hereinafter referred to as “FPS of the plurality of second objects”). can be set (or adjusted).

In one embodiment, the processor 460 may determine (eg, calculate) the speed at which the first object moves within the first image. For example, when the coordinates of the first object change within the first image (e.g. video), the processor 460 determines the speed of the first object based on the change rate of the coordinates of the first object (e.g. : Calculation) can be done. For example, when only the background is changed without changing the position of the first object corresponding to the car in the first image (e.g., video), such as in a car game, the processor 460 determines the change speed of the background. can be determined as the speed of the first object.

In one embodiment, the processor 460 may set (eg, adjust) the FPS of a plurality of second objects based on the speed of the first object. For example, the processor 460 may set the FPS of the plurality of second objects to be high as the speed of the first object is fast, and set the FPS of the plurality of second objects to be low as the speed of the first object is slow. .

FIG. 14 is a flowchart 1400 for explaining a method of providing an image, according to an embodiment.

FIG. 15 is a diagram 1500 for explaining a method of providing an image, according to an embodiment.

In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel.

According to one embodiment, operations 1401 to 1411 may be understood as being performed by a processor (e.g., processor 460 of FIG. 4) of an electronic device (e.g., electronic device 401 of FIG. 4).

Referring to FIGS. 14 and 15 , in operation 1401, in one embodiment, the processor 460 may obtain the distance to an external electronic device.

In operation 1403, in one embodiment, the processor 460 may obtain an input for displaying the first image displayed on the external electronic device using a display (e.g., the display 420 of FIG. 4).

Since operations 1401 and 1403 are at least partially the same or similar to operations 501 and 503, detailed descriptions thereof will be omitted.

At operation 1405, in one embodiment, the processor 460 may detect movement of the external electronic device relative to the movement of the electronic device 401.

In one embodiment, the processor 460 may detect the movement (and posture) of the electronic device 401 through a second sensor (e.g., the second sensor 442 in FIG. 4).

In one embodiment, the processor 460 may receive, from an external electronic device, the movement (and posture) of the external electronic device detected by the external electronic device. However, it is not limited thereto, and for example, the processor 460 may process an image of an external electronic device acquired through a camera (e.g., the camera 430 of FIG. 4) (e.g., acquired through the camera 430). Based on the change in size of the part corresponding to the external electronic device within the image, the movement of the external electronic device can be detected.

In one embodiment, the processor 460 may detect movement of the external electronic device relative to the movement of the electronic device 401 based on obtaining an image switching input to the electronic device. For example, the processor 460 may detect the movement of the external electronic device relative to the movement of the electronic device 401 at the time of obtaining the image change input to the electronic device. However, the present invention is not limited to this, and the movement of the external electronic device relative to the movement of the electronic device 401 may be detected before or while performing operation 1401.

In operation 1407, in one embodiment, the processor 460 may acquire a plurality of second images that are related to the first image and correspond to the distance (distance to the external electronic device).

Since operation 1407 is at least partially the same or similar to operation 505 of FIG. 5, a detailed description thereof will be omitted.

In operation 1409, in one embodiment, the processor 460 generates a plurality of second images based on the movement (e.g., movement of the external electronic device relative to the movement of the electronic device 401 detected through operation 1405). You can set (e.g. adjust) their FPS.

In one embodiment, the processor 460 operates at a specified FPS (e.g. : A higher FPS (FPS set when the size of the movement of the external electronic device relative to the movement of the electronic device 401 is less than or equal to a specified size) may be set as the FPS of the plurality of second images. For example, as shown in FIG. 15, the processor 460 moves the external electronic device 1510 in a direction (e.g., the direction indicated by the arrow) toward the electronic device 401 without moving the electronic device 401. When moving to (1521)), the user selects images (e.g., images 1531 to 1532 and images 1541 to 1542) corresponding to a plurality of second images related to the first image 1510. To make the images appear faster to the user's eyes, the FPS of the plurality of second images may be set to a higher FPS than the designated FPS.

In operation 1411, in one embodiment, the processor 460 may continuously display the number of second images through the display 420 based on the FPS of the set plurality of second images. there is.

FIG. 16 is a diagram 1600 for explaining a method of providing an image based on binocular disparity, according to an embodiment.

Referring to FIG. 16, in one embodiment, a processor (e.g., processor 460 of FIG. 4) generates a plurality of second images in 2D based on binocular parallax for the first image 1620. Even in the case of images, a plurality of second images can be displayed in 3D form. For example, the processor 460 may determine the positions of the plurality of second images in 2D so that the plurality of second images in 3D are displayed in 3D form, based on binocular parallax with respect to the first image 1620.

In one embodiment, the processor 460 includes a camera (e.g., camera 430 of FIG. 4) (e.g., eye tracking (ET) camera 311 of FIG. 3, one or more second cameras 212 of FIG. 2). Using -1, 212-2)), the positions of both eyes of the user can be obtained. The processor 460 may obtain (eg, calculate) binocular disparity for the first image 1620 using a stereoscopic method, based on the acquired positions of both eyes of the user. The processor 460 may determine the positions of the plurality of 2D second images so that they are displayed in 3D form, based on binocular parallax with respect to the first image 1620. For example, the processor 460 may display the second image 1631 to be viewed by the left eye at a position to the right of the position on the display area 1611 corresponding to the position of the user's left eye. -2 The position of the image 1631 can be determined. The processor 460 displays the 2-2 image 1632 to be viewed by the right eye at a position to the left of the position on the display area 1612 corresponding to the position of the user's right eye. 1632) can be determined. The processor 460 determines the location of the 2-1 image to be viewed by the left eye, the location of the 2-1 image to be viewed by the right eye, the location of the 2-2 image 1631, and the location of the 2-2 image 1632. Based on the location, the locations of the remaining second images, which will be viewed by the left and right eyes, can be determined.

In one embodiment, the processor 460 may determine the positions of the plurality of second images further based on the distance to the external electronic device. For example, the closer the distance to the external electronic device 1601 is, the more the processor 460 displays the 2-2 image 1631 to be shown to the left eye and the 2-2 image 1631 to be shown to the right eye. The positions of the 2-2 image 1631 and the 2-2 image 1632 can be determined so that the distance between the images 1632 becomes closer. Through this, the processor 460 can display the plurality of 2D second images in a more 3D form as the distance to the external electronic device 1601 becomes closer.

FIG. 17 is a diagram 1700 for explaining a method of providing an image, according to an embodiment.

Referring to FIG. 17, in FIGS. 4 to 16 described above, a plurality of images are used to provide the user with the feeling that the first image is approaching from the location of the external electronic device to the user's eyes (or the transparent member of the electronic device 401). Although the method for determining the positions of the second images has been described, the method is not limited thereto.

In one embodiment, the processor (e.g., processor 460 of FIG. 4) may be configured to provide the user with the impression that the first image is moving in various directions, such as upward, downward, left, or right. The positions of the second images of can be determined.

In one embodiment, the processor 460 is configured to provide the user with a feeling that the first image is moving in the direction (and speed) of the user's gesture, based on the direction (and speed) of the user's gesture. The positions of the second images of can be determined. For example, as shown in FIG. 17, the processor 460 determines the direction in which the user's hand 1720 moves based on an image acquired through a camera (e.g., the camera 430 in FIG. 4). : The direction indicated by the arrow 1731 (and speed) can be detected. The processor 460 displays the first image 1711 in the direction (and speed) toward which the user's gesture is headed, such as the images (1712, 1713, 1714, and 1715) that are felt as moving in the direction (and speed) toward which the user's gesture is headed. and speed), the positions of the plurality of second images may be determined to provide the user with the feeling of moving.

In one embodiment, the processor 460 may determine the positions of the plurality of second images to provide the user with the feeling that the first image is moving to a specific point in space in the real world. For example, the processor 460 provides the user with the feeling that the first image is moving from the location of the electronic device 401 to a specific point located further than the location of the external electronic device 1701 in the space of the real world. To provide this, positions of the plurality of second images may be determined. In this case, the processor 460 may set the FPS of the positions of the plurality of second images to be lower than the designated FPS. For example, the processor 460 provides the user with the feeling that the first image moves from the location of the electronic device 401 to a specific point located closer to the location of the external electronic device in the space of the real world, The positions of the plurality of second images may be determined. In this case, the processor 460 may set the FPS of the positions of the plurality of second images to be higher than the designated FPS.

FIG. 18 is a diagram for explaining a method of providing an image, according to an embodiment.

Referring to FIG. 18, in one embodiment, the processor (e.g., processor 460 of FIG. 4) includes a plurality of processors to provide the user with the feeling that the first image (or first object) is moving to the location of a specific object. The positions of the second images of can be determined. For example, the processor 460 uses the display 420 to provide the user with the feeling that the first object moves to a specific object 1820, as shown in reference numerals 1801 and 1802, A plurality of second objects (1841-1, 1841-2, 1841-3, 1841-4, 1841-5, 1841-6, 1841-7, 1842) can be displayed.

In one embodiment, the processor 460 sets the FPS of the plurality of second objects to be lower as the distance between the specific object 1820 and the external electronic device 1810 increases, and sets the FPS of the plurality of second objects to be lower as the distance between the specific object 1820 and the external electronic device 1810 increases. 1810) The closer the distance between objects is, the higher the FPS of the plurality of second objects can be set. For example, the distance between the specific object 1820 and the external electronic device 1810 shown at 1801 may be greater than the distance between the specific object 1820 and the external electronic device 1810 shown at 1802. In this case, the processor 460 sets the FPS of the plurality of second objects to the first FPS at reference numeral 1801, and sets the FPS of the plurality of second objects to the second FPS higher than the first FPS at reference numeral 1802. You can. Through this, the processor 460 causes the first object to reach the specific object 1820 more slowly as the distance between the specific object 1820 and the external electronic device 1810 increases, and the specific object 1820 and the external electronic device 1810 ( The closer the distance between objects (1810), the faster the first object can reach the specific object (1820).

In one embodiment, the processor 460 may determine the size of the feature object 1820 based on the distance from the external electronic device 1810. For example, the processor 460 determines the size of the feature object 1820 to be smaller as the distance to the external electronic device 1810 increases, and determines the size of the feature object 1820 to be larger as the distance to the external electronic device is smaller. You can decide.

FIG. 19 is a diagram 1900 for explaining a method of providing an image, according to an embodiment.

Referring to FIG. 19, in one embodiment, a processor (e.g., processor 460 of FIG. 4) displays information related to information displayed on the external electronic device 1910 (e.g., display 420 of FIG. 4). It can be displayed using . For example, as shown in FIG. 19, when the external electronic device 1910 displays information 1920 including text, the processor 460 displays information 1930 related to the information 1920. It can be displayed using (420). The processor 460 selects the text 1921 included in the information 1920 while the information 1920 is displayed on the external electronic device 1910 and obtains user input to retrieve information related to the text 1921. can do. Based on the acquired user input, the processor 460 generates information 1930 related to the text 1921 retrieved by the search if at least part of the information 1930 overlaps with the location of the external electronic device 1910 or ) The information 1930 can be displayed using the display 420 so that it appears to be displayed in a position adjacent to ). In one embodiment, the processor 460 may determine the size of the information 1930 (e.g., an area on the transparent member where the information 1930 is displayed) based on the distance from the external electronic device 1910. For example, the processor 460 determines the size of the information 1930 to be small as the distance to the external electronic device 1910 increases, and determines the size of the information 1930 to be large as the distance to the external electronic device 1910 is small. You can decide.

In one embodiment, the processor 460 may transmit the information 1930 from the external electronic device 1910, just as the first image described in the examples above approaches the electronic device 401 from the external electronic device 1910. An operation may be performed to provide the user with a feeling of approaching the electronic device 401.

4 to 19 , examples performed based on image conversion input to the electronic device have been described, but the present invention is not limited thereto. Examples of operations performed based on an image change input to an external electronic device can be similarly applied to operations performed based on an image change input to an electronic device. For example, the processor 460 performs an operation to provide the user with the feeling that the first image is approaching from the external electronic device to the electronic device 401 based on the image switching input to the electronic device, and the external electronic device Based on the image change input to the electronic device 401, an operation may be performed to provide the user with the feeling that the image displayed on the electronic device 401 is moving from the external electronic device to the electronic device 401.

In one embodiment, a method of providing an image from the wearable electronic device 401 may include obtaining a distance to an external electronic device. The method may include obtaining an input for displaying the first image displayed on the external electronic device using the display 420 of the wearable electronic device. The method may include obtaining, based on the input, a plurality of second images related to the first image and corresponding to the distance. The method may include continuously displaying the plurality of second images using the display 420 .

In one embodiment, the operation of acquiring a plurality of second images related to the first image and corresponding to the distance includes, based on the distance to the external electronic device, the plurality of second images in proportion to the distance. may include determining the number of second images and determining sizes of the plurality of second images based on the distance to the external electronic device.

In one embodiment, the operation of acquiring the distance to the external electronic device includes obtaining the distance to the external electronic device using a depth sensor and/or a proximity sensor of the wearable electronic device 401. It can be included.

In one embodiment, the operation of acquiring a plurality of second images related to the first image and corresponding to the distance includes determining the size of the plurality of second images based on the distance to the external electronic device. It may include actions to determine the

In one embodiment, the method includes detecting movement of the external electronic device relative to movement of the wearable electronic device 401 based on the input and detecting movement of the external electronic device based on the movement of the external electronic device. 2 The operation of setting the FPS for displaying images may be further included.

In one embodiment, the method includes an operation of acquiring the posture of the wearable electronic device 401 and the posture of the external electronic device, based on the posture of the wearable electronic device 401 and the posture of the external electronic device, The method may further include determining positions of the plurality of second images, and setting an FPS for displaying the plurality of second images based on the positions of the plurality of second images.

In one embodiment, the method includes obtaining a first object included in the first image, obtaining a plurality of second objects related to the first object and corresponding to the distance, and An operation of continuously displaying the plurality of second objects using the display 410 may be further included.

In one embodiment, the method further includes obtaining binocular parallax for the first image and determining positions of the plurality of second images based on the obtained binocular parallax for the first image. It can be included.

In one embodiment, the method may further include obtaining a user's gesture and determining positions of the plurality of second images based on the direction in which the user's gesture is directed.

In one embodiment, the operation of acquiring the distance to the external electronic device includes obtaining the posture of the wearable electronic device 401 and the posture of the external electronic device, and the posture of the wearable electronic device 401 and the posture of the external electronic device. When the posture of the external electronic device satisfies a specified condition, the operation may include obtaining the distance to the external electronic device.

In one embodiment, the operation of acquiring the distance to the external electronic device includes obtaining an image of the external electronic device through the camera 430 of the wearable electronic device 401, and adding Based on the input, an operation of checking content displayed on the external electronic device, an operation of receiving the content from the external electronic device through the communication module 410 of the wearable electronic device 401 based on the input, and If the confirmed content and the received content are the same, the operation may include obtaining the distance to the external electronic device.

An electronic device according to an embodiment disclosed in this document may be of various types. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to embodiments of this document are not limited to the above-described devices.

An embodiment of this document and the terms used herein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various changes, equivalents, or substitutes for the embodiment. In connection with the description of the drawings, similar reference numbers may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the above items, unless the relevant context clearly indicates otherwise. As used herein, “A or B”, “at least one of A and B”, “at least one of A or B”, “A, B or C”, “at least one of A, B and C”, and “A Each of phrases such as “at least one of , B, or C” may include any one of the items listed together in the corresponding phrase, or any possible combination thereof. Terms such as "first", "second", or "first" or "second" may be used simply to distinguish one component from another, and to refer to that component in other respects (e.g., importance or order) is not limited. One (e.g., first) component is said to be “coupled” or “connected” to another (e.g., second) component, with or without the terms “functionally” or “communicatively.” When mentioned, it means that any of the components can be connected to the other components directly (e.g. wired), wirelessly, or through a third component.

The term “module” used in one embodiment of this document may include a unit implemented in hardware, software, or firmware, and may be interchangeable with terms such as logic, logic block, component, or circuit, for example. can be used A module may be an integrated part or a minimum unit of the parts or a part thereof that performs one or more functions. For example, according to one embodiment, the module may be implemented in the form of an application-specific integrated circuit (ASIC).

One embodiment of the present document is one or more instructions stored in a storage medium (e.g., built-in memory 136 or external memory 138) that can be read by a machine (e.g., electronic device 101). It may be implemented as software (e.g., program 140) including these. For example, a processor (e.g., processor 120) of a device (e.g., electronic device 101) may call at least one command among one or more commands stored from a storage medium and execute it. This allows the device to be operated to perform at least one function according to the at least one instruction called. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. A storage medium that can be read by a device may be provided in the form of a non-transitory storage medium. Here, “non-transitory” simply means that the storage medium is a tangible device and does not contain signals (e.g. electromagnetic waves), and this term refers to cases where data is semi-permanently stored in the storage medium. There is no distinction between temporary storage cases.

According to one embodiment, a method according to an embodiment disclosed in this document may be provided and included in a computer program product. Computer program products are commodities and can be traded between sellers and buyers. The computer program product may be distributed in the form of a machine-readable storage medium (e.g. compact disc read only memory (CD-ROM)) or via an application store (e.g. Play Store ^TM ) or on two user devices (e.g. It can be distributed (e.g. downloaded or uploaded) directly between smart phones) or online. In the case of online distribution, at least a portion of the computer program product may be at least temporarily stored or temporarily created in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

According to one embodiment, each component (e.g., module or program) of the above-described components may include a single or multiple entities, and some of the multiple entities may be separately placed in other components. . According to one embodiment, one or more of the above-described corresponding components or operations may be omitted, or one or more other components or operations may be added. Alternatively or additionally, multiple components (eg, modules or programs) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each component of the plurality of components in the same or similar manner as those performed by the corresponding component of the plurality of components prior to the integration. . According to one embodiment, operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or , or one or more other operations may be added.

Additionally, the data structure used in the above-described embodiments of this document can be recorded on a computer-readable recording medium through various means. The computer-readable recording media includes storage media such as magnetic storage media (eg, ROM, floppy disk, hard disk, etc.) and optical read media (eg, CD-ROM, DVD, etc.).

Claims (20)

In the wearable electronic device 401,
display 410; and
At least one processor (460) operatively connected to the display (410),
The at least one processor 460,
Obtaining the distance from an external electronic device,
Obtaining an input for displaying a first image displayed on the external electronic device using the display 410,
Based on the input, obtain a plurality of second images related to the first image and corresponding to the distance, and
A wearable electronic device (401) configured to continuously display the plurality of second images using the display (410).
According to claim 1,
The at least one processor 460,
Based on the distance from the external electronic device, determine the number of the plurality of second images to be proportional to the distance, and
A wearable electronic device (401) configured to determine sizes of the plurality of second images based on the distance from the external electronic device.
The method of claim 1 or 2,
Further comprising a depth sensor and/or a proximity sensor,
The at least one processor 460,
A wearable electronic device 401 configured to obtain the distance to the external electronic device using the depth sensor and/or proximity sensor.
The method according to any one of claims 1 to 3,
The at least one processor 460,
Detecting movement of the external electronic device relative to movement of the wearable electronic device 401 based on the input, and
A wearable electronic device 401 configured to set a frame per second (FPS) for displaying the plurality of second images based on the movement of the external electronic device.
The method according to any one of claims 1 to 4,
The at least one processor 460,
Obtaining the posture of the wearable electronic device 401 and the posture of the external electronic device,
Based on the posture of the wearable electronic device 401 and the posture of the external electronic device, determine the positions of the plurality of second images, and
A wearable electronic device (401) configured to set an FPS for displaying the plurality of second images based on the positions of the plurality of second images.
The method according to any one of claims 1 to 5,
The at least one processor 460,
Obtaining a first object included in the first image,
Obtaining a plurality of second objects associated with the first object and corresponding to the distance, and
A wearable electronic device (401) configured to continuously display the plurality of second objects using the display (410).
The method according to any one of claims 1 to 6,
The at least one processor 460,
Obtain binocular disparity for the first image, and
A wearable electronic device 401 configured to determine positions of the plurality of second images based on binocular parallax with respect to the acquired first image.
The method according to any one of claims 1 to 7,
The at least one processor 460,
acquire the user's gestures, and
A wearable electronic device (401) configured to determine positions of the plurality of second images based on the direction in which the user's gesture is directed.
The method according to any one of claims 1 to 8,
The at least one processor 460,
Obtaining the posture of the wearable electronic device 401 and the posture of the external electronic device, and
A wearable electronic device (401) configured to obtain a distance to the external electronic device when the posture of the wearable electronic device (401) and the posture of the external electronic device satisfy specified conditions.
The method according to any one of claims 1 to 9,
The at least one processor 460,
Obtaining an image of the external electronic device through the camera 430 of the wearable electronic device 401,
Based on the acquired image, confirm the content displayed on the external electronic device,
Based on the input, receive the content from the external electronic device through the communication module 410 of the wearable electronic device 401, and
A wearable electronic device 401 configured to obtain a distance from the external electronic device when the confirmed content and the received content are the same.
In a method of providing an image from a wearable electronic device 401,
Obtaining a distance from an external electronic device;
Obtaining an input for displaying a first image displayed on the external electronic device using the display 410 of the wearable electronic device 401;
Based on the input, obtaining a plurality of second images related to the first image and corresponding to the distance; and
A method comprising sequentially displaying the plurality of second images using the display 410.
According to claim 11,
The operation of acquiring a plurality of second images related to the first image and corresponding to the distance includes:
Based on the distance to the external electronic device, determining the number of the plurality of second images to be proportional to the distance; and
A method comprising determining sizes of the plurality of second images based on the distance from the external electronic device.
The method of claim 11 or 12,
The operation of acquiring the distance to the external electronic device is:
A method comprising acquiring the distance to the external electronic device using a depth sensor and/or a proximity sensor of the wearable electronic device 401.
The method according to any one of claims 11 to 13,
detecting movement of the external electronic device relative to movement of the wearable electronic device 401 based on the input; and
The method further includes setting an FPS for displaying the plurality of second images based on movement of the external electronic device.
The method according to any one of claims 11 to 14,
Obtaining the posture of the wearable electronic device 401 and the posture of the external electronic device;
determining positions of the plurality of second images based on the posture of the wearable electronic device 401 and the posture of the external electronic device; and
The method further includes setting an FPS for displaying the plurality of second images based on the positions of the plurality of second images.
The method according to any one of claims 11 to 15,
Obtaining a first object included in the first image;
Obtaining a plurality of second objects related to the first object and corresponding to the distance; and
The method further includes continuously displaying the plurality of second objects using the display 410.
The method according to any one of claims 11 to 16,
Obtaining binocular disparity for the first image; and
The method further includes determining positions of the plurality of second images based on binocular parallax with respect to the acquired first image.
The method according to any one of claims 11 to 17,
An operation to acquire a user's gesture; and
The method further includes determining positions of the plurality of second images based on the direction in which the user's gesture is directed.
The method according to any one of claims 11 to 18,
The operation of acquiring the distance to the external electronic device is:
Obtaining the posture of the wearable electronic device 401 and the posture of the external electronic device; and
A method comprising obtaining a distance to the external electronic device when the posture of the wearable electronic device 401 and the posture of the external electronic device satisfy specified conditions.
The method according to any one of claims 11 to 19,
The operation of acquiring the distance to the external electronic device is:
Obtaining an image of the external electronic device through the camera 430 of the wearable electronic device 401;
An operation of checking content displayed on the external electronic device based on the acquired image;
Based on the input, receiving the content from the external electronic device through the communication module 410 of the wearable electronic device 401; and
When the confirmed content and the received content are the same, a method comprising obtaining a distance to the external electronic device.

Images