KR20240058729A - Electronic device and method for acquiring an image with a changed rendering style - Google Patents

Abstract

A wearable electronic device is provided. The wearable electronic device may include a camera. The wearable electronic device may include a display. The wearable electronic device may include a processor. The processor may be configured to identify a first image containing a visual object via a software application. The processor may be configured to identify a color effect to be applied to a portion of the background image corresponding to the visual object, based on a property of the visual object in the first image. The processor may be configured to identify a rendering style applied to the first image. The processor may be configured to obtain a third image as the background image by converting the second image representing the environment around the wearable electronic device, obtained through the camera, based on the color effect and the rendering style. You can. The processor may be configured to display the first image together with the third image through the display. The properties may include luminescence properties, texture properties, or temperature properties. The rendering style may include a cartoon style, a retro style, an outline style, or a black and white style.

Description

Electronic device and method for acquiring an image with a changed rendering style {ELECTRONIC DEVICE AND METHOD FOR ACQUIRING AN IMAGE WITH A CHANGED RENDERING STYLE}

The descriptions below relate to an electronic device and method for obtaining an image with a changed rendering style.

In a virtual reality (VR) or augmented reality (AR) environment, an electronic device can provide users with a more diverse experience by combining information about the real space and information about the virtual space. Pass-through VR is a technology that creates images of real space through a camera and mixes and displays images of virtual space.

A wearable electronic device is provided. The wearable electronic device may include a camera. The wearable electronic device may include a display. The wearable electronic device may include a processor. The processor may be configured to identify a first image containing a visual object via a software application. The processor may be configured to identify a color effect to be applied to a portion of the background image corresponding to the visual object, based on a property of the visual object in the first image. The processor may be configured to identify a rendering style applied to the first image. The processor may be configured to obtain a third image as the background image by converting a second image representing the environment around the wearable electronic device, obtained through the camera, based on the color effect and the rendering style. You can. The processor may be configured to display the first image together with the third image through the display. The properties may include luminescence properties, texture properties, or temperature properties. The rendering style may include a cartoon style, a retro style, an outline style, or a black and white style.

A method performed by a wearable electronic device may include identifying a first image including a visual object through a software application. The method may include identifying a color effect to be applied to a portion of the background image corresponding to the visual object, based on a property of the visual object in the first image. The method may include identifying a rendering style applied to the first image. The method converts a second image representing the environment around the wearable electronic device, obtained through a camera of the wearable electronic device, based on the color effect and the rendering style, thereby converting the third image into the background image. It may include acquisition operations. It may include displaying the first image together with the third image through a display of the wearable electronic device. The properties may include luminescence properties, texture properties, or temperature properties. The rendering style may include a cartoon style, a retro style, an outline style, or a black and white style.

1 is a block diagram of an electronic device in a network environment according to various embodiments.
FIG. 2A shows an example of a wearable electronic device according to an embodiment of the present disclosure.
FIG. 2B shows an example of hardware included in a wearable electronic device according to an embodiment of the present disclosure.
FIG. 2C illustrates an example of obtaining an image with a changed rendering style through a wearable electronic device according to an embodiment of the present disclosure.
FIG. 3A is a flowchart illustrating an example of a method for obtaining a composite image based on information about a rendering style according to an embodiment of the present disclosure.
FIG. 3B illustrates an example of obtaining information about a rendering style through a plurality of images according to an embodiment of the present disclosure.
FIG. 3C shows an example of obtaining a composite image based on information about the rendering style according to an embodiment of the present disclosure.
FIG. 4A is a flowchart illustrating an example of a method for obtaining information about a rendering style according to an embodiment of the present disclosure.
FIG. 4B illustrates an example of providing a preset for a rendering style to a user according to an embodiment of the present disclosure.
FIG. 5 is a flowchart illustrating an example of a method for obtaining a composite image based on information about a rendering style in a plurality of software applications according to an embodiment of the present disclosure.
Figures 6A to 6C show examples of images with changed rendering styles according to an embodiment of the present disclosure.
Figure 7 shows another example of obtaining a composite image based on information about the rendering style according to an embodiment of the present disclosure.
FIG. 8 illustrates an example of obtaining a composite image in which some areas have been changed based on information about the rendering style according to an embodiment of the present disclosure.
9A to 9C illustrate examples of obtaining a composite image based on information about a rendering style in a plurality of software applications according to an embodiment of the present disclosure.
FIG. 10 illustrates an example of applying a rendering style based on a visual object including the light-emitting attribute of a software application according to an embodiment of the present disclosure.

Terms used in the present disclosure are merely used to describe specific embodiments and may not be intended to limit the scope of other embodiments. Singular expressions may include plural expressions unless the context clearly indicates otherwise. Terms used herein, including technical or scientific terms, may have the same meaning as commonly understood by a person of ordinary skill in the technical field described in this disclosure. Among the terms used in this disclosure, terms defined in general dictionaries may be interpreted to have the same or similar meaning as the meaning they have in the context of related technology, and unless clearly defined in this disclosure, have an ideal or excessively formal meaning. It is not interpreted as In some cases, even terms defined in the present disclosure cannot be interpreted to exclude embodiments of the present disclosure.

In various embodiments of the present disclosure described below, a hardware approach method is explained as an example. However, since various embodiments of the present disclosure include technology using both hardware and software, the various embodiments of the present disclosure do not exclude software-based approaches.

Terms referring to the configuration of the device used in the following description (e.g. processor, camera, display, module, etc.), terms for the operation state (e.g. step, operation) (operation, procedure), terms referring to signals (e.g. signal, information, etc.), terms referring to data (e.g. parameters, values, etc.) is an example for convenience of explanation. Accordingly, the present disclosure is not limited to the terms described below, and other terms having equivalent technical meaning may be used.

In addition, in the present disclosure, the expressions greater than or less than may be used to determine whether a specific condition is satisfied or fulfilled, but this is only a description for expressing an example, and the description of more or less may be used. It's not exclusion. Conditions written as ‘more than’ can be replaced with ‘more than’, conditions written as ‘less than’ can be replaced with ‘less than’, and conditions written as ‘more than and less than’ can be replaced with ‘greater than and less than’. In addition, hereinafter, 'A' to 'B' means at least one of the elements from A to (including A) and B (including B).

1 is a block diagram of an electronic device in a network environment according to various embodiments.

1 is a block diagram of an electronic device 101 in a network environment 100, according to various embodiments. Referring to FIG. 1, in the network environment 100, the electronic device 101 communicates with the electronic device 102 through a first network 198 (e.g., a short-range wireless communication network) or a second network 199. It is possible to communicate with at least one of the electronic device 104 or the server 108 through (e.g., a long-distance wireless communication network). According to one embodiment, the electronic device 101 may communicate with the electronic device 104 through the server 108. According to one embodiment, the electronic device 101 includes a processor 120, a memory 130, an input module 150, an audio output module 155, a display module 160, an audio module 170, and a sensor module ( 176), interface 177, connection terminal 178, haptic module 179, camera module 180, power management module 188, battery 189, communication module 190, subscriber identification module 196 , or may include an antenna module 197. In some embodiments, at least one of these components (eg, the connection terminal 178) may be omitted or one or more other components may be added to the electronic device 101. In some embodiments, some of these components (e.g., sensor module 176, camera module 180, or antenna module 197) are integrated into one component (e.g., display module 160). It can be.

The processor 120, for example, executes software (e.g., program 140) to operate at least one other component (e.g., hardware or software component) of the electronic device 101 connected to the processor 120. It can be controlled and various data processing or operations can be performed. According to one embodiment, as at least part of data processing or computation, the processor 120 stores commands or data received from another component (e.g., sensor module 176 or communication module 190) in volatile memory 132. The commands or data stored in the volatile memory 132 can be processed, and the resulting data can be stored in the non-volatile memory 134. According to one embodiment, the processor 120 includes a main processor 121 (e.g., a central processing unit or an application processor) or an auxiliary processor 123 that can operate independently or together (e.g., a graphics processing unit, a neural network processing unit ( It may include a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, if the electronic device 101 includes a main processor 121 and a secondary processor 123, the secondary processor 123 may be set to use lower power than the main processor 121 or be specialized for a designated function. You can. The auxiliary processor 123 may be implemented separately from the main processor 121 or as part of it.

The auxiliary processor 123 may, for example, act on behalf of the main processor 121 while the main processor 121 is in an inactive (e.g., sleep) state, or while the main processor 121 is in an active (e.g., application execution) state. ), together with the main processor 121, at least one of the components of the electronic device 101 (e.g., the display module 160, the sensor module 176, or the communication module 190) At least some of the functions or states related to can be controlled. According to one embodiment, co-processor 123 (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module 180 or communication module 190). there is. According to one embodiment, the auxiliary processor 123 (eg, neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. Artificial intelligence models can be created through machine learning. For example, such learning may be performed in the electronic device 101 itself on which the artificial intelligence model is performed, or may be performed through a separate server (e.g., server 108). Learning algorithms may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but It is not limited. An artificial intelligence model may include multiple artificial neural network layers. Artificial neural networks include deep neural network (DNN), convolutional neural network (CNN), recurrent neural network (RNN), restricted boltzmann machine (RBM), belief deep network (DBN), bidirectional recurrent deep neural network (BRDNN), It may be one of deep Q-networks or a combination of two or more of the above, but is not limited to the examples described above. In addition to hardware structures, artificial intelligence models may additionally or alternatively include software structures.

The memory 130 may store various data used by at least one component (eg, the processor 120 or the sensor module 176) of the electronic device 101. Data may include, for example, input data or output data for software (e.g., program 140) and instructions related thereto. Memory 130 may include volatile memory 132 or non-volatile memory 134.

The program 140 may be stored as software in the memory 130 and may include, for example, an operating system 142, middleware 144, or application 146.

The input module 150 may receive commands or data to be used in a component of the electronic device 101 (e.g., the processor 120) from outside the electronic device 101 (e.g., a user). The input module 150 may include, for example, a microphone, mouse, keyboard, keys (eg, buttons), or digital pen (eg, stylus pen).

The sound output module 155 may output sound signals to the outside of the electronic device 101. The sound output module 155 may include, for example, a speaker or a receiver. Speakers can be used for general purposes such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part of it.

The display module 160 can visually provide information to the outside of the electronic device 101 (eg, a user). The display module 160 may include, for example, a display, a hologram device, or a projector, and a control circuit for controlling the device. According to one embodiment, the display module 160 may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of force generated by the touch.

The audio module 170 can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module 170 acquires sound through the input module 150, the sound output module 155, or an external electronic device (e.g., directly or wirelessly connected to the electronic device 101). Sound may be output through the electronic device 102 (e.g., speaker or headphone).

The sensor module 176 detects the operating state (e.g., power or temperature) of the electronic device 101 or the external environmental state (e.g., user state) and generates an electrical signal or data value corresponding to the detected state. can do. According to one embodiment, the sensor module 176 includes, for example, a gesture sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, It may include a temperature sensor, humidity sensor, or light sensor.

The interface 177 may support one or more designated protocols that can be used to connect the electronic device 101 directly or wirelessly with an external electronic device (eg, the electronic device 102). According to one embodiment, the interface 177 may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

The connection terminal 178 may include a connector through which the electronic device 101 can be physically connected to an external electronic device (eg, the electronic device 102). According to one embodiment, the connection terminal 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (eg, a headphone connector).

The haptic module 179 can convert electrical signals into mechanical stimulation (e.g., vibration or movement) or electrical stimulation that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

The camera module 180 can capture still images and moving images. According to one embodiment, the camera module 180 may include one or more lenses, image sensors, image signal processors, or flashes.

The power management module 188 can manage power supplied to the electronic device 101. According to one embodiment, the power management module 188 may be implemented as at least a part of, for example, a power management integrated circuit (PMIC).

Battery 189 may supply power to at least one component of electronic device 101. According to one embodiment, the battery 189 may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

Communication module 190 is configured to provide a direct (e.g., wired) communication channel or wireless communication channel between electronic device 101 and an external electronic device (e.g., electronic device 102, electronic device 104, or server 108). It can support establishment and communication through established communication channels. Communication module 190 operates independently of processor 120 (e.g., an application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module 190 is a wireless communication module 192 (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (e.g. : LAN (local area network) communication module, or power line communication module) may be included. Among these communication modules, the corresponding communication module is a first network 198 (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network 199 (e.g., legacy It may communicate with an external electronic device 104 through a telecommunication network such as a cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., LAN or WAN). These various types of communication modules may be integrated into one component (e.g., a single chip) or may be implemented as a plurality of separate components (e.g., multiple chips). The wireless communication module 192 uses subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module 196 to communicate within a communication network such as the first network 198 or the second network 199. The electronic device 101 can be confirmed or authenticated.

The wireless communication module 192 may support 5G networks after 4G networks and next-generation communication technologies, for example, NR access technology (new radio access technology). NR access technology provides high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and access to multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low latency). -latency communications)) can be supported. The wireless communication module 192 may support a high frequency band (eg, mmWave band), for example, to achieve a high data rate. The wireless communication module 192 uses various technologies to secure performance in high frequency bands, for example, beamforming, massive array multiple-input and multiple-output (MIMO), and full-dimensional multiplexing. It can support technologies such as input/output (FD-MIMO: full dimensional MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module 192 may support various requirements specified in the electronic device 101, an external electronic device (e.g., electronic device 104), or a network system (e.g., second network 199). According to one embodiment, the wireless communication module 192 supports peak data rate (e.g., 20 Gbps or more) for realizing eMBB, loss coverage (e.g., 164 dB or less) for realizing mmTC, or U-plane latency (e.g., 164 dB or less) for realizing URLLC. Example: Downlink (DL) and uplink (UL) each of 0.5 ms or less, or round trip 1 ms or less) can be supported.

The antenna module 197 may transmit or receive signals or power to or from the outside (eg, an external electronic device). According to one embodiment, the antenna module 197 may include an antenna including a radiator made of a conductor or a conductive pattern formed on a substrate (eg, PCB). According to one embodiment, the antenna module 197 may include a plurality of antennas (eg, an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network such as the first network 198 or the second network 199 is, for example, connected to the plurality of antennas by the communication module 190. can be selected. Signals or power may be transmitted or received between the communication module 190 and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (eg, radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module 197.

According to various embodiments, the antenna module 197 may form a mmWave antenna module. According to one embodiment, a mmWave antenna module includes a printed circuit board, an RFIC disposed on or adjacent to a first side (e.g., bottom side) of the printed circuit board and capable of supporting a designated high frequency band (e.g., mmWave band); And a plurality of antennas (e.g., array antennas) disposed on or adjacent to the second side (e.g., top or side) of the printed circuit board and capable of transmitting or receiving signals in the designated high frequency band. can do.

At least some of the components are connected to each other through a communication method between peripheral devices (e.g., bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI)) and signal ( (e.g. commands or data) can be exchanged with each other.

According to one embodiment, commands or data may be transmitted or received between the electronic device 101 and the external electronic device 104 through the server 108 connected to the second network 199. Each of the external electronic devices 102 or 104 may be of the same or different type as the electronic device 101. According to one embodiment, all or part of the operations performed in the electronic device 101 may be executed in one or more of the external electronic devices 102, 104, or 108. For example, when the electronic device 101 needs to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device 101 may perform the function or service instead of executing the function or service on its own. Alternatively, or additionally, one or more external electronic devices may be requested to perform at least part of the function or service. One or more external electronic devices that have received the request may execute at least part of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device 101. The electronic device 101 may process the result as is or additionally and provide it as at least part of a response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology can be used. The electronic device 101 may provide an ultra-low latency service using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device 104 may include an Internet of Things (IoT) device. Server 108 may be an intelligent server using machine learning and/or neural networks. According to one embodiment, the external electronic device 104 or server 108 may be included in the second network 199. The electronic device 101 may be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

VR (virtual reality) is a technology that makes objects that can exist appear to be objects that exist in an artificial environment. AR (augmented reality) is a technology that synthesizes arbitrary objects or information with real objects to make them appear as if they exist in the original environment. Electronic devices for VR or AR can provide virtualized or augmented information based on real-world objects.

For example, the electronic device may include a wearable electronic device for providing a VR or AR environment to the user. For example, a wearable electronic device may be a pass-through VR device that can display both a virtual environment and a real environment through a camera. Here, pass-through VR can refer to a technology that creates images of real space through a camera and displays them by mixing them with images of a virtual environment.

Below, for convenience of explanation, the video see-through (VST) method is used to generate and provide new information about the actual environment seen through the camera. However, the present disclosure is not limited thereto. The present disclosure can also be applied to an OST (optical see-through) method that provides information by adding generated information to an actual visible environment.

FIG. 2A shows an example of a wearable electronic device according to an embodiment of the present disclosure. FIG. 2B shows an example of hardware included in a wearable electronic device according to an embodiment of the present disclosure.

2A and 2B, the electronic device 101 (e.g., the electronic device 101 of FIG. 1) according to an embodiment includes some or all of the components of the electronic device 101 of FIG. 1. It can be included. According to one embodiment, the electronic device 101 includes a housing 210, at least one display 220, at least one optical device 230, and at least one camera 240 (e.g., the camera module of FIG. 1 ( 180)), at least one microphone 250, speaker 260, antenna module 270 (e.g., antenna module 197 of FIG. 1), battery 280 (e.g., battery 189 of FIG. 1), and/or a printed circuit board 290. According to one embodiment, the electronic device 101 may be referred to as a wearable device worn on part of the user's body. The electronic device 101 may be configured to provide augmented reality (AR), virtual reality (VR), or mixed reality (MR) that combines augmented reality and virtual reality to the user. there is. For example, the electronic device 101 uses augmented reality through a display 220 that displays a real image provided by light received from the outside of the electronic device 101 and a virtual object superimposed on the real image. It can be configured to provide. A real image can be implemented by light received from outside the electronic device 101 without additional data processing by the electronic device 101. The virtual object may include at least one of text and images corresponding to various information related to the object in the real image. However, the virtual object is not limited thereto, and may include at least one of text and images corresponding to various information related to other objects that are not included in the real image.

The housing 210 may define at least a portion of the outer surface of the electronic device 101. At least a portion of the housing 210 may be in contact with a part of the user's body when the electronic device 101 is worn by the user. According to one embodiment, the housing 210 may support components included in the electronic device 101. For example, some of the components included in the electronic device 101 may be disposed inside the housing 210. For example, other parts of the electronic device 101 may be exposed to the outside of the housing 210 . According to one embodiment, the housing 210 includes at least one rim 211, a bridge 212, at least one nose pad 213, at least one temple 214, and/or a hinge structure 215. may include.

At least one rim 211 may support the display 220. When the electronic device 101 is worn by a user, at least one rim 211 may be worn at a position corresponding to the user's eyes. According to one embodiment, at least one rim 211 may include a first rim 211a and a second rim 211b. The first rim 211a and the second rim 211b may be spaced apart from each other. For example, when the electronic device 101 is worn by a user, the first limb 211a is positioned to correspond to the user's left eye, and the second limb 211b is positioned to correspond to the user's right eye. You can.

Bridge 212 may be coupled to at least one rim 211. According to one embodiment, the bridge 212 may connect the first rim 211a and the second rim 212b. For example, the bridge 212 may extend between the edge of the first rim 211a and the edge of the second rim 211b. When the electronic device 101 is worn by a user, the bridge 212 may be positioned between the user's left and right eyes.

At least one nose pad 213 may maintain the position of the electronic device 101 when the electronic device 101 is worn by a user. For example, when the electronic device 101 is worn by a user, at least one nose pad 213 may be in contact with the user's nose. According to one embodiment, at least one nose pad 213 may include a first nose pad 213a and a second nose pad 213b spaced apart from the first nose pad 213a. For example, when the electronic device 101 is worn by a user, the first nose pad 213a may be disposed on the edge of the first rim 211a facing the user. For example, when the electronic device 101 is worn by a user, the second nose pad 213b may be disposed on the edge of the second rim 211b facing the user.

At least one temple 214 may maintain the position of the electronic device 101 when the electronic device 101 is worn by a user. For example, when the electronic device 101 is worn by a user, at least one temple 214 may be in contact with the user's ear. According to one embodiment, at least one temple 214 may be coupled to at least one rim 211. For example, the at least one temple 214 may include a first temple 214a coupled to the first rim 211a, and a second temple 214b coupled to the second rim 211b. . According to one embodiment, at least one temple 214 may provide a space where components of the electronic device 101 are arranged.

The hinge structure 215 may connect at least one rim 211 and at least one temple 214. The hinge structure 215 may couple at least one temple 214 and at least one rim 211 such that the at least one temple 214 can rotate with respect to the at least one rim 211 . For example, the first temple 214a may be rotatably coupled to the first rim 211a through the hinge structure 215. For example, the second temple 214b may be rotatably coupled to the second rim 211b through the hinge structure 215.

At least one display 220 may be configured to display visual information to the user. At least one display 220 may include a substantially transparent or translucent lens. For example, at least one display 220 may be referred to as a see-through display, but is not limited thereto. According to one embodiment, at least one display 220 may be surrounded by at least one rim 211. For example, at least one display 220 may include a first display 221 and a second display 222. The second display 220 may be spaced apart from the first display 221 . The first display 221 may be coupled to the first rim 211a, and the second display 222 may be coupled to the second rim 211b.

Referring to FIG. 2B , according to one embodiment, at least one display 220 may include a first surface 220a and a second surface 220b opposite to the first surface 220a. At least one display 220 may include a display area disposed on the second surface 220b. For example, light incident from the outside of the electronic device 101 through the first surface 220a may pass through the first surface 220a and the second surface 220b. Light passing through the second surface 220b may be transmitted to the user.

According to one embodiment, at least one display 220 may include a wave guide 223. The wave guide 223 may transmit light transmitted from at least one optical device 230 to the user based on changing the direction of travel of the light transmitted from the at least one optical device 230. Light emitted from at least one optical device 230 may be transmitted to the user by passing through the second side 220b of at least one display 220 by the wave guide 223. For example, the wave guide 223 may transmit light emitted from at least one optical device 230 to the user by diffracting the light within the wave guide 223. When the wave guide 223 diffracts light emitted from the at least one optical device 230, the wave guide 223 is a diffractive element (e.g., a diffractive optical element (DOE)) disposed within the wave guide 223. Alternatively, the wave guide 223 may include a holographic optical element (HOE) by reflecting the light emitted from the at least one optical device 230 within the wave guide 223 to transmit it to the user. When the wave guide 223 reflects light emitted from at least one optical device 230, the wave guide 223 may include a plurality of mirrors disposed within the wave guide 223.

At least one optical device 230 may emit light to display a virtual object on at least one display 220. For example, at least one optical device 230 may include a projector. For example, the electronic device 101 may transmit data for displaying a virtual object to at least one optical device 230. At least one optical device 230 may emit light toward the wave guide 223 based on receiving data for displaying a virtual object. The wave guide 223 may transmit light received from the at least one optical device 230 to the display area on the second side 220b of the display 220. Light emitted by at least one optical device 230 may pass through the second surface 220b and be transmitted to the user along with light embodying a real image transmitted from the outside of the electronic device 101. Light emitted from at least one optical device 230 and light transmitted from the outside of the electronic device 101 may provide augmented reality to the user. According to one embodiment, at least one optical device 230 may be disposed on at least one display 220. For example, at least one optical device 230 may be surrounded by at least one rim 211, but is not limited thereto. For example, at least one optical device 230 may be included in at least one display 220 .

At least one camera 240 may be configured to acquire an image based on receiving light from an object external to the electronic device 101 . According to one embodiment, the at least one camera 240 may include an eye tracking camera 241, a motion recognition camera 242, and/or a photographing camera 243.

The gaze tracking camera 241 may output data representing the gaze of the user wearing the electronic device 101. For example, the electronic device 101 may detect the gaze from an image including the user's pupils obtained through the gaze tracking camera 241. According to one embodiment, the eye tracking camera 241 may include a plurality of eye tracking cameras corresponding to the user's left eye and the user's right eye, respectively. According to one embodiment, the electronic device 101 tracks the user's gaze and at least one display 220 through a gaze tracking camera 241 for tracking the gaze of a user wearing the electronic device 101. ) can match the visual information provided. Eye tracking camera 241 may be configured to capture an image of the user's pupils to determine the user's gaze. For example, eye tracking camera 241 may be configured to capture an image of the user's pupils to determine the user's gaze. For example, the gaze tracking camera 241 may receive gaze detection light reflected from the user's pupil and track the user's gaze based on the position and movement of the received gaze detection light. According to one embodiment, the eye tracking camera 241 may be disposed within the housing 210. The eye tracking camera 241 may be placed in the housing 210 to be directed to the user's left and/or right eye. For example, the eye tracking camera 241 may be disposed on at least one rim 211, but is not limited thereto. For example, the eye tracking camera 241 may be placed on the bridge 212.

The motion recognition camera 242 can provide a specific event to the screen provided on at least one display 220 by recognizing the movement of at least a part of the user's body (e.g., the user's torso, hands, or face). there is. The gesture recognition camera 242 acquires a signal corresponding to the gesture for the electronic device 101 to recognize the user's gesture, and displays a display corresponding to the signal on at least one display 220. can be provided. The electronic device 101 may identify a signal corresponding to the operation and perform a designated function based on the identification. For example, the electronic device 101 can be turned on/off based on identifying a signal corresponding to the operation. For example, the electronic device 101 may display a virtual object on at least one display 220 through at least one optical device 230 based on identifying a signal corresponding to the operation. . According to one embodiment, the motion recognition camera 242 may be disposed within the housing 210. For example, the motion recognition camera 242 may be disposed inside at least one rim 211, but is not limited thereto. For example, the motion recognition camera 242 may be placed inside the bridge 212.

The shooting camera 243 can capture a real image or background to be matched with a virtual image to implement augmented reality or mixed reality content. The capturing camera 243 captures an image of a specific object that exists at a location where the user is looking, and the electronic device 101 provides the image obtained from the capturing camera 243 to at least one display 220. You can. At least one display 220 displays information about a real image or background including an image of the specific object acquired using a shooting camera 243 and a virtual image provided through at least one optical device 230. A single overlapping image can be displayed. According to one embodiment, the photographing camera 243 may be disposed inside at least one rim 211. For example, the photographing camera 243 may include a plurality of photographing cameras spaced apart from each other.

At least one microphone 250 may be configured to acquire audio from outside the electronic device 101. According to one embodiment, at least one microphone 250 may be placed inside the housing 210. For example, the at least one microphone 250 may include a first microphone 251, a second microphone 252, and/or a third microphone 253. The first microphone 251 may be placed inside the first rim 211a. The second microphone 252 may be placed inside the second rim 211b. The third microphone 253 may be placed between the first microphone 251 and the second microphone 252. For example, the third microphone 253 may be placed within the bridge 212 of the housing 210. However, the present invention is not limited thereto, and the number and arrangement of at least one microphone 250 may be changed differently from FIG. 2B. For example, at least one microphone 250 may be placed inside at least one temple 214. According to one embodiment, when the number of at least one microphone 250 included in the electronic device 101 is two or more, the electronic device 101 includes a plurality of microphones disposed on different parts of the housing 200. Using the fields 251, 252, and 253, the direction of the sound signal can be identified.

The speaker 260 may be configured to output audio to the outside of the electronic device 101. According to one embodiment, the speaker 260 may be positioned adjacent to the user's ear while the electronic device 101 is worn by the user. For example, the speaker 260 may be placed inside at least one temple 214 that is in contact with the user's ear. For example, the speaker 260 may be placed inside the first temple 214a so as to be adjacent to the user's left ear. For example, the speaker 260 may be placed inside the second temple 214b so as to be adjacent to the user's right ear.

The antenna module 270 may transmit a signal or power to the outside of the electronic device 101 or receive a signal or power from the outside of the electronic device 101 . Antenna module 270 may be electrically and/or operatively connected to a communication circuit. According to one embodiment, the antenna module 270 may be disposed within at least one temple 214. For example, the antenna module 270 may be placed inside the first temple 214a. For example, the antenna module 270 may be placed inside the second temple 214b. According to one embodiment, at least a portion of the antenna module 270 may be exposed to the outside of at least one temple 214, but is not limited thereto.

The battery 280 may supply power to electronic components of the electronic device 101. According to one embodiment, the battery 280 may be disposed within at least one temple 214. For example, the battery 280 may be placed inside the first temple 214a. For example, the battery 280 may be placed inside the second temple 214b. According to one embodiment, the battery 280 may be disposed on the other end of the at least one temple 214, which is opposite to one end of the at least one temple 214 connected to the at least one rim 211. . For example, the battery 280 may be disposed inside the other end of the first temple 214a, which is opposite to one end of the first temple 214a connected to the first rim 211a. For example, the battery 280 may be disposed inside the other end of the second temple 214b, which is opposite to one end of the second temple 214b connected to the second rim 211b.

The printed circuit board 290 may form electrical connections between electronic components within the electronic device 101 . For example, the printed circuit board 290 may form electrical connections between electronic components disposed on the printed circuit board 290. For example, the printed circuit board 290 includes electronic components disposed on the outside of the printed circuit board 290 and electronic components disposed on the printed circuit board 290 (e.g., processor 120 of FIG. 1 ). ) can form an electrical connection between According to one embodiment, the printed circuit board 290 may be disposed inside at least one temple 214. For example, the printed circuit board 290 may include a plurality of printed circuit boards disposed inside the first temple 214a and the second temple 214b, respectively. The printed circuit board 290 may include a flexible printed circuit board (FPCB) for connecting a plurality of printed circuit boards.

FIG. 2C illustrates an example of obtaining an image with a changed rendering style through a wearable electronic device according to an embodiment of the present disclosure.

The rendering style may indicate a style applied when rendering a two-dimensional or three-dimensional scene as an image. The wearable electronic device may be the electronic device 101 of FIGS. 1, 2A, and 2B for providing a VR or AR environment to the user. Hereinafter, for convenience of explanation, a wearable electronic device for providing a VR environment will be described as an example. However, the present disclosure is not limited thereto.

Referring to FIG. 2C, an image acquired through a software application that provides a VR environment (hereinafter referred to as VR image) and an image of the surrounding environment of the wearable electronic device acquired through a camera of the wearable electronic device (hereinafter referred to as VR image) Examples of synthesizing images of a real environment are shown. The environment may also be referred to as space.

The first example 200-1 shows an example in which an image 205-1 of a real environment with a changed rendering style is synthesized with a VR image 203. Here, the image 205-1 of the real environment may represent a virtual image to which a rendering style that is the same or similar to that of the VR image 203 (eg, cartoon style) is applied. In contrast, the second example 200-2 shows an example in which an image 205-2 of a real environment in which the rendering style has not been changed is synthesized with a VR image 203. Here, the image 205-2 of the real environment may represent an actual image of the surrounding environment as seen by the user of the wearable electronic device.

Referring to the second example 200-2, the wearable electronic device may provide an image 205-2 of the real environment along with the VR image 203 to a user using a software application that provides a VR environment. . In other words, the user can be provided with an image that is a composite of the cartoon-style VR image 203 and the image 205-2 of the real environment. Due to images of different styles included in the composite image, the user can feel the heterogeneity of the composite image displayed through the display of the wearable electronic device. That is, the second example 200-2 combines the image 205-2 of the real environment and the VR image 203 as is, or only applies simple filtering to the image 205-2 of the real environment. Since information is not exchanged between the environment and the VR environment, heterogeneous composite images may be derived. Accordingly, the user may feel less immersed when experiencing the VR environment.

Hereinafter, embodiments of the present disclosure, as in the first example 200-1, identify the rendering style of the VR image 203 and the rendering style identified in the image 205-1 for the real environment viewed by the user. By applying , we propose a method to obtain an image with a changed rendering style. Through this, embodiments of the present disclosure can provide a more natural and immersive VR experience by combining images with changed rendering styles and VR images and providing them to the user.

FIG. 3A is a flowchart illustrating an example of a method for obtaining a composite image based on information about a rendering style according to an embodiment of the present disclosure.

The rendering style may indicate a style applied when rendering a two-dimensional or three-dimensional scene as an image. The composite image may represent an image that is a composite of an image acquired through a software application that provides a VR environment and an image of the environment around the wearable electronic device. The wearable electronic device may be the electronic device 101 of FIGS. 1, 2A, and 2B for providing a VR or AR environment to the user. Hereinafter, for convenience of explanation, a wearable electronic device for providing a VR environment will be described as an example. However, the present disclosure is not limited thereto.

The flowchart of FIG. 3A may be performed by the electronic device 101 of FIGS. 1, 2A, and 2B. For example, the flowchart of FIG. 3A may be performed by the processor 120 of the electronic device 101 of FIGS. 1, 2A, and 2B.

Referring to Figure 3A, in operation 301, processor 120 may identify whether the number of software applications is 1. Here, the software application may represent a software application related to a VR or AR environment. For example, the software application may be a software application that can be used in a VR environment or an AR environment. The processor 120 may identify the number of software applications that can be displayed through the display of the electronic device 101 (eg, the display module 150 of FIG. 1). For example, when one software application is displayed through the entire display area of the electronic device 101, the processor 120 may identify the number of software applications as one. A software application may also be referred to as an immersive software application that utilizes the entire area of the display. For example, when three software applications are displayed on the display of the electronic device 101, the processor 120 may identify the number of software applications as three. The three software applications may also be referred to as widget-type software applications that utilize some areas of the display. In operation 301, if the processor 120 identifies the number of software applications as 1, operation 303 may be performed. In operation 301, if the processor 120 identifies the number of software applications as 2 or more (i.e., multiple), operation 305 may be performed.

In operation 303, processor 120 may obtain information about the rendering style of one software application. Here, information about the rendering style may indicate the rendering style applied to the software application. The rendering style applied to a software application may also be referred to as a rendering style defined by the software application. For example, information about the rendering style may be identified based on file information or code information of a software application. Specific details related to this are described in FIG. 4A below. The rendering style may include various types of styles. For example, the rendering style may be a cartoon style, a physical based rendering (PBR) style, a black/white style, an outline style, a voxel style, a comic style, or May include retro styles. However, it is not limited to this, and other rendering styles may also be applied to embodiments of the present disclosure.

In operation 305, the processor 120 may obtain information about the rendering style of each of the plurality of software applications. Here, information about the rendering style may indicate the rendering style applied to the software application. In other words, the processor 120 may obtain information about a plurality of rendering styles corresponding to a plurality of software applications. For example, information about each rendering style may be identified based on file information or code information of a software application. Specific details related to this are described in FIG. 4A below. Additionally, in operation 305, the processor 120 may obtain boundary information for each of a plurality of software applications. For example, boundary information may indicate information about the area of each software application displayed through the display of the electronic device 101. The processor 120 may map information about the rendering style applied to each of the divided areas based on the boundary information. Additionally, the processor 120 may store information about the rendering style applied to each of the divided areas in a memory (eg, memory 130 in FIG. 1).

At operation 307, processor 120 may identify the first image via a software application. For example, the first image may refer to a VR or AR image that the electronic device 101 provides to the user through the software application. For example, the first image may include the VR image 203 of FIG. 2C. According to one embodiment, the first image is an image of a VR or AR environment provided through the software application and may include at least one visual object. For example, at least one visual object may include a light-emitting property, as shown in FIG. 10 below.

According to one embodiment, the processor 120 may identify a plurality of first images through one software application. For example, the processor 120 may identify a plurality of n first images at m second intervals. For example, the processor 120 may acquire a plurality of first images displayed through a software application. Additionally, the processor 120 may identify a plurality of first images for each of a plurality of software applications. For example, the processor 120 may identify a first set of first images through a first software application included in the plurality of software applications. Additionally, the processor 120 may identify the second set of first images through a second software application included in the plurality of software applications. Here, the set for each plurality of first images may represent a set in which the same first image is identified at different times.

According to the above description, the processor 120 may identify at least one first image for a software application. Additionally, the processor 120 may store at least one identified first image in a memory (eg, memory 130 of FIG. 1).

At operation 309, processor 120 may identify a color effect to be applied to the background image based on properties of visual objects in the first image. For example, processor 120 may identify a color effect to be applied to a portion of the background image corresponding to the visual object. The properties may include luminescence properties, temperature properties, texture properties, etc. expressed from the visual object. For example, if some of the visual objects in the first image are virtual light sources, some of the visual objects may be understood to have luminescent properties. For example, the background image may be composited with the first image and may represent an image that serves as a background of the first image. The background image may represent an image of the surrounding environment of the wearable electronic device (e.g., an image of the actual environment or a second image) or a third image including the same visual object as the second image and to which a rendering style is applied. For example, the background image to which the color effect will be applied may be a second image, and the background image to which the color effect will be applied may be a third image. The color effect may include the number of colors, histogram, presence or absence of outline, size of pixel with the same color, shadow color, contrast, sharpening, blurring or masking, etc. For example, the color effect may be set considering the location of a virtual light source with luminous properties included in the first image, the brightness of the virtual light source, the color temperature of the virtual light source, or the brightness of ambient light. there is. Details of the color effect are described below in FIGS. 6A to 6C.

At operation 311, processor 120 may identify a rendering style based on the first image. For example, the processor 120 may identify the rendering style of the software application based on the visual object included in the first image. According to one embodiment, when the processor 120 identifies a plurality of first images, the processor 120 may identify a rendering style of a software application based on the plurality of first images. there is. For example, the processor 120 uses the manifest file, executable file information, code information, deep learning, histogram, or image information of the software application to create the first You can identify the rendering style applied to an image. According to the above description, the processor 120 may identify at least one first image for a software application and, based on the identified at least one first image, identify a rendering style for the software application. If information about the rendering style is identified through information about the software application through operations 301 to 305, operation 311 may be omitted. Specific details related to this are described in FIG. 4A below.

In operation 313, processor 120 may obtain a third image by converting the second image. For example, the second image may represent an image of the surrounding environment of the electronic device 101. That is, the second image may represent an image of the real environment that can be identified through the camera of the electronic device 101 (eg, the camera module 180 of FIG. 1). Here, the camera may include a pass-through camera.

According to one embodiment, the processor 120 may acquire a second image, which is an image of the actual environment, through the camera. For example, the processor 120 may obtain a second image of the actual environment in the direction the user faces with respect to the surrounding environment. The processor 120 may obtain information about the surrounding environment in order to obtain the second image. For example, the information about the surrounding environment may include the color temperature of an environmental light source, the brightness of an environmental light source, or fog information. The environmental light source exists in a real environment and may represent a light source related to the second image. Accordingly, the processor 120 may obtain the second image based on information about the surrounding environment.

According to one embodiment, the processor 120 may convert the second image into a third image based on the color effect and the rendering style. For example, the processor 120 may convert the second image of the real environment into a third image based on the color effect of the VR environment provided through the software application and the rendering style of the software application. According to one embodiment, when a plurality of software applications are used, the processor 120 identifies an area of each of the plurality of software applications based on the boundary information, and sets the rendering style for the identified area to the second The third image to which the image is applied can be identified. Here, the third image can be used as a background image for the first image. According to the above description, the third image may be changed to the same or similar style as the first image.

At operation 315, processor 120 may display a first image and a third image. For example, the processor 120 may synthesize the first image and the third image and display them on the display of the electronic device 101. For example, the processor 120 may display the third image along with the first image through the display of the electronic device 101.

According to the above, embodiments of the present disclosure acquire an image (e.g., a third image) to which a rendering style is applied to an image of a real environment (e.g., a second image), and use the acquired third image and a software application. An image that is a composite of images of the VR environment (e.g., the first image) acquired through the VR environment may be provided to the user. The rendering style may be identified from information about a software application or from the first image. Embodiments of the present disclosure may display the display by combining the first image with a third image to which a rendering style related to the first image is applied, rather than directly combining the second image and the first image. Accordingly, the electronic device 101 can provide the user with a highly immersive VR environment through more naturally rendered images.

FIG. 3B illustrates an example of obtaining information about a rendering style through a plurality of images according to an embodiment of the present disclosure. FIG. 3C shows an example of obtaining a composite image based on information about the rendering style according to an embodiment of the present disclosure.

The rendering style may indicate a style applied when rendering a two-dimensional or three-dimensional scene as an image. The composite image may represent an image that is a composite of an image acquired through a software application that provides a VR environment and an image of the environment around the wearable electronic device. The wearable electronic device may be the electronic device 101 of FIGS. 1, 2A, and 2B for providing a VR or AR environment to the user. Hereinafter, for convenience of explanation, a wearable electronic device for providing a VR environment will be described as an example. However, the present disclosure is not limited thereto.

Referring to FIG. 3B, the processor 120 may acquire the first image through a software application. In the example 330 of FIG. 3B, for convenience of explanation, a plurality of first images 340-1, 340-2, 340-3, and 340-n of one software application are shown. However, the present disclosure is not limited thereto. For example, the processor 120 may acquire one first image through a software application. Alternatively, the processor 120 may acquire a plurality of first images through a software application. According to one embodiment, when using a plurality of software applications, the processor 120 may acquire at least one first image for each of the plurality of software applications. For example, the processor 120 may acquire at least one first image displayed through a software application.

According to one embodiment, each of the plurality of first images 340-1, 340-2, 340-3, and 340-n may include the same visual object. For example, each of the plurality of first images 340-1, 340-2, 340-3, and 340-n may include a visual object that is a cartoon character. The plurality of first images 340-1, 340-2, 340-3, and 340-n may represent images including the same visual object over time.

Although not shown in FIG. 3B, the processor 120 renders a software application based on the plurality of first images 340-1, 340-2, 340-3, and 340-n of the first example 330. Styles can be identified. For example, the processor 120 may identify the rendering style of a software application through deep learning based on an artificial neural network. For the example of the plurality of first images 340-1, 340-2, 340-3, and 340-n including visual objects that are cartoon characters, the processor 120 determines that the rendering style of the software application is cartoon style. can be identified.

Referring to FIG. 3C, in the second example 350, the processor 120 may acquire a second image 355 of the real environment through the camera of the electronic device 101. For example, the processor 120 may acquire a second image 355 of the actual environment in the direction the user faces with respect to the surrounding environment. Additionally, the processor 120 may obtain information about the surrounding environment in order to obtain the second image 355. For example, the information about the surrounding environment may include the color temperature of an environmental light source, the brightness of an environmental light source, or fog information. The environmental light source exists in a real environment and may represent a light source related to the second image 355. Accordingly, the processor 120 may obtain the second image 355 based on information about the surrounding environment.

Referring to FIG. 3C, in the third example 360, the processor 120 may convert the second image 355 into a third image 365 based on color effects and rendering styles. In other words, the processor 120 can obtain the third image 365, which is a corrected virtual image, by applying a rendering style to the second image 355 of the real space. For example, the processor 120 converts the second image 355 of the real environment into a third image 365 based on the color effect of the VR environment provided through the software application and the rendering style of the software application. It can be converted. The color effect may be identified based on the plurality of first images 340-1, 340-2, 340-3, and 340-n of FIG. 3B. The color effect may include the number of colors, histogram, presence or absence of outline, size of pixel with the same color, shadow color, contrast, sharpening, blurring or masking, etc. For example, the color effect may be set considering the location of a virtual light source with luminous properties included in the first image, the brightness of the virtual light source, the color temperature of the virtual light source, or the brightness of ambient light. there is.

The rendering style is identified based on information about a software application, identified based on a plurality of first images 340-1, 340-2, 340-3, and 340-n, or identified according to user input. It can be. Specific details related to this are described in FIG. 4A below. The rendering style may include various types of styles. For example, the rendering style may be a cartoon style, a physical based rendering (PBR) style, a black/white style, an outline style, a voxel style, a comic style, or May include retro styles. The third example 360 of FIG. 3C shows a case where the rendering style is cartoon style.

Referring to the third example 360, the processor 120 may obtain the third image 365 by applying a cartoon-style rendering style and color effects to the second image 355. In other words, the third image 365 may represent an image in which the second image 355 has been changed to a cartoon style and a shader has been applied based on a color effect.

Referring to FIG. 3C , in the fourth example 370, the processor 120 may display a first image 340 and a third image 365. For example, the processor 120 may synthesize and display the first image 340 and the third image 365 through the display of the electronic device 101. For example, the processor 120 may display the third image 365 along with the first image 340 through the display of the electronic device 101.

According to the above, embodiments of the present disclosure acquire an image to which a rendering style is applied (e.g., the third image 365) with respect to an image of the real environment (e.g., the second image 355), and obtain the obtained image (e.g., the third image 365). A composite image of the 3 image 365 and an image of the VR environment acquired through a software application (eg, the first image 340) may be provided to the user. The rendering style may be identified from information about a software application or from the first image. Embodiments of the present disclosure do not synthesize the second image 355 and the first image 340 as is, but instead combine the third image 365 and the third image 365 to which a rendering style related to the first image 340 is applied. The first image 340 can be synthesized and displayed. Accordingly, the electronic device 101 can provide the user with a highly immersive VR environment through more naturally rendered images.

FIG. 4A is a flowchart illustrating an example of a method for obtaining information about a rendering style according to an embodiment of the present disclosure.

The rendering style may indicate a style applied when rendering a two-dimensional or three-dimensional scene as an image. The composite image may represent a composite image of an image acquired through a software application that provides a VR environment and an image of the environment surrounding the wearable electronic device. The wearable electronic device may be the electronic device 101 of FIGS. 1, 2A, and 2B for providing a VR or AR environment to the user. Hereinafter, for convenience of explanation, a wearable electronic device for providing a VR environment will be described as an example. However, the present disclosure is not limited thereto.

The flowchart of FIG. 4A may be performed by the electronic device 101 of FIGS. 1, 2A, and 2B. For example, the flowchart of FIG. 4A may be performed by the processor 120 of the electronic device 101 of FIGS. 1, 2A, and 2B.

Referring to Figure 4A, at operation 401, processor 120 may identify a rendering style based on configuration file information of the software application. For example, processor 120 may identify a rendering style based on a manifest file of a software application. The manifest file defines the components that make up the software application, defines the name of the package of the software application, defines the permission request of the software application, or defines device compatibility of the software application. It can indicate files that are present. According to the above, the processor 120 can read the manifest file of the software application and identify the rendering style based on the manifest file. If in operation 401 the processor 120 does not identify the rendering style, operation 403 may be performed. If the processor 120 identifies a rendering style in operation 401, it may store the rendering style identified in operation 413 in memory (e.g., memory 130 of FIG. 1).

In operation 403, processor 120 may identify a rendering style based on executable file information of the software application. For example, the processor 120 may identify the rendering style by comparing the title, file name, and package name of the executable file of the software application with information on the server. Here, the server information may include a database uploaded to the server. According to the above description, the processor 120 may search the rendering style of the software application within the server's database based on the executable file information of the software application. Accordingly, the processor 120 can identify the rendering style of the software application. In operation 403, if processor 120 does not identify the rendering style, operation 405 may be performed. If the processor 120 identifies a rendering style in operation 403, it may store the rendering style identified in operation 413 in memory (e.g., memory 130 of FIG. 1).

In operation 405, the processor 120 may identify a rendering style based on code information of a software application. For example, the processor 120 may identify code information about the rendering style included in code information of a software application. Here, code information about the rendering style may include string data. Additionally, code information about the rendering style may also be referred to as code information indicating a shader. According to the above, the processor 120 may search the rendering style of the software application in the server's database based on the code information of the software application. Accordingly, the processor 120 can identify the rendering style of the software application. In operation 405, if the processor 120 does not identify the rendering style, operation 407 may be performed. If processor 120 identifies a rendering style at operation 405, it may store the rendering style identified at operation 413 in memory (e.g., memory 130 of FIG. 1).

At operation 407, processor 120 may identify a rendering style based on the first image identified through the software application. For example, the first image may refer to a VR or AR image that the electronic device 101 provides to the user through the software application. For example, the first image may include the VR image 203 of FIG. 2C. For example, the first image is an image of a VR or AR environment provided through the software application and may include at least one visual object. The processor 120 may identify a rendering style for the identified first image using deep learning, histogram, or image information. In FIG. 4A, the description is made with one first image, but the present disclosure is not limited thereto. For example, processor 120 may identify a rendering style for a plurality of first images for one software application. Alternatively, the processor 120 may identify a rendering style for at least one first image for each of a plurality of software applications. In operation 407, if processor 120 cannot identify the rendering style, operation 409 may be performed. If processor 120 identifies a rendering style in operation 407, it may store the rendering style identified in operation 413 in memory (e.g., memory 130 of FIG. 1).

In operation 409, processor 120 may display a preset image associated with a software application to the user and identify a rendering style based on user input. For example, a preset image related to a software application may include images to which various rendering styles are applied to the first image. The various rendering styles for the first image may represent styles related to the rendering style identified according to the rendering result for the first image obtained according to the rendering result of the software application. The processor 120 may display the preset image through the display of the electronic device 101. The processor 120 may identify the selected preset image by identifying receipt of user input regarding the displayed preset image. Accordingly, the processor 120 may identify the rendering style for the preset image as the rendering style of the software application. In operation 409, if processor 120 does not identify the rendering style, operation 411 may be performed. If processor 120 identifies a rendering style in operation 409, it may store the rendering style identified in operation 413 in memory (e.g., memory 130 of FIG. 1).

In operation 411, the processor 120 may identify that the rendering style related to the software application is not applied to the image of the real environment (eg, the second image). Accordingly, the processor 120 may apply the basic rendering style to the second image. The basic rendering style can be saved in advance by the user.

At operation 413, processor 120 may store the identified rendering style. Additionally, the processor 120 may convert the second image into a third image based on the stored rendering style.

FIG. 4B illustrates an example of providing a preset for a rendering style to a user according to an embodiment of the present disclosure.

The rendering style may indicate a style applied when rendering a two-dimensional or three-dimensional scene as an image. The composite image may represent an image that is a composite of an image acquired through a software application that provides a VR environment and an image of the environment around the wearable electronic device. The wearable electronic device may be the electronic device 101 of FIGS. 1, 2A, and 2B for providing a VR or AR environment to the user. Hereinafter, for convenience of explanation, a wearable electronic device for providing a VR environment will be described as an example. However, the present disclosure is not limited thereto.

FIG. 4B shows preset images 450 displayed to the user through the display of the electronic device 101. Preset images 450 of FIG. 4B may represent an example of a preset image presented in operation 409 of FIG. 4A. For example, the preset images 450 may represent images to which various rendering styles are applied to the first image. In the example of FIG. 4B, the first image may be an image that includes visual objects such as a tree, the sun, a hill, and a road. In the process of acquiring the first image, the electronic device 101 may obtain a rendering result of a software application related to the first image. For example, the rendering result for acquiring the first image may include a pencil style. Accordingly, preset images 450 may be determined based on the pencil style.

Preset images 450 may include preset images to which various rendering styles are applied. For example, the preset images 450 may include a first preset image 451, a second preset image 452, and a third preset image 453. For example, the first preset image 451 may be an image to which a style that displays a plurality of letters is applied to the first image. Additionally, the second preset image 452 may be an image to which a pencil style has been applied to the first image. Additionally, the third preset image 453 may be an image to which a cartoon style is applied to the first image.

According to one embodiment, the processor 120 may display preset images 450 through a display. The processor 120 may receive a user's input for one of the preset images 450 and identify the preset image corresponding to the received user's input. Accordingly, the processor 120 can identify the rendering style of the software application.

FIG. 5 is a flowchart illustrating an example of a method for obtaining a composite image based on information about a rendering style in a plurality of software applications according to an embodiment of the present disclosure.

The rendering style may indicate a style applied when rendering a two-dimensional or three-dimensional scene as an image. The composite image may represent an image that is a composite of an image acquired through a software application that provides a VR environment and an image of the environment around the wearable electronic device. The wearable electronic device may be the electronic device 101 of FIGS. 1, 2A, and 2B for providing a VR or AR environment to the user. Hereinafter, for convenience of explanation, a wearable electronic device for providing a VR environment will be described as an example. However, the present disclosure is not limited thereto.

The flowchart of FIG. 5 may be performed by the electronic device 101 of FIGS. 1, 2A, and 2B. For example, the flowchart of FIG. 5 may be performed by the processor 120 of the electronic device 101 of FIGS. 1, 2A, and 2B.

Referring to Figure 5, at operation 501, processor 120 may identify an area in which a software application is displayed on the display. For example, the processor 120 may identify an area in which each software application among a plurality of software applications is displayed on the display of the electronic device 101. For example, assume that the plurality of software applications include a first software application, a second software application, and a third software application. For the entire area of the display of electronic device 101, processor 120 identifies a first area for a first software application, a second area for a second software application, and a third area for a third software application. can do. At this time, the processor 120 may identify the area based on boundary information of the software application.

At operation 503, processor 120 may identify information about the rendering style of each software application. For example, the processor 120 may identify information about the rendering style of each software application based on the operations included in the flowchart of FIG. 4A. For example, processor 120 may identify a first rendering style of a first software application. Processor 120 may identify a second rendering style of the second software application. Processor 120 may identify a third rendering style of a third software application. Here, the first rendering style, the second rendering style, and the third rendering style may be the same, different from each other, or partially identical to each other.

In operation 505, the processor 120 may obtain a plurality of third images by converting the second image for each software application. For example, the processor 120 may obtain the third image by converting the second image, which is an image of the surrounding environment of the electronic device 101, by applying the first rendering style to the first area. The processor 120 may obtain a third image by converting the second image, which is an image of the surrounding environment of the electronic device 101, by applying a second rendering style to the second area. The processor 120 may obtain the third image by converting the second image, which is an image of the surrounding environment of the electronic device 101, by applying the third rendering style to the third area.

In operation 507, processor 120 may display a first image and a plurality of third images. For example, the processor 120 may synthesize and display the first image and the plurality of third images through the display of the electronic device 101. For example, the processor 120 may display the plurality of third images together with the first image through the display of the electronic device 101. For example, the processor 120 may display a third image synthesized with the first image through the display of the electronic device 101.

According to one embodiment, the synthesized third image may represent one image composed of the plurality of third images. For example, the synthesized third image may represent an image created by combining the plurality of third images corresponding to the areas identified in operation 501. The areas may be identified based on boundary information of each of a plurality of software applications. According to one embodiment, the processor may identify areas so that there is no separation between the areas in order to minimize heterogeneity between a plurality of software applications. Alternatively, if there is a gap between the areas, the processor may convert the spaced portion based on information about the rendering style identified from a plurality of software applications. For example, if the spaced portion is a portion between the first area of the first software application and the second area of the second software application, it is included in both the first software application and the second software application. Based on information about the current rendering style or similar rendering styles, the rendering style for the spaced portion may be applied.

According to the above, embodiments of the present disclosure acquire images (e.g., a plurality of third images) to which a rendering style is applied to an image of a real environment (e.g., a second image), and A composite image of 3 images and an image of the VR environment acquired through a software application (e.g., the first image) can be provided to the user. The rendering style may be identified from information about a software application or from the first image. Embodiments of the present disclosure may display the first image by combining a plurality of third images to which a rendering style related to the first image is applied, rather than directly combining the second image and the first image. . Accordingly, the electronic device 101 can provide the user with a highly immersive VR environment through more naturally rendered images.

Figures 6A to 6C show examples of images with changed rendering styles according to an embodiment of the present disclosure. The rendering style may indicate a style applied when rendering a two-dimensional or three-dimensional scene as an image.

Referring to FIG. 6A, a first example 600 shows an image including a visual object in 3D (dimensional). In contrast, the image of the second example 610 shows an image obtained by applying a rendering style (eg, cartoon style) to the image of the first example 600. For example, in the image of the second example 610, the colors used may be simplified compared to the first example 600. That is, the image of the second example 610 may use fewer colors compared to the image of the first example 600. For example, compared to the first example 600, the image of the second example 610 may directly display the outline of a visual object. Additionally, the image of the second example 610 may be expressed in a simplified state in which the shading level of the shadow is compared to the first example 600. By applying the effects described above, a cartoon-style rendering style can be applied to the image. Accordingly, images to which cartoon style is applied can provide users with a cartoon-like feel.

Referring to FIG. 6B, the third example 620 shows an image to which a retro style rendering style has been applied. For example, the image of the third example 620 may be expressed by reducing the resolution of pixels by n times so that the visual object appears to be composed of pixels. Additionally, the colors used in the image of the third example 620 may be simplified. Additionally, the image of the third example 620 may use a histogram color map such as a 3D image. Accordingly, an image with a retro style can provide a classic feel to the user.

Referring to FIG. 6C, the fourth example 630 shows an image to which a pencil style rendering style has been applied. For example, in the image of the fourth example 630, the outline of a visual object may be displayed in black. Additionally, the image of the fourth example 630 may be displayed by connecting points included in the visual object with short straight lines. Accordingly, an image to which a pencil style is applied can provide a drawing feeling to the user.

Referring to FIGS. 6A to 6C, the image may be changed based on a color effect. The color effect may include the number of colors, histogram, presence or absence of outline, size of pixel with the same color, shadow color, contrast, sharpening, blurring or masking, etc. An image can be altered by changing the number of colors. For example, an image can be altered by changing 16-bit or 32-bit colors to 8-bit colors. Additionally, the image can be changed by using a histogram. For example, the image can be altered by using the camera image color and histogram of the 3D rendered image. Additionally, the image may change depending on the presence or absence of an outline. Additionally, the image can be changed by changing the number of the plurality of pixels when the plurality of pixels have the same color. Additionally, the image can be altered by changing the level of shading of the shadows. For example, an image can be altered by simplifying the shadow colors by grading them. Additionally, the image can be altered by adjusting the contrast and applying sharpening. Additionally, the image can be altered by blurring the image. Additionally, the image can be altered by sharpening the image using masking. The rendering style of the image may be identified based on the combination of the color effects. For example, the cartoon-style rendering style of FIG. 6A may represent a style in which the number of applied colors is small, outlines are drawn directly, and shading levels are simple.

Figure 7 shows another example of obtaining a composite image based on information about the rendering style according to an embodiment of the present disclosure.

The rendering style may indicate a style applied when rendering a two-dimensional or three-dimensional scene as an image. The composite image may represent an image that is a composite of an image acquired through a software application that provides a VR environment and an image of the environment around the wearable electronic device. The wearable electronic device may be the electronic device 101 of FIGS. 1, 2A, and 2B for providing a VR or AR environment to the user. Hereinafter, for convenience of explanation, a wearable electronic device for providing a VR environment will be described as an example. However, the present disclosure is not limited thereto.

7 shows that, rather than converting the real environment image 205-2, which is a background image, into the image 205-1, as shown in FIG. 2C, the VR image 203 is converted into the real environment image 205-2. ) shows an example of conversion to be similar to

Referring to FIG. 7 , the first example 700 shows an image for a VR environment (eg, a game), and the second example 750 shows an image for a real environment. Referring to the first example 700, the image may include a visual object 703 and a background image 705. Here, the visual object 703 may include a car. The image of the first example 700 may have the same rendering style as a game screen applied. According to one embodiment, referring to the second example 750, the processor 120 may obtain a new visual object 753 by applying a rendering style to the visual object 703. For example, the processor 120 may identify a rendering style for the image 755 of the real environment and apply the identified rendering style to the visual object 703 of the first example 700 . Accordingly, the processor 120 may obtain a visual object 753 to which the rendering style is applied. Afterwards, the processor 120 may synthesize the visual object 753 and the image 755 of the real environment, and the synthesized image may be provided to the user through the display of the electronic device 101. At this time, identifying the rendering style for the image 755 of the real environment is performed by selecting the acquired image 755 of the real environment, as shown in operation 311 of FIG. 3A or operation 407 of FIG. 4A. This can be done by analyzing.

Accordingly, embodiments of the present disclosure, such as the second example 750, identify a rendering style of an image 755 of a real-world environment and apply the identified rendering style to a visual object 703 of the image for a VR environment. By doing so, an image with a changed rendering style can be obtained. Through this, embodiments of the present disclosure can provide a more natural and immersive VR experience by combining an image of the real environment and a VR image to which a rendering style similar to the real environment is applied and provided to the user.

FIG. 8 illustrates an example of obtaining a composite image in which some areas have been changed based on information about the rendering style according to an embodiment of the present disclosure.

The rendering style may indicate a style applied when rendering a two-dimensional or three-dimensional scene as an image. The composite image may represent an image that is a composite of an image acquired through a software application that provides a VR environment and an image of the environment around the wearable electronic device. The wearable electronic device may be the electronic device 101 of FIGS. 1, 2A, and 2B for providing a VR or AR environment to the user. Hereinafter, for convenience of explanation, a wearable electronic device for providing a VR environment will be described as an example. However, the present disclosure is not limited thereto.

FIG. 8 shows a first example 800 and a second example 850 in which a rendering style is applied only to some areas of the image. The first example 800 shows an image of a real environment to which a rendering style is applied only to the first area 810. Here, the first area 810 may represent a sky area included in the actual environment. For example, the rendering style applied to the first area 810 may be a cartoon style. According to one embodiment, the processor 120 may identify the user's input for the first area 810 and, upon identification, may separate the first area 810 from the image. Afterwards, the processor 120 may acquire an image to which the identified rendering style is applied to the first area 810, and combine the obtained image for the first area 810 with the image for the actual environment to create a composite image. can be obtained.

In the second example 850, an image of a real environment including a plurality of people is shown with a rendering style applied only to the second area 860. Here, the second area 860 may represent a human area included in the real environment. For example, the rendering style applied to the second area 860 may be a cartoon style. According to one embodiment, the processor 120 may identify the user's input to the second area 860 and, upon identification, may separate the second area 860 from the image. Afterwards, the processor 120 may acquire an image to which the identified rendering style is applied to the second area 860, and combine the obtained image for the second area 860 with the image for the actual environment to create a composite image. can be obtained.

9A to 9C illustrate examples of obtaining a composite image based on information about a rendering style in a plurality of software applications according to an embodiment of the present disclosure.

The rendering style may indicate a style applied when rendering a two-dimensional or three-dimensional scene as an image. The wearable electronic device may be the electronic device 101 of FIGS. 1, 2A, and 2B for providing a VR or AR environment to the user. Hereinafter, for convenience of explanation, a wearable electronic device for providing a VR environment will be described as an example. However, the present disclosure is not limited thereto.

9A to 9C show an example of an electronic device 101 that uses multiple software applications. Referring to FIG. 9A, a first example 900 shows an example of executing three software applications. In the first example 900, the processor 120 may execute a first software application 910, a second software application 920, and a third software application 930. In addition to the image of the real environment, the processor 120 generates a first image for the first software application 910, a first image for the second software application 920, and a first image for the third software application 930. 1 Images can be displayed through the display. Processor 120 may identify an area of the first image for each software application. For example, the processor 120 may identify a first area 915 for the first software application 910 . For example, processor 120 may identify a second area 925 for a second software application 920 . For example, the processor 120 may identify a third area 935 for a third software application 930. At this time, the first region 915 may also be referred to as a first rendering style region. The second region 925 may also be referred to as a second rendering style region. The third region 935 may also be referred to as a third rendering style region.

Referring to FIG. 9B, a second example 940 shows an example of applying a rendering style to a surrounding area of a software application. According to one embodiment, processor 120 may identify a region of the image for each software application. For example, the processor 120 may identify the first area 915 where the first software application 910 is located on the display of the electronic device 101. Additionally, the processor 120 may identify a first peripheral area 917 that includes the first area 915 surrounding the first area 915 . Here, the first peripheral area 917 may represent an area to which the rendering style applied to the first software application 910 of the first area 915 can be applied. The size of the first peripheral area 917 may be identified based on the importance or size of the first software application 910, or the user's interest. For example, when the size of the first software application 910 is physically large, the first peripheral area 917 may also become larger. For example, as the importance of the first software application 910 increases, the first peripheral area 917 may also become wider.

According to one embodiment, the first software application 910 may be displayed with visual emphasis compared to an image of the actual environment. For example, the first area 915 of the first software application 910 may be displayed with visual emphasis compared to an image of the actual environment. For example, the processor 120 may apply a rendering style of a filtering style to areas other than the first area 915. At this time, the processor 120 may apply a rendering style of a relatively weak filtering style to areas other than the first area 915 or areas within the first peripheral area 917. Accordingly, the user can identify the first software application 910 more effectively. In FIG. 9B , for convenience of explanation, application of a rendering style to emphasize one software application (eg, the first software application 910) is shown, but the present disclosure is not limited thereto. The present disclosure may include the application of a rendering style to highlight a plurality of software applications.

Referring to FIG. 9C, a third example 970 shows an example of applying a rendering style to areas other than some of the three software applications used by the user. In the third example 970, assume that the user of the electronic device 101 uses the first software application 910. According to one embodiment, the processor 120 may identify the first software application 910 that the user is using. For example, the processor 120 may identify the first software application 910 that the user is focusing on for a certain period of time or more by tracking the user's gaze. At this time, the processor 120 may identify the first area 915 of the first software application 910. For example, a certain time may be preset by the user or determined depending on the type of software application used. For example, if a software application requires the user to respond quickly, the certain amount of time may be shortened.

According to one embodiment, a rendering style may be applied to the first area 915 of the first software application 910. For example, the processor 120 may apply the rendering style of the first software application 910 to the first area 915 of the first software application 910. Additionally, the processor 120 may expand the area to which the rendering style is applied to the first peripheral area 917 when the user watches the first software application 910 for more than the predetermined period of time. Afterwards, when it is identified that the user is looking at the first software application 910 for more than a certain period of time (e.g., longer than the above-mentioned period of time), the processor 120 sets a rendering style for the entire area displayed through the display. can be applied.

According to the above, the present disclosure acquires images (e.g., a plurality of third images) to which a rendering style is applied to an image of a real environment (e.g., a second image), and the obtained plurality of third images An image that is a composite of images (e.g., first image) of the VR environment acquired through a software application may be provided to the user. The rendering style may be identified from information about a software application or from the first image. Specifically, not only may the entire area of the second image be applied, but the rendering style may also be applied to the area of the second image corresponding to the surrounding area of the first image acquired through a software application. Embodiments of the present disclosure may display the first image by combining a plurality of third images to which a rendering style related to the first image is applied, rather than directly combining the second image and the first image. . Accordingly, the electronic device 101 can provide the user with a highly immersive VR environment through more naturally rendered images.

FIG. 10 illustrates an example of applying a rendering style based on a visual object including the light-emitting attribute of a software application according to an embodiment of the present disclosure.

The rendering style may indicate a style applied when rendering a two-dimensional or three-dimensional scene as an image. The wearable electronic device may be the electronic device 101 of FIGS. 1, 2A, and 2B for providing a VR or AR environment to the user. Hereinafter, for convenience of explanation, a wearable electronic device for providing a VR environment will be described as an example. However, the present disclosure is not limited thereto. The luminous property refers to the fact that when some of the visual objects in the image (e.g., the first image) of the VR environment of the software application are virtual light sources, some of the visual objects have luminous properties. It can be understood. The luminescence property may be included in the properties of a visual object. For example, the properties may include luminescence properties, temperature properties, or texture properties.

10 shows a first example 1000 showing a first image including a visual object 1010 that is a virtual light source and a second image of a real environment, and a visual object 1010 that is a virtual light source for the second image. A second example 1050 is shown in which a third image and a first image to which a rendering style is applied based on are synthesized.

Referring to the first example 1000, the processor 120 may identify a color effect to be applied to the background image based on the luminous properties of the visual object in the first image. The first image may include a visual object 1010. The visual object may include luminous properties. That is, the visual object may be a virtual light source. According to one embodiment, the processor 120 may acquire the first image including the visual object 1010 having light-emitting properties and identify a color effect from the obtained first image. For example, the color effect may include the number of colors, histogram, presence or absence of outline, size of pixels with the same color, shadow color, contrast, sharpening, blurring or masking, etc. You can. For example, the color effect may be set considering the location of a virtual light source with luminous properties included in the first image, the brightness of the virtual light source, the color temperature of the virtual light source, or the brightness of ambient light. there is.

Referring to the second example 1050, the processor 120 may apply a rendering style to the second image based on the color effect obtained from the first image. For example, the processor 120 may apply a rendering style to the surrounding area 1060 of the visual object 1010. At this time, the applied rendering style may represent a rendering style applied by the visual object 1010, which is a virtual light source. For example, when the location of the visual object 1010, which is a virtual light source, changes, the surrounding area 1060 may change, and the processor 120 applies a rendering style in consideration of the changing surrounding area 1060. can do. At this time, the closer the location of the visual object 1010, which is a virtual light source, is, the brighter the rendering style can be. Additionally, the processor 120 may adjust the color temperature of the rendering style by considering information about the color temperature of the visual object 1010, which is a virtual light source. Additionally, the processor 120 may adjust the brightness of the rendering style by considering information about the brightness of the visual object 1010, which is a virtual light source. Additionally, the processor 120 may adjust the brightness of the ambient light in the first image to be the same as the brightness of the ambient light in the second image. Here, ambient light can refer to light that illuminates all surroundings without a specific direction. In other words, ambient light may represent light in which a light source exists, but the direction is lost because the light emitted from the light source is reflected by multiple elements. As described above, the processor 120 may apply a rendering style based on a color effect identified based on information about the visual object 1010, which is a virtual light source. The processor 120 may synthesize a first image including the visual object 1010 and a third image to which a rendering style is applied to the second image and display them on a display.

Referring to FIGS. 1 to 10 , the present disclosure acquires an image (e.g., a third image) to which a rendering style is applied to an image of a real environment (e.g., a second image), and uses the acquired third image and a software application. An image that is a composite of images of the VR environment (e.g., the first image) acquired through can be provided to the user. The rendering style may be identified from information about a software application or from the first image. Embodiments of the present disclosure may display the display by combining the first image with a third image to which a rendering style related to the first image is applied, rather than directly combining the second image and the first image. Accordingly, the electronic device 101 can provide the user with a highly immersive VR environment through more naturally rendered images. In addition, the present disclosure can clearly indicate boundaries between software applications when using a plurality of software applications, so the usability of the user's electronic device 101 can be improved. Additionally, the present disclosure can provide users with a more natural VR (or AR) environment through information exchange between image information about the actual environment and images acquired from a software application in the VR (or AR) environment.

As described above, the wearable electronic device 101 may include a camera 180. The wearable electronic device 101 may include a display 160. The wearable electronic device 101 may include a processor 120. The processor 120 may be configured to identify a first image including a visual object through a software application. When the visual object in the first image has a light-emitting property, the processor 120 may be configured to identify a color effect to be applied to the background image based on the light-emitting property. The processor 120 may be configured to identify a rendering style applied to the first image. The processor 120 converts the second image representing the environment around the wearable electronic device, obtained through the camera 180, based on the color effect and the rendering style, thereby converting the third image into the background. It may be configured to acquire an image. The processor 120 may be configured to display the first image together with the third image through the display. The properties may include luminescence properties, texture properties, or temperature properties. The rendering style may include a cartoon style, a retro style, an outline style, or a black and white style.

According to one embodiment, the processor 120 may be configured to identify an area where the software application is displayed through the display 160. The processor 120 may be configured to obtain information about the rendering style of the software application when the area is the entire area of the display 160.

According to one embodiment, the processor 120 may be configured to identify an area where each of a plurality of software applications including the software application and another software application is displayed through the display 160. The processor 120 may be configured to obtain first information about the rendering style of the software application for a first area where the software application is displayed with respect to the entire area of the display 160. The processor 120 may be configured to obtain second information about the rendering style of the other software application for a second area where the other software application is displayed with respect to the entire area of the display 160.

According to one embodiment, the processor 120 may be configured to obtain a fourth image based on the color effect and the first information with respect to the first area of the second image. The processor 120 may determine a different color effect for the second area of the second image based on a different visual object having different properties in another first image obtained through the different software application and It may be configured to acquire a fifth image based on the second information. The processor 120 may be configured to obtain the third image by combining the fourth image and the fifth image.

According to one embodiment, the processor 120 may be configured to obtain first boundary information about the first area from the software application. The processor 120 may be configured to obtain second boundary information for the second area from the other software application.

According to one embodiment, the processor 120 may be configured to identify the software application among the plurality of software applications based on identifying the area on which the user's gaze of the wearable electronic device focuses. . The processor 120 may be configured to identify a first rendering style region including the first region in which the software application is displayed. The third image may be obtained by converting the second image based on the first information with respect to the first rendering style area of the second image.

According to one embodiment, the processor 120 creates an area expanded with respect to the first rendering style area when the gaze is located within the area corresponding to the display 160 for a certain period of time or more. It can be configured to identify. The third image may be obtained by converting the second image based on the first information with respect to the expanded area of the second image.

According to one embodiment, the processor 120 obtains information about the rendering style based on manifest file information of the software application or determines the rendering style based on executable file information of the software application. It may be configured to obtain information about the rendering style or obtain information about the rendering style based on code information indicating a shader of the software application.

According to one embodiment, when the processor 120 fails to obtain information about the rendering style through the manifest file information, the executable file information, code information indicating the shader, or the first image, the It may be configured to display a plurality of presets determined based on the rendering results of the software application to the user through the display 160. The processor 120 may be configured to obtain information about the rendering style of the software application based on the user's response to one of the plurality of presets.

According to one embodiment, the processor 120 may be configured to identify a partial area of the second image based on an input from a user of the wearable electronic device. The processor 120 may be configured to obtain the third image by converting the partial area of the second image based on information about the color effect and the rendering style.

As described above, the method performed by the wearable electronic device 101 may include an operation 307 of identifying a first image including a visual object through a software application. The method may include an operation 309 of identifying a color effect to be applied to a portion of the background image corresponding to the visual object, based on a property of the visual object in the first image. there is. The method may include an operation 311 of identifying a rendering style applied to the first image. The method converts a second image representing the environment around the wearable electronic device, obtained through the camera 180 of the wearable electronic device 101, based on the color effect and the rendering style, to a third image. An operation 313 of acquiring an image as the background image may be included. It may include an operation 315 of displaying the first image along with the third image through a display of the wearable electronic device. The properties may include luminescence properties, texture properties, or temperature properties. The rendering style may include a cartoon style, a retro style, an outline style, or a black and white style.

According to one embodiment, the method may include an operation of identifying an area where the software application is displayed through the display 160. The method may include an operation of obtaining information about the rendering style of the software application when the area is the entire area of the display 160.

According to one embodiment, the method may include an operation of identifying an area where each of a plurality of software applications including the software application and another software application is displayed through the display 160. The method may include an operation of obtaining first information about a rendering style of the software application for a first area where the software application is displayed with respect to the entire area of the display 160. The method may include an operation of obtaining second information about a rendering style of the other software application for a second area where the other software application is displayed with respect to the entire area of the display 160.

According to one embodiment, the method may include obtaining a fourth image based on the color effect and the first information with respect to the first area of the second image. The method is based on a different color effect and the second information based on a different visual object with different properties in another first image obtained through the different software application for the second region of the second image. This may include an operation of acquiring a fifth image. The method may include obtaining the third image by combining the fourth image and the fifth image.

According to one embodiment, the method may include obtaining first boundary information about the first area from the software application. The method may include obtaining second boundary information for the second area from the other software application.

According to one embodiment, the method may include identifying the software application among the plurality of software applications based on identifying an area on which the user's gaze of the wearable electronic device focuses. The method may include identifying a first rendering style region that includes the first region in which the software application is displayed. The third image may be obtained by converting the second image based on the first information with respect to the first rendering style area of the second image.

According to one embodiment, the method includes an operation of identifying an extended area with respect to the first rendering style area when the gaze is located within the area corresponding to the display 160 for a certain period of time or more. may include. The third image may be obtained by converting the second image based on the first information with respect to the expanded area of the second image.

According to one embodiment, the method An operation of obtaining information about the rendering style based on manifest file information of the software application, or an operation of obtaining information about the rendering style information based on executable file information of the software application. contains, or It may include obtaining information about the rendering style based on code information indicating a shader of the software application.

According to one embodiment, when the method fails to obtain information about the rendering style through the manifest file information, the executable file information, the code information indicating the shader, or the first image, the software application An operation of displaying a plurality of presets determined based on rendering results to the user may be included. The method may include obtaining information about the rendering style of the software application based on the user's response to one of the plurality of presets.

According to one embodiment, the method may include identifying a partial area of the second image based on an input from a user of the wearable electronic device 101. The method may include obtaining the third image by converting the partial area of the second image based on information about the color effect and the rendering style.

Electronic devices according to embodiments disclosed in this document may be of various types. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to embodiments of this document are not limited to the above-described devices.

The various embodiments of this document and the terms used herein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various changes, equivalents, or replacements of the embodiments. In connection with the description of the drawings, similar reference numbers may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the relevant context clearly indicates otherwise. As used herein, “A or B”, “at least one of A and B”, “at least one of A or B”, “A, B or C”, “at least one of A, B and C”, and “A Each of phrases such as “at least one of , B, or C” may include any one of the items listed together in the corresponding phrase, or any possible combination thereof. Terms such as "first", "second", or "first" or "second" may be used simply to distinguish one element from another, and may be used to distinguish such elements in other respects, such as importance or order) is not limited. One (e.g. first) component is said to be "coupled" or "connected" to another (e.g. second) component, with or without the terms "functionally" or "communicatively". Where mentioned, it means that any of the components can be connected to the other components directly (e.g. wired), wirelessly, or through a third component.

The term “module” used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and is interchangeable with terms such as logic, logic block, component, or circuit, for example. It can be used as A module may be an integrated part or a minimum unit of the parts or a part thereof that performs one or more functions. For example, according to one embodiment, the module may be implemented in the form of an application-specific integrated circuit (ASIC).

Various embodiments of the present document are one or more instructions stored in a storage medium (e.g., built-in memory 136 or external memory 138) that can be read by a machine (e.g., electronic device 101). It may be implemented as software (e.g., program 140) including these. For example, a processor (e.g., processor 120) of a device (e.g., electronic device 101) may call at least one command among one or more commands stored from a storage medium and execute it. This allows the device to be operated to perform at least one function according to the at least one instruction called. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. A storage medium that can be read by a device may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' only means that the storage medium is a tangible device and does not contain signals (e.g. electromagnetic waves). This term refers to cases where data is stored semi-permanently in the storage medium. There is no distinction between temporary storage cases.

According to one embodiment, methods according to various embodiments disclosed in this document may be provided and included in a computer program product. Computer program products are commodities and can be traded between sellers and buyers. The computer program product may be distributed in the form of a machine-readable storage medium (e.g. compact disc read only memory (CD-ROM)) or through an application store (e.g. Play Store™) or on two user devices (e.g. It can be distributed (e.g. downloaded or uploaded) directly between smart phones) or online. In the case of online distribution, at least a portion of the computer program product may be at least temporarily stored or temporarily created in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

According to various embodiments, each component (e.g., module or program) of the above-described components may include a single or plural entity, and some of the plurality of entities may be separately placed in other components. there is. According to various embodiments, one or more of the components or operations described above may be omitted, or one or more other components or operations may be added. Alternatively or additionally, multiple components (eg, modules or programs) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each component of the plurality of components identically or similarly to those performed by the corresponding component of the plurality of components prior to the integration. . According to various embodiments, operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, or omitted. Alternatively, one or more other operations may be added.

Claims (20)

In the wearable electronic device 101,
camera (180);
display (160); and
Includes a processor 120,
The processor 120,
Identifying a first image containing a visual object through a software application,
Based on properties of the visual object in the first image, identify a color effect to be applied to a portion of the background image corresponding to the visual object,
Identifying a rendering style applied to the first image,
Obtaining a third image as the background image by converting a second image representing the environment around the wearable electronic device, obtained through the camera 180, based on the color effect and the rendering style,
configured to display the first image together with the third image through the display,
The properties include luminescence properties, texture properties, or temperature properties,
The rendering style includes a cartoon style, a retro style, an outline style, or a black and white style,
Wearable electronic devices (101).
In claim 1,
The processor 120:
The software application identifies an area displayed through the display 160,
configured to obtain information about the rendering style of the software application when the area is the entire area of the display 160,
Wearable electronic devices (101).
The method of any one of claims 1 and 2,
The processor 120:
Identifying an area where each of a plurality of software applications including the software application and another software application is displayed through the display 160,
Obtaining first information about a rendering style of the software application for a first area where the software application is displayed with respect to the entire area of the display 160,
configured to obtain second information about a rendering style of the other software application for a second area where the other software application is displayed with respect to the entire area of the display 160,
Wearable electronic devices (101).
In claim 3,
The processor 120:
Obtaining a fourth image based on the color effect and the first information for the first area of the second image,
Based on the second information and a different color effect determined based on a different visual object having different properties in another first image obtained through the different software application for the second area of the second image Acquire a fifth image,
configured to obtain the third image by combining the fourth image and the fifth image,
Wearable electronic devices (101).
In claim 3,
The processor 120:
Obtain first boundary information for the first area from the software application,
configured to obtain second boundary information for the second area from the other software application,
Wearable electronic devices (101).
In claim 3,
The processor 120:
Identifying the software application among the plurality of software applications based on identifying an area on which the user's gaze of the wearable electronic device focuses,
configured to identify a first rendering style region comprising the first region in which the software application is displayed;
The third image is obtained by converting the second image based on the first information with respect to the first rendering style area of the second image,
Wearable electronic devices (101).
In claim 6,
The processor 120:
The gaze is configured to identify an extended area with respect to the first rendering style area when the software application is located within an area corresponding to the display 160 for a certain period of time or more,
The third image is obtained by converting the second image based on the first information with respect to the expanded area of the second image,
Wearable electronic devices (101).
The method of any one of claims 1 to 7,
The processor 120,
Obtain information about the rendering style based on manifest file information of the software application, or
Obtain information about the rendering style based on executable file information of the software application, or
Configured to obtain information about the rendering style based on code information indicating a shader of the software application,
Wearable electronic devices (101).
In claim 8,
The processor 120,
If the information on the rendering style is not obtained through the manifest file information, the executable file information, the code information indicating the shader, or the first image, a plurality of images determined based on the rendering result of the software application Displaying presets to the user through the display 160,
configured to obtain information about the rendering style of the software application based on the user's response to one of the plurality of presets,
Wearable electronic devices (101).
The method of any one of claims 1 to 9,
The processor 120,
Based on input from a user of the wearable electronic device, identify a partial region of the second image,
configured to obtain the third image by converting the partial area of the second image based on information about the color effect and the rendering style,
Wearable electronic devices (101).
In a method performed by a wearable electronic device 101,
Identifying a first image containing a visual object through a software application (307),
An operation 309 of identifying a color effect to be applied to a portion of a background image corresponding to the visual object, based on a property of the visual object in the first image,
An operation 311 of identifying a rendering style applied to the first image,
By converting the second image representing the environment around the wearable electronic device, obtained through the camera 180 of the wearable electronic device, based on the color effect and the rendering style, the third image is converted to the background image. comprising an operation 313 of acquiring, and
An operation 315 of displaying the first image together with the third image on a display of the wearable electronic device,
The properties include luminescence properties, texture properties, or temperature properties,
The rendering style includes a cartoon style, a retro style, an outline style, or a black and white style,
method.
In claim 11,
The above method is:
An operation of identifying an area where the software application is displayed through the display 160;
When the area is the entire area of the display 160, comprising obtaining information about the rendering style of the software application,
method.
The method of any one of claims 1 and 2,
The above method is:
identifying an area where each of a plurality of software applications including the software application and another software application is displayed through the display 160;
Obtaining first information about a rendering style of the software application for a first area where the software application is displayed with respect to the entire area of the display 160;
Comprising an operation of acquiring second information about the rendering style of the other software application for a second area where the other software application is displayed with respect to the entire area of the display 160,
method.
In claim 13,
The above method is:
An operation of acquiring a fourth image based on the color effect and the first information for the first area of the second image,
A fifth color effect based on the second information and a different color effect based on a different visual object with different properties in another first image acquired through the different software application with respect to the second area of the second image. An operation to acquire an image;
Comprising an operation of obtaining the third image by combining the fourth image and the fifth image,
method.
In claim 13,
The above method is:
Obtaining first boundary information for the first area from the software application;
Comprising the operation of obtaining second boundary information for the second area from the other software application,
method.
In claim 13,
The above method is:
An operation of identifying the software application among the plurality of software applications based on identifying an area on which the user's gaze of the wearable electronic device is focused;
Identifying a first rendering style region including the first region in which the software application is displayed,
The third image is obtained by converting the second image based on the first information with respect to the first rendering style area of the second image,
method.
In claim 16,
The above method is:
When the gaze is located within the area corresponding to the display 160 for a certain period of time or more, an operation of identifying an extended area with respect to the first rendering style area,
The third image is obtained by converting the second image based on the first information with respect to the expanded area of the second image,
method.
The method of any one of claims 11 to 17,
The above method is:
Obtaining information about the rendering style based on manifest file information of the software application;
Obtaining information about the rendering style information based on executable file information of the software application; or
Comprising an operation of obtaining information about the rendering style based on code information indicating a shader of the software application,
method.
In claim 18,
The above method is:
If the information on the rendering style is not obtained through the manifest file information, the executable file information, the code information indicating the shader, or the first image, a plurality of images determined based on the rendering result of the software application Displaying presets to a user;
Comprising the operation of obtaining information about the rendering style of the software application based on the user's response to one of the plurality of presets,
method.
The method of any one of claims 11 to 19,
The above method is:
identifying a partial area of the second image based on an input from a user of the wearable electronic device 101;
Comprising an operation of obtaining the third image by converting the partial area of the second image based on information about the color effect and the rendering style,
method.

Images